ksh93s+ (ast-ksh.2008-02-02) is out (see release annoucement)
AT&T Laboratories has released a new version of the AST Toolkit which contains ksh93, nmake, and about 150 UNIX tools as well as several libraries including SFIO, (SafeFast/IO).
ksh93, an implementation of the POSIX/IEEE Std 1003.1 shell specification, is the latest version of KornShell ; since the last major release the following changes were made:
ksh93/libshell changes:
08-01-31 A bug in which command substitution inside ((...)) could cause
syntax errors or lead to core dumps has been fixed.
08-01-17 A bug in which discipline functions could be deleted when invoked
from a subshell has been fixed.
08-01-17 A bug in which a command substitution consisting only of
assignments was treated as a noop has been fixed.
08-01-17 A bug in which discipline functions invoked from withing a
compound assignment could fail has been fixed.
08-01-16 Incomplete arithmetic assigments, for example ((x += )), now
generate an error message.
08-01-16 A bug in which a set discipline defined for a variable before
an array assignment could cause a core dump has been fixed.
08-01-03 A bug in on some systems in which exit status 0 is incorrectly
returned by a process that catches the SIGCONT signal is stopped
and then continued.
07-12-13 A race condition in which a program that has been stopped and then
continued could loose the exit status has been fixed.
07-12-12 Code to check for file system out of space write errors for all
writes has been added.
07-12-11 A bug in the macro expander for multibyte characters in which
part of the character contains a file pattern byte has been fixed.
07-12-06 A bug in the emacs edit mode when multiline was set that output
a backspace before the newline to the screen has been fixed.
07-12-04 A bug in which using <n>TAB after a variable name listing expansion
in the edit modes would cause the $ to disappear has been fixed.
07-11-28 A bug in which setting IFS to readonly could cause a subsequent
command substitution to fail has been fixed.
07-11-27 A work around for a gcc 4.* C99 "feature" that could cause a job
control shell to go into an infinite loop by adding the volatile
attribute to some auto vars in functions that call setjmp().
07-11-27 A bug in which the shell could read ahead on a pipe causing the
standard input to be incorrectly positioned has been fixed.
07-11-27 A bug in which compound variable UTF-8 multibyte values were not
expanded or traced properly has been fixed.
07-11-21 A bug where an unbalanced '[' in a command argument was not treated
properly has been fixed.
07-11-15 A bug in which compatibility mode (no long option names) getopts(1)
incorrectly set the value of OPTARG for flag options has been fixed.
07-11-15 A bug in which "hash -- name" treated "--" as an invalid name operand
has been fixed.
07-11-15 typeset now handles "-t -- [-r] [--]" for s5r4 hash(1) compatibility.
07-11-15 A bug in which the umask builtin mis-handled symbolic mode operands
has been fixed.
07-11-15 Bugs in which shell arithmetic and the printf builtin mis-handled the
signs of { -NaN -Inf -0.0 } have been fixed.
07-11-15 The full { SIGRTMIN SIGRTMIN+1 ... SIGRTMAX-1 SIGRTMAX } range
of signals, determined at runtime, are now supported.
07-11-15 A bug in which creating an index array with only subscript 0 created
only a simple variable has been fixed.
07-11-14 A bug in which appending to an indexed array using the form
name+=([sub]=value) could cause the array to become an associative
array has been fixed.
07-11-14 A bug in which typeset without arguments could coredump if a
variable is declared as in indexed array and has no elements has
been fixed.
07-11-14 A bug in which creating a local SECONDS variable with typeset in
a function could corrupt memory has been fixed.
07-11-14 A bug which could cause a core dump when a script invoked by name
from a function used compound variables has been fixed.
07-11-05 A bug in which printf %d "'AB" did not diagnose unconverted characters
has been fixed.
07-11-05 printf %g "'A" support added for all floating point formats.
libcmd changes:
08-01-30 expr.c: fix <=0 type that broke substr * 1 * -- wow
07-12-13 cp.c: fix builtin state reinitialization
07-11-29 rev.c: honor multibyte locales
07-11-27 cp.c: open non-existent destination with O_EXCL
07-11-27 stty.c: add -t,--terminal-group to list tty pgrp
07-11-27 cksum.c: --silent -s => -S, -s == -x sys5 for gnu compatibility
07-11-11 tee.c: drop ancient bsd compatibility "-" operand => SIGINT
libast changes:
08-02-02 string/strmatch.c: fix str="" pat="" sub values
08-01-31 comp/conf.sh,conf.tab: handle /bin/sh \ in read data, redir subshell
08-01-18 misc/magic.tab: amd-x68, 64-bit => x86-64
08-01-18 string/strnton.c,strntonll.c: add
07-12-10 string/strelapsed.c: "0" is a valid elapsed time!
07-12-02 sfio/sfreserve.c: preserve SF_SHARE sfrd() via sfreserve(f,0,0)
07-11-21 comp/setlocale.c: add sjis_mbtowc() to work around [\~] translation
07-11-15 features/signal.c: RT(1) .. RT(MAX-1) => RTMIN+1 .. RTMAX-1
07-11-14 features/float: favor sscanf() due to gnu strto[l]d() nan bugs
Happy hacking! :-)
The upcoming Indiana Preview2 is going to ship ksh93 as /usr/bin/sh and /sbin/sh.
Please test this change extensively and report any bugs to the project bugzilla.
Today I posted a prototype patch which adds SAX-style XML parsing support to ksh93 via a new "xmlsaxparse" builtin command.
The idea is to provide a builtin XML parsing facility to handle the
shortcomings of a XML parser written in ksh93 code, e.g. …
- … it's difficult to handle encoding conversions (e.g. document inASCII, ISO8859-1 or UTF-8 while the shell runs in the current user's
locale. It could be solved via
/usr/bin/iconv but this is _slow_ because
we would need to do it per word-basis)
- … it's slow (assuming the parser should really handle all details of XML)
- … doesn't handle any extra XML features like external entity support, namespaces, DTD, Schemas etc. unless the ksh93 script code becomes very large&&complex
AFAIK it may be better to handle the issue via builtin XML support based
on libxml2 which is loaded via libbusybox.so.1
Example usage (the usage assumes some knowledge how the SAX API works):
- Create an associative array which contains the sames of shell functions which should be called per SAX callback
- Load the "
xmlsaxparse" builtin
- Run the "
xmlsaxparse" command to parse a specific document
("myfile.xml" in the example below). The builtin will call the matching
callbacks for each SAX event hit.
Example code fragment:
function run_xml_sax
{
builtin -f libshell.so.1 xmlsaxparse || fatal_error "xmlsaxparse builtin not found."
typeset -A xcallbacks=(
["startelement"]="mystartelement_callback"
["startdocument"]="mystartdocument_callback"
["enddocument"]="myenddocument_callback"
["characters"]="mytextdata_callback"
)
xmlsaxparse -u "user-defined_data_string" -c xcallbacks "myfile.xml"
}
The callback functions are called with the callback name itself passed
as arg1 (if "xmlsaxparse" is called with the "-c" argument, otherwise
the callback name is obmitted) and a name of a (temporary) compound
variable as arg2 (and the variable "\<compoundvar\>.userdata" contains the
string passed via the "-u" option to "xmlsaxparse"). The SAX parser will abort the SAX run if any registered callback function returns a non-zero
return code.
Notes:
- I've modified the "
xmldocumenttree1" and "rssread" demos to use the"xmlsaxparse" builtin if they are called using the "-S" option
- The code automagically converts the document encoding to the currentlocale/encoding of the shell, even if LC_*/LANG is changed in one of the
callbacks
- The code is fully re-entrant, e.g. a callback function may itselfstart another "
xmlsaxparse" session/run
- The patch is only a technology demonstrator. AFAIK a better way may be to use libxml2's "xmlreader" API which won't require a complex set of callbacks. I'll try to post a prototype patch for that either tonight or later next week
Comments/rants/etc welcome…
ksh93s+ (ast-ksh.2007-11-05) is out (see release annoucement)
AT&T Laboratories has released a new version of the AST Toolkit which contains ksh93, nmake, and about 150 UNIX tools as well as several libraries including SFIO, (SafeFast/IO).
ksh93, an implementation of the POSIX/IEEE Std 1003.1 shell specification, is the latest version of KornShell ; since the last major release the following changes were made:
ksh93 changes:
07-11-05 --- Release ksh93s+ ---
07-11-05 A bug in which printf %d "'AB" did not diagnose unconverted characters
has been fixed.
07-11-05 printf %g "'A" support added for all floating point formats.
07-11-01 A bug in which typeset -f fun did not display the function definition
when invoked in a subshell has been fixed.
07-10-29 The sleep builtin was fixed so that all floating point constants
are valid operands.
07-10-10 A bug in which the locale was not being restored after
LANG=value command has been fixed.
07-09-20 A bug in which a nameref to a compound variable that was local
to the calling function would not expand correctly when displaying
is value has been fixed.
07-09-19 A bug which cause cause a core dump if .sh.edchar returned
80 characters or more from a keyboard trap has been fixed.
07-09-14 A bug in which could cause a core dump when more than 8 file
descriptors were in use has been fixed.
07-09-10 A bug in which creating a name reference to an instance of
an array when the array name is itself a reference has been fixed.
07-09-10 The file completion code has been modified so that after an = in
any word, each : will be considered a path delimiter.
07-09-06 A bug in which subprocess cleanup could corrupt the malloc() heap
has been fixed.
07-08-26 A bug in which a name reference to an associatve array instance
could cause the subscript to be evaluated as an arithmetic expression
has been fixed.
07-08-22 A bug in which the value of an array instance was of a compound
variable was not expanded correctly has been fixed.
07-08-14 A bug which could cause a core dump when a compound assignment was
made to a compound variable element with a typeset -a attribute
has been fixed.
07-08-08 A bug in which a trap ignored in a subshell caused it to be
ignored by the parent has been fixed.
07-08-07 A bug in which the set command would generated erroneous output
for a variable with the -RZ attribute if the variable name had been
passed to a function has been fixed.
07-08-02 A bug in which read x[1] could core dump has been fixed.
07-08-02 A second bug in which after read x[sub] into an associative array
of an element that hasn't been assigned could lead to a core dump
has been fixed.
07-07-31 A bug in which a pipeline that completed correctly could have
an exit status of 127 when pipefail was enabled has been fixed.
07-07-09 The SHOPT_AUDIT compile option has been added for keyboard logging.
07-06-25 In vi insert mode, ksh no longer emits a backspace character
before the carraige return when the newline is entered.
07-06-25 A bug in which pipefail would cause a command to return 0
when the pipeline was the last command and the failure happened
on a component other than the last has been fixed.
07-06-25 A bug in the expansion of ${var/pattern/rep} when pattern or rep
contained a left parenthesis in single quotes has been fixed.
07-06-18 The braces for a subscripted variable with ${var[sub]} are now
optional when inside [[...]], ((...)) or as a subscript.
07-05-28 A bug in brace expansion in which single and double quotes did
not treat the comma as a literal character has been fixed.
07-05-24 The -p option of whence now disables -v.
07-05-23 Several bug fixes in compound variables and arrays of arrays
have been made.
07-05-15 A bug in which the %B format of printf was affected by the
locale has been fixed.
07-05-14 A bug in which \ was not removed in the replacement pattern with
${var/pattern/rep} when it was not followed by \ or a digit has
been fixed.
07-05-10 A bug in which ksh -R file core dumped if no script was specified
has been fixed. it not displays an error message.
07-05-07 Added additional Solaris signals to signal table.
07-04-30 A bug in which a pipeline with command substitution inside a
function could cause a pipeline that invokes this function to
hang when the pipefail option is on has been fixed.
07-04-30 Added -q to whence.
libcmd changes:
07-10-29 cksum.c: add SUM_LEGACY for -r
07-10-12 cp.c: plug usage string memory leak by using per-builtin state
07-09-21 cksum.c: add sumprint() default scale arg, --scale, --bsd for solaris
07-09-10 chmod.c: add --show,-n
07-07-27 wclib.c: bias <wchar.h> checks for modern unix
07-07-17 cat.c: fix --squeeze-blank to reduce multiple blank lines to *one*
07-05-20 cmd.h: handle msvc's balk at if(0)0=0;
07-05-20 cksum.c: #include <modex.h>
07-05-11 cmd.h: add _CMD_CONTEXT_OK() to verify >= 20070511 context
07-05-09 fds.c: handle ipv6 sockets
07-05-09 cmd.h: <shbltin.h> : cmdquit() => sh_checksig(context)
07-04-25 mkdir.c: force (S_ISVTX|S_ISUID|S_ISGID) after mkdir(2)
07-04-24 procrun.c: add -last intercept => sh_run() and whence -q
07-04-19 uname.c: name operands first checked for CS_NAME, then NAME
07-03-28 date.c: add --unelapsed=scale, -U: fmtelapsed() => strelapsed()
libast changes:
07-10-31 regex/regcomp.c: fix REX_COLL_CLASS node allocation size
07-10-31 sfio/sfcvt.c: use signbit() if available
07-10-31 features/isoc99: _ISOC99_SOURCE tests
07-10-31 port/astmath.c: add -DN=8 for signbit()
07-10-31 sfio/sfstrtod.h: don't forget about -0.0
07-10-26 features/map.c: add { optopt optarg optind opterr }
07-10-26 features/stdio: add _filbuf => _ast__filbuf
07-10-26 comp/getsubopt.c: fix #undef that interfered with <ast_map.h>
07-10-26 regex/regcomp.c: fix bug that missed ')' in ~(F)…
07-10-12 port/astconf.c: fix CONF_ALLOC 16 bit overflow
07-10-12 misc/fts.c: fix fts_close() to free the handle -- doh
07-10-11 comp/setlocale.c: second and subsequent setlocale(*,"") reverts to previous
07-10-11 path/pathprobe.c: add vfs ST_NOSUID check
07-10-10 comp/conf.tab: add a few more xpg6 deferrals
07-09-28 astsa: update to share with mainline src via _PACKAGE_astsa
07-09-25 sfio/sfgetr.c: no limit on string stream line size
07-09-25 sfio/sfextern.c: increase _Sfmaxr to 256*1024
07-09-18 misc/procopen.c: tighten up SIGCHLD logic between parent/child
07-09-18 misc/signal.c: unblock SIG_DFL after setting handler, sig<0 => don't unblock
07-09-13 misc/fs3d.c: no $LD_PRELOAD => no 3d and avoids invalid mount(2) call
07-09-11 vmalloc: vmstat(0,0)==1 => region in use, drop VM_primary|VM_secondary
07-09-05 misc/recstr.c: handle [lL] gobbled by strtol() -- ouch
07-08-17 path/pathprobe.c: handle '\r' in VERSION string
07-07-17 regex/regcache.c: regcache(0,n,0) extends cache to size n (no shrinking)
07-07-16 tm/tmdata.c: add 2005-12-31, drop 1999-12-31 (where did that come from?)
07-05-21 tm/tmxfmt.c,tmxscan.c: %F => %L (TM_DEFAULT); %F => %Y-%m-%d
07-05-15 sfio/sfvprintf.c: %h? and SFFMT_SHORT => raw bytes
07-05-09 features/signal.c,features/siglist: use kill -l & strsignal()
07-04-25 misc/optctx.c: add for opt_info switching
07-04-24 misc/cmdarg.c,include/cmdarg.h: add CMD_CHECKED, CMD_SILENT
07-04-24 misc/procopen.c,include/proc.h: add PROC_CHECK
07-04-24 misc/procrun.c: add flags arg (current use PROC_ARGMOD)
07-04-24 misc/cmdarg.c,include/cmdarg.h: move from src/cmd/tw
07-04-20 port/(lclang.h|lc.c|mc.c|lclib.h|lcgen.c): separate lctab.c
07-04-20 comp/conf.sh: defer to systems without 'grep -q' -- sigh
07-04-20 comp/conf.sh: probe for LL integer constant initializer suffix
07-04-20 include/syslog.h: <namval.h> => <ast_namval.h> for win32
07-04-20 ast_namval.h: add as copy of include/namval.h for win32
07-04-19 comp/conf.tab: fix SVID SI entries to probe SI_* (not _SI_*)
Note:
The ksh93-integration 2007-11-05 release is based on ast-ksh-2007-11-01 (which is more or less identical to ast-ksh-2007-11-05).
Happy hacking! :-)
These tarballs are intended to be installed over an existing OpenSolaris
i386 or SPARC installation (>= Nevada B72) to update the provided ksh93
installation to ksh93s+_20071101.
Note that the binaries are provided for testing and evaluation purposed
ONLY.
Please report any problems/errors/bugs/comments to the ksh93-integration
project bugzilla or the ksh93-integration mailinglist.
- "
multiline" editor mode allows to edit lines in a similar fashion as"readline"-based applications like "bash"
/etc/ksh.kshrc now sets a default prompt for interactive sessions- Many bugfixes
- Many closed-source binaries are replaced with opensource versions
- The shell script compiler "
shcomp" is now included
- Many new demos including:
- "
shnote" (utilty to read/write opensolaris.pastebin.ca)
- "
shtwitter" (access twitter.com service and push blog entries)
- "
shpiano" (audio demo)
- "
shtinyurl" (tool to generate short tinyurl.com URLs)
- "
shman" (early prototype of a /usr/bin/man rewrite)
- "
svcproptree1" (SMF demo)
- "
xmldocumenttree1" (XML parser demo)
- "
test_net_sctp" (SCTP demo)
- "
multifollow" (poll(1) demo which run multiple "tail -f")
- … and much more…
The full annoucement can be found here.
Happy testing… :-)
