Have added features to the server command.

The new features should be tested in the test suite.

-unescape should convert escape sequences into their proper form:

string -unescape ${ This\tis\na\    test \ . }$

turns into

"This	is
a test  . "

Note the trailing space, and that there are two spaces after "test" because of
the position of the backslash, and only one after "a" because normal string
parsing rules would still apply before the unescape, turning that segment into
"a\ test")

Escape would return the same string as was unescaped (not in the same
pre-parsed representation, though, this would still be just "This\tis\na
test . ", as there is no way of knowing exactly which spaces were escaped
in which way). The guarantee is that @string == unescape(escape(@string)),
but not that @string == escape(unescape(@string)).

Unknown escapes should throw an error, in order to leave illegal sequences that
can be extended into legal sequences in the future.

Should be able to store keys. Currently user has to provide a keystore manually.

Because the invocation of the jvm doesn't quote the gensh_runtime_opts, any splats or other relevant characters in passed-in values will glob on the filesystem. This is particularly troublesome for regex operation. Maybe a set -o noglob or something should be specified, or a way to pass gensh_runtime_opts into ublu as a single raw string should be devised.

Sample of problem:

foo.ublu:

FUNC foo ( argument ) $[
    put @@argument
]$

ublu exec:

> gensh -to foo.sh -path /opt/ublu/ublu.jar -includepath $SCRIPTDIR -optr a ARGUMENT @argument ${ Example argument }$ ${ foo.sh: example script }$ foo.ublu ${ foo ( @argument ) }$

$ ls
alpha  beta  foo.sh  foo.ublu

$ sh foo.sh silent -a 'this is a test'  
this is a test

$ sh foo.sh silent -a 'this is a * test'
this is a alpha beta foo.sh foo.ublu test

$ sh foo.sh silent -a "'this is a * test'"
'this is a alpha beta foo.sh foo.ublu test'

$ sh foo.sh silent -a "'this is a \* test'" 
'this is a \* test'

$ sh foo.sh silent -a 'this is a \* test' 
this is a \* test

If we put set -o noglob above the invocation:

$ sh foo.sh silent -a 'this is a * test' 
this is a * test

When using ftp -get with an absolute path, the command fetches from an absolute path to an absolute path, rather than to the local directory (which is more expected behavior with FTP). This is very likely not desired. (for instance, if I use ftp -get /tmp/testfile.txt, it fetches it from the system's /tmp/testfile.txt to the local machine's /tmp/testfile.txt, regardless of the current working directory. This won't work if you're trying to fetch a file by absolute path to a directory that doesn't exist on the local machine. I just ran into this issue with trying to fetch /QIBM/qzrdhasm/qzrdhasm.log from an as400 machine)

The subsystem command doesn't populate most of the subsystem's attributes without a -refresh being run:

> subsys -as400 @as400 -to @subsys -new -subsyspath /QSYS.LIB/QINTER.SBSD
> @subsys -query status
null
> @subsys -refresh
> @subsys -query status
*ACTIVE

This isn't necessarily a bug in the code, but the documentation doesn't explain this, so it can be confusing. This should either be changed in the code (to refresh the subsystem before returning it in getSubsystem) or in the documentation (explaining that getting the subsystem doesn't populate most of its data, and an initial -refresh is necessary). I think the former makes a little more sense, as this means that there isn't a way to access a subsystem's status or some other attributes without assigning to a tuple (though that may be desirable).

MessageQueue notes that getMessages fetches in blocks of 1000 messages, but it looks like that's an implementation detail, and not an invitation to call getMessages over and over again:

> put -to @queue /qsys.lib/qsysopr.msgq
> calljava -to @mq -new com.ibm.as400.access.MessageQueue -arg @as400 -arg @queue
> calljava -- @mq -method getLength
41668
> calljava -to @enum -- @mq -method getMessages
> num -to @i -long 0
> tuple -true @keepgoing
> \\ ${ Iterate all messages in enum, incrementing @i for each }$
> WHILE @keepgoing $[ calljava -to message -- @enum -method nextElement
calljava -to @keepgoing -- @enum -method hasMoreElements
eval -to @i + @i 1 ]$
> put @i
41668
> WHILE @keepgoing $[ calljava -to message -- @enum -method nextElement
calljava -to @keepgoing -- @enum -method hasMoreElements
eval -to @i + @i 1 ]$
> put @i
83336

Here in MsgQ, though, it will apparently iterate the message queue 1000 times for every message that is in the queue, because getMessages always returns an enumerable to the beginning of the queue: https://github.com/jwoehr/ublu/blob/master/src/ublu/command/CmdMsgQ.java#L398-L404
It should be fixable just by getting rid of the outer for loop and iterating the enumeration.

For one of these hosts, we have 40k messages, which should be manageable, but it ends up getting an OutOfMemoryError even when given a Java heap size of 2G:

> msgq -as400 @as400 -to @messages -all /qsys.lib/qsysopr.msgq
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at com.ibm.as400.access.DataStreamCompression.decompressRLE(DataStreamCompression.java:446)
	at com.ibm.as400.access.RCCallProgramReplyDataStream.getParameterList(RCCallProgramReplyDataStream.java:55)
	at com.ibm.as400.access.RemoteCommandImplRemote.runProgramOffThread(RemoteCommandImplRemote.java:581)
	at com.ibm.as400.access.RemoteCommandImplRemote.runProgram(RemoteCommandImplRemote.java:532)
	at com.ibm.as400.access.ProgramCall.run(ProgramCall.java:785)
	at com.ibm.as400.access.ListUtilities.retrieveListEntries(ListUtilities.java:302)
	at com.ibm.as400.access.MessageQueue.getMessages(MessageQueue.java:513)
	at com.ibm.as400.access.QueuedMessageEnumeration.nextElement(QueuedMessageEnumeration.java:72)
	at ublu.command.CmdMsgQ.readMessageQueue(CmdMsgQ.java:424)
	at ublu.command.CmdMsgQ.msgq(CmdMsgQ.java:288)
	at ublu.command.CmdMsgQ.cmd(CmdMsgQ.java:454)
	at ublu.util.Interpreter.loop(Interpreter.java:1222)
	at ublu.util.Interpreter.interpret(Interpreter.java:1422)
	at ublu.Ublu.runMainInterpreter(Ublu.java:337)
	at ublu.Ublu.niam(Ublu.java:397)
	at ublu.Ublu.main(Ublu.java:366)

Right now, I'm working around this by doing it all with calljava manually.

An include inside an include pointing to an non-existent file causes a system exit.

$ cat foo.ublu 
include bar.ublu

bar doesn't exist.

Ublu will exit.

Given the following functions:

FUNC first ( ) $[
    second ( )
]$

FUNC second ( ) $[
    put -to @val foo
    TASK -to NULL: -start $[
        put @val
    ]$
]$

A call to first produces:

> first ( )
foo

But, if I add a LOCAL @Val to first, even if it does nothing:

FUNC first ( ) $[
    LOCAL @val
    second ( )
]$

FUNC second ( ) $[
    put -to @val foo
    TASK -to NULL: -start $[
        put @val
    ]$
]$

> first ( )
null

This breaks the ability of threads to reliably deal with the tuple stack, as a local of the same name can kill it. I've also noticed that it varies; if I add a sleep to first after the call to second, it will work as expected, if there was previously a non-local @val, that previous value will be accessed instead of the local one. My guess is that this occurs because the main thread kills the tuple stack for @val before the child thread has a chance to access it.

I couldn't find a way for threads to access tuple variables that they don't exclusively control without risk of the tuples being mutated under them. Example of weird behavior with global state existing:

put -to @val baz

FUNC localfirst ( ) $[
    LOCAL @val
    second ( )
]$

FUNC first ( ) $[
    second ( )
]$

FUNC second ( ) $[
    put -to @val foo
    TASK -to NULL: -start $[
        put @val
    ]$
]$

> localfirst ( )
baz
> first ( )
foo
> localfirst ( )
foo

So which tuple is accessed also changes based on order of calls.

    LOCAL @ftp
    ftp -cd /tmp -session @ftp @host @username @password

This generates the following exception:

Ublu:1:UbluInterpreter.Thread[main,5,main]:SEVERE:ublu.command.CmdFTP.ftp():Session tuple @ftp does not reference FTP.
I can't find a way to create an ftp session that doesn't depend on a nonexistent global tuple.

> put -to @string ${ this is a test }$
> put -to @test test
> calljava -obj @string -method contains -arg @test
Ublu:1:UbluInterpreter.Thread[main,5,main]:SEVERE:ublu.command.CmdCallJava.cmdCallJava():Error looking up method in calljava/0 [-to @datasink] [-new ~@{classname}] [--,-obj ~@object] [-method ~@{methodname}] [-arg ~@argobj [-arg ..]] [-primarg ~@argobj [-primarg ..]] : call Java method
java.lang.String.contains(java.lang.String)
java.lang.NoSuchMethodException: java.lang.String.contains(java.lang.String)
	at java.lang.Class.getDeclaredMethod(Class.java:2130)
	at ublu.util.JavaCallHelper.<init>(JavaCallHelper.java:70)
	at ublu.command.CmdCallJava.cmdCallJava(CmdCallJava.java:118)
	at ublu.command.CmdCallJava.cmd(CmdCallJava.java:173)
	at ublu.util.Interpreter.loop(Interpreter.java:1140)
	at ublu.util.Interpreter.interpret(Interpreter.java:1317)
	at ublu.Ublu.runMainInterpreter(Ublu.java:194)
	at ublu.Ublu.niam(Ublu.java:249)
	at ublu.Ublu.main(Ublu.java:229)
> 

This should work. I've noticed some similar issues using calljava to create and work with HashMaps.

The rs command needs some work, e.g., the -metadata dashcommand (using -from and ignoring -- value).