Would you add an uninstall script? I assume just deleting the folder wouldn't be enough?

Inspiration: galaxyproject/tools-iuc#141 (diff)

Maybe "output"

parser.add_argument('--mode', choices=['rock', 'paper', 'scissors'])

actually all of the rest

Fix and enhance

one more request on this way .. the tool I want to wrap uses the parent feature of subparser, i.e. it defines subparsers that are reused in many other subbarsers.

check for the "parent" argument (in combination with add_help=False) here :

checkm.py.txt

I guess macros would be a great way to do this.

for now I will do this manually

I make a virtualenv, clone the repo, run python setup.py install and it seems to terminate successfully, but when I try the test script, it fails, and I'm not quite sure why.

Finished processing dependencies for gxargparse==0.2.5
(venv) computer:gxargparse Alex$ gxargparse_check_path
gxargparse not installed

Any ideas?

• enable py3 & py3 running of the tests in .travis.yml (possible use tox to manage python versions?)
• don't use Mock if passing in arguments directly to argparse is sufficient
• for remaining cases just monkeypatch the library as needed
  See #32 (comment)

Anton, if you don't have time for this soonish let me know and I'll do it.

type: string? vs: type: [ "null", string ]

parser.add_argument('--new_name', nargs='?', help='New name for the fasta sequence')

works, while

parser.add_argument('--new_name',  help='New name for the fasta sequence')

breaks with

  File "cpt_fasta_rename.py", line 12, in <module>
    args = parser.parse_args()
  File "/home/users/cpt/cpt/esr/Projects/cpt/dev/software/python-galaxy-tools/.venv/local/lib/python2.7/site-packages/argparse/__init__.py", line 109, in parse_args
    gxt_parameter = methodToCall(result, tool=self.tool)
  File "/home/users/cpt/cpt/esr/Projects/cpt/dev/software/python-galaxy-tools/.venv/local/lib/python2.7/site-packages/argparse/argparse_translation.py", line 139, in _StoreAction
    self.__args_from_nargs(param, repeat_name, repeat_var_name, positional, flag_wo_dashes)
  File "/home/users/cpt/cpt/esr/Projects/cpt/dev/software/python-galaxy-tools/.venv/local/lib/python2.7/site-packages/argparse/argparse_translation.py", line 85, in __args_from_nargs 
    raise Exception("TODO: Handle argparse.REMAINDER")
Exception: TODO: Handle argparse.REMAINDER

The last issue that blocks successful travis checks is the resolving of default values in select fields.

example.py

...
parser.add_argument('--mode', choices=['rock', 'paper', 'scissors'], default='scissors')
...

In the example the default value is placed on the first place in the list of choices
example.xml

    <param argument="--mode" label="Author did not provide help for this parameter... " name="mode" optional="true" type="select">
      <option selected="true" value="scissors">scissors</option>
      <option value="paper">paper</option>
      <option value="rock">rock</option>
    </param>

The current version of argparse2tool resolves the default value in the order it appeared in argparse choices:

    <param argument="--mode" label="Author did not provide help for this parameter... " name="mode" optional="true" type="select">
      <option value="paper">paper</option>
      <option value="rock">rock</option>
      <option selected="true" value="scissors">scissors</option>
    </param>

If the behavior of the current version is correct than we should simple update example.xml.

type=argparse.FileType('w') should be detected as an output file. If we can suggest that authors use this, output files can be specified at the cli and auto-detected.

As a workaround for
https://github.com/erasche/argparse2tool/blob/master/README.rst#limitations

Custom types and custom actions are not supported.

I would suggest that a string is a safe fallback.

• find real world example
• test this
• explain in the readme that custom types and custom actions are implemented using CWL's type: string

Here some possible improvements (for now I have a solution for all of them):

• remove spaces in tool name (problem comes for subcommands)
• <command interpreter="python"><!\[CDATA\[ -> <command><!\[CDATA\[python
• non optional positional arguments are currently treated as optional in the galaxy xml file
• Is it possible to use the name of positional arguments instead of naming them positional_NUMBER? If there are multiple positionals with the same name one could still number them.

This would simplify automated postprocessing quite a bit.

Example: checkm outliers -h

positional arguments:
out_folder folder specified during qa command
bin_folder folder containing bins (fasta format)
tetra_profile tetranucleotide profiles for each sequence (see tetra command)
output_file print results to file

Are now positional1,..., positional 4

I would like to use sed/grep to replace the corresponding parts of the xml with appropriate content.

I think we may be at the point that a PR should be created always, no direct pushing to the main branch unless absolutely necessary.

@erasche @anton-khodak What are your thoughts?

Hi Eric,
I'm ashamed to admit, I'm not exactly sure how to use this code. I don't think its explicitly specified anywhere. I have a program, which I've stripped down to just the argument parser, and I want to generate xml for it. Am I supposed to run the program without specifying the flags?

python parseME.py --generate-galaxy-xml

Or should I specify the flags as well? Does the program need to terminate successfully in order to generate the xml? Where does the xml get generated?

I suggest that whatever the PyPI release of argparse2tool is on 2020-01-01 be the last to include Python 2.7 support.

@erasche If you approve, then I am happy to implement a warning in the codebase and README, etc.. (as was done for cwltool and friends).

The tool does not provide any output if the parser is instantiated with the prog keyword. e.g. for foo.py that contains parser = argparse.ArgumentParser(description='A description', prog='some prog name') and running python foo.py --generate_cwl_tool > foo.cwl foo.cwl will be empty. If prog='some prog name' is removed, everything works fine.

From https://github.com/erasche/argparse2tool/blob/master/README.rst#limitations

Mutual exclusion is not supported.

See http://www.commonwl.org/v1.0/UserGuide.html#Record_inputs,_dependent_and_mutually_exclusive_parameters

they support foo.py --generate and foo.py foo --generate which does just a single subparser named foo rather than all. Need to figure out what they're doing differently and perhaps unify the code.

https://github.com/pypa/twine

https://argcomplete.readthedocs.io/en/latest/
https://github.com/kislyuk/argcomplete

From https://github.com/erasche/argparse2tool/blob/master/README.rst#limitations

Argument groups don't work in CWL as arguments are sorted with a special algorithm

According to https://docs.python.org/2/library/argparse.html#argument-groups argument groups are just presentational, they don't themselves dictate any particular sort order.

One could mimic the grouping by creating a http://www.commonwl.org/v1.0/CommandLineTool.html#CommandInputRecordSchema mapping the argparse title to the CWL label field (and eventually the argparse description to the CWL doc field)

From https://github.com/erasche/argparse2tool/blob/master/README.rst#limitations

const argument of add_argument(). All constants must be specified in job files.

My reading of https://docs.python.org/2/library/argparse.html#const is that const is an internal implementation detail -- I suspect that it can be safely ignored for the purpose of generating command line invocations.

• find real world usage of const
• test ignoring const
• update readme and perhaps add a comment in the code if this is indeed ignorable

Conditionals

Need some way to support conditionals.

parser = argparse.ArgumentParser()
parser.add_argument('bar', choices=('a', 'b', 'c'))

def bar_test(value):
    return value == 'a'

parser.add_argument('baz', depends_on='bar', when=bar_test)

????

Please comment/edit issue for input.

We should at least iterate over all action groups.

While following the CWL v1.0 spec, I was implementing a hello world example python script, and figured i'd try out argparse2tool. The generated yml seems to be inconsistent with the v1.0 spec. Changing 'str' to 'string' in the yaml seems to fix the issue.

error:

...
Resolved 'hello_world.yml' to file:///home/MatthewRalston/sandbox/Toil/hello_world.yml'
INFO:cwltool:resolve 'hello_world.yml' to 'file:///home/MatthewRalston/sandbox/Toil/hello_world.yml'
Tool definition failed validation:
While checking field `inputs`
  While checking object `file:///home/MatthewRalston/sandbox/Toil/hello_world.yml#message`
    Field `type` contains undefined reference to `str`, tried  [u'file:///home/MatthewRalston/sandbox/Toil/hello_world.yml#str']
ERROR:cwltool:Tool definition failed validation:
While checking field `inputs`
  While checking object `file:///home/MatthewRalston/sandbox/Toil/hello_world.yml#message`
    Field `type` contains undefined reference to `str`, tried  [u'file:///home/MatthewRalston/sandbox/Toil/hello_world.yml#str']

hello_world.py

...
parser.add_argument('--message', help="Required argument", required=True)
parser.add_argument('--flag', help="This is a flag", action="store_true")
...

--help/usage info

usage: hello_world.py [-h] --message MESSAGE [--flag]

optional arguments:
  -h, --help                       show this help message and exit
  --message MESSAGE     Required argument (default: None)
  --flag                              This is a flag (default: False)

Generated yml:

inputs:

  message:
    type: str
    doc: Required argument
    inputBinding:
      prefix: --message

  flag:
    type: ["null", boolean]
    default: False
    doc: This is a flag
    inputBinding:
      prefix: --flag

I'm soliciting feedback on this, we need an API that /looks/ like argparse commands to handle output files.

My idea is this:

• "normal" programs will use argparse as-is.
• people who want the gxargparse functionality will do import gxargparse as argparse.
  • those same people will then be able to use some "extended" syntax that might break under normal argparse. Namely would be an output API to handle writing out datasets.

Any comments/thoughts on what it might look like? I've toyed with the idea of:

import gxargparse as argparse
...
parser.add_output('fa_output', format='fasta', help='Some fasta output of this tool')
... # some parsing later
SeqIO.write(parser.fa_output, my_seqio_object)

Any suggestions? Very much in planning stage but I keep writing tools that output multiple files and need a better way to handle than stdout.

I think it is reasonable to change the short description to

drop in replacement for argparse with Galaxy XML and CWL tool generation on the side

From https://github.com/erasche/argparse2tool/blob/master/README.rst#limitations

nargs=N. Number of arguments can not be specified in CWL (yet).

This can be implemented using a http://www.commonwl.org/v1.0/CommandLineTool.html#CommandInputRecordSchema

Have N entries named 1..N, set the inputBinding at the record level.

I think some exceptions need to be made for positional arguments to treat them differently, as they're a hard requirement.

e.g.

  <command interpreter="python"><![CDATA[phantasm_data_comparison.py 
#if $positional_1 and $positional_1 is not None:
$positional_1
#end if

#if $positional_2 and $positional_2 is not None:
$positional_2
#end if

#if $positional_3 and $positional_3 is not None:
$positional_3
#end if

#if $positional_4 and $positional_4 is not None:
$positional_4
#end if
> $default]]></command>
  <inputs>
    <param label="First Tabular Dataset (positional_1)" name="positional_1" type="data"/>
    <param label="Second Tabular Dataset (positional_2)" name="positional_2" type="data"/>
    <param label="Method for comparison (positional_3)" name="positional_3">

xref #46