This is not a place of honour.
hexylena / argparse2tool Goto Github PK
View Code? Open in Web Editor NEWtransparently build CWL and Galaxy XML tool definitions for any script that uses argparse
License: Apache License 2.0
transparently build CWL and Galaxy XML tool definitions for any script that uses argparse
License: Apache License 2.0
This is not a place of honour.
Would you add an uninstall script? I assume just deleting the folder wouldn't be enough?
Inspiration: galaxyproject/tools-iuc#141 (diff)
Maybe "output"
parser.add_argument('--mode', choices=['rock', 'paper', 'scissors'])
actually all of the rest
Fix and enhance
one more request on this way .. the tool I want to wrap uses the parent feature of subparser, i.e. it defines subparsers that are reused in many other subbarsers.
check for the "parent" argument (in combination with add_help=False) here :
I guess macros would be a great way to do this.
for now I will do this manually
I make a virtualenv, clone the repo, run python setup.py install
and it seems to terminate successfully, but when I try the test script, it fails, and I'm not quite sure why.
Finished processing dependencies for gxargparse==0.2.5
(venv) computer:gxargparse Alex$ gxargparse_check_path
gxargparse not installed
Any ideas?
Anton, if you don't have time for this soonish let me know and I'll do it.
Traceback (most recent call last):
File "cpt_shinefind.py", line 24, in <module>
parser.add_argument('--lookahead_min', action='store_const', help='Number of bases upstream of CDSs to end search', default=5)
File "/home/users/cpt/cpt/esr/Projects/cpt/dev/software/python-galaxy-tools/.venv/local/lib/python2.7/site-packages/argparse/__init__.py", line 44, in add_argument
result = self.parser.add_argument(*args, **kwargs)
File "/usr/lib/python2.7/argparse.py", line 1283, in add_argument
action = action_class(**kwargs)
TypeError: __init__() takes at least 4 arguments (5 given)
type: string?
vs: type: [ "null", string ]
parser.add_argument('--new_name', nargs='?', help='New name for the fasta sequence')
works, while
parser.add_argument('--new_name', help='New name for the fasta sequence')
breaks with
File "cpt_fasta_rename.py", line 12, in <module>
args = parser.parse_args()
File "/home/users/cpt/cpt/esr/Projects/cpt/dev/software/python-galaxy-tools/.venv/local/lib/python2.7/site-packages/argparse/__init__.py", line 109, in parse_args
gxt_parameter = methodToCall(result, tool=self.tool)
File "/home/users/cpt/cpt/esr/Projects/cpt/dev/software/python-galaxy-tools/.venv/local/lib/python2.7/site-packages/argparse/argparse_translation.py", line 139, in _StoreAction
self.__args_from_nargs(param, repeat_name, repeat_var_name, positional, flag_wo_dashes)
File "/home/users/cpt/cpt/esr/Projects/cpt/dev/software/python-galaxy-tools/.venv/local/lib/python2.7/site-packages/argparse/argparse_translation.py", line 85, in __args_from_nargs
raise Exception("TODO: Handle argparse.REMAINDER")
Exception: TODO: Handle argparse.REMAINDER
The last issue that blocks successful travis checks is the resolving of default values in select fields.
example.py
...
parser.add_argument('--mode', choices=['rock', 'paper', 'scissors'], default='scissors')
...
In the example the default value is placed on the first place in the list of choices
example.xml
<param argument="--mode" label="Author did not provide help for this parameter... " name="mode" optional="true" type="select">
<option selected="true" value="scissors">scissors</option>
<option value="paper">paper</option>
<option value="rock">rock</option>
</param>
The current version of argparse2tool resolves the default value in the order it appeared in argparse choices:
<param argument="--mode" label="Author did not provide help for this parameter... " name="mode" optional="true" type="select">
<option value="paper">paper</option>
<option value="rock">rock</option>
<option selected="true" value="scissors">scissors</option>
</param>
If the behavior of the current version is correct than we should simple update example.xml
.
type=argparse.FileType('w')
should be detected as an output file. If we can suggest that authors use this, output files can be specified at the cli and auto-detected.
As a workaround for
https://github.com/erasche/argparse2tool/blob/master/README.rst#limitations
Custom types and custom actions are not supported.
I would suggest that a string is a safe fallback.
type: string
Here some possible improvements (for now I have a solution for all of them):
<command interpreter="python"><!\[CDATA\[
-> <command><!\[CDATA\[python
This would simplify automated postprocessing quite a bit.
Example: checkm outliers -h
positional arguments:
out_folder folder specified during qa command
bin_folder folder containing bins (fasta format)
tetra_profile tetranucleotide profiles for each sequence (see tetra command)
output_file print results to file
Are now positional1,..., positional 4
I would like to use sed/grep to replace the corresponding parts of the xml with appropriate content.
I think we may be at the point that a PR should be created always, no direct pushing to the main branch unless absolutely necessary.
@erasche @anton-khodak What are your thoughts?
Hi Eric,
I'm ashamed to admit, I'm not exactly sure how to use this code. I don't think its explicitly specified anywhere. I have a program, which I've stripped down to just the argument parser, and I want to generate xml for it. Am I supposed to run the program without specifying the flags?
python parseME.py --generate-galaxy-xml
Or should I specify the flags as well? Does the program need to terminate successfully in order to generate the xml? Where does the xml get generated?
I suggest that whatever the PyPI release of argparse2tool
is on 2020-01-01 be the last to include Python 2.7 support.
@erasche If you approve, then I am happy to implement a warning in the codebase and README, etc.. (as was done for cwltool
and friends).
The tool does not provide any output if the parser is instantiated with the prog keyword. e.g. for foo.py that contains parser = argparse.ArgumentParser(description='A description', prog='some prog name')
and running python foo.py --generate_cwl_tool > foo.cwl
foo.cwl will be empty. If prog='some prog name'
is removed, everything works fine.
From https://github.com/erasche/argparse2tool/blob/master/README.rst#limitations
Mutual exclusion is not supported.
they support foo.py --generate
and foo.py foo --generate
which does just a single subparser named foo
rather than all. Need to figure out what they're doing differently and perhaps unify the code.
Traceback (most recent call last):
File "cpt_fasta_rename.py", line 10, in <module>
parser.add_argument('new_name', nargs='1', help='New name for the fasta sequence')
File "/home/users/cpt/cpt/esr/Projects/cpt/dev/software/python-galaxy-tools/.venv/local/lib/python2.7/site-packages/argparse/__init__.py", line 44, in add_argument
result = self.parser.add_argument(*args, **kwargs)
File "/usr/lib/python2.7/argparse.py", line 1295, in add_argument
raise ValueError("length of metavar tuple does not match nargs")
ValueError: length of metavar tuple does not match nargs
From https://github.com/erasche/argparse2tool/blob/master/README.rst#limitations
Argument groups don't work in CWL as arguments are sorted with a special algorithm
According to https://docs.python.org/2/library/argparse.html#argument-groups argument groups are just presentational, they don't themselves dictate any particular sort order.
One could mimic the grouping by creating a http://www.commonwl.org/v1.0/CommandLineTool.html#CommandInputRecordSchema mapping the argparse title
to the CWL label
field (and eventually the argparse description
to the CWL doc
field)
From https://github.com/erasche/argparse2tool/blob/master/README.rst#limitations
const
argument ofadd_argument()
. All constants must be specified in job files.
My reading of https://docs.python.org/2/library/argparse.html#const is that const
is an internal implementation detail -- I suspect that it can be safely ignored for the purpose of generating command line invocations.
const
const
Need some way to support conditionals.
parser = argparse.ArgumentParser()
parser.add_argument('bar', choices=('a', 'b', 'c'))
def bar_test(value):
return value == 'a'
parser.add_argument('baz', depends_on='bar', when=bar_test)
Please comment/edit issue for input.
We should at least iterate over all action groups.
While following the CWL v1.0 spec, I was implementing a hello world example python script, and figured i'd try out argparse2tool. The generated yml seems to be inconsistent with the v1.0 spec. Changing 'str' to 'string' in the yaml seems to fix the issue.
error:
...
Resolved 'hello_world.yml' to file:///home/MatthewRalston/sandbox/Toil/hello_world.yml'
INFO:cwltool:resolve 'hello_world.yml' to 'file:///home/MatthewRalston/sandbox/Toil/hello_world.yml'
Tool definition failed validation:
While checking field `inputs`
While checking object `file:///home/MatthewRalston/sandbox/Toil/hello_world.yml#message`
Field `type` contains undefined reference to `str`, tried [u'file:///home/MatthewRalston/sandbox/Toil/hello_world.yml#str']
ERROR:cwltool:Tool definition failed validation:
While checking field `inputs`
While checking object `file:///home/MatthewRalston/sandbox/Toil/hello_world.yml#message`
Field `type` contains undefined reference to `str`, tried [u'file:///home/MatthewRalston/sandbox/Toil/hello_world.yml#str']
hello_world.py
...
parser.add_argument('--message', help="Required argument", required=True)
parser.add_argument('--flag', help="This is a flag", action="store_true")
...
--help
/usage info
usage: hello_world.py [-h] --message MESSAGE [--flag]
optional arguments:
-h, --help show this help message and exit
--message MESSAGE Required argument (default: None)
--flag This is a flag (default: False)
Generated yml:
inputs:
message:
type: str
doc: Required argument
inputBinding:
prefix: --message
flag:
type: ["null", boolean]
default: False
doc: This is a flag
inputBinding:
prefix: --flag
I'm soliciting feedback on this, we need an API that /looks/ like argparse commands to handle output files.
My idea is this:
import gxargparse as argparse
.
Any comments/thoughts on what it might look like? I've toyed with the idea of:
import gxargparse as argparse
...
parser.add_output('fa_output', format='fasta', help='Some fasta output of this tool')
... # some parsing later
SeqIO.write(parser.fa_output, my_seqio_object)
Any suggestions? Very much in planning stage but I keep writing tools that output multiple files and need a better way to handle than stdout.
I think it is reasonable to change the short description to
drop in replacement for argparse with Galaxy XML and CWL tool generation on the side
From https://github.com/erasche/argparse2tool/blob/master/README.rst#limitations
nargs=N
. Number of arguments can not be specified in CWL (yet).
This can be implemented using a http://www.commonwl.org/v1.0/CommandLineTool.html#CommandInputRecordSchema
Have N entries named 1
..N
, set the inputBinding
at the record level.
I think some exceptions need to be made for positional arguments to treat them differently, as they're a hard requirement.
e.g.
<command interpreter="python"><![CDATA[phantasm_data_comparison.py
#if $positional_1 and $positional_1 is not None:
$positional_1
#end if
#if $positional_2 and $positional_2 is not None:
$positional_2
#end if
#if $positional_3 and $positional_3 is not None:
$positional_3
#end if
#if $positional_4 and $positional_4 is not None:
$positional_4
#end if
> $default]]></command>
<inputs>
<param label="First Tabular Dataset (positional_1)" name="positional_1" type="data"/>
<param label="Second Tabular Dataset (positional_2)" name="positional_2" type="data"/>
<param label="Method for comparison (positional_3)" name="positional_3">
xref #46
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.