add Static code analysis via mypy to the project.

depends on CycloneDX/cyclonedx-python-lib#52

• have mypy configured
• have mypy run in tox
• have mypy run in CI/CT for pull requests & commits to master

Currently planning to refactor the code to be able to publish a library that would contain CycloneDX data models, serialization, deserialization, and validation. And a separate tool to generate a BOM for a Python project.

It would also help with issue #211 to relax the version constraints.

as of CycloneDX/cyclonedx-node-module#192 (comment)

coderpatros needs to be changed to $DOCKERHUB_USERNAME
$DOCKER_TOKEN needs to be changed to $DOCKERHUB_TOKEN.

Both DOCKERHUB_USERNAME and DOCKERHUB_TOKEN are secrets defined at the GitHub organization level. I assume DOCKER_TOKEN is likely unique to this repo and should be deleted in favor of the organizational secrets.

tasks

• use the organisation's secrets instead of repos secrets - via #272
• remove the repo's secrets via https://github.com/CycloneDX/cyclonedx-python/settings/secrets/actions
  (this requires admin prvilege) via #271 (comment)

Example of authors metadata in JSON format

{
  "bomFormat": "CycloneDX",
  "specVersion": "1.2",
  "serialNumber": "urn:uuid:3e671687-395b-41f5-a30f-a58921a69b79",
  "version": 1,
  "metadata": {
    "authors": [
      {
        "name": "Samantha Wright",
        "email": "[email protected]",
        "phone": "800-555-1212"
      }
    ]
  },
  "components": []
}

Example of manufacturer metadata in JSON format

{
  "bomFormat": "CycloneDX",
  "specVersion": "1.2",
  "serialNumber": "urn:uuid:3e671687-395b-41f5-a30f-a58921a69b79",
  "version": 1,
  "metadata": {
    "manufacture": {
      "name": "Acme, Inc.",
      "url": [
        "https://example.com"
      ],
      "contact": [
        {
          "name": "Acme Professional Services",
          "email": "[email protected]"
        }
      ]
    }
  },
  "components": []
}

Example of supplier metadata in JSON format...

{
  "bomFormat": "CycloneDX",
  "specVersion": "1.2",
  "serialNumber": "urn:uuid:3e671687-395b-41f5-a30f-a58921a69b79",
  "version": 1,
  "metadata": {
    "supplier": {
      "name": "Acme, Inc.",
      "url": [
        "https://example.com"
      ],
      "contact": [
        {
          "name": "Acme Distribution",
          "email": "[email protected]"
        }
      ]
    }
  },
  "components": []
}

Manufacturer and supplier are both "organizationalEntity" elements. Author, manufacturer contact and supplier contact are all "organizationalContact" elements. They should be created as a class under models much like component is.

Issue Type:

• defect report
• enhancement request

Current Behavior:

When cyclonedx-python encounters an implicit post release entry it will crash and won't generate a BOM.

PEP-0440 declares the two versioning variations to be equivalent.

Implicit post releases

Post releases allow omitting the post signifier all together. When using this form the separator MUST be - and no other form is allowed. This allows versions such as 1.0-1 to be normalized to 1.0.post1. This particular normalization MUST NOT be used in conjunction with the implicit post release number rule. In other words, 1.0- is not a valid version and it does not normalize to 1.0.post0.

It seems that pip uses the 3.2.3.post2 format in the frozen requirements file and the PyPi restful web service responses use the 3.2.3-2 format. PyPi accepts either format, per the spec.

Steps to Reproduce (if defect):

For example, see functools32 (requires python 2.7) currently version 3.2.3-2 https://pypi.org/project/functools32/#history.

$ curl -s https://pypi.org/pypi/functools32/json | python -m json.tool | grep "\"version\":"
        "version": "3.2.3-2"

Release versions:

$ curl -s https://pypi.org/pypi/functools32/json | python -m json.tool
...
    "releases": {
        "3.2.3": [],
...
        "3.2.3-1": [
...
        "3.2.3-2": [
...

Generate the Requirements file

1. Load library and generate a requirements file

$ pip install functools32
$ pip freeze > requirements_default.txt

2. Note the version is listed as 3.2.3.post2

$ cat requirements_default.txt
functools32==3.2.3.post2

Reproduce the error

1. Run CycloneDX

$ cyclonedx-py -i requirements_default.txt 
Input file: requirements_default.txt
Output BOM: bom.xml
Generating CycloneDX BOM
Traceback (most recent call last):
  File "/tmp/cyclone/cycdx/bin/cyclonedx-py", line 10, in <module>
    sys.exit(main())
  File "/tmp/cyclone/cycdx/lib/python3.6/site-packages/cyclonedx/cli/generateBom.py", line 69, in main
    version_release = releases[version]
KeyError: '3.2.3.post2'

The "releases" array was displayed above and the keys were in implicit version string format.

Expected Behavior:

It should treat 3.2.3.post2 the same as 3.2.3-2.

Environment:

• cyclonedx-bom Version: 0.2.0
• python Version: 3.6.7

Other Details:

(e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, gitter, etc)

Possibly check for implicit version strings if there is a KeyError when working with the PyPi response?

Verify PyPi works with either format

The etag HTTP response header is the same value with either format so internally PyPi treats both version strings as the same resource.

Post format

$ curl -s --head https://pypi.org/pypi/functools32/3.2.3.post2/json | grep "etag:"
etag: "TvkoNOBe/zwpNL1J9zhKjg"

Implicit format

$ curl -s --head https://pypi.org/pypi/functools32/3.2.3-2/json | grep "etag:"
etag: "TvkoNOBe/zwpNL1J9zhKjg"

Hi all,

I have multiple requirement files in a folder. This is done because of different environments, e.g.:
/requirements/base.txt
/requirements/development.txt
/requirements/production.txt

The base.txt has all dependencies which are needed in all environments. The other files (e.g. development.txt or production.txt) refer to the base.txt and add additional dependencies, e.g.

# base requirements
-r base.txt

#additional requirements
pylint==2.2.0

Could you advance cyclonedx-python to scan multiple requirement files at once?
Furthermore, cyclonedx-python fails if there is "-r base.txt" in a requirements file?

Traceback (most recent call last):
  File "/home/test/.local/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 98, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/home/test/.local/lib/python3.8/site-packages/pkg_resources/_vendor/pyparsing.py", line 1654, in parseString
    raise exc
  File "/home/test/.local/lib/python3.8/site-packages/pkg_resources/_vendor/pyparsing.py", line 1644, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/home/test/.local/lib/python3.8/site-packages/pkg_resources/_vendor/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/test/.local/lib/python3.8/site-packages/pkg_resources/_vendor/pyparsing.py", line 3417, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/home/test/.local/lib/python3.8/site-packages/pkg_resources/_vendor/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/test/.local/lib/python3.8/site-packages/pkg_resources/_vendor/pyparsing.py", line 3739, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/home/test/.local/lib/python3.8/site-packages/pkg_resources/_vendor/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/test/.local/lib/python3.8/site-packages/pkg_resources/_vendor/pyparsing.py", line 3400, in parseImpl
    loc, resultlist = self.exprs[0]._parse( instring, loc, doActions, callPreParse=False )
  File "/home/test/.local/lib/python3.8/site-packages/pkg_resources/_vendor/pyparsing.py", line 1406, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/test/.local/lib/python3.8/site-packages/pkg_resources/_vendor/pyparsing.py", line 2711, in parseImpl
    raise ParseException(instring, loc, self.errmsg, self)
pkg_resources._vendor.pyparsing.ParseException: Expected W:(abcd...) (at char 0), (line:1, col:1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/test/.local/bin/cyclonedx-bom", line 8, in <module>
    sys.exit(main())
  File "/home/test/.local/lib/python3.8/site-packages/cyclonedx_py/client.py", line 211, in main
    CycloneDxCmd(args).execute()
  File "/home/test/.local/lib/python3.8/site-packages/cyclonedx_py/client.py", line 88, in execute
    output = self.get_output()
  File "/home/test/.local/lib/python3.8/site-packages/cyclonedx_py/client.py", line 52, in get_output
    parser = self._get_input_parser()
  File "/home/test/.local/lib/python3.8/site-packages/cyclonedx_py/client.py", line 203, in _get_input_parser
    return RequirementsParser(requirements_content=input_data)
  File "/home/test/.local/lib/python3.8/site-packages/cyclonedx/parser/requirements.py", line 32, in __init__
    for requirement in requirements:
  File "/home/test/.local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3080, in parse_requirements
    yield Requirement(line)
  File "/home/test/.local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3090, in __init__
    super(Requirement, self).__init__(requirement_string)
  File "/home/test/.local/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 100, in __init__
    raise InvalidRequirement(
pkg_resources.extern.packaging.requirements.InvalidRequirement: Parse error at "'-r base.'": Expected W:(abcd...)

The following command is an example to this issue:
cyclonedx-bom -r -i base.txt -r -i development.txt --format json -o test.json

Thank you

Non-XML characters may get into a BOM file.

$ pip3 freeze | grep cyclonedx
cyclonedx-bom==0.4.2

$ cat requirements.txt
fuzzywuzzy==0.15.0

$ cyclonedx-py
Input file: requirements.txt
Output BOM: bom.xml
JSON output: False
Package info url: https://pypi.org/pypi/{package_name}/{package_version}/json
Generating CycloneDX BOM
Validating BOM
Traceback (most recent call last):
  File "/usr/local/bin/cyclonedx-py", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/site-packages/cyclonedx/client.py", line 85, in main
    validate_bom(args)
  File "/usr/local/lib/python3.8/site-packages/cyclonedx/client.py", line 70, in validate_bom
    if validator.is_valid(args.output_file, args.json):
  File "/usr/local/lib/python3.8/site-packages/cyclonedx/bom/validator.py", line 39, in is_valid
    return xml_bom_schema.is_valid(output_filename)
  File "/usr/local/lib/python3.8/site-packages/xmlschema/validators/schema.py", line 1435, in is_valid
    error = next(self.iter_errors(source, path, schema_path, use_defaults, namespaces), None)
  File "/usr/local/lib/python3.8/site-packages/xmlschema/validators/schema.py", line 1456, in iter_errors
    source = XMLResource(source, defuse=self.defuse, timeout=self.timeout, lazy=False)
  File "/usr/local/lib/python3.8/site-packages/xmlschema/resources.py", line 349, in __init__
    self.source = source
  File "/usr/local/lib/python3.8/site-packages/xmlschema/resources.py", line 366, in __setattr__
    self._root, self._text, self._url = self._fromsource(value)
  File "/usr/local/lib/python3.8/site-packages/xmlschema/resources.py", line 430, in _fromsource
    return self.parse(resource).getroot(), None, url
  File "/usr/local/lib/python3.8/site-packages/xmlschema/resources.py", line 590, in parse
    return ElementTree.parse(source)
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/xml/etree/ElementTree.py", line 1202, in parse
    tree.parse(source, parser)
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/xml/etree/ElementTree.py", line 595, in parse
    self._root = parser._parse_whole(source)
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 73, column 0

as a <> i want build artifacts added to a release, so PyPI is not my single point of source.

use https://github.com/actions/upload-release-asset
to add artifacts to the release

how to do it: see an example here: https://github.com/k4cg/nichtparasoup/blob/3.0-dev/.github/workflows/release.yaml

build artifacts to store on the build and to be added to the release:

• dist release - wheel(build/*.whl)
• source release - (build/*.tar.gz)
• sbom of the docker image
  simply use our own image and do a pip freeze and generate the python sbom from it

In environments where proxy setup is required, cyclonedx-bom fails to generate SBOM.

No output is given and the program hangs when running

cyclonedx-py -r -o -

I'm using 1.0.2

The program outputs nothing and does not exit, we ran it for 12 minutes on our CI with that config until realizing it hung.
Since both r and rf flags are optional arguments in the help command I believed there would be defaults, maybe those flags should be described as required, not optional.

When rf flag is added everything works fine :)

Hey, I think it would be really great if it were possible to list out the licenses of each package as well, It is supported in the CycloneDX BOM format and there are ways to obtain the licenses using the package name like shown here : https://pypi.org/project/pip-licenses/ Just one issue that I see here is if the package is not installed, then it could be troublesome.

I've used this part of your code to validate user provided BOM files

import xmlschema  # same as your used version
bom_schema = xmlschema.XMLSchema("http://cyclonedx.org/schema/bom/1.1")

I've changed the version to 1.1 here and in my test code since I get this error when using 1.0:

  File "/home/glozmantal/PycharmProjects/Android-iOS-Code-OSS-Report/venv/lib64/python3.8/site-packages/xmlschema/validators/schema.py", line 885, in get_schema
    raise XMLSchemaKeyError('the namespace {!r} is not loaded'.format(namespace)) from None
xmlschema.exceptions.XMLSchemaKeyError: "the namespace 'http://cyclonedx.org/schema/bom/1.1' is not loaded"

When I use

bom_schema.is_valid(bom.name):  # I use pathlib so 'bom.name' is equivalent to 'bom' in your code.

And when I try to validate this broken test bom.xml file that was created using the ORT tool which uses BOM schema 1.1 against "http://cyclonedx.org/schema/bom/1.1", I get a positive validation. I've changed the license text value, apart from distorting the XML format integrity for the validation test.

<?xml version="1.0" encoding="UTF-8"?>
<bom xmlns="http://cyclonedx.org/schema/bom/1.1" serialNumber="urn:uuid:1817d96f-20c1-41a6-b8fa-fdb16bdc97de" version="1">
    <components>
        <component type="library">
        <group>androidx.annotation</group>
        <name>annotation
        <version>1.1.0</version>
        <description>The Support Library is a static library that you can add to your Android application in order to use APIs that are either not available for older platform versions or utility APIs that aren't a part of the framework APIs.</description>
        <scope>required</scope>
        <hashes>
            <hash alg="SHA-1">e3a6fb2f40e3a3842e6b7472628ba4ce416ea4c8</hash>
        </hashes>
        <licenses>
            <license>
                <id>Apache-2.0</id>
                <text content-type="plain/text"><![CDATA[ TEST
LICENSE TEXT.
]]></text>

            </license>
        </licenses>
        <purl>pkg:maven/androidx.annotation/[email protected]</purl>

    </externalReferences>

i have run the cyclonedx tool and it rewrites the requirements file and then create the bom of it. but i want the bom of my requirements.txt file and dont want it to get rewrited when you run this command-cyclonedx-py -i /path/ -j, can you tell me whats wrong in it?
@stevespringett

https://github.com/CycloneDX/cyclonedx-python/tree/master/.devcontainer
needs some maintenance.

it tries to install dependencies via requirements.txt which is no longer present.
instead poetry is used.

Just a heads up, Python 3.5 is end of support September 2020.

We haven't hit any maintainability issues yet. But if we do we might drop support for it.

And I certainly expect to drop support for it in the next major release.

Version
cyclonedx-bom==1.5.3

Executed command
cyclonedx-py -r -o bom.xml

Use case
Our requirements.txt file references other local python packages. Example of requirements file:

../local-python-package1
../local-python-package2
../local-python-package3

package1==0.0.0
package2==0.0.0
package3==0.0.0

Exception

Traceback (most recent call last):
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 98, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1654, in parseString
    raise exc
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1644, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 3417, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 3739, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 3400, in parseImpl
    loc, resultlist = self.exprs[0]._parse( instring, loc, doActions, callPreParse=False )
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1406, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 2711, in parseImpl
    raise ParseException(instring, loc, self.errmsg, self)
pkg_resources._vendor.pyparsing.ParseException: Expected W:(abcd...) (at char 0), (line:1, col:1)

cyclonedx-bom version: 1.0.2

$ cyclonedx-py -r -rf equirements.freeze -Fo bom.xml
[ERROR] - 2021-09-23 09:51:58.704323 - The requirements.txt file path provided does not exist (equirements.freeze)

FILE_PATH in option -rf and in error message are not equivalent.
In command line: equirements.freeze
in nessage line: requirements.txt

Right now, the BOM is dumped unformatted and thus unfriendly to humans. Either there should be a --pretty or --format option (see xmllint), or prettifying should be the default.

ValueError: ('Could not find a matching normalized version string', 'FormEncode', '2.0.0a1')

I debugged into _get_pypi_version.
The version '2.0.0a1' is in release_dict, but canonicalize_version changes it to '2a1'.
'2.0.0a1' is thus not found.

Can we change it to
if special_version == pypi_version or special_version == release:
?

In preparation of v1.2 of the spec we need to support JSON output.

Is it on the roadmap to support the install_dependencies set in setup.py? It would mean fewer steps from project to bom.xml for projects that utilize a setup.py file.

Situation:

Running commands

cyclonedx-py -X -r -o bom.xml  # or alias
cyclonedx-bom -X -r -o bom.xml

on version 1.4.1 fails with an error:

Running same command on version 1.3.1 succeeds:

Environment

image: mcr.microsoft.com/playwright:focal

python --version
Python 3.8.10

Expected behaviour:

Command should succeed on version 1.4.1 as well or
a hint about the reason should be provided.

How to add project python version to bom.xml?

Dependabot can't evaluate your Python dependency files.

As a result, Dependabot couldn't check whether any of your dependencies are out-of-date.

The error Dependabot encountered was:

InstallationError("Invalid requirement: 'chardet=3.0.4' (from line 7 of /home/dependabot/dependabot-updater/dependabot_tmp_dir/requirements.txt)\nHint: = is not a valid operator. Did you mean == ?")

You can mention @dependabot in the comments below to contact the Dependabot team.

I encountered an issue running cyclonedx-py 0.3.5 in an environment with no internet access. The tool fails with the stack trace below, apparently trying to connect to http://cyclonedx.org/schema/bom/1.0.

We had excellent experience with CycloneDX for .NET, Java and NPM, which do not require internet access, when configured with on-premise package feeds.

Is it possible to configure cyclonedx-py to run without internet access? An on-premise Pip source is available, if necessary, but no other external resources are accessible.

If the answer is no currently, and code changes are required to support this case, I would be interested to contribute a PR, as long as someone can provide a few pointers on how to disable internet access.

Traceback (most recent call last):
  File "python\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "python\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "python\Scripts\cyclonedx-py.exe\__main__.py", line 4, in <module>
  File "python\lib\site-packages\cyclonedx\cli\generateBom.py", line 25, in <module>
    from cyclonedx import BomValidator
  File "python\lib\site-packages\cyclonedx\BomValidator.py", line 19, in <module>
    bom_schema = xmlschema.XMLSchema("http://cyclonedx.org/schema/bom/1.0")
  File "python\lib\site-packages\xmlschema\validators\schema.py", line 270, in __init__
    self.source = XMLResource(source, base_url, defuse, timeout, lazy=False)
  File "python\lib\site-packages\xmlschema\resources.py", line 279, in __init__
    self.source = source
  File "python\lib\site-packages\xmlschema\resources.py", line 303, in __setattr__
    self._root, self._text, self._url = self._fromsource(value)
  File "python\lib\site-packages\xmlschema\resources.py", line 381, in _fromsource
    resource = urlopen(url, timeout=self.timeout)
  File "python\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "python\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
  File "python\lib\urllib\request.py", line 640, in http_response
    response = self.parent.error(
  File "python\lib\urllib\request.py", line 569, in error
    return self._call_chain(*args)
  File "python\lib\urllib\request.py", line 502, in _call_chain
    result = func(*args)
  File "python\lib\urllib\request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

Environment information:

• Windows 10 (2004) 64-bit
• Python 3.8.5 x64
• cyclonedx_bom 0.3.5

When generating an SBOM a timestamp should be generated and added to the metadata element.

Example in JSON format

{
  "bomFormat": "CycloneDX",
  "specVersion": "1.2",
  "serialNumber": "urn:uuid:3e671687-395b-41f5-a30f-a58921a69b79",
  "version": 1,
  "metadata": {
    "timestamp": "2020-04-13T20:20:39+00:00"
  },
  "components": []
}

Example in XML format

<?xml version="1.0"?>
<bom serialNumber="urn:uuid:3e671687-395b-41f5-a30f-a58921a69b79" version="1" xmlns="http://cyclonedx.org/schema/bom/1.2">
    <metadata>
        <timestamp>2020-04-07T07:01:00Z</timestamp>
    </metadata>
    <components />
</bom>

Changes to the JSON output are required here

Changes to the XML output are required here

Hi everybody.
Found the problem, my requirements.txt has packages with no version specified.
When executing the command:

cyclonedx-py -r -rf requirements.txt -o bom.xml 

I have an error:

Traceback (most recent call last):
  File "/usr/local/bin/cyclonedx-py", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/cyclonedx_py/client.py", line 152, in main
    CycloneDxCmd().execute()
  File "/usr/local/lib/python3.9/site-packages/cyclonedx_py/client.py", line 63, in execute
    output = self.get_output()
  File "/usr/local/lib/python3.9/site-packages/cyclonedx_py/client.py", line 55, in get_output
    bom=Bom.from_parser(self._get_input_parser()),
  File "/usr/local/lib/python3.9/site-packages/cyclonedx_py/client.py", line 139, in _get_input_parser
    return RequirementsFileParser(requirements_file=self._arguments.input_requirements_file)
  File "/usr/local/lib/python3.9/site-packages/cyclonedx/parser/requirements.py", line 51, in __init__
    super(RequirementsFileParser, self).__init__(requirements_content=r.read())
  File "/usr/local/lib/python3.9/site-packages/cyclonedx/parser/requirements.py", line 41, in __init__
    (op, version) = requirement.specs[0]
IndexError: list index out of range 

If I remove packages without versions, the process goes without errors.

pip3 list | grep cyclonedx
cyclonedx-bom        1.0.2
cyclonedx-python-lib 0.4.0

When attempting to parse a UTF16 requirements txt the tool throws an UnicodeDecodeError. One solution could be to ignore UTF16 chars with with open(path, encoding="utf8", errors='ignore') as f: (source).

The complete message:

+ cyclonedx-py
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/requirements/parser.py", line 19, in parse
    if not isinstance(reqstr, basestring):
NameError: name 'basestring' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/cyclonedx-py", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/cyclonedx/cli/generateBom.py", line 48, in main
    for req in requirements.parse(fd):
  File "/usr/local/lib/python3.6/site-packages/requirements/parser.py", line 24, in parse
    reqstr = reqstr.read()
  File "/usr/lib64/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
Input file: requirements.txt
Output BOM: bom.xml
Generating CycloneDX BOM

The workaround is to generate the requirements.txt with pip again to get rid of the UTF16.

python3 -m venv venv
source venv/bin/activate
pip freeze > requirements.txt
deactivate

However, it would be nice if cyclonedx-python can handle UTF16 like pip.

Release v0.4.0 breaks when pypi returns a null license. Sometimes packages will have empty strings for a license and sometimes they will have null.

Actual Results

(cyclonedx_venv) -bash-4.2$ cyclonedx-py -i output/xad -o output/RICK_bom.xml
Input file: output/xad
Output BOM: output/RICK_bom.xml
JSON output: False
Package info url: https://pypi.org/pypi/{package_name}/{package_version}/json
Generating CycloneDX BOM
Traceback (most recent call last):
  File "/var/lib/jenkins/venvs/cyclonedx_venv/bin/cyclonedx-py", line 8, in <module>
    sys.exit(main())
  File "/var/lib/jenkins/venvs/cyclonedx_venv/lib/python3.6/site-packages/cyclonedx/client.py", line 83, in main
    bom_contents = generate_bom(args)
  File "/var/lib/jenkins/venvs/cyclonedx_venv/lib/python3.6/site-packages/cyclonedx/client.py", line 59, in generate_bom
    bom_contents = reader.read_bom(fd, args.package_info_url, args.json)
  File "/var/lib/jenkins/venvs/cyclonedx_venv/lib/python3.6/site-packages/cyclonedx/bom/reader.py", line 23, in read_bom
    for component in all_components:
  File "/var/lib/jenkins/venvs/cyclonedx_venv/lib/python3.6/site-packages/cyclonedx/bom/reader.py", line 18, in <genexpr>
    all_components = (get_component(req, package_info_url) for req in requirements.parse(fd))
  File "/var/lib/jenkins/venvs/cyclonedx_venv/lib/python3.6/site-packages/cyclonedx/bom/reader.py", line 67, in get_component
    if package_license != 'UNKNOWN' and len(package_license.strip()) > 0:
AttributeError: 'NoneType' object has no attribute 'strip'

Expected Results

I expect the SBOM to be generated successfully.

Examples

I found a couple packages that demonstrate the issue.

Alabaster 0.7.9 and 0.7.10

Sample requirements.txt

alabaster==0.7.9

pypi response

Note: Building the SBOM fails when the license is null but is successful when the license is an empty string.

$ curl -s https://pypi.org/pypi/alabaster/0.7.9/json | python -m json.tool | grep license
        "license": null,
$ curl -s https://pypi.org/pypi/alabaster/0.7.10/json | python -m json.tool | grep license
        "license": null,
$ curl -s https://pypi.org/pypi/alabaster/0.7.11/json | python -m json.tool | grep license
        "license": "",
$ curl -s https://pypi.org/pypi/alabaster/0.7.12/json | python -m json.tool | grep license
        "license": "",

pbr 1.10.0

Sample requirements.txt

pbr==1.10.0

pypi response

Note: Building the SBOM fails when the license is null but is successful when the license is an empty string.

$ curl -s https://pypi.org/pypi/pbr/1.10.0/json | python -m json.tool | grep license
        "license": null,
$ curl -s https://pypi.org/pypi/pbr/2.0.0/json | python -m json.tool | grep license
        "license": "",

From CycloneDX/gh-python-generate-sbom#2

The generator mentions using requirements.txt. However, the Python community is moving away from using requirements.txt to integrating the build system and all requirements into a pyproject.toml file. Any chance this will be added to gh-python-generate-sbom?

As noted in #247 (comment) by @llamahunter.

I am getting below error, when I upgraded to v1.4

 cyclonedx-py -pip --pip-file app/Pipfile.lock -o bom/bom.json --format json
usage: cyclonedx-py [-h] (-c | -cj | -e | -p | -pip | -r) [-i FILE_PATH]
                    [--format {json,xml}] [--schema-version {1.3,1.2,1.1,1.0}]
                    [-o FILE_PATH] [-F] [-X]
cyclonedx-py: error: unrecognized arguments: --pip-file app/Pipfile.lock

when i revert back to 1.3.1 it works fine

pip install cyclonedx-bom==1.31 
cyclonedx-py 
usage: cyclonedx-py [-h] (-e | -p | -pip | -r) [-pf FILE_PATH] [--pip-file FILE_PATH] [-rf FILE_PATH] [--format {json,xml}] [--schema-version {1.3,1.2,1.1,1.0}]
                    [-o FILE_PATH] [-F] [-X]

As per issue #284 raised by @Jonas-vdb.

Some development teams add local references to other packages in their requirements.txt files. This is currently causing an exception:

Traceback (most recent call last):
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 98, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1654, in parseString
    raise exc
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1644, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 3417, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 3739, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 3400, in parseImpl
    loc, resultlist = self.exprs[0]._parse( instring, loc, doActions, callPreParse=False )
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 1406, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/jvdb/repos/venv-tmp/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 2711, in parseImpl
    raise ParseException(instring, loc, self.errmsg, self)
pkg_resources._vendor.pyparsing.ParseException: Expected W:(abcd...) (at char 0), (line:1, col:1)

Could you publish a docker image for the tool? Then it is smooth to integrate in pipelines.

When running cyclonedx-py -i requirements.txt -o bom.xml

I got:

Input file: requirements.txt
Output BOM: bom.xml
Generating CycloneDX BOM
Traceback (most recent call last):
  File "/home/user/.local/share/virtualenvs/project-mOBEjDrB/bin/cyclonedx-py", line 8, in <module>
    sys.exit(main())
  File "/home/user/.local/share/virtualenvs/project-mOBEjDrB/lib/python3.6/site-packages/cyclonedx/cli/generateBom.py", line 142, in main
    component_elements = read_bom(fd)
  File "/home/user/.local/share/virtualenvs/project-mOBEjDrB/lib/python3.6/site-packages/cyclonedx/cli/generateBom.py", line 103, in read_bom
    raise ValueError("Could not find a matching normalized version string", name, version)
ValueError: ('Could not find a matching normalized version string', 'coverage', '5.0a8')

Tracking it down, looks like there is a call to canonicalize_version that converts 5.0a8 to 5a8 not giving a match inside _get_pypi_version which ultimately raises the exception.

Doing some Q&D editing and removing the call seems too fix it.

Any ideas?

Thx

Python 3.10 officially released on 09-Nov-2021 - support for this should be included / verified.

acc crit:

• have CI/CT running in py3.10 - and pass
• have tox running in py3.10 - and pass
• if it applies: added the py3.10 trove / classifier "Programming Language :: Python :: 3.10"

depends on CycloneDX/cyclonedx-python-lib#60

Hi,

cyclonedx-py fails on a requirements.txt, when called like this:

$ cyclonedx-py -i requirements.txt -o bom.xml
Traceback (most recent call last):
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\urllib\request.py", line 1317, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\http\client.py", line 1229, in request
self._send_request(method, url, body, headers, encode_chunked)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\http\client.py", line 1275, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\http\client.py", line 1224, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\http\client.py", line 1016, in _send_output
self.send(msg)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\http\client.py", line 956, in send
self.connect()
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\http\client.py", line 1384, in connect
super().connect()
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\http\client.py", line 932, in connect
self._tunnel()
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\http\client.py", line 906, in _tunnel
(version, code, message) = response._read_status()
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\http\client.py", line 265, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\site-packages\xmlschema\validators\schema.py", line 218, in init
self.source = XMLResource(source, base_url, defuse, timeout, lazy=False)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\site-packages\xmlschema\resources.py", line 240, in init
self.source = source
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\site-packages\xmlschema\resources.py", line 264, in setattr
self._root, self._document, self._text, self._url = self._fromsource(value)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\site-packages\xmlschema\resources.py", line 342, in _fromsource
resource = urlopen(url, timeout=self.timeout)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\urllib\request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\urllib\request.py", line 531, in open
response = meth(req, response)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\urllib\request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\urllib\request.py", line 563, in error
result = self._call_chain(*args)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\urllib\request.py", line 503, in _call_chain
result = func(*args)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\urllib\request.py", line 755, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\urllib\request.py", line 525, in open
response = self._open(req, data)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\urllib\request.py", line 543, in _open
'_open', req)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\urllib\request.py", line 503, in _call_chain
result = func(*args)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\urllib\request.py", line 1360, in https_open
context=self._context, check_hostname=self._check_hostname)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\urllib\request.py", line 1319, in do_open
raise URLError(err)
urllib.error.URLError:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\klp1wa3\AppData\Local\Programs\Python\Python37-32\Scripts\cyclonedx-py.exe_main.py", line 5, in
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\site-packages\cyclonedx\cli\generateBom.py", line 23, in
from cyclonedx import BomValidator
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\site-packages\cyclonedx\BomValidator.py", line 19, in
bom_schema = xmlschema.XMLSchema("http://cyclonedx.org/schema/bom/1.0")
File "c:\users\klp1wa3\appdata\local\programs\python\python37-32\lib\site-packages\xmlschema\validators\schema.py", line 220, in init
raise type(err)('cannot create schema: %s' % err)
urllib.error.URLError: <urlopen error cannot create schema: >

The requirements.txt looks a bit weird but seems to valid, see URL to blog post as comment:

$ cat requirements.txt
--index-url https://pypi.python.org/simple/

-e .

see https://caremad.io/posts/2013/07/setup-vs-requirement/ for more information on -e . ..

$

Is this something in the tool or in my environment?
Any idea? Thanks,

Peter

We currently utilise pkg_resources.parse_requirements to parse lines from requirements.txt files.

This seemingly has a number of short comings that it does not support. See:

• #315
• CycloneDX/cyclonedx-python-lib#8

This feature will look at alternatives to the above method to attempt to support these other formats. CycloneDX/cyclonedx-python-lib#97 may have identified a candidate in requirements-parser - TBC.

The current version constraints (all pinned) make it impractical for reuse as a library. It would be helpful to relax these in the setup.py (it is fine to keep these pinned in your requirements.txt)
This would help with nexB/scancode-toolkit#1888

When you do on Windows on Powershell a
pip freeze > requirments.txt
the resulting file will have the encoding UCS-2 LE BOM.
Cyclonedx-py crashes when processing sucha file. Reason is that the requirments library can't process a file with such encoding.

One workaround would be to use on Powershell the command
pip freeze | Out-File -Filepath r4.txt -Encoding ASCII
but then you'd need an extra description for Windows.

It would be easier to let cyclonedx-py detect the encoding.

poetry failed to install in CI from time to time. the a administrator/maintainer needs to click the "re-run all jobs" button to kick tests manually. 🔍 TODO: find a breaking workflow and link it here
this seams to be caused by the poetry cache.

as a maintainer i want the poetry install to pass, so that dont have to re-run them manually.

Pip supports including hashes in requirements.txt, e.g. generated with pip-tools:

$ echo requests > requirements.in
$ pip-compile --generate-hashes
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile --generate-hashes
#
certifi==2021.5.30 \
    --hash=sha256:2bbf76fd432960138b3ef6dda3dde0544f27cbf8546c458e60baf371917ba9ee \
    --hash=sha256:50b1e4f8446b06f41be7dd6338db18e0990601dce795c2b1686458aa7e8fa7d8
    # via requests
chardet==4.0.0 \
    --hash=sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa \
    --hash=sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5
    # via requests
idna==2.10 \
    --hash=sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6 \
    --hash=sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0
    # via requests
requests==2.25.1 \
    --hash=sha256:27973dd4a904a4f13b263a19c866c13b92a39ed1c964655f025f3f8d3d75b804 \
    --hash=sha256:c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e
    # via -r requirements.in
urllib3==1.26.5 \
    --hash=sha256:753a0374df26658f99d826cfe40394a686d05985786d946fbe4165b5148f5a7c \
    --hash=sha256:a7acd0977125325f516bda9735fa7142b909a8d01e8b2e4c8108d0984e6e0098
    # via requests

This seems to make cyclonedx-python crash:

$ cyclonedx-py -i requirements.txt
Input file: requirements.txt
Output BOM: bom.xml
JSON output: False
Package info url: https://pypi.org/pypi/{package_name}/{package_version}/json
Generating CycloneDX BOM
Traceback (most recent call last):
  File "/usr/local/bin/cyclonedx-py", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/cyclonedx/client.py", line 83, in main
    bom_contents = generate_bom(args)
  File "/usr/local/lib/python3.8/dist-packages/cyclonedx/client.py", line 59, in generate_bom
    bom_contents = reader.read_bom(fd, args.package_info_url, args.json)
  File "/usr/local/lib/python3.8/dist-packages/cyclonedx/bom/reader.py", line 23, in read_bom
    for component in all_components:
  File "/usr/local/lib/python3.8/dist-packages/cyclonedx/bom/reader.py", line 18, in <genexpr>
    all_components = (get_component(req, package_info_url) for req in requirements.parse(fd))
  File "/usr/local/lib/python3.8/dist-packages/requirements/parser.py", line 50, in parse
    yield Requirement.parse(line)
  File "/usr/local/lib/python3.8/dist-packages/requirements/requirement.py", line 220, in parse
    return cls.parse_line(line)
  File "/usr/local/lib/python3.8/dist-packages/requirements/requirement.py", line 198, in parse_line
    pkg_req = Req.parse(line)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3150, in parse
    req, = parse_requirements(s)
ValueError: not enough values to unpack (expected 1, got 0)

Will there be added support for the dependency graph extension, or will that be a separate library?

currently the code goes trough a pep8 test via flake8 that must be passed in CI

as a developer i want a tool available, that automatically fixes my code according to pep8

options:

• autopep8: https://pypi.org/project/autopep8/
• black: https://pypi.org/project/black/
• yapf: https://pypi.org/project/yapf/

I, @jkowalleck, prefer autopep8 as it fixes pep8 only. black is to much opinionated.

I am getting following error: Pasting little extra information for context and clarity:

2019-10-28T23:20:16.1096691Z Successfully built requirements-parser
2019-10-28T23:20:16.1102564Z Installing collected packages: requirements-parser, urllib3, certifi, chardet, idna, requests, elementpath, xmlschema, six, attrs, pyparsing, packaging, packageurl-python, cyclonedx-bom
2019-10-28T23:20:17.1299044Z Successfully installed attrs-19.3.0 certifi-2019.9.11 chardet-3.0.4 cyclonedx-bom-0.3.2 elementpath-1.2.2 idna-2.8 packageurl-python-0.8.7 packaging-19.1 pyparsing-2.4.2 requests-2.22.0 requirements-parser-0.1.0 six-1.12.0 urllib3-1.25.6 xmlschema-1.0.14
2019-10-28T23:20:17.3477654Z You are using pip version 8.1.1, however version 19.3.1 is available.
2019-10-28T23:20:17.3479418Z You should consider upgrading via the 'pip install --upgrade pip' command.
2019-10-28T23:20:17.3702738Z + pip show cyclonedx-bom
2019-10-28T23:20:17.8761536Z ---
2019-10-28T23:20:17.8765697Z Metadata-Version: 2.1
2019-10-28T23:20:17.8766202Z Name: cyclonedx-bom
2019-10-28T23:20:17.8766413Z Version: 0.3.2
2019-10-28T23:20:17.8769902Z Summary: CycloneDX software bill-of-material (SBOM) generation utility
2019-10-28T23:20:17.8770437Z Home-page: https://github.com/CycloneDX/cyclonedx-python
2019-10-28T23:20:17.8770629Z Author: Steve Springett
2019-10-28T23:20:17.8774491Z Author-email: [email protected]
2019-10-28T23:20:17.8774946Z Installer: pip
2019-10-28T23:20:17.8775323Z License: Apache-2.0
2019-10-28T23:20:17.8778455Z Location: /home/vsts/.local/lib/python2.7/site-packages
2019-10-28T23:20:17.8779012Z Requires: requirements-parser, requests, xmlschema, packaging, packageurl-python
2019-10-28T23:20:17.8779198Z Classifiers:
2019-10-28T23:20:17.8782981Z Intended Audience :: Developers
2019-10-28T23:20:17.8783440Z Intended Audience :: Information Technology
2019-10-28T23:20:17.8783608Z Intended Audience :: Legal Industry
2019-10-28T23:20:17.8786665Z Intended Audience :: System Administrators
2019-10-28T23:20:17.8787179Z Topic :: Security
2019-10-28T23:20:17.8787358Z Topic :: Software Development
2019-10-28T23:20:17.8790255Z Topic :: System :: Software Distribution
2019-10-28T23:20:17.8791144Z License :: OSI Approved :: Apache Software License
2019-10-28T23:20:17.8791334Z Programming Language :: Python :: 2.7
2019-10-28T23:20:17.8795009Z Entry-points:
2019-10-28T23:20:17.8795283Z [console_scripts]
2019-10-28T23:20:17.8795699Z cyclonedx-py = cyclonedx.cli.generateBom:main
2019-10-28T23:20:18.0775388Z You are using pip version 8.1.1, however version 19.3.1 is available.
2019-10-28T23:20:18.0777645Z You should consider upgrading via the 'pip install --upgrade pip' command.
2019-10-28T23:20:18.0986202Z + echo '/usr/share/rust/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/.local/bin'
2019-10-28T23:20:18.0988032Z + cyclonedx-py
2019-10-28T23:20:18.0988825Z /usr/share/rust/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/.local/bin
2019-10-28T23:20:24.7208894Z Input file: requirements.txt
2019-10-28T23:20:24.7210400Z Output BOM: bom.xml
2019-10-28T23:20:24.7211081Z Traceback (most recent call last):
2019-10-28T23:20:24.7212274Z File "/home/vsts/.local/bin/cyclonedx-py", line 11, in
2019-10-28T23:20:24.7212852Z sys.exit(main())
2019-10-28T23:20:24.7213651Z File "/home/vsts/.local/lib/python2.7/site-packages/cyclonedx/cli/generateBom.py", line 137, in main
2019-10-28T23:20:24.7214486Z with open(args.input_file, 'r', encoding=result['encoding']) as fd:
2019-10-28T23:20:24.7215292Z TypeError: 'encoding' is an invalid keyword argument for this function
requirements.txt

as a developer i want all imports sorted, so i read and maintain them easily

proposal:

• have isort as a tool available in poetry-dev-env
• have isort configured -- see below
• have flake8 plugin flake8-isort` installed and setup in poetry-dev-env
• have isort applied to all python files (once)
• have the use of isort mentioned i the CONTRIBUTING docs

proposed isort config file .isort.cfg:

[isort]
## read the docs: https://github.com/timothycrosley/isort/wiki/isort-Settings
known_first_party = cyclonedx_py
skip_glob =
    build,dist,__pycache__,.eggs,*_cache
    .git,.tox,.venv,venv
    _OLD,_TEST,
    docs
combine_as_imports = true
default_section = THIRDPARTY
include_trailing_comma = true
line_length = 120
multi_line_output = 5

Hey @stevespringett, I was going to do some basic happy path tests for this implementation.

Do you have any existing requirements files with corresponding BOM files you have already validated?

And any current limitations? i.e. are transitive dependencies correctly added to the BOM? Never mind, I can see that pip freeze correctly reports all dependencies.

During the reincarnation of this application, we disabled building the docker image for ease.

It's now time to bring this back!