From a user's perspective, it would be beneficial to have more details (or at least an option for it) in the output. It should include:

• Information from the publisher object
• role
• Path of the provider-metadata.json found as link
• whether the provider-metadata.json is valid
• Path of the security.txt tested/found
• whether ROLIE feeds exist, where and whether they are valid

In order to implement checks for the generated ROLIE documents, e.g. #41 and #47
it would be good to have a json schema (which includes the required fields).

https://www.rfc-editor.org/rfc/rfc8322.html#appendix-A has a
complete informative RELAX NG Compact Schema for the new elements introduced by ROLIE
based on the Atom scheme, but it is for XML.

One should also be able to provide a TLS client certificate to use for the TLP:AMBER and TLP:RED feeds.

Originally posted by @tschmidtb51 in #42 (comment)

I found a few issues with the documentation of the provider setup:

• csaf_distribution/docs/provider-setup.md

  Line 10 in c57de75

  systemctl status fcgiwrap.servic

  is missing an e
• It is unclear what is checked with https://github.com/csaf-poc/csaf_distribution/blob/main/docs/provider-setup.md?plain=1#L10-L13
• From the description before https://github.com/csaf-poc/csaf_distribution/blob/main/docs/provider-setup.md?plain=1#L22-L50 it is not clear that fcgiwrap.conf should be modified.

• csaf_distribution/docs/provider-setup.md

  Line 52 in c57de75

  Add to `/etc/nginx/sites-enabled/default`:

  : Add should be bold.

• csaf_distribution/docs/provider-setup.md

  Line 56 in c57de75
  should at least contain an additional

       # Other config
       # ... 
  

  to show that there is additional stuff in the file.
• As the directory /usr/lib/cgi-bin/ may not exist, there should be a step to create it and set the correct permissions / ownership.
• The binary, when compiled does not end with .go. Either it should be renamed or

  csaf_distribution/docs/provider-setup.md

  Line 74 in c57de75

  Place the binary under `/usr/lib/cgi-bin/csaf_provider.go`.

  should be changed.
• web should be in the config show here.

• csaf_distribution/docs/provider-setup.md

  Line 88 in c57de75

  with suitable replacements.

  should also link to #32

Each application should get a short description, e.g. in the main README file.

Maybe 3-4 sentences.

We should use the CSD02's JSON schema.

My reading of https://docs.oasis-open.org/csaf/csaf/v2.0/cs01/csaf-v2.0-cs01.html#51-filename is that filenames MUST NOT contain upper case characters. It seem we still accept them, to help introducing the standard we probably should reject them on all levels.

This would be an enhancement, as the example files of the standardisation group still have upper case characters oasis-tcs/csaf#508
So first my reading has to be confirmed.

The aggregator has a mistake in

It should be instead:

 csafURL := w.cfg.Domain + "/.well-known/csaf-aggregator/" + 

It looks like tlp=csaf is an invalid config option as it results in:

Errors:
	unsupported TLP type 'csaf'

Was that intended?

Only some valid TLS client certificate should allow an upload of information (e.g. access to the API and webinterface of the csaf_provider.)

Technically we need

• a config option to set an Issuer
• a compariston of this issuer to os.Getenv("SSL_CLIENT_I_DN") in cmd/csaf_provider/controller.go
• and extension of the documentation that this CA has to be part of the general list of allowed CS in the ssl_client_certificate bundle of nginx.

Please align the table with the help function. E.g.

• The --insecure flag seems to exist in the csaf_uploader but is not documented in the table.
• -x gives more info in the help than the table has

If possible, it would be nice of the provider would tell me that it is a cgi binary when called directly.
(And potentially the version, number, see #66).

The provider should be configurable for indicating that it wants to be

• listed
• mirrored

Would be nice to give usage information if options or parameters fail or if -h or --help is given explitly at all.
Currently:

./csaf_checker -t html example.com -h
unknown flag `t'
2022/02/25 17:46:33 error: unknown flag `t'

So I need to call it all just with -h without extra parameters.
If -h is present it should be executed and then execution should stop.

Would be nice to use -v and --version to see which version of the binary I have.

technical considerations

We need one way to build this for released versions (if they are tagged and this is git we could take it from the tag)
and for development versions.

I was testing the aggregator and ran into the error

However, I was unable to figure out where the result was set / defined.

If that's just a "Not implemented" - feel free to provide a short response and ignore the issue.

There are forthcoming changes towards CSAF-2.0 cs2 and it would be nice if the provider could write out the improved ROLIE values (in the area of hashes and sigs).

To authenticate with a TLS client certificate, the client needs to have access to the private key. Private keys can be protected by a passphrase. It should be possible to use such protected client certificates with the uploader.

Currently the config files for the provider are placed in /usr/lib/csaf, e.g. see (but also in other places)

As those config files are not likely to change in normal operations, they probably belong to /etc, either
/etc/csaf or /etc/csaf_provider at least that is my interpretation of https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard

@s-l-teichmann what do you think?

Here is a checker run, which encountered a ROLIE document with

{
"updated" : "2022-02-22T00:00Z"
}

which is a valid ISO 8601 format (see second paragraph in https://en.wikipedia.org/wiki/ISO_8601#Combined_date_and_time_representations for an example)
and returned a
Loading ROLIE feed failed: parsing time "2022-02-22T00:00Z" as "2006-01-02T15:04:05Z07:00": cannot parse "Z" as ":".

Expectation: this format is accepted.

The value of role in provider-metadata.json should be csaf_trusted_provider by default.

oasis-tcs/csaf#430 suggests retrieving steps which the aggregator should follow and the checker may check more than this (like retrieving the DNS and see if it is there and different).

It looks like some options of the csaf_uploader can't be stored in the config (e.g. settings for interactive password). Is that correct? Shouldn't that be possible with password-interactive=true?

Sometimes the web interface shows, after calling `https://example.com/cgi-bin/csaf_provider.go, the following message:

CSAF-Provider - Directory structure created

Everything is setup fine now.

If I reload the page, it comes up with the upload interface. What am I missing?

When trying to check a domain with the csaf_checker, I get the following error message:

$ ./csaf_checker www.domain.com
2022/01/11 15:51:08 error: no document to extract data from

Why does the command fail?

Thanks in advance!
Tobi

For a correct CSAF Trusted Provider, only one method of providing several things is needed.

It would be nice to have an option to only publish as ROLIE and to have this option being activated by default.
Because ROLIE is considered the preferred way of publishing via CSAF-2.0 and by using it only, a Trusted Provider sets a good example for others.

Maybe out of scope for 1.0

Wish to have a csaf_provider setup option that a password is required together with a client certificate.

We should extend the documentation regarding the password option (as we do not store it in plain text ;-).

Missing a property in the provider-metadata.json, that is marked as required key in the schema does not display an error.

• csaf_distribution/cmd/csaf_provider/actions.go

  Line 175 in 934d175

  return nil, fmt.Errorf("not a valid TL: %s", ex.tlpLabel)

  says: not a valid TL. It would be nice if it differentiates between a non-existing TLP and an invalid one. Moreover, the message should be improved to TLP label missing in the document resp. not a valid TLP label: %s

As protection against an overly long running process, another start of csaf_aggregator should be detected, prevented and warned about. This shall protect against two instances running and conflicting.

Using the current build instructions

 cd csaf_distribution
 go build -v ./cmd/...

there are sometimes no binaries build.
(Test done with go version go1.17.6 linux/amd64 and c57de75 (current head) )

Allow the csaf_uploader to use a TLS Client certificate to authenticate itself to the csaf_provider API.

technical considerations

It is okay to use whatever can be more easily implemented when telling the tool which client certificate to take, one setting for the p12 format or two settings for key and cert.

Needs checking if this also works with client certificates which are password protected.

Some CSAF providers may block an IP address if there are too many documents downloaded in a certain time frame.

To stay below this threshold, a config option in the checker and per provider in the aggreator shall limit the rate of how many documents are downloaded per time unit. E.g. in documents per minute.

We should improve the logging when using TLS client certificates. Even if I just access the WebUI I get a message in the log:

2022/04/08 17:30:26 [error] 11918#11918: *23 FastCGI sent in stderr: "2022/04/08 17:30:26 SSL_CLIENT_VERIFY: SUCCESS
2022/04/08 17:30:26 ca: C=DE,O=C=DE,O=CSAF Tools Development (internal),CN=Tester
2022/04/08 17:30:26 user: C=DE,O=CSAF Tools Development (internal),CN=TLS Uploader Client 1" while reading response header from upstream, client: xxx.xxx.xxx.xxx, server: xxxxxx.xxxx, request: "GET /cgi-bin/csaf_provider.go/ HTTP/2.0", upstream: "fastcgi://unix:/var/run/fcgiwrap.socket:", host: "xxxxxx.xxxx"

Noticed during #83 testing.

If the value of the category attribute in provide_metadata.publisher is not one of the allowed ones a 502 Bad Gateway error message is shown.
This message should be enhanced to indicate the error of wrong value for the category.

11ed0e8 changed the schema and thus the requirement and contents of the provider-metadata.json. Now our checker shows that the provider-metadata.json we have written with the provider does not validate with:

 "messages": [
  "https://localhost:8443/.well-known/csaf/provider-metadata.json: Validating against JSON schema failed:",
  "https://docs.oasis-open.org/csaf/csaf/v2.0/provider_json_schema.json#/additionalProperties: additionalProperties 'pgp_keys' not allowed",

so at least the pgp_keys have to be changed to public_openpgp_keys, but there maybe other changes.

@Fadiabb one for you?

We should have options to set those values from the provider-metadata.json which we can't set right now through the config...

The CSAF provider comes with many options. They should be documented (including their default value) and explained.

When trying to upload without the password, it results correctly in 403. But the message seems a little bit weird:

Upload failed: 403 Forbidden
error: invalid character 'F' looking for beginning of value

I guess

There are forthcoming changes towards CSAF-2.0 cs2 and it would be nice if the checker could check for the improved ROLIE values (in the area of hashes and sigs).

maybe out of scope for 1.0

CSAF Aggregators must at least do two mirrors, but can also have entries that are just listed, see
oasis-tcs/csaf#470

technical consideration

A config option per provider could handle this.

It looks like the following config could work to fulfill requirement 10:

# Special DNS requirement
server {
        listen 443 ssl http2;
        listen [::]:443 ssl http2;

        ssl_certificate {SSL-CERT};
        ssl_certificate_key {SSL_KEY};

        root /var/www/domain;
        
        server_name csaf.data.security.domain.tld;

        location / {
                try_files /.well-known/csaf/provider-metadata.json =404;
        }

        access_log /var/log/nginx/dns-domain_access.log;
        error_log /var/log/nginx/dns-domain_error.log;

}

But maybe there is a better solution?

Split out from #24 .

The checker has to check if the provider-metadata.json can be found in at least one of the following to be valid:

• The security.txt through the CSAF field Requirment 8
• The .well-known path Requirment 9
• Through the DNS path Requirment 10

These should be fulfilled according to the second group of the requirements in Role: CSAF provider

The ROLIE feeds currently generated by the csaf_provider are missing the surrounding feed element.

Expected output:

{
  "feed": {
    "id": "example-csaf-feed-tlp-white",
    "title": "Example CSAF feed (TLP:WHITE)",
    //...
  }
}

Current output:

{
  "id": "csaf-feed-tlp-white",
  "title": "CSAF feed (TLP:WHITE)",
  // ...
}

The ROLIE feeds generated by the csaf_provider are missing the category field

"category": [ 
        {
          "scheme": "urn:ietf:params:rolie:category:information-type",
          "term": "csaf"
        }
      ]

which is required as per RFC 8322 Section 6.1.

Some functions implemented in the csaf_aggregator are useful when implementing a downloader.
The should be provided that they can be used easily.

With secvisogram/csaf-validator-service a service exists that could run the business level test on CSAF documents and provide the information whether the document is valid. We should add a config option to send the CSAF document to such a service and parse the result. If isValid is true, the upload should proceed, otherwise an error should be thrown.
See the documentation for details on how to send the document and receive the response.

It would be nice, if the security.txt is updated (adding the CSAF field) if it already exists.

As clarified from @tschmidtb51 and others in January, the public OpenPGP key used for signing the CSAF files shall be placed in a directory openpgp/ next to the provider-metadata.json by the provider.

Accordingly this is where the pgp_keys url should point to.