enthec / webappanalyzer Goto Github PK

Is your feature request related to a problem? Please describe.
As part of the CI workflow for PRs (etc) would it be possible to validate the regex patterns or dom selectors ?

In the past we've found that upstream of AliasIO we encountered invalid regex patterns added to the technology files, or invalid selectors.

The two "normal" cases seemed to be:

• when a regex contained curly braces which either weren't matched in a repetition declaration or when they weren't escaped.
• when a dom selector had unbalanced single or double quotes.

There are plenty of other things that can make a regex or dom selector invalid, it would be good to catch and fix these early.

Describe the solution you'd like
I believe this could be added to the existing Python based validation. In Java a pattern can be compiled (Pattern.compile(String)) at which point an exception would be thrown if invalid. We also came up with something similar for DOM selectors. I assume something similar can be done with Python.

Describe alternatives you've considered

• Live with the errors and correct them as they're noticed.

Additional context
Not sure what else to say here. Mainly I was thinking that catching potential errors as close to introduction as possible would be the easiest way to address/prevent them.

Platform
MacOS 12.7 with python 3.10.

Describe the bug
The Wagtail DOM regexs [style*='images/'], img[src*='images/'], etc. Are not compiling in python 3.10.

To Reproduce

import re
re.compile("(?:\\.[a-z]+|/media)(?:/[\\w-]+)?/(?:original_images/[\\w-]+|images/[\\w-.]+\\.(?:(?:fill|max|min)-\\d+x\\d+(?:-c\\d+)?|(?:width|height|scale)-\\d+|original))\\.")

Expected behavior
The regex should compile.

Additional context
The source of the bug is [\\w-.]. When modified to [\\w-\\.] everything works accordingly. Before doing a PR fixing this problem, I wanted to be sure I didn't miss something about how Javascript regexs work.

Platform
MacOS

Describe the bug
After await wappalyzer.init(); if I do a await wappalyzer.destroy(); it hangs forever despite checking I'm calling it the right way.

When doing console.log(wappalyzer.destroy) I get [AsyncFunction: destroy].

It was in their last public documentation https://www.npmjs.com/package/wappalyzer/v/6.10.66 so I don't understand what I'm doing wrong 😢 .

Additional context
Version v6.10.66

Describe the bug
The definition of the schema in schema.json specifies that some fields may be one of several types.

For instance, the field implies can either be a string or an array (of strings):

"implies": {
  "oneOf": [
    {
      "type": "array",
      "items": {
        "$ref": "#/definitions/non-empty-non-blank-string"
      }
    },
    {
      "$ref": "#/definitions/non-empty-non-blank-string"
    }
  ]
},

This makes it really hard to parse the JSON object in languages like Go where you need to define the types statically:

type Techno struct {
    Categories  []int    `json:"cats"`
    Implies     []string `json:"implies"`  // This may just be a string!
}

Trying to deserialize a JSON object into an instance of this struct will return an error if the JSON input is using a string type instead of an array of strings:

json: cannot unmarshal string into Go struct field Techno.implies of type []string

Expected behavior
Have a single type for each field.

The CPE for IIS is defined in actual signatures as
cpe:2.3:a:microsoft:internet_information_server::::::::

It should be
cpe:2.3:a:microsoft:internet_information_services::::::::

It would probably be a good idea for this repo to include the latest schema.json from the original project.

Platform
macOS 12.7 with Python 3.10.

Describe the bug
After my last Issue, I tried to test ALL regex with python and found 3861 errors.

To Reproduce
See this gist

Expected behavior
All regex should compile in Python

Additional context
No

Describe the solution you'd like
Included with each tech found, indicate if it's vulnerable and a link to the vulnerability such as a CWE. Expose this data in such a way that another tool, such as ZAP, can leverage it.

Describe alternatives you've considered
Searching using other tools.

Hiya ZAP team here (inc @thc202, @kingthorin, @ricekot),

As you may know we have a ZAP add-on which wraps the old Wappalyzer functionality: https://www.zaproxy.org/docs/desktop/addons/technology-detection/

Obviously this is not now being updated ☹️

Are you ok with us migrating to use webappanalyzer instead?
We will update that link to give you credit.

FYI we have a github action which automatically pulls the source. This also has some unit tests - we've found bugs in Wappalyzer regexes quite a few times, and always report these back 😁