jhinch / nginx-linter Goto Github PK

According to https://www.nginx.com/resources/wiki/start/topics/depth/ifisevil/ (currently unavailable, but archived here: https://archive.md/V82FV), the if directive has problems when used in location context. However, the linter shows errors even if it is used in the server context.

It seems like the the built in options always get used, even if passing in a path to a custom config.

When location regexes such as ~^ doesn't contain a space ie:

location ~*^/if-and-alias/

nginx accepts the configuration but the linter will show an incorrect error message:
"error Expected string when using 'location', got regular expression"

This message disappears as soon as the location is using a space.

location ~* ^/if-and-alias/

It would be nice if the linter suggested to add the space instead.

I have a generated nginx configuration file that is 3.7MB / 130K LOC.
Trying to run nginx-linter on that file fails;

nginx-linter --include nginx.conf 

nginx.conf
Unexpected error: RangeError: Maximum call stack size exceeded
    at flatten (/opt/homebrew/lib/node_modules/nginx-linter/node_modules/table/dist/src/utils.js:112:15)
    at mapDataUsingRowHeights (/opt/homebrew/lib/node_modules/nginx-linter/node_modules/table/dist/src/mapDataUsingRowHeights.js:49:32)
    at table (/opt/homebrew/lib/node_modules/nginx-linter/node_modules/table/dist/src/table.js:24:64)
    at outputInnerResults (/opt/homebrew/lib/node_modules/nginx-linter/bin/_cli/commands.js:148:20)
    at outputResults (/opt/homebrew/lib/node_modules/nginx-linter/bin/_cli/commands.js:127:5)
    at /opt/homebrew/lib/node_modules/nginx-linter/bin/_cli/commands.js:83:13
    at Array.map (<anonymous>)
    at validate (/opt/homebrew/lib/node_modules/nginx-linter/bin/_cli/commands.js:80:32)
    at execute (/opt/homebrew/lib/node_modules/nginx-linter/bin/_cli/commands.js:51:20)
    at Object.main (/opt/homebrew/lib/node_modules/nginx-linter/bin/_cli/commands.js:168:16)

I also have a generated configuration file that is 800KB / 27500 LOC.
Running the linter on this file takes about 30 minutes on an Apple M1 CPU.
The file has a lot of whitespace issues, if that's a hint.

Example 1
I'm using json logging, but the linter does not like my formatting.

http {
    log_format json_analytics escape=json
      '{'
        '"msec": "$msec", ' # request unixtime in seconds with a milliseconds resolution
        '"connection": "$connection", ' # connection serial number
        '"connection_requests": "$connection_requests", ' # number of requests made in connection
        '"pid": "$pid", ' # process pid
      '}';
}

The output of this is:

6:29 error Expected 4 spaces, found 9 spaces indentation
7:41 error Expected 4 spaces, found 9 spaces indentation
8:59 error Expected 4 spaces, found 9 spaces indentation
9:27 error Expected 4 spaces, found 9 spaces indentation

That's weird, I only see 8 spaces. Let's make it 4.

6:25 error Expected 4 spaces, found 5 spaces indentation
7:37 error Expected 4 spaces, found 5 spaces indentation
8:55 error Expected 4 spaces, found 5 spaces indentation
9:23 error Expected 4 spaces, found 5 spaces indentation

Okay, let's make it 3 spaces then.

user nginx;

http {
    log_format json_analytics escape=json
      '{'
   '"msec": "$msec", ' # request unixtime in seconds with a milliseconds resolution
   '"connection": "$connection", ' # connection serial number
   '"connection_requests": "$connection_requests", ' # number of requests made in connection
   '"pid": "$pid", ' # process pid
      '}';
}

Validation succeeded! Files: 1, Errors: 0

That doesn't look right.

Example 2
Another log_format setup. Here I'm trying to provoke an error by inserting multiple spaces on the log_format line, as well as using weird indentation on the following lines.

stream {
    include conf.d/stream/*.conf;

    log_format    basic  '$remote_addr [$time_local] '
                             '$protocol $status $bytes_sent $bytes_received '
                           '$session_time';

    access_log  /data/logs/stream.log basic buffer=1m flush=10s;
}

Validation succeeded! Files: 1, Errors: 0

I was expecting it to complain about multiple spaces between log_format and basic and the weird indentation.

Currently the rules are documented but you need to inspect the source code to see how to apply the override. Both the configuration file and the comment based overrides

I started out with a long regex containing curly brackets, but realized only the braces containing something is enough.

map $http_x_forwarded_for $realip {
"{a}" $1;
}

Curiously empty braces works:

map $http_x_forwarded_for $realip {
"{}" $1;
}

At the moment the only default location for the configuration file .nginx-linter.json is in the home directory (~/.nginx-linter.json).
This means if you use one configuration file per repository/project to have this consistent and use it in CI, you will always have to add --config .nginx-linter.json.

My proposal would be to:

• check for .nginx-linter.json in current directory
• if not found check ~/.nginx-linter.json

I've encountered crashes for the linter. It would be helpful to get what line it parsed before it crashed.

First I got this issue, perhaps due to paranthesis in regex expressions.

$ nginx-linter --include site.conf
Unexpected error: Error: Failed to parse: site.conf
Caused by:
SyntaxError: Expected " ", "#", ";", "\n", "\r", "\t", or "{" but ")" found.
at parseFile (/opt/homebrew/lib/node_modules/nginx-linter/lib/parser/index.js:22:15)
at Array.map ()
at Object.parseFiles (/opt/homebrew/lib/node_modules/nginx-linter/lib/parser/index.js:31:23)
at validate (/opt/homebrew/lib/node_modules/nginx-linter/bin/_cli/commands.js:75:28)
at execute (/opt/homebrew/lib/node_modules/nginx-linter/bin/_cli/commands.js:51:20)
at Object.main (/opt/homebrew/lib/node_modules/nginx-linter/bin/_cli/commands.js:168:16)
at Object. (/opt/homebrew/lib/node_modules/nginx-linter/bin/nginx-linter.js:6:24)
at Module._compile (node:internal/modules/cjs/loader:1105:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
at Module.load (node:internal/modules/cjs/loader:981:32)

Then I got this issue, not sure what's up here, since I get no hint on what line that fails.

$ nginx-linter --include anothersite.conf
Unexpected error: Error: Failed to parse: anothersite.conf
Caused by:
SyntaxError: Expected " ", "#", "\n", "\r", "\t", "access_by_lua_block", "balancer_by_lua_block", "body_filter_by_lua_block", "content_by_lua_block", "header_filter_by_lua_block", "init_by_lua_block", "init_worker_by_lua_block", "log_by_lua_block", "rewrite_by_lua_block", "set_by_lua_block", "ssl_certificate_by_lua_block", "ssl_session_fetch_by_lua_block", "ssl_session_store_by_lua_block", "}", or any character but end of input found.
at parseFile (/opt/homebrew/lib/node_modules/nginx-linter/lib/parser/index.js:22:15)
at Array.map ()
at Object.parseFiles (/opt/homebrew/lib/node_modules/nginx-linter/lib/parser/index.js:31:23)
at validate (/opt/homebrew/lib/node_modules/nginx-linter/bin/_cli/commands.js:75:28)
at execute (/opt/homebrew/lib/node_modules/nginx-linter/bin/_cli/commands.js:51:20)
at Object.main (/opt/homebrew/lib/node_modules/nginx-linter/bin/_cli/commands.js:168:16)
at Object. (/opt/homebrew/lib/node_modules/nginx-linter/bin/nginx-linter.js:6:24)
at Module._compile (node:internal/modules/cjs/loader:1105:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
at Module.load (node:internal/modules/cjs/loader:981:32)

Currently, nginx-linter only works with the builtin rules. However, the validator itself can work with custom rules. Allow for the configuration to import custom rules using a similar strategy to eslint.

Hi there, thanks for creating this project.

The following line broke the parser

SecDefaultAction "phase:1,log,auditlog,pass"

It would be nice to add a 'file X broke the parser' error, or maybe just a log 'parsing file X'.

Also the globs have many problems. I have the following structure in /etc/nginx

.
  |-config
  |  |-mime.types
  |  |-include
  |  |  |-internal_locations.conf
  |  |  |-logformat.conf
  |  |  |-subdomains
  |  |  |  |-a.conf
  |  |  |  |-b.conf
  |  |  |  |-c.conf
  |  |  |-compression.conf
  |  |-nginx.conf
  |  |-modsec
  |  |  |-rules
  |  |  |  |-REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.aconf
  |  |  |-modsecurity.aconf
  |  |  |-crs-setup.conf

** works only for one level, it should work recursive. So if I'm on /etcthis only include one file: nginx-linter --include nginx/**/*.conf

If I include an extra **, like nginx-linter --include nginx/**/**/*.conf, I get Unknown option: nginx/config/include/logformat.conf

There are similar problems with exclude.

If a configuration block isn't closed with '}' nginx-linter will crash with the following message:

Unexpected error: Error: Failed to parse: test.conf
Caused by:
SyntaxError: Expected " ", "#", "\n", "\r", "\t", "access_by_lua_block", "balancer_by_lua_block", "body_filter_by_lua_block", "content_by_lua_block", "header_filter_by_lua_block", "init_by_lua_block", "init_worker_by_lua_block", "log_by_lua_block", "rewrite_by_lua_block", "set_by_lua_block", "ssl_certificate_by_lua_block", "ssl_session_fetch_by_lua_block", "ssl_session_store_by_lua_block", "}", or any character but end of input found.
at parseFile (/opt/homebrew/lib/node_modules/nginx-linter/lib/parser/index.js:22:15)
at Array.map ()
at Object.parseFiles (/opt/homebrew/lib/node_modules/nginx-linter/lib/parser/index.js:31:23)
at validate (/opt/homebrew/lib/node_modules/nginx-linter/bin/_cli/commands.js:75:28)
at execute (/opt/homebrew/lib/node_modules/nginx-linter/bin/_cli/commands.js:51:20)
at Object.main (/opt/homebrew/lib/node_modules/nginx-linter/bin/_cli/commands.js:168:16)
at Object. (/opt/homebrew/lib/node_modules/nginx-linter/bin/nginx-linter.js:6:24)
at Module._compile (node:internal/modules/cjs/loader:1120:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1174:10)
at Module.load (node:internal/modules/cjs/loader:998:32)

It would be helpful if the linter could give some hints on what character it's looking for or where in the configuration this is happening.

Most modern linters implement the ability to automatically fix your lint errors. This is no simple task to implement but would be very useful.

These are just suggestions. Feel free to close or disregard if it's invalid.

Currently the linter warns about incorrect indentation levels and trailing spaces.
I've been looking at example configurations from nginx to see if there is some common style guidelines that could be added.

1. One space before the opening brace.

events {
   worker_connections  4096;  ## Default: 1024
}

Warn when it isn't.

events{

2. One space between if parameters.

if ($true) {
    add_header X-First 1;
}

Warn when there are multiple spaces.

if ($true) {
     add_header   X-First   1;
}

3. Comments uses space.

# A comment.

Warn when it itsn't

#A comment.

4. One newline between blocks

  upstream big_server_com {
    server 127.0.0.3:8000 weight=5;
  }

  server { # simple load balancing
    listen          80;
  }

Warn when there are none or multiple:

  upstream big_server_com {
    server 127.0.0.3:8000 weight=5;
  }
  server { # simple load balancing
    listen          80;
  }

  upstream big_server_com {
    server 127.0.0.3:8000 weight=5;
  }



  server { # simple load balancing
    listen          80;
  }

5. location regexes should use space between modifier and regex.

location ~* ^/if-and-alias/

Warn:

location ~*^/if-and-alias/

The example without space currently produces another warning, see #24

A commented line ending with whitespace is not reported when 'trailing-whitespace' is true.

# some explanatory text with whitespace at the end____
worker_processes auto;___

In the example above, underlines ('_') are spaces. The commented line will not be reported, while the uncommented will be.

I'm assuming you ignore commented lines right off the bat, so I'm not sure if anything can be done about this.

Hi,

I just tried using nginx-linter on an existing project and got 6287 errors 🥲.
After setting indentation and trailing-whitespace to warn i get significantly less, but it is hard to find the errors, if there are so many warning messages.

Is it possible to complete disable a certain rule via config?

Hi again, I found Megalinter interesting, although I'm not using it yet. Just wanted to let you know!

oxsecurity/megalinter#891 (comment)

At the moment there is no documentation about the format or the settings of the nginx-linter configuration file.

There is only test/examples/nginx-linter.config.json

https://www.nginx.com/blog/avoiding-top-10-nginx-configuration-mistakes/