Let's implement this rule:

{ 
"request_rate": "1 / 60 / IP",
}

where:

• 1 is the number of requests of the window
• 60 the duration in seconds of the window
• IP the method to distinguish clients. (can be IP or FF (forwarded-for) )

Extra headers can be used to uniquely identify a client - they are separated by +.

For example, if your web services uses browser id assertions, you can do:

{ 
"request_rate": "1 / 60 / IP+Authorization+User-Agent",
}

Notice that since DDos attacks will likely not use valid authorization headers, this
rate limiting has to be used for limiting legit clients.

DDos attacks needs to be prevented using only the remote address

http://blog.cloudflare.com/cloudflares-new-waf-compiling-to-lua

as per https://github.com/mozilla/videur/blob/master/spec/mig_example.json#L71

what about adopting an existing schema validation system ?

I am thinking about http://json-schema.org/examples.html

I'm wondering if we need support for resource inheritance, aka defining validation rules and limits on a top-level resource, and have sub-resource directly inherit them.

Take the list of resources below:

1. /resource
2. /resource/something
3. /resource/users
4. /resource/users/
5. /resource/users/<userid>
6. /resource/users/<userid>?param=somethingelse

Some information, like rate limiting, request size or TLS parameters, can easily be shared at the top-level and inherited by the children. Some other options, like query parameters validation, are probably harder to reuse. But a session parameter will look the same across all resources, and thus would be interesting to define only once.

Another issue is the one of selecting the most specific path when a query comes in. Should we only do exact matching of request path? Or should we try to find the most specific path and use that for validation? I think the later will be required for resources like 5 and 6. The inspection algorithm could parse a resource tree, loading and overloading parameters along the way, and using the most-specific resource.

Or is this just too complicated?

We want ops to be able to do

$ luarocks install videur

And get all the dependencies pulled in the Lua environment

As of January 1 2019, Mozilla requires that all GitHub projects include this CODE_OF_CONDUCT.md file in the project root. The file has two parts:

1. Required Text - All text under the headings Community Participation Guidelines and How to Report, are required, and should not be altered.
2. Optional Text - The Project Specific Etiquette heading provides a space to speak more specifically about ways people can work effectively and inclusively together. Some examples of those can be found on the Firefox Debugger project, and Common Voice. (The optional part is commented out in the raw template file, and will not be visible until you modify and uncomment that part.)

If you have any questions about this file, or Code of Conduct policies and procedures, please see Mozilla-GitHub-Standards or email [email protected].

(Message COC001)

Let's make sure the way we read the body is not impacting performances

see https://groups.google.com/forum/#!topic/openresty-en/qgSjFSSDVrg

@jvehent

we have some parameters like rfc3339date and also some regexp.

It makes it hard to distinguish constants from regexps or other things

I'd like to clarify the syntax, and add a prefix to provide the type when it's not a constant, and also simple matches to avoid writhing regexps

• RFC3339 => must be a matching date value
• regexp:[a-zA-Z0-9]{1,64} => regexp (^ and $ are implicit)
• digits:1,20 => must be a positive number with 1 to 20 digits
• digits:4 => must be a positive number, must be 4 digits
• values: action|command|agent|12 => a discrete list of values

We want to send back a 412 if max_req_size is reached

We discussed inspecting POST body a few times, and so far concluded that it was too difficult and to costly to do, and we should stick to body size validation instead.

Discussing with @psiinon today, he raised a good point: If we add the format of a POST request into the specification, ZAP could use it as a template to do security tests/fuzzing. The cost issue could be solved by enabling/disabling the filtering at the resource level, and them simply turn it off in production.

Example from https://github.com/mozilla/videur/blob/master/spec/mig_example.json#L73-L106

The flag must_validate controls whether validate is enforced or not, and thus can be turned off in production.
The format field indicates that the POST body, and thus validation field, contains valid JSON.
The validation field is then a standardized version of the expected body, with values replaced with placeholders such as [[datetime]] or [[text]] or [[json]].

{
    "service": {
        "resources": {
            "/action/create": {
                "POST": {
                    "format": "json",
                    "must_validate": false,
                    "validation": {
                        "id": "[[float64]]",
                        "name": "[[text]]",
                        "target": "[[text]]",
                        "description": {
                            "author": "[[text]]",
                            "email": "[[email]]",
                            "url": "[[text]]",
                            "revision": "[float64]"
                        },
                        "threat": {
                            "level": "[[text]]",
                            "family": "[[text]]",
                            "type": "[[text]]"
                        },
                        "validfrom": "[[datetime]]",
                        "expireafter": "[[datetime]]",
                        "operations": [
                            {
                                "module": "[[text]]",
                                "parameters": "[[json]]"
                            }
                        ],
                        "pgpsignatures": [
                            "[[text]]"
                        ],
                        "starttime": "[[datetime]]",
                        "finishtime": "[[datetime]]",
                        "lastupdatetime": "[[datetime]]",
                        "counters": "[[json]]",
                        "syntaxversion": "[[float64]]"
                    }
                }
            }
        }
    }
}

datetime could be a simpler notation than RFC3339

can probably get some inspiration from https://github.com/jvehent/haproxy-aws#61automated-rate-limiting

Let's drop our adhoc spec and use Swagger.

@jvehent what do you think?