• update type definition

• update logic

• paths should be relative to parent path defined in the rule

• open question, how to we handle comparing values of paths? since in the below example body and formData are not keys

example:

{
   "lint:formData-or-body": {
      "type": "xor",
      "path": "$..paths.*.*.parameters",
      "enabled": true,
      "description": "example should have either a value or externalValue member",
      "xor": {
          properties: ["orJustThisProperty"],
          paths: ["$.in.body", "$.in.formData"],
      }
   }
}

Should leverage relevant types (like IValidation) from @stoplight/types. Will require some small modifications, and can remove all the duplicate types from Spectral during the process.

We will have a breaking release as it is due to #57 - we can bundle all this together into one major.

Here is an example of OpenAPIv2 schema:

https://github.com/stoplightio/designer/blob/master/src/graph/nodes/oas2/schema.ts

Here is an example of using AJV to validate:

https://github.com/stoplightio/designer/blob/master/src/graph/nodes/oas2/validation-provider.ts#L7-L14

I'm submitting a...

• bug report
• feature request

What is the current behavior?

You will get all errors for everything in whatever ruleset, and there is no way to disable it.

What is the expected behavior?

As a Spectral CLI user, sometimes I like a whole ruleset but a specific rule is really annoying me for some reason. I would like to be able to pass --skip=valid-example and spectral would not even bother running that.

What is the motivation / use case for changing the behavior?

Plenty of folks have pointed out that the default ruleset is a bit more opinionated than they might like, so turning off a few errors temporarily whilst their team catch up on fixing them would help.

OAS2

FormData

• Parameter in: ${value.in} is invalid did you mean in: formData?
• Parameters with type: file must have in: formData
• Operations with parameters of type: file must include multipart/form-data in their consumes property
• Operations with Parameters of in: formData must include application/x-www-form-urlencoded or multipart/form-data in their consumes property
• Parameters cannot have both a in: body and in: formData as formData will be the body

Security

• Security requirements must match a security definition
• Security scope definition ${scope} could not be resolved

Schema

• minimum must be lower value than maximum
• minLength must be lower value than maxLength
• minProperties must be lower value than maxProperties
• Schema type key must be a string
• Read only properties cannot be marked as required by a schema.
• \Z anchors are not allowed in regular expression patterns

Paths

• Empty path parameter declarations are not valid
• Equivalent paths are not allowed.

Parameters

• Path parameters must have required: true. You can always create another path/operation without this parameter to get the same behaviour.
• Multiple body parameters are not allowed.

Operations

• Operations must have unique operationIds.

In Spectral 2.0 the CLI code will autodetect which ruleset to apply from the two rulesets available:

• oas2
• oas3

A new feature is being added #144 which will allow custom rulesets to be required. When we add this new --ruleset= option in 2.1.0 we want to avoid automatically shoving our opinions in, as well as the opinions of the user defined in the ruleset files they are adding in.

For example, if you don't care about adding descriptions to tags, and our ruleset is demanding that you do that, you might get a bit annoyed, especially as we do not have --skip-rule yet. (even when that is added you might not want to have to turn off a bunch of things that you never asked to be turned on).

Issue #144 will convert the existing oas2 and oas3 rulesets to the new declarative YAML format and add the require feature to these rulesets. This means custom rulesets can require the core oas2 or oas3 rulesets if requested.

One downside here is the inability toto load both oas2 and oas3 (schema validation would clash) and lose the autodetect logic. Autodetect will remain by default, so this is not a breaking change, but anyone using --ruleset will lose autodetect functionality, and they will only be able to write custom rulesets that build on top of oas2 or oas3.

If this is a problem (it might not be...) then we need to fix it. One approach might be merging oas2 and oas3 into just "oas", and adding some sort of version based filter for the rules.

Ideas in the comments please!

These are the primary breaking changes that we have identified to get in before public release, if possible.

The goal with these changes is to significantly smooth over the end user DX, and simplify the internals while we're at it.

When all is said and done, almost every rule in #20 should be possible without too much effort.

@casserni need a quick sanity check that making rules enabled by default is OK for our use cases. I still don't quite understand why we made the decision to disable by default, which feels odd.

Changes

0. Update readme examples (Marc has done this).

1. remove:

   • Spectral constructor argument
   • IRuleset
   • IRuleEntry.format

2. update Spectral.run signature

   public run(target: object, opts?: IRunOpts): IRunResult;
   
   // returning an object is better than an array, so that we can add properties later without breaking API
   interface IRunResult {
     results: types.IRuleResult[];
   }

3. simplify IRunOpts

   interface IRunOpts {
     /**
      * The fully-resolved version of the target object.
      *
      * Some functions require this in order to operate.
      */
     resolvedTarget?: object;
   }

4. move spectral class to spectral.ts

5. move the types defined in spectral.ts to the types folder

6. rename IRuleStore to IRuleCollection (or IRuleset now that that name is open?)

7. rename Spectral.setRules to Spectral.addRules, update signature to take IRuleCollection. This does an Object.assign, overwriting existing rules with the same key.

8. rename Spectral.updateRules to Spectral.mergeRules, update signature to take IRuleCollection. This does a deep merge, modifying existing rules with the same key.

9. add Spectral.addFunctions

10. update IRule

    export interface IRule<O = any> {
      // Defaults to RuleType.STYLE
      type?: RuleType;
    
      // A short summary of the rule and its intended purpose
      summary: string;
    
      // The severity of results this rule generates
      severity?: ValidationSeverity;
      severityLabel?: ValidationSeverityLabel;
    
      // A long-form description of the rule formatted in markdown
      description?: string;
    
      // Tags attached to the rule, which can be used for organizational purposes
      tags?: string[];
    
      // set to false to disable the rule (enabled by default)
      enabled?: boolean;
    
      // Filter the target down to a subset[] with a JSON path
      given: string;
    
      when?: {
        // the `path.to.prop` to field, or special `@key` value to target keys for matched `given` object
        // EXAMPLE: if the target object is an oas object and given = `$..responses[*]`, then `@key` would be the response code (200, 400, etc)
        field: string;
    
        // a regex pattern
        pattern: string;
      };
    
      then: {
        // the `path.to.prop` to field, or special `@key` value to target keys for matched `given` object
        // EXAMPLE: if the target object is an oas object and given = `$..responses[*]`, then `@key` would be the response code (200, 400, etc)
        field?: string;
    
        // a regex pattern
        pattern?: string;
    
        // name of the function to run
        function?: string;
    
        // Options passed to the function
        functionOptions?: O;
      };
    }

11. rules should be enabled by default

12. update custom function signature. no more ensureRule needed, functions should return void or an array of result objects that will be transformed into complete result objects by spectral internally.

    export type IFunction<O = any> = (targetValue: any, options: O, paths: IFunctionPaths, otherValues: IFunctionValues) => void | IFunctionResult[];
    
    export interface IFunctionPaths {
      given: Path;
      target: Path;
    }    
    
    export interface IFunctionValues {
      original: any;
      resolved?: any;
      given: any;
    }
    
    export interface IFunctionResult {
      message: string;
      path?: Path; // NOTE: Path should come from @stoplight/types
    }

13. remove Pattern rule. QUESTION: can we? The current function takes options omit and split - are those needed, is there another way to accomplish whatever they do?

14. Final pass through codebase to simplify! The above changes should allow us to simplify a lot of the internals and organization of oas rulesets. The included "rulesets" are now really just a collection of rules and functions that anybody can import and add to their spectral instance via Spectral.addRules and Spectral.addFunctions. See suggested export structure for rulesets in ## Linting an OAS 2 document: section of readme.

It seems like the included rules for OAS2/3 can generate paths that do not exist in the original document.

An easy way to reproduce this is to simply install the default rules and create an OAS document with a path that's missing the description property.

which has a way better description of the validation error than message and title. I discovered this while debugging the piece of software and we should investigate whether it's a leftover and it should be added — or maybe deleted.

Linting this document:

openapi: 3.0.8
info:
  title: occaecat ex exercitation consequat
  version: sunt Duis aliqua qui
x-testcase: 'should not work without required property: paths'

Results in stylish format output:

linting ../openapi3-examples/fail/fuzz1/331be1bf-781d-407f-93d6-1f4b390ae32b.yaml
OpenAPI 3.x detected

/home/mike/node/openapi3-examples/fail/fuzz1/331be1bf-781d-407f-93d6-1f4b390ae32b.yaml
 1:1    error  oas3-schema       Validate structure of OpenAPIv3 specification
 1:1  warning  api-servers       OpenAPI `servers` must be present and non-empty array
 2:6  warning  info-contact      Info object should contain `contact` object
 2:6  warning  info-description  OpenAPI object info `description` must be present and non-empty string

✖ 4 problems (1 error, 3 warnings, 0 infos)

But the json format output gives the more helpful message for the error:

	{
		"code": "oas3-schema",
		"path": [],
		"message": "should have required property 'paths'",
		"severity": 0
	},

Originally posted by @MikeRalphson in #112 (comment)

Make sure path parameters specified in a path are actually included as parameters.

I'm submitting a...

• bug report
• feature request

What is the current behavior?

Running Spectral lint with JSON output and then piping standard output to another program fails because the logging to standard output gets sent along with the JSON output and fails to parse.

For example, imagine an invalid OpenAPI document consisting only of:

openapi: 3.0.1

Running an incredibly simple pipeline like this fails:

$ spectral lint --format=json c.yaml | jq '.[]'                                                    
parse error: Invalid numeric literal at line 1, column 8 

Because stdout looks like this:

Linting c.yaml                                                                        
OpenAPI 3.x detected                                                          
[                                                                                     
        {                                                                             
                "code": "info-contact",   

What is the expected behavior?

Standard output with JSON output should be parseable so that spectral can be used in scripts.

What is the motivation / use case for changing the behavior?

Using Spectral in a script. It is possible to work around this by dumping the JSON to a temp file with -o, but that complicates some scripts.

Please tell us about your environment:

• Version: 2.1.0
• Framework: [ ]
• Language: [all]

> $ spectral --version
@stoplight/spectral/2.1.0 linux-x64 node-v10.12.0

Other information

N/A

/domains/{domainCode}:
  parameters:
    - $ref: "#/components/parameters/domainCode"

The domainCode is defined in components/parameters/domainCode but it complains:

- name: path-params
  summary: Path parameters are correct and valid.
  message: >-
    The path "**/domains/{domainCode}/models**" uses a parameter
    "**{domainCode}**" that does not have a corresponding definition.


    To fix, add a path parameter with the name "**domainCode**".
  path:
    - paths
    - '/domains/{domainCode}/models'
  severity: 50
  severityLabel: error

We will have some separate validation rules that can be used to enable/disable validations for specific exceptions that OAS makes to their JSON schema support. These exceptions are documented in the OASv2 specification.

Change rule structure slightly:

• type -> function
• category -> type
• description -> summary
• add optional description field which is long-form, markdown-formatted text describing the rule and it's functionality
• adding input, which is always an object with the parameters required by the corresponding function
• add an optional tags field, with type string[]

So this:

"some-rule": {
        "category": "style",
        "type": "truthy",
        "path": "$..paths.*.*.parameters",
        "enabled": true,
        "description": "parameter objects should have a description",
        "truthy": "description"
}

To this:

"some-rule": {
        "type": "style",
        "path": "$..paths.*.*.parameters",
        "enabled": true,
        "summary": "parameter objects should have a description",
        "function": "truthy",
        "input": { "property": "description" }
        "tags": [ "parameters" ]
}

rules taken from
sway

• Operations cannot have both a body parameter and a formData parameter | Error
• Operations must have only one body parameter | Error
• Operations must have unique (name + in combination) parameters | Error (also in swagger)
• Operations must have unique operationId | Error
• Path parameters declared in the path string need matching parameter definitions (Either at the path-level or the operation) | Error
• Path parameter declarations do not allow empty names (/path/{} is not valid) | Error
• Path parameters definition (Either at the path-level or the operation) need matching paramater declarations | Error
• Path strings must be (equivalently) different (Example: /pet/{petId} and /pet/{petId2} are equivalently the same and would generate an error) | Error
• Paths must have unique (name + in combination) parameters | Error
• Referenceable definitions should be used by being referenced in the appropriate way | Warning
• References must point to existing documents or document fragments | Error
• The default property for Schema Objects, or schema-like objects (non-body parameters), must validate against the respective JSON Schema | Error
• Circular composition/inheritance for Schema Objects is not allowed (You can have circular references everywhere except in composition/inheritance.) | Error
• The items property for Schema Objects, or schema-like objects (non-body parameters), is required when type is set to array (See OAI/OpenAPI-Specification/issues/174) | Error
• The required properties for a Schema Object must be defined in the object or one of its ancestors | Error

I'm submitting a...

• bug report

What is the current behavior?

When the "operationId" field in Swagger specification is filled with Russian text, Stoplight doesn't mark it as a mistake. However "operationId" is to form the URL address and must contain English words only.

What is the expected behavior?

An error occurred for the "operationId" field with Russian text.

What is the motivation / use case for changing the behavior?

The "operationId" field is also to name the API method. When I wrote Russian text in this field, I couldn't open pages with methods but could open only the "ROOT" page.

We should use the same format for both defining rules as well as toggling. Sample format:

{
  rules: {
    '*': {
      'object-literal-sort-keys': true,
    },
    'oas2|oas3|jschema': {

    },
    oas3: false,
    oas2: {
      'validate:object-literal-sort-keys': true,
      'radix': false,
      'variable-name': false,
      'curly': false,
      'no-console': false,
      'no-empty-interface': false,
      'ban-types': false,
      'max-classes-per-file': [true, 3],
      'no-var-requires': false,
    },
  }
}

Open Questions

• If rules/oas3 is set to false, doesn't that conflict with a union rule containing oas3 (ie, oas2|oas3? No, since all rules should be unique.
• Should we have a single to for rule definitions to ensure all rule names are unique (ie, something like a ruleDefinitions object, and everything else references the names)? Can evaluate at a later date.

Need to add the concept of severity to Spectral rules.

Note: For support questions, please use the Stoplight Community forum. This repository's issues are reserved for feature requests and bug reports. If you are unsure if you are experiencing a bug, the Community forum is a great place to start.

We realize there is a lot of data requested here. We ask only that you do your best to provide as much information as possible so we can better help you.

Feel free to delete this section above before creating the issue.

I'm submitting a...

• bug report
• feature request

What is the current behavior?

Building by cloning the repo is not possible.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem.

What is the expected behavior?

Buildinig by cloning the repo is possible. 😸

What is the motivation / use case for changing the behavior?

Being able to build spectral.

Please tell us about your environment:

• Version: 2.0.0-beta.X
• Framework: [ ]
• Language: ES6/7

Other information

(e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, issues outside of the repo, forum, etc.)

Below is a sample rule set provided by Aspect, who are using this. We should review the rules and identify any areas where our rule format may be insufficient:

{
  "rules": {
     "responses-custom": [
        {
        "rule": "Response definition names should not have an http method in them",
        "whenField": "$key",
        "whenPattern": "^(201|200)$",
        "thenField": "schema.$ref",
        "thenPattern": "^((?!([pP]ost|[gG]et|[pP]atch|[pP]ut|[dD]elete)).)*$"
        },
        {
        "rule": "Responses that are references must be of type object so they can easily be added to in the future",
        "whenField": "type",
        "whenPattern": "^((?!undefined).)*$",
        "thenField": "type",
        "thenPattern": "^object$"
        },
        {
        "rule": "Successful responses that are references must include a kind property and it must be non-empty",
        "whenField": "$key",
        "whenPattern": "^2*$",
        "thenField": "properties.kind",
        "thenPattern": "^(?!undefined).*$"
        },
        {
        "rule": "Non-empty responses must be of type object so they can easily be added to in the future",
        "whenField": "schema.type",
        "whenPattern": "^((?!undefined).)*$",
        "thenField": "schema.type",
        "thenPattern": "^object$"
        },
        {
        "rule": "Non-emtpy responses must include a kind property and it must be non-empty",
        "whenField": "schema.type",
        "whenPattern": "^((?!undefined).)*$",
        "thenField": "schema.properties.kind",
        "thenPattern": "^(?!undefined).*$"
        }
    ],
    "info-custom": [
        {
        "rule": "The info object must have a version property set to a specific value",
        "whenField": "version",
        "whenPattern": ".*",
        "thenField": "version",
        "thenPattern": "^2.0.0$"
        },
        {
        "rule": "License object must exist and contain a specific name value",
        "whenField": "license",
        "whenPattern": ".*",
        "thenField": "license.name",
        "thenPattern": "^License: Creative Commons Attribution 4.0 International Public License$" 
        },
        {
        "rule": "License object must exist and contain a specific url value",
        "whenField": "license",
        "whenPattern": ".*",
        "thenField": "license.url",
        "thenPattern": "^https://creativecommons.org/licenses/by/4.0/legalcode$" 
        },
        {
        "rule": "The info object must exist and contain a specific title property value and no trademark characters",
        "whenField": "title",
        "whenPattern": ".*",
        "thenField": "title",
        "thenPattern": "^Aspect Via .* REST API$"
        }
    ],
    "path-style":
        {
        "rule": "All paths must adhere to camel case.",
        "case": "camel"
        },
    "tags-style": {
        "rule": "All paths must have a tag and the tag name must follow a specific format"
        },
    "properties-style": {
		"rule": "All properties must adhere to camel case.",
		"case": "camel"
		},
    "properties-custom": [
        {
        "rule": "All id properties must be of type string",
        "whenField": "$key",
        "whenPattern": ".*[iI]d$",
        "thenField": "type",
        "thenPattern": "^(string)$"
        },
        {
        "rule": "All time properties must have a format of type date-time specified",
        "whenField": "$key",
        "whenPattern": ".*Time$",
        "thenField": "format",
        "thenPattern": "date-time"
        },
        {
        "rule": "All properties must have a non-empty example",
        "whenField": "type",
        "whenPattern": "^(?!(array|items|object)).*$",
        "thenField": "example",
        "thenPattern": "^(?!undefined).*$" 
        },
        {
        "rule": "All id properties must have a similar description format",
        "whenField": "$key",
        "whenPattern": "^.*[iI]d$",
        "thenField": "description",
        "thenPattern": "^Aspect-generated unique identifier for the .*$" 
        },
        {
        "rule": "Property names cannot be OpenAPI reserved words",
        "whenField": "description",
        "whenPattern": "^(?!.*\\b(deprecated)\\b).*$",
        "thenField": "$key",
        "thenPattern": "^(?!.*\\b(name|names|description|example|default|schema|response|responses|type|summary|parameters|format|items|array|object|string|integer|boolean|enum|required)\\b).*$"
        },
        {
        "rule": "All properties must exist in the API property dictionary",
        "whenField": "description",
        "whenPattern": "^(?!.*\\b(deprecated)\\b).*$",
        "thenField": "$key",
        "thenPattern": "^(acceptParms|account|accountControlItem|accountControlItems|accountControlQueryItems|acdInstanceId|acdInstanceIds|acdInstances|acdLogins|activeGreetingId|activateParms|active|agentFormulaSetIds|agentFormulaSets|allowAssignment|allowLeader|ani|apFormulaId|api|aps|artifacts|attach|attachmentCount|attemptTrackerEvents|bmp|body|callBackTime|callLegId|callType|callbackFlag|calldata|callsBusy|cancelParms|cancelType|channel|channelType|channels|chat|childObjectKeys|code|codes|color|conferenceInteractionId|conferenceParms|connectParms|consultParms|consultType|consultationInteractionId|contactAddress|contactDataCollection|contactDataKey|contactDataValue|contactEmail|contactList|contactListName|contactLists|contactName|contactRecords|contactType|content|contents|continuationToken|createdBy|creationTime|currentSize|customFieldsStringCollection|customInformation|customer|customerId|customers|datasetName|datasets|datasetType|dataType|date|dateRanges|days|defaultDate|defaultDateTime|defaultDuration|defaultFloat|defaultInteger|defaultString|delayStartDuration|deploymentId|detailStatuses|dialParms|dialType|digits|digitsParms|disconnectParms|displayName|disposeParms|dispositionId|dnis|duration|dynamicFilterName|eMailAddress|effectiveHireDate|ehubSessionId|email|emailAddress|emailTemplates|employeeExtraFieldTabs|employeeExtraFields|employeeFilter|employeeGroupClass|employeeGroupClassId|employeeGroupClassIds|employeeGroupClasses|employeeGroupFilter|employeeGroupId|employeeGroupIds|employeeGroupTreeId|employeeGroups|employeeIdentifier|employeeIdentifiers|employeeIds|employeeSelector|employeeSkill|employeeSkillIds|employeeSkills|employeeSuperGroupIds|employeeSuperGroups|employees|enabled|endPattern|endTime|entityName|entityType|error|errors|events|exclusionFilter|exclusionStartTime|exclusionType|executionStatus|exists|expirationTimeForAttemptTracker|externalRouteId|extraFieldFilter|extraFieldId|extraFieldIds|extraFieldTabId|extraFields|fastPathPostResponseCollection|fastPathRecord|fastPathRecordCollection|fastPathRecords|field|fieldKey|fieldValue|fields|filterName|firstName|fontColor|forwardCountAction|forwardProperties|forwardToMailbox|friendlyName|from|geoCode|greetingActive|greetingType|group|handle|hideInactiveEmployees|hireDateFilter|hitRate|holdCount|holdParms|htmlColor|id|ids|image|inactivityDelay|includeAcdLogins|includeAssociatedEmployeeGroups|includeEmployeeGroups|includeExtraFields|includeSkills|includeStates|initialState|initialWorkType|instantMessageAddress|interactionDataCollection|interactionId|interactionType|isAuthenticated|isMasked|jobName|jobs|jobStatus|jobType|key|keyHash|keyName|keyType|kind|label|lastBuildTime|lastExecutionTime|lastModifiedBy|lastModifiedTime|lastName|leader|list|listPenetration|listSaturation|listType|locale|location|logging|login|lookupKey|lookupValue|mailboxLimit|mailboxNumber|maximum|maximumDuration|maximumHoldTime|memo|message|messageCount|messageId|messageParms|messageSize|messageTime|minimum|minimumDuration|model|monitorParms|monitorType|monitoringId|monitoringStationId|multitaskingParms|newMessageCount|nextValidRecallTime|notReadyParms|notReadyType|notes|numberType|objectKeys|operator|orgId|organizationId|packages|parent|parentEmployeeGroupId|parentRecordingId|partialCodes|passcountPenetration|passcountSaturation|payload|penetration|perspectiveIds|perspectives|phone|phoneDialed|phoneNumber|pictureMask|pin|playMessageType|png|precision|preferredName|priority|priorityRange|progressStatus|projects|propertyName|propertyValue|publishedTime|purposeName|pushUrlParms|rank|readMessageCount|readyParms|reason|reasonId|recordFields|recordingId|recordingReasons|records|recordsAttempted|recordsAvailable|recordsInActive|rejectParms|relatedRecordingIds|replaceExisting|requestId|requireWrap|requiresMemo|respectFilterProfile|responseStatus|result|retrieveParms|routeaccessId|rows|rule|salesFlag|saturation|schedule|schemaVersion|scope|screenCaptureFileStatus|segmentDefinitionId|segmentDefinitionIds|segmentDefinitions|segmentFilter|selector|sendTextParms|sender|seniority|sentTime|sequenceNumber|service|services|sessionId|settings|shortName|signatureId|signedDownloadUrl|signedUploadUrl|site|size|skillFilter|skillId|skillIds|skills|sms|sortName|sortOrderType|speedDial|ssn|start|startDate|startDateTime|startTime|state|stateId|states|station|status|statusDescription|stop|stopDate|stopDateTime|stopTime|strategy|stream|subject|switch|systemExternal|systemExternals|takenWorkType|teamId|teamMembers|teamOwners|teamReference|teams|templateType|termDateFilter|terminateParms|terminationDate|text|timeOfAttempt|timeZoneId|timeZones|to|totalHoldTime|totalItems|totalMessagesForwarded|transactionId|transferParms|transferType|umId|unit|url|user|userDefined|userId|users|value|version|view|voiceFileStatus|voicemails|warningParms|warningType|windowsColor|workHandlers|workType|workTypeId|workTypes|wrapRequired)$"
        }
    ],
    "parameters-custom": [
        {
        "rule": "All non-header parameter names must be camel case",
        "whenField": "in",
        "whenPattern": "^(?!.*(header)).*$",
        "thenField": "name",
        "thenPattern": "(x-api-key|Authorization|^[a-z.]+([0-9]|([A-Z.0-9][a-z.0-9]+))*([A-Z.])?$)"
        },
        {
        "rule": "Only headers in the API header dictionary are allowed",
        "whenField": "in",
        "whenPattern": "header",
        "thenField": "name",
        "thenPattern": "^(Authorization|x-api-key|via-client-sessionid)$" 
        },
        {
        "rule": "Authorization header must have specific description",
        "whenField": "name",
        "whenPattern": "Authorization",
        "thenField": "description",
        "thenPattern": "^Authentication token with the value: 'Bearer {accessToken}', where\\n{accessToken} was returned from a call to the authorization endpoint.*$" 
        },
        {
        "rule": "Organization parameter must have specific description",
        "whenField": "name",
        "whenPattern": "organization",
        "thenField": "description",
        "thenPattern": "Name of a customer organization" 
        },
        {
        "rule": "x-api-key parameter must have specific description",
        "whenField": "name",
        "whenPattern": "x-api-key",
        "thenField": "description",
        "thenPattern": "Aspect-provided value used to track API endpoint usage" 
        },
        {
        "rule": "via-client-sessionid parameter must have specific description",
        "whenField": "name",
        "whenPattern": "^via-client-session[iI]d$",
        "thenField": "description",
        "thenPattern": "Identifier representing a long-lived HTTP interchange between Aspect Via® and an entity when the entity is a contact, external client application, or authenticated user. This interchange is called the streaming session. Streaming sessions are used for operations such as exchanging text messages and retrieving events related to user activity." 
        },
        {
        "rule": "Request definition names should not have an http method in them",
        "whenField": "in",
        "whenPattern": "body",
        "thenField": "schema.$ref",
        "thenPattern": "^((?!([pP]ost|[gG]et|[pP]atch|[pP]ut|[dD]elete)).)*$"
        },
        {
        "rule": "maxResults parameter must have specific description",
        "whenField": "name",
        "whenPattern": "maxResults",
        "thenField": "description",
        "thenPattern": "Maximum results. This is used to specify the maximum number of records to return in the result set. You can use this to set a result set limit that is less than the system/API-specific limit." 
        },
        {
        "rule": "All id parameters must be of type string",
        "whenField": "name",
        "whenPattern": ".*[iI]d$",
        "thenField": "type",
        "thenPattern": "string"
        }
    ],
    "no-query-operations": [
        {
        "rule": "Query parameters are not allowed in POST/PUT/PATCH operations",
        "whenField": "$key",
        "whenPattern": "^(post|patch|put)$",
        "thenField": "in",
        "thenPattern": "^((?!(query)).)*$"
        }
    ],
    "operation-response-codes": [
        {
        "rule": "GET response codes must not include update and delete codes",
        "whenHttpMethod": "get",
        "thenResponseCodePattern": "^((?!(409|204|201)).)*$"
        },
        {
        "rule": "DELETE response codes must not include GET/POST/PATCH codes",
        "whenHttpMethod": "delete",
        "thenResponseCodePattern": "^((?!(200|201)).)*$"
        }
    ],
    "operation-response-put": {
        "rule": "The PUT response payload must match the GET response payload"
    },
    "operation-response-post": {
        "rule": "The POST response payload must match the GET response payload"
    },
    "operation-response-patch": {
        "rule": "The PATCH response payload must match the GET response payload"
    }
  }
}

We have a public Docker image available for Spectral available on Docker hub as:

stoplight/spectral

We might want to add that to the installation section with a little snippet, like:

docker run --rm -it stoplight/spectral lint "${URL}"

Spectral's IRuleResult interface which is the type returned from the validation process extends IValidationResult which has a nullable position reference.

While it is correct — Spectral does not have (yet?) the ability of generating the position information — these are up to the caller.

In order to reduce the confusion (which I personally had) it would be better to simply make sure such interface does not have that property at all.

For the first phase of Spectral integration, we'll use a hard-coded set of rules before allowing them to be customizable (or stored inside the project). We need to come up with a set of 'best-practice' rules (nothing too opinionated) that should prove out the concept of the lint rules.

If the rule configuration wants tries to enable a rule that doesn't exist, how should be handled?

• Should we extend the current rule result format for this kind of error?
• Should we do nothing and log a warning?
• Should we throw an error?

Shouldn't be an issue at first, but we might want to come up with a strategy sooner rather than later.

Right now the error messages are generic and not too helpful.

Note: For support questions, please use the Stoplight Community forum. This repository's issues are reserved for feature requests and bug reports. If you are unsure if you are experiencing a bug, the Community forum is a great place to start.

We realize there is a lot of data requested here. We ask only that you do your best to provide as much information as possible so we can better help you.

Feel free to delete this section above before creating the issue.

I'm submitting a...

• bug report
• feature request

What is the current behavior?

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem.

JSONPath is used, which can be complex, confusing, non-cross-platform. and insecure

What is the expected behavior?

Use JMesPath which is going to be the basis of OpenAPI 3.1 Overlays.

What is the motivation / use case for changing the behavior?

Improvements to readability, maintainability, cross-platform support and security.

Please tell us about your environment:

• Version: 2.0.0-beta.X
• Framework: [ ]
• Language: [ES6/7]

Other information

(e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, issues outside of the repo, forum, etc.)

!!!HOLD on this!!!!

• create a new existsIn rule
• options
  • properties to determine whether the key or some property? or do we assume its key
  • in with path to new path and the property

      "swagger:security-requirement-has-definition": {
        "type": "existsIn",
        "path": "$..paths.*.*.security.*",
        "enabled": true,
        "description": "security requirements must match a security definition",
        "existsIn": {
          "properties": "*",
          "in": { 
            "path": "$.securityDefinitions", 
            "properties": "*" 
          }
        }
      },

Running spectral OAS2 rules and functions sets on a OAS2 .yaml file on all tests (2 different yaml files) results in a cryptic error without a "path" parameter:

{
    "name": "oas2-schema",
    "summary": "Validate structure of OpenAPIv2 specification.",
    "message": "should NOT have additional properties: 0",
    "path": [],
    "severity": 50,
    "severityLabel": "error"
}

Those files don't throw any errors in any other validation tool (swagger editor, apiary, etc.). I am happy to provide them if needed.

I'm submitting a...

• bug report
• feature request

What is the current behavior?

Running:

spectral.addFunctions(oas2Functions());
spectral.addRules(oas2Rules());
const results = spectral.run(myOAS);

on a OAS2 .yaml file

What is the expected behavior?

Do not throw this error or be more specific on it's origin and place in the file.

What is the motivation / use case for changing the behavior?

User experience and compliance with OAS2 specification

Please tell us about your environment:

• Version: @stoplight/[email protected]
• Framework: [ ]
• Language: [all]

Other information

It also throws

oasOpParams expects a resolved object, but none was provided. Results may not be correct.
oasPathParam expects a resolved object, but none was provided. Results may not be correct.

which does not say anything of value without more explanation.

I'm submitting a...

• bug report
• feature request

What is the current behavior?

Given a following repro/two.yaml file

swagger: '2.0'

info:
  title: Repro two
  description: Endpoint definition
  version: "1.0.0"
  contact:
    name: toto
host: not-example.com
schemes:
  - https
basePath: /repro/two

paths:
  /quotes_requests:
    post:
      summary: Gets nothing either.
      operationId: "12"
      description: Cf. summary
      tags: [yep]
      parameters:
        - name: body
          description: Content.
          in: body
          schema:
            $ref: '#/definitions/AnotherDefinition'
      responses:
        204:
          description: Zip

definitions:
  AnotherDefinition:
    type: object
    properties:
      special:
        description:

linting from the cli returns a error.

$ yarn spectral lint ./repro/two.yaml
yarn run v1.15.2
$ C:\REDACTED\node_modules\.bin\spectral lint ./repro/two.yaml
linting ./repro/two.yaml
OpenAPI 2.0 (Swagger) detected
Encountered error when running rule 'oas2-schema' on node at path '$':
TypeError: Cannot read property 'parent' of null
No errors or warnings found!
Done in 1.01s.

Note: Providing a valued description in the very last node makes the issue disappear.

What is the expected behavior?

Either:

• Remove the error message if it's not supposed to happen
• Provide a more user-friendly/actionable error message (if another error is blocked from popping up by this one)

What is the motivation / use case for changing the behavior?

N/A

Please tell us about your environment:

• Version: @stoplight/spectral/2.0.6
• Framework: win32-x64 node-v10.14.2
• Language: all

With the changes for #21, all rule/function inputs are now under an inputs object, however now some of the naming is inconsistent and redundant. We should review each rule and fix values to be more sane.

For example:

"input": {
  "truthy": "tags"
}

Should be converted to:

"input": {
  "property": "tags"
}

Right now the lint results are pretty specific to lint issues. We should have a more generic result type that allows for both validation and lint results (and that allows consumers to differentiate between the two).

This change will most likely impact #4.

Request for feedback!

We would love to hear from users what syntax you would like to used -- please comment with your feedback!

I'm submitting a...

feature request

What is the current behavior?

Currently, the CLI detects if you are using a OpenAPI 2 or 3 based document and uses the default rulesets. No specifications or rulesets outside of that works with it.

What is the expected behavior?

You have the ability to use any custom ruleset when using the Spectral CLI.

One potential syntax could be:

spectral lint foo.yaml --ruleset=whatever.json --ruleset=advanced.js

What is the motivation / use case for changing the behavior?

One of the things that makes Spectral so great is the ability to build out custom rulesets, which can already be done now, but not in the CLI.

Other information

We are also exploring adding a ESLint or TSLint like config file (i.e. .spectralrc) with all the plugins/rules/presets setup. Current plan is that this would come after adding the custom ruleset option to the CLI.

Right now we just verify that the default rules return errors on invalid specs, but it would be good to expand the test checks to verify which tests are failing on the specific paths.

Requires #5

Same as #12, but for OASv3.

In addition to accepting the un-resolved objects, we should be able to pass in fully-resolved and stringified objects.

Overview

The purpose of Phase 1 is to be an MVP of integrating Spectral into the Stoplight UI. This will include:

• Add JSON schema validation (required, discussed below) #1
• A common rule-set that applies to all projects (not extendable), where specific rules can be enabled/disabled through UI. Which rules are enabled/disabled is tied to the session, so it won't live in the project. #5
• Functionality for running rules within the context of the UI (ie, replace sway with Spectral).
• Functionality for returning rule results and categorizing based on severity and lint/validation. Also interpreting the Spectral result format into results needed by UI.

Validation

• JSON schema validation (tied to the current file-type, ie oas2, oas3), with a valid/not-valid response
• Items marked as invalid are errors (not warnings)
• Should not be able to disable validation

Linting

• Should be able to be enabled/disabled #2
• Be able to mark certain rule severity (error vs warning) #6
• [future] Should be able to customize/extend functionality
• [future] Should be able to store rules in project

Open Questions

• How to differentiate which rules apply to which file types? Should we add it it to the rule object, or should there be some other method used (ie, something similar to tslint, where you can group rule names by file type but disconnected from rule definition)?
• Should the default rule set mimic the current sway warning rules?
• How to define validation vs lint rules? Should validations even be customizable?
• How to return validation vs lint results?
• Should validations just be a rule type (similar to the other lint rules), or should we have a separate rule format for validations?

Right now they will be hard-coded imports, but we should investigate making them dynamic based on the incoming rule configuration.

Combination of rules taken from sway, swaggerHub, and in-house

API Info

• API title must be present and non-empty string~~ (included in spec schema)
• API description must be present and non-empty string
• API license must be present and non-empty string
• API contact must be present and non-empty string

Paths

• Query strings in paths are not allowed.
• Path parameter declarations do not allow empty names (/path/{} is not valid)
• (custom) Path parameter definition need a matching parameter declaration
• (custom) Path parameter declaration need a matching parameter definition
• (custom) Path strings must be (equivalently) different (Example: /pet/{petId} and /pet/{petId2} are the same)
• (custom/sibling?) Path parameters must have required: true

Operations

• operationId must be present and non-empty string
• (custom/unique?)operationId values must be unique within the API definition
• Operation summary must be present and non-empty string
• summary should start with upper case and end with a dot
• Operation description must be present and non-empty string
• Operation must have a tag and non-empty string
• Operation must have one and only one tag
• (custom/atLeast?) Operation must have at least one 2xx response
• Operation must have a default response
• (xor with path extension) Operation cannot have both a in:body and a in:formData` parameter
• (custom/max?) Operation must have only one in:body parameter
• (custom/unique?) Operations must have unique (name + in combination) parameters
• (custom/includes?) Operations with Parameters of in: formData must include application/x-www-form-urlencoded or multipart/form-data in their consumes property

Parameters

• (custom/sibling/includes?) Default values must be present in enum
• (skip since we don't allow type file in the ui) Parameters with type: file must have in: formData

Models

• skip (custom/truthy? but how nested?) All model properties must have examples
• API must not have local definitions (i.e. only $refs are allowed)
• skip (custom) Combine models when practical
• Every model should have a description
• skip (custom) List all required properties before any optional properties
• (custom) Declared definition was not used in the document

Security

• Security requirements must match a security definition
• (covered by resolver) Security scope definition ${scope} could not be resolved

References

• (xor?) Sibling values are not allowed alongside $refs
• (resolver) References must point to existing documents or document fragments
• $ref paths must begin with #/
• [covered by resolver] $refs must reference a valid location in the document
• (pattern) $ref values must be RFC3986-compliant percent-encoded URIs

Schemas

• Schemas with type: array require a sibling items: field
• items must be an object
• The required properties for a Schema Object must be defined in the object or one of its ancestors
• [PUNT] minimum must be lower value than maximum
• [PUNT] minLength must be lower value than maxLength
• [PUNT] minProperties must be lower value than maxProperties
• Schema type key must be a string
• Read only properties cannot be marked as required by a schema.
• \Z anchors are not allowed in regular expression patterns

Naming strategy

• (custom/pattern?) Enforce naming strategy for paths, parameters and property names: snake_lower | Snake_Upper | camelLower | CamelUpper
• (custom/pattern?) Avoid using reserved words for model/property names (e.g. error, return, type, input) for paths, parameters and property names

I'm submitting a bug report

What is the current behavior?

I am currently migrating my API specification and running into this warning:

oasOpParams expects a resolved object, but none was provided. Results may not be correct.
oasPathParam expects a resolved object, but none was provided. Results may not be correct.

There are no $ref mentions in my spec, so I am confused as to why I am getting this error. The code is as follows.

What is the expected behavior?

For this not to error

Please tell us about your environment:

• Version: @stoplight/spectral": "^1.0.1"
• Framework: Node v11.7.0
• Language: Node v11.7.0

Other information

My code:

// Load Spectral and its OAS3 validations
const { Spectral } = require('@stoplight/spectral')
const { oas3Functions, oas3Rules } = require('@stoplight/spectral/rulesets/oas3')

// Find all OpenAPI files
const fs = require('fs')
const glob = require('glob')

const lint = async () => {
  const fileNames = await new Promise((resolve, reject) => {
    glob('./v2.0/*.openapi.json', {}, (error, match) => {
      if (error) { reject(error) }
      else { resolve(match) }
    })
  })
  
  // Initialize Spectral
  const spectral = new Spectral()
  spectral.addFunctions(oas3Functions())
  spectral.addRules(oas3Rules())
  
  fileNames.forEach(fileName => {
    let specification = JSON.parse(fs.readFileSync(fileName))
    const results = spectral.run(specification)
    // console.dir(results)
  })  
}

lint()

Verify contents of the document being parsed are valid JSON (as defined by the RFC), and flag anything that doesn't conform.

For example, if a JSON string contains un-escaped control characters, flag the string.

I'm submitting a...

• bug report
• feature request

What is the current behavior?

Given two files:

• repro/nested/one.yaml

swagger: '2.0'

info:
  title: Repro one
  description: Endpoint definition
  version: "1.0.0"
  contact:
    name: toto
host: not-example.com
schemes:
  - https
basePath: /repro/nested/one

paths:
  /test:
    post:
      summary: Gets nothing
      operationId: "17"
      tags: [ one ]
      description: Cf. summary
      parameters:
        - name: body
          description: Content
          in: body
          schema:
            $ref: '#/definitions/TheDefinition'
          required: true
      responses:
        204:
          description: No content

definitions:
  TheDefinition:
    type: object
    properties:
      email:
        description: Email address.
        example: "[email protected]"
        $ref: '../library.yaml#/definitions/email'

• repro/library.yaml

definitions:
  email:
    type: string

linting from the cli returns an error

$ yarn spectral lint ./repro/nested/one.yaml
yarn run v1.15.2
$ C:\REDACTED\node_modules\.bin\spectral lint ./repro/nested/one.yaml
linting ./repro/nested/one.yaml
OpenAPI 2.0 (Swagger) detected
Encountered error when running rule 'valid-example' on node at path '$,definitions,TheDefinition,properties,email':
Error: can't resolve reference ../library.yaml#/definitions/email from id #
No errors or warnings found!
Done in 1.18s.

What is the expected behavior?

Teach valid-example rule to honor relative remote references

What is the motivation / use case for changing the behavior?

N/A

Please tell us about your environment:

• Version: @stoplight/spectral/2.0.6
• Framework: win32-x64 node-v10.14.2
• Language: all

From what I can find online (and my experience in general), jmespath is generally regarded as more powerful than jsonpath.

• While development is not active, it has a much larger install base (https://www.npmjs.com/package/jsonpath versus https://www.npmjs.com/package/jmespath).
• It also happens to be less than 20% the size (https://bundlephobia.com/[email protected] versus https://bundlephobia.com/[email protected]). jsonpath is surprisingly pretty much the same size as AJV (large!).

http://jmespath.org

@chris-miaskowski, I think this is a relatively simple change, but would be breaking - if it is simple, we could consider bundling it with the two other breaking changes we're working on (#57 and #58).

Currently we are importing the rules from spectral, in platform, and this is no good

• create a get rules function to return rules
  • should be a flattened array with rule attributes, name, and format
• allow to filter by format
  • basic functionality is there but we may want to rethink how we are currently matching on format both here and during apply
• add tests

Right now disabled rules are pruned when registering them. We want to be able to register rules and then check enabled/disabled at run-time.

I'm submitting a...

• bug report

What is the current behavior?

The linter warning path-keys-no-trailing-slash is triggered on a path key of /which is incorrect as all path keys MUST start with a / character.

src/cli/bin/run lint .minimal.yaml

where minimal.yaml is:

openapi: 3.0.0
info:
  title: ''
  version: ''
paths:
  /: {}

Results:

 ›   Warning: { name: 'path-keys-no-trailing-slash',
 ›      summary: 'given keys should not end with a slash.',
 ›      message: 'must not match the pattern \'/$\'',
 ›      path: [ 'paths', '/' ],
 ›      severity: 40,
 ›      severityLabel: 'warn' }

What is the expected behavior?

This warning is not triggered.

What is the motivation / use case for changing the behavior?

Eliminating spurious warnings.

Please tell us about your environment:

• Version: 0.0.0
• Language: [TypeScript 3.2.2]

Other information

oas-kit (the source of the default rules in Speccy, and hence the inspiration for a number of rules in spectral) uses an omit parameter to the pattern rule to chomp off the leading / character before comparison. Doubtless this could be done within the regex by someone with the appropriate skills.

Should we add a force property to rule definitions? What this means is that if force: true, the rule toggle will be disabled in the ui more importantly spectral would apply that rule regardless of the configuration file.

Use Cases

• An organization might have a set of presets, or a subset of our own presets that they want to apply to every project without allowing their developers to disable.
• Schema validation is technically a rule, so without this it could potentially be disabled in the ui or through the raw configuration file

Current Thoughts

• since in phase one of spectral we are not displaying the code view, we can blacklist certain rules to not show in the ui and then they can't get disabled that way
  • issue with this is that a more advanced user can still tinker with the configuration file through git
  • I also want to avoid "hiding" the rules from users, as it might create a confusion around what is being applied during linting/validation
• allow users to disable/enable what they want, if for whatever reason they wanted to disable oas2 schema validation entirely they should be allowed to
  • in the case of an organization, if a developer disables a rule enabled by their organization preset, we can flash a little message or have an extra confirm box to inform them that they are doing something they probably shouldn't be
  • along with the above, maybe instead of a force property we include an important: true which does the above confirm box flow

...Discuss

I'm submitting a...

feature request

What is the expected behavior?

Have an example that clearly shows how to convert from YAML to JSON using https://github.com/stoplightio/yaml

What is the motivation / use case for changing the behavior?

Make it easier to use specification files that are in YAML with Spectral

I'm submitting a...

• bug report
• feature request

What is the current behavior?

The default behavior of Spectral v2.0 is to operate only on the original description object for 95% of the rules, and the uses the resolvedTarget in a few rules which specifically request that resolved version of the object (like oasOpsParams).

Spectral CLI uses json-ref-parser to created that resolvedTarget, and by default json-ref-parser only processes internal $ref. This is a safe default, because $ref can be potentially used maliciously if you are working with description files you don't control.

This is unexpected for people transitioning from Speccy, which did external by default. See #108.

What is the expected behavior?

Maybe we could have a new CLI option for the lint command, maybe --external.

The main Spectral constructor takes an object with a resolver key, which defaults to a stock standard json-ref-parser instance. The lint command should take advantage of this and change the resolver based on CLI options.

if (flags.external) {
    const spectral = new Spectral(resolver: <fancy-resolver-however-that-works>);
}

Or something like that.

Thoughts @tbarn @marbemac @MikeRalphson?