examples files about rest-api-fuzz-testing HOT 10 OPEN

You can pass the folder with examples to docker container by mounting it. See this example:

from rest-api-fuzz-testing.

that example shows how to pass compile output as input to later RESTler stages. How do you tell the compiler to use the file share mount that you set up in the config? If I only mount a share as shown in the example, I still get the error.

from rest-api-fuzz-testing.

you can mount any file share. The example passes a result of compile step to run.

But you can create an example share same way and pass it to compile step.

see this example:
https://github.com/microsoft/rest-api-fuzz-testing/blob/main/cli/samples/restler/replay-a-bug/replay-common-file-share.json

this example mounts a file share to replay a RESTler bug.

So you can mount any file share and pass it to RESTler

from rest-api-fuzz-testing.

I have this configuration:

{
"readOnlyFileShareMounts": [
{
"fileShareName": "specifications",
"mountPath": "/specifications"
},
{
"fileShareName": "examples",
"mountPath": "/examples"
}
],...
"tasks": [
{
"toolName": "RESTler",
"outputFolder": "restler",
"keyVaultSecrets": [ "TOKEN" ],
"authenticationMethod": {
"Token": "Token"
},
"targetConfiguration": {
"apiSpecifications": [
"http://host.docker.internal:8000/openapi_cams_only.yaml"
],
"endpoint": "http://host.docker.internal/managementserver/rest"
},
"toolConfiguration": {
"tasks": [
{
"task": "Compile",
"runConfiguration": {
"inputFolderPath": "/examples"
},...

but I still get the same error - does the swagger compile step know to look in the share mount or do I need to do something else in my configuration?

from rest-api-fuzz-testing.

you need to change your Compile task and ExamplesDirectory there:

        /// The directory where the compiler should look for examples.
        /// If 'discoverExamples' is true, this directory will contain the
        /// example files that have been discovered.
        /// If 'discoverExamples' is false, every time an example is used in the
        /// Swagger file, RESTler will first look for it in this directory.
        ExamplesDirectory : string

"tasks": [
  {
    "toolName": "RESTler",
    "outputFolder": "compile",
    "toolConfiguration": {
      "task": "Compile",
    "compileConfiguration" : {
     "exampleConfigFilePath " : "/examples"
    }
    }
  }
]

Here is a good example that uses compile configuration section 👍
https://github.com/microsoft/rest-api-fuzz-testing/blob/main/cli/samples/restler/running-against-raft/compile.json

from rest-api-fuzz-testing.

I added this to my configuration:

"compileConfiguration":
{
"useRefreshableToken": true,
"discoverExamples": false,
"useBodyExamples": true,
"useQueryExamples": true,
"examplesDirectory": "Examples",
"exampleConfigFilePath": "/examples"
}

and I changed the reference in my yaml file to:

examples:
camera:
$ref: '../examples/examples.json'

now I get a different error so i assume that worked:

Unhandled exception. System.AggregateException: One or more errors occurred. (Unable to cast object of type 'NJsonSchema.JsonSchema' to type 'NSwag.OpenApiExample'.)
---> System.InvalidCastException: Unable to cast object of type 'NJsonSchema.JsonSchema' to type 'NSwag.OpenApiExample'.
at NJsonSchema.References.JsonReferenceBase1.NJsonSchema.References.IJsonReferenceBase.set_Reference(IJsonReference value) at NJsonSchema.JsonSchemaReferenceUtilities.JsonReferenceUpdater.VisitJsonReferenceAsync(IJsonReference reference, String path, String typeNameHint) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action`1 replacer)

my examples.json file looks like this:

{
"parameters": {
"id": "09b11ea3-085c-4641-a84a-ce2f68a09a53"
}
}

from rest-api-fuzz-testing.

Where do you get the failure from ? is it happening in RAFT or RESTler ?

from rest-api-fuzz-testing.

It's coming from RESTler- here's the full stack trace:

Unhandled exception. System.AggregateException: One or more errors occurred. (Unable to cast object of type 'NJsonSchema.JsonSchema' to type 'NSwag.OpenApiExample'.)
---> System.InvalidCastException: Unable to cast object of type 'NJsonSchema.JsonSchema' to type 'NSwag.OpenApiExample'.
at NJsonSchema.References.JsonReferenceBase1.NJsonSchema.References.IJsonReferenceBase.set_Reference(IJsonReference value) at NJsonSchema.JsonSchemaReferenceUtilities.JsonReferenceUpdater.VisitJsonReferenceAsync(IJsonReference reference, String path, String typeNameHint) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj, String path, String typeNameHint, ISet1 checkedObjects, Action1 replacer) at NJsonSchema.Visitors.AsyncJsonReferenceVisitorBase.VisitAsync(Object obj) at NJsonSchema.JsonSchemaReferenceUtilities.JsonReferenceUpdater.VisitAsync(Object obj) at NJsonSchema.JsonSchemaReferenceUtilities.UpdateSchemaReferencesAsync(Object rootObject, JsonReferenceResolver referenceResolver, IContractResolver contractResolver) at NJsonSchema.Infrastructure.JsonSchemaSerialization.FromJsonAsync[T](String json, SchemaType schemaType, String documentPath, Func2 referenceResolverFactory, IContractResolver contractResolver)
at NSwag.OpenApiDocument.FromJsonAsync(String data, String documentPath, SchemaType expectedSchemaType, Func2 referenceResolverFactory) at NSwag.OpenApiDocument.FromFileAsync(String filePath) --- End of inner exception stack trace --- at Microsoft.FSharp.Control.AsyncResult1.Commit() in F:\workspace_work\1\s\src\fsharp\FSharp.Core\async.fs:line 349
at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInCurrentThread[a](CancellationToken cancellationToken, FSharpAsync1 computation) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\async.fs:line 882 at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync1 computation, FSharpOption1 timeout) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\async.fs:line 890 at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync1 computation, FSharpOption1 timeout, FSharpOption1 cancellationToken) in F:\workspace_work\1\s\src\fsharp\FSharp.Core\async.fs:line 1154
at [email protected](String fp) in D:\a\1\s\src\compiler\Restler.Compiler\Workflow.fs:line 88
at Microsoft.FSharp.Primitives.Basics.List.map[T,TResult](FSharpFunc2 mapping, FSharpList1 x) in F:\workspace_work\1\s\src\fsharp\FSharp.Core\local.fs:line 248
at Microsoft.FSharp.Collections.ListModule.Map[T,TResult](FSharpFunc2 mapping, FSharpList1 list) in F:\workspace_work\1\s\src\fsharp\FSharp.Core\list.fs:line 75
at Restler.Workflow.generateGrammarFromSwagger(String grammarOutputDirectoryPath, FSharpOption1 swaggerDoc, Config config) in D:\a\1\s\src\compiler\Restler.Compiler\Workflow.fs:line 88 at Restler.Workflow.generateRestlerGrammar(FSharpOption1 swaggerDoc, Config config) in D:\a\1\s\src\compiler\Restler.Compiler\Workflow.fs:line 247
at Program.main(String[] argv) in D:\a\1\s\src\compiler\Restler.CompilerExe\Program.fs:line 37

from rest-api-fuzz-testing.

@marina-p
Does this crash look like an obvious fix in user configs ? Or RESTler update required ?

from rest-api-fuzz-testing.

@LYDE1234 If you are referencing the example from the specification, and they are in the same directory (or a sub-directory with relative path from the specification), you do not need the ExampleConfigFilePath or an example config file. If that works for you, just omit the above file path from config.json, put the example in the same directory as the spec, and hopefully the issue will be resolved.

I believe the reason for your error is you are referencing the example config file in your specification yaml. The specification should only reference actual examples (like the one you provided which contains "id"). The example config file should only be referenced in config.json, and the example config file contains a list of request types, which point to one or more examples.

See https://github.com/microsoft/restler-fuzzer/blob/main/docs/user-guide/Examples.md for more details.

from rest-api-fuzz-testing.