go-openapi / spec Goto Github PK

This refers to go-openapi/validate#102

When we are using expand with root with some circular references, the resulting resolved $ref unduly refers to the internal reference in cache, starting with pseudo key "root".
This makes it unusable for further analysis.

This does not affect go-swagger and go-openapi/analysis, which expand things using a base path, but may affect users of the go-openapi/validate lib .

The validate package $ref resolution mechanism is outdated and requires to use expansion methods from spec.
These funcs should be exported

It is not possible to initialize parent types that uses these types because they are not exported. I do not see any reason any reason not to export them. Any ideas?

Show a short description or tip for help the user enter correctly the API Key.

This is now supported but not used.

Current:

// APIKeyAuth creates an api key auth security scheme
func APIKeyAuth(fieldName, valueSource) *SecurityScheme {
	return &SecurityScheme{SecuritySchemeProps: SecuritySchemeProps{Type: apiKey, Name: fieldName, In: valueSource}}
}

Maybe is not common to show up a description for API Key authentication, but the current implementation does not allow to use it.

A number of test assertions fail when running tests on master with go1.12 on windows

I am wondering why IsCirclar method in the following line skips the check when basePath is not set?
https://github.com/go-openapi/spec/blob/master/expander.go#L628

func isCircular(ref *Ref, basePath string, parentRefs ...string) bool {
	return basePath != "" && swag.ContainsStringsCI(parentRefs, ref.String())
}

That means, if I load a swagger file with circular definitions from a file using loads.Spec, as in

doc, _ := loads.Spec(filepath)
doc.Expanded()

That will work because the filepath will be used as the above basePath in the circular check.

But if I have a swagger file in memory and load it using loads.Analyzed, as in

doc, _  := loads.Analyzed(rawspec, "")
doc.Expanded()

This will not work as the empty basePath is passed to the above check and the circle will not be detected.

I could avoid this problem by adding the expansion option explicitly as in

doc, _  := loads.Analyzed(rawspec, "")
doc.Expanded(&spec.ExpandOptions{RelativeBase: "/inmemory"})

But if IsCircular did not omit the check even if basePath is empty, I wouldn't need to pass the expansion option.

Could someone tell me the reason for this basePatch nil check?

regards, aki

The file https://github.com/go-openapi/spec/blob/master/debug.test is saved in the git repository, but it looks like an error.

$ file debug.test 
debug.test: Mach-O 64-bit x86_64 executable, flags:<NOUNDEFS>

RelativeBase property of ExpandOptions is used inconsistently through the codebase. In some places, there are checks that it is not empty, in others not.

For example:

if opts != nil {
	specBasePath, _ = absPath(opts.RelativeBase)
}

This code does not check that opts.RelativeBase is empty. It leads to absPath returning current working directory as a specBasePath, which is obviously incorrect.

How I got into this problem: I tried to validate an in-memory spec.

0. My code:

   doc, _ := loads.Analyzed(jsn, "2.0")
   
   // this works fine, but without "nil" it does not. 
   // I think it SHOULD work without nil.
   // And this is the SAME problem I describe in this issue.
   doc, _ = doc.Expanded(nil) 
   
   if err := validate.Spec(doc, strfmt.Default); err != nil {
       // this fails. See steps below.
   	log.Fatalf("failed to validate spec: %v", err) 
   }

1. The validation code internally expands the spec

2. Then this code sets RelativeBase to empty string (note d.specFilePath is empty because my spec is in-memory)

3. Next ExpandSpec sets specBasePath to the current directory

4. Finally validation fails with read <current_working_directory_here>: is a directory.

go-swagger throws error on Windows from this lib.

I can't reveal all the details. Stacktrace follows:

2018/03/02 14:10:37
basePath: C:\Go\my_code\src\bitbucket.org\repoowner\reponame\api\core\root
base: c:\Go\my_code\src\bitbucket.org\repoowner\reponame\api\core\root
2018/03/02 14:10:37 calling Resolve with target: {"$ref":"#/definitions/appSettingsParamsBody"}
2018/03/02 14:10:37 loading schema from url: c:\Go\my_code\src\bitbucket.org\repoowner\reponame\api\core\root
[c:]
c:\Go\my_code\src\bitbucket.org\repoowner\reponame\api\core\root
2018/03/02 14:10:37 getting "c:\\Go\\my_code\\src\\bitbucket.org\\repoowner\\reponame\\api\\core\\root" from resolution cache
2018/03/02 14:10:37 got "c:\\Go\\my_code\\src\\bitbucket.org\\repoowner\\reponame\\api\\core\\root" from resolution cache: false
2018/03/02 14:10:37 fetching document at "c:\\Go\\my_code\\src\\bitbucket.org\\repoowner\\reponame\\api\\core\\root"
base: c:\Go\my_code\src\bitbucket.org\repoowner\reponame\api\core\root
2018/03/02 14:10:37 normalizing #/definitions/appSettingsParamsBody against C:\Go\my_code\src\bitbucket.org\repoowner\reponame\api\core\root (#/definitions/appSettingsParamsBody)
2018/03/02 14:10:37 current ref is: #/definitions/appSettingsParamsBody
2018/03/02 14:10:37 current ref normalized file: c:\Go\my_code\src\bitbucket.org\repoowner\reponame\api\core\root#/definitions/appSettingsParamsBody
2018/03/02 14:10:37 loading schema from url: c:\Go\my_code\src\bitbucket.org\repoowner\reponame\api\core\root#/definitions/appSettingsParamsBody
[c:]
c:\Go\my_code\src\bitbucket.org\repoowner\reponame\api\core\root
2018/03/02 14:10:37 getting "c:\\Go\\my_code\\src\\bitbucket.org\\repoowner\\reponame\\api\\core\\root" from resolution cache
2018/03/02 14:10:37 got "c:\\Go\\my_code\\src\\bitbucket.org\\repoowner\\reponame\\api\\core\\root" from resolution cache: false
2018/03/02 14:10:37 fetching document at "c:\\Go\\my_code\\src\\bitbucket.org\\repoowner\\reponame\\api\\core\\root"
+! Panic in Spec Setup (BeforeEach) [0.151 seconds]
Testing API
C:/Go/my_code/src/bitbucket.org/repoowner/reponame/api/core/core_apps_test.go:73
  app registration method [BeforeEach]
  C:/Go/my_code/src/bitbucket.org/repoowner/reponame/api/core/core_apps_test.go:149
    checks that OK
    C:/Go/my_code/src/bitbucket.org/repoowner/reponame/api/core/core_apps_test.go:166

    Test Panicked
    Invalid schema provided to SchemaValidator: open c:\Go\my_code\src\bitbucket.org\repoowner\reponame\api\core\root: The system cannot find the file specified.
    C:/Go/src/runtime/panic.go:505

    Full Stack Trace

I dive into the lib and found that the problem is the CaSe of the Windows drive letter.

Implemented a fix PR #67

The package has introduced a regression on Windows platform.

• This is no more blocking go-swagger vendor update.

Running go test -v on Windows 10 with go 1.9, I get:

C:\Users\root\go\src\github.com\go-openapi\spec>go test -v -timeout 20m
=== RUN TestSerialization_AuthSerialization
--- PASS: TestSerialization_AuthSerialization (0.00s)
=== RUN TestSerialization_AuthDeserialization
--- PASS: TestSerialization_AuthDeserialization (0.00s)
=== RUN TestIntegrationContactInfo
--- PASS: TestIntegrationContactInfo (0.00s)
=== RUN TestNormalizePaths
--- FAIL: TestNormalizePaths (0.00s)
expander_test.go:73:
Error Trace: expander_test.go:73
Error: Not equal:
expected: "/another/base/path.json#/definitions/Pet"
actual : "\another\base\path.json#\definitions\Pet"
Test: TestNormalizePaths
expander_test.go:73:
Error Trace: expander_test.go:73
Error: Not equal:
expected: "/base/another/base/path.json#/definitions/Pet"
actual : "\base\another\base\path.json#/definitions/Pet"
Test: TestNormalizePaths
=== RUN TestExpandsKnownRef
--- PASS: TestExpandsKnownRef (0.01s)
=== RUN TestExpandResponseSchema
--- PASS: TestExpandResponseSchema (0.02s)
=== RUN TestSpecExpansion
--- PASS: TestSpecExpansion (0.02s)
=== RUN TestResolveRef
--- PASS: TestResolveRef (0.00s)
=== RUN TestResponseExpansion
2018/02/22 16:13:17 {"description":"pet response","schema":{"required":["id","name"],"properties":{"id":{"type":"integer","format":"int64"},"name":{"type":"string"},"tag":{"type":"string"}}}}
2018/02/22 16:13:17 {"description":"pet response","schema":{"required":["id","name"],"properties":{"id":{"type":"integer","format":"int64"},"name":{"type":"string"},"tag":{"type":"string"}}}}
--- PASS: TestResponseExpansion (0.14s)
=== RUN TestExportedResponseExpansion
2018/02/22 16:13:17 {"description":"pet response","schema":{"required":["id","name"],"properties":{"id":{"type":"integer","format":"int64"},"name":{"type":"string"},"tag":{"type":"string"}}}}
2018/02/22 16:13:17 {"description":"pet response","schema":{"required":["id","name"],"properties":{"id":{"type":"integer","format":"int64"},"name":{"type":"string"},"tag":{"type":"string"}}}}
--- PASS: TestExportedResponseExpansion (0.01s)
=== RUN TestIssue3
--- PASS: TestIssue3 (0.02s)
=== RUN TestParameterExpansion
--- PASS: TestParameterExpansion (0.01s)
=== RUN TestExportedParameterExpansion
--- PASS: TestExportedParameterExpansion (0.00s)
=== RUN TestCircularRefsExpansion
panic: test timed out after 20m0s
goroutine 36 [running]:
testing.startAlarm.func1()
D:/Go/src/testing/testing.go:1145 +0x100
created by time.goFunc
D:/Go/src/time/sleep.go:170 +0x4b

goroutine 1 [chan receive, 20 minutes]:
testing.(*T).Run(0xc0423de000, 0x7d94b6, 0x19, 0x7ee658, 0x482e01)
D:/Go/src/testing/testing.go:790 +0x303
testing.runTests.func1(0xc0423de000)
D:/Go/src/testing/testing.go:1004 +0x6b
testing.tRunner(0xc0423de000, 0xc04205dde0)
D:/Go/src/testing/testing.go:746 +0xd7
testing.runTests(0xc04215c7a0, 0x9c2380, 0x3b, 0x3b, 0x0)
D:/Go/src/testing/testing.go:1002 +0x2df
testing.(*M).Run(0xc04212bf18, 0xc04205df70)
D:/Go/src/testing/testing.go:921 +0x118
main.main()
github.com/go-openapi/spec/_test/_testmain.go:160 +0xe2

goroutine 51 [runnable]:
encoding/json.(*decodeState).scanWhile(0xc16c7a65a0, 0x9, 0xc15ec7d54c)
D:/Go/src/encoding/json/decode.go:346 +0xad
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1092 +0x1d3
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x7, 0x7a17ddd)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xa, 0x7a17df8)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a17e06)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x7, 0x7a17e1f)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xa, 0x7a17e3a)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a17e48)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xa, 0x7a17e64)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a17eb4)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x7, 0x7a17ecd)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x9, 0x7a17ed8)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a18051)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x8, 0x7a1806b)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a18584)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x7, 0x7a1859d)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x9, 0x7a185a8)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a18730)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x16, 0x7a18758)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a18766)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a188b8)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x7, 0x7a188d1)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x9, 0x7a188dc)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a18a76)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x16, 0x7a18a9e)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a18aac)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a18c04)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x7, 0x7a18c1d)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x9, 0x7a18c28)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a18dce)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x7, 0x7a18de7)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x9, 0x7a18df2)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a2abb1)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x7, 0x7a2abca)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x9, 0x7a2abd5)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a2ad8a)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x7, 0x7a2ada3)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x9, 0x7a2adae)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a60af6)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x7, 0x7a60b0f)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x9, 0x7a60b1a)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a60cf3)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x16, 0x7a60d1b)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a60d29)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a60e96)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x7, 0x7a60eaf)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x9, 0x7a60eba)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a610b7)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0x16, 0x7a610df)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a610ed)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
encoding/json.(*decodeState).valueInterface(0xc16c7a65a0, 0xc, 0x7a61266)
D:/Go/src/encoding/json/decode.go:1023 +0xd0
encoding/json.(*decodeState).objectInterface(0xc16c7a65a0, 0x9)
D:/Go/src/encoding/json/decode.go:1089 +0x156
...additional frames elided...
created by testing.(*T).Run
D:/Go/src/testing/testing.go:789 +0x2e5
exit status 2
FAIL github.com/go-openapi/spec 1201.971s

C:\Users\root\go\src\github.com\go-openapi\spec>

Let's make things official about this limitation, mentioned in the discussion following #75 (credits @elakito):

• when $ref do form several cycles (i.e. out of the root document)
• when some of those cycles have an intersection

then the expander chose some random expansion path from the common cycle

This behavior is illustrated by this fixture (not run by tests).

First, thank you for go-openapi, you've made our life so much easier!

But we think we've found something incorrect. A user of our documentation tool DapperDox has raised an issue with the handling of description members in referenced definitions, and I've tracked the behaviour down to go-openapi, but need clarification as to whether this is expected behaviour or not.

In case it matters, DapperDox does the following to load a specification:

func loadSpec(url string) (*loads.Document, error) {
    document, err := loads.Spec(url)
    if err != nil {
        return nil, err
    }

    err = spec.ExpandSpec(document.Spec(), nil)
    if err != nil {
        return nil, err
    }

    return document, nil
}

After which, it processes document.Spec() and does it's stuff.

However, consider the following simple OpenAPI specification (note the subtle difference in the reference declaration between first and second, where second wraps the $ref in an "items": { } construct:

{
  "x-navigateMethodsByName": true,
  "definitions": {
    "ItemsTest": {
      "type": "object",
      "title": "ItemsTest",
      "properties": {
        "first": {
          "type": "object",
          "description": "first property",
          "$ref": "#/definitions/MyObject"
        },
        "second": {
          "type": "object",
          "description": "second property",
          "items": {
            "$ref": "#/definitions/MyObject"
          }
        }

      }
    },
    "MyObject": {
      "type": "object",
      "title": "ThisIsTheSubDefinition",
      "properties": {
        "leaf": {
          "description": "leaf",
          "type": "string"
        }
      }
    }
  },
  "info": {
    "title": "minimal"
  },
  "swagger": "2.0",
  "paths": {
    "/items": {
      "get": {
        "summary": "go-openapi test",
        "consumes": [
          "application/json"
        ],
        "produces": [
          "application/json"
        ],
        "responses": {
          "200": {
            "description": "Success",
            "schema": {
              "$ref": "#/definitions/ItemsTest"
            }
          }
        }
      }
    }
  }
}

Having carefully looked at the OpenAPI 2.0 specification (which isn't very explicit in this case) and looked at petstore and a number of other specifications, we have concluded that the items member should not be used when referring to other definitions, only in the case where type is array.

Now, if we dump out document.Spec() and trace through, we find that the property first does not have a Description value (empty):

     Properties: (map[string]spec.Schema) (len=2) {
      (string) (len=5) "first": (spec.Schema) {
       VendorExtensible: (spec.VendorExtensible) {
        Extensions: (spec.Extensions) <nil>
       },
       SchemaProps: (spec.SchemaProps) {
        ID: (string) "",
        Ref: (spec.Ref) ,
        Schema: (spec.SchemaURL) "",
        Description: (string) "",
        Type: (spec.StringOrArray) (len=1 cap=1) {
         (string) (len=6) "object"
        },
        Format: (string) "",
        Title: (string) (len=22) "ThisIsTheSubDefinition",

In fact, the string "second property" does not appear in the dump of document.Spec() at all (it does if we dump out document itself, but the information at that level is not usable).

Conversely, property second does have a correctDescription, and is the expected case (taking the description from the referring property and not the referred to object):

      (string) (len=6) "second": (spec.Schema) {
       VendorExtensible: (spec.VendorExtensible) {
        Extensions: (spec.Extensions) <nil>
       },
       SchemaProps: (spec.SchemaProps) {
        ID: (string) "",
        Ref: (spec.Ref) ,
        Schema: (spec.SchemaURL) "",
        Description: (string) (len=15) "second property",
        Type: (spec.StringOrArray) (len=1 cap=1) {
         (string) (len=6) "object"
        },

I note, however that the structures are different - wrapping $ref in "items":{} in the second case produces a more complicated tree.

If we add a description member to the MyObject definition, then that is the description seen for first instead of blank.

"MyObject": {
      "type": "object",
      "title": "ThisIsTheSubDefinition",
      "description": "Does This Override?",
      "properties": {
        "leaf": {
          "description": "leaf",
          "type": "string"
        }
      }
    }

gives:

     Properties: (map[string]spec.Schema) (len=2) {
      (string) (len=5) "first": (spec.Schema) {
       VendorExtensible: (spec.VendorExtensible) {
        Extensions: (spec.Extensions) <nil>
       },
       SchemaProps: (spec.SchemaProps) {
        ID: (string) "",
        Ref: (spec.Ref) ,
        Schema: (spec.SchemaURL) "",
        Description: (string) (len=19) "Does This Override?",

Essentially, it appears that without the "items":{} wrapper, the referenced definition is directly merging into the referring property, thus overriding Description with an empty field (and probably other members too).

Can you confirm our analysis?

Hopefully we can find a solution soon, as the documentation DapperDox is producing for users is wrong.

Best regards!

I just upgrade some dependencies in my project and I noticed that this package is now requiring github.com/go-openapi/swag which in turn causes many other dependencies to be included in my project.

I expected a specification package to be and stay as minimal as possible. WDYT?

I think this is another regression that was introduced in #38

Consider the following part of the schema:

paths:
  /pet:
    post:
      tags:
      - "pet"
      summary: "Add a new pet to the store"
      operationId: "addPet"
      consumes:
      - "application/json"
      produces:
      - "application/json"
      parameters:
      - in: "body"
        name: "body"
        description: "Pet object that needs to be added to the store"
        required: true
        schema:
          $ref: "#/definitions/Pet"

If basePath is empty (as I described in #44), then this is always true, and the schema refs are never get expanded.

After coming across the same or similar issue to go-swagger #537 I looked into it a little further. If I understand it correctly expander.go has a check for circular references.

pRefs := strings.Join(parentRefs, ",")
pRefs += ","
if strings.Contains(pRefs, currentSchema.Ref.String()+",") {
  break
}

However it seems like it's appending the current schema too late. I tested it with the swagger definition below which has a Book definition with related Book items, and the overall definition structure is Store > Category > Book.

fmt.Println(pRefs) // #/definitions/Store,#/definitions/Category,

fmt.Println(currentSchema.Ref.String()+",") // #/definitions/Book,

It only appears to happen if the circular reference a few levels deep. For instance, if the store has books directly without categories, it's fine.

Swagger Definition

swagger: "2.0"
info:
  title: Swagger Sample
  description: Sample API Playground.
  version: 1.0.0
basePath: /v1
schemes:
- http
consumes:
- application/vdn.sample.v1+json
produces:
- application/vdn.sample.v1+json

paths:
  /books:
    get:
      summary: List all books
      operationId: listBooks
      tags:
        - books
      responses:
        200:
          headers:
            Link:
              type: string
          description: An array of books
          schema:
            type: array
            items:
              $ref: "#/definitions/Book"
        default:
          description: generic error response
          schema:
            $ref: "#/definitions/Error"

definitions:
  # Store
  Store:
    type: object
    properties:
      title:
        type: string
        example: Book Shop
      categories:
        type: array
        items:
          $ref: "#/definitions/Category"

  # Category
  Category:
    type: object
    properties:
      title:
        type: string
        example: Drama
      books:
        type: array
        items:
          $ref: "#/definitions/Book"

  # Book
  Book:
    type: object
    required:
      - title
      - summary
    properties:
      title:
        type: string
        example: Winnie the Pooh
      summary:
        type: string
        example: Famous children's book
      # related_books is causing an infinite loop
      related_books:
        type: array
        items:
          $ref: "#/definitions/Book"


  # Error
  Error:
    type: object
    readOnly: true
    properties:
      code:
        type: integer
        format: int64
        example: 400
      message:
        type: string
        example: Unexpected error
    required:
      - message

When running expand on a spec with a circular $ref, the resulting $ref is left with the full absolute path of the file.

Example: https://github.com/go-swagger/go-swagger/fixtures/canary/bitbucket.org/swagger.json

There are two circular $ref in this spec:

#/definitions/repository
#/definitions/base_commit

If I use expand fromthe swagger expand command, or independently from go-openapi/analysis/flatten(), the resolved circular $ref leaves the full absolute path of the file, instead of just #/definitions/...`

In both case, the RelativeBase expand option parameter is set to the absolute path of the spec file.

This is annoying when in go-openapi/analysis, we try to sort out $ref's which are plain definitions.

Hi,
We've had a user of our OpenAPI documentation tool DapperDox report a Stack Overflow with our latest release. This was built today using master of go-openapi.

From the log, the stack overflow appears to be caused somewhere in go-openapi/spec/expander.go and we assume it may be caused by the size of the specification, as it is quite big, at 84K lines.

He has uploaded his specification and error log giving the stacktrace in a DapperDox issue:
DapperDox Issue - Stack Overflow

To re-cook the binary releases, we have temporarily wound go-openapi/spec back to commit e51c28f07 and go-openapi/analysis back to 7427b3df9, which was stable for us previously.

Are you aware of issues with large specifications?

Cheers!

Go 1.19 treats file paths that have file:/ rather than file:/// differently than previous versions of Go. See https://tip.golang.org/doc/go1.19

This causes TestNormalize to fail with Go 1.19:

--- FAIL: TestNormalizer_NormalizeBase (0.00s)
    --- PASS: TestNormalizer_NormalizeBase/#00 (0.00s)
    --- PASS: TestNormalizer_NormalizeBase/file:///folder//subfolder///file.json/ (0.00s)
    --- PASS: TestNormalizer_NormalizeBase/file:///. (0.00s)
    --- PASS: TestNormalizer_NormalizeBase/file://folder/file.json (0.00s)
    --- PASS: TestNormalizer_NormalizeBase/http://www.anotherexample.com/another/base/path/swagger.json#/definitions/Pet (0.00s)
    --- PASS: TestNormalizer_NormalizeBase/https://user:[email protected]:123/base/sub/file.json (0.00s)
    --- PASS: TestNormalizer_NormalizeBase/base/sub/file.json (0.00s)
    --- PASS: TestNormalizer_NormalizeBase/. (0.00s)
    --- PASS: TestNormalizer_NormalizeBase//base/sub/file.json?query=param (0.00s)
    --- PASS: TestNormalizer_NormalizeBase//base/sub/file.json (0.00s)
    --- PASS: TestNormalizer_NormalizeBase/.. (0.00s)
    --- PASS: TestNormalizer_NormalizeBase/gs://bucket/folder/file.json (0.00s)
    --- FAIL: TestNormalizer_NormalizeBase/file:/. (0.00s)
    --- FAIL: TestNormalizer_NormalizeBase/file:/base/sub/file.json#01 (0.00s)
    --- PASS: TestNormalizer_NormalizeBase/./base/sub/file.json (0.00s)
    --- PASS: TestNormalizer_NormalizeBase/# (0.00s)
    --- PASS: TestNormalizer_NormalizeBase/https:///host/base/sub/file.json?query=param (0.00s)
    --- PASS: TestNormalizer_NormalizeBase//.. (0.00s)
    --- PASS: TestNormalizer_NormalizeBase/\x7f� (0.00s)
    --- PASS: TestNormalizer_NormalizeBase/smb://host (0.00s)
    --- PASS: TestNormalizer_NormalizeBase////folder//subfolder///file.json/ (0.00s)
    --- FAIL: TestNormalizer_NormalizeBase/file:/base/sub/file.json (0.00s)
    --- FAIL: TestNormalizer_NormalizeBase/file:/.. (0.00s)

golang.org/x/text which has a here vulnerability as reported here:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-14040

Short-term and Long-term upgrade fix is available in v0.3.7

Impacted Items:
spec
loads
jsonreference

This is not an exhaustive list , there may be more

Failure with current master when building on Windows 10 OS (go 1.9):
all refs fragments are considered paths. with \ everywhere.
One would expect that only the file path gets \ replaced.

expected: "/another/base/path.json#/definitions/Pet"
actual : "\another\base\path.json#\definitions\Pet" 

Actually expected on windows, in my opinion:

expected: "c:\another\base\path.json#/definitions/Pet"

C:\Users\root\go\src\github.com\go-openapi\spec>go test -v -run Normalize
=== RUN TestNormalizePaths
--- FAIL: TestNormalizePaths (0.00s)
expander_test.go:73:
Error Trace: expander_test.go:73
Error: Not equal:
expected: "/another/base/path.json#/definitions/Pet"
actual : "\another\base\path.json#\definitions\Pet"
Test: TestNormalizePaths
expander_test.go:73:
Error Trace: expander_test.go:73
Error: Not equal:
expected: "/base/another/base/path.json#/definitions/Pet"
actual : "\base\another\base\path.json#/definitions/Pet"
Test: TestNormalizePaths
FAIL
exit status 1
FAIL github.com/go-openapi/spec 0.593s

Hi there.

This might be an unusual "issue" beeing reported.

There has been a security incident in codecov.io with the bash-uploader script (see [1] for details) which potentially exposed secrets to 3rd parties.

It seems you are using the referenced bash uploader in your .travis.yml file. I wanted to draw your attention to this incident in case you missed it.

It would be great if you could verify that no code has been altered and check the impact of this security incident on your repository.

Regards,
Robert

[1] https://about.codecov.io/security-update/

If I look at the SchemaProps definition, I see that the ID is marshalled into json:

type SchemaProps struct {
	ID                   string            `json:"id,omitempty"`
}

Is there a specific reason for that? Because when I marshal the definitions to json, this ID is also added, but according to openapi the id field is not part of the spec and the validation in some validators fails.

Can we change that to json:"-,omitempty" or is there another way of marshalling the spec?

/cc @emicklei @wsong

Hi ! Thanks for this very good work.

I have a question though: why are the keys for the extensions lower-cased at the moment of adding an extension ?

For example, I need to use the tag groups. I build a tag group list, then add it with the key "x-tagGroups".

But when I JSON-marshall my Swagger object, I get a "x-taggroups" tag, which is then not recognised by libraries that can handle this OpenAPI extension.

For now, I have no solution in mind other than forking this repo and using my fork. But I would be pleased to have your answers.

Thanks again

Just starting a test binary, compiled with -race, which depends on this library:

$ hyperfine './xds.test -test.run ^$' -w 1 -r 25
Benchmark #1: ./xds.test -test.run ^$
  Time (mean ± σ):     248.4 ms ±  15.1 ms    [User: 224.9 ms, System: 42.7 ms]
  Range (min … max):   216.8 ms … 276.5 ms    25 runs

Modifying the library so that MustLoadJSONSchemaDraft04 and MustLoadSwagger20Schema do nothing:

$ hyperfine './xds.test -test.run ^$' -w 1 -r 25
Benchmark #1: ./xds.test -test.run ^$
  Time (mean ± σ):     144.2 ms ±  14.0 ms    [User: 112.3 ms, System: 41.2 ms]
  Range (min … max):   123.9 ms … 174.4 ms    25 runs

You can see its almost 2x faster

Currently the library is embedding a json string from go:generate. Maybe it would be more efficient to generate the go struct directly?

Following the study of go-swagger/go-swagger#1429, I found out that expanding a spec with SkipSchemas: true wrongly resolves remote $refs in schemas.

The problem is that when skipping schemas, $refs in parameters and responses are properly resolved,
but if the expanded parameter or response has a schema with a $ref (relative to another root), it is simply resolved "as is" without rebasing this ref to the current root. This results in a schema with unresolvable $ref, since the root is lost.

I added a testcase derived from go-swagger/go-swagger#1429 to illustrate this.

Main spec:
swagger.yaml.txt
Remote file:
responses.yaml.txt

This is especially annoying for spec flattening in go-swagger, since it is relying on the SkipSchema feature.

I think that another SkipRemoteSchemas option should be made available to tune the expand behavior:

• default: SkipRemoteSchemas: false: remote $ref in schemas are expanded, even though SkipSchemas is true. This ensures that an ExpandSpec() with SkipSchemas: true does not leave remote $ref's behind.
• SkipRemoteSchemas: true: remote $ref in schemas are rebased to the current root, but not expanded. This adheres to a true SkipSchemas; true (no expansion at all in schemas)

Are there any plans to support the 3rd version of the OpenAPI spec?

See title.

Hi,
I noticed this problem when generating a go client from one of the odata-openapi samples. The generation gets stuck and looking into this problem revealed that the call to spec.EpandSpec is not coming back. While looking at the recursive iterations that go into expandSchema in expander.go, I am wondering why we are not doing something like isSeen check in addition to isCircular check.

I put a test case that shows this issue at my forked repo.
https://github.com/elakito/spec/blob/issue-expand/expander_test.go#L454
and this is the swagger file used.
https://github.com/elakito/spec/blob/issue-expand/fixtures/expansion/circularSpec2.json

If someone can comment on this issue, that would be appreciated.

regards, aki

Unit test CircularRefsExpansion fails to detect circular ref and hang on Windows,
possibly because of #63.

C:\Users\root\go\src\github.com\go-openapi\spec>go test -v -run CircularRefsExpansion
=== RUN TestCircularRefsExpansion

When looking for circular reference given by non-normalized Ref at

the circular ref check fails to identify that the ref has already been encountered

Latest spec with Go 1.13:

Testing    in: /builddir/build/BUILD/spec-0.19.2/_build/src
         PATH: /builddir/build/BUILD/spec-0.19.2/_build/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin
       GOPATH: /builddir/build/BUILD/spec-0.19.2/_build:/usr/share/gocode
  GO111MODULE: off
      command: go test -buildmode pie -compiler gc -ldflags "-X github.com/go-openapi/spec/version=0.19.2 -extldflags '-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld '"
      testing: github.com/go-openapi/spec
github.com/go-openapi/spec
--- FAIL: Test_ExpandSwaggerSchema (0.03s)
    expander_test.go:667: 
        	Error Trace:	expander_test.go:667
        	            				expander_test.go:665
        	            				expander_test.go:633
        	Error:      	Received unexpected error:
        	            	object has no key "default"
        	Test:       	Test_ExpandSwaggerSchema
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/default"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/description"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/enum"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/exclusiveMaximum"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/exclusiveMinimum"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/default"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/description"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/definitions/stringArray"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/title"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/definitions/positiveInteger"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/definitions/positiveInteger"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/maximum"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/definitions/positiveIntegerDefault0"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/definitions/positiveIntegerDefault0"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/minimum"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/multipleOf"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/pattern"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/default"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/description"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/enum"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/exclusiveMaximum"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/exclusiveMinimum"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/definitions/positiveInteger"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/definitions/positiveInteger"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/definitions/positiveInteger"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/maximum"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/definitions/positiveIntegerDefault0"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/definitions/positiveIntegerDefault0"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/definitions/positiveIntegerDefault0"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/minimum"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/multipleOf"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/pattern"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/definitions/stringArray"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/title"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/type"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/uniqueItems"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/title"
    expander_test.go:639: 
        	Error Trace:	expander_test.go:639
        	Error:      	Should be true
        	Test:       	Test_ExpandSwaggerSchema
        	Messages:   	expected $ref to be inlined, got: "$ref": "http://json-schema.org/draft-04/schema#/properties/uniqueItems"
FAIL
exit status 1
FAIL	github.com/go-openapi/spec	2.815s

json-schema semantics with "id" fail to be expanded properly when references based on "id" define cycle.

This is not much of a problem for swagger specs ("id" is not supported), but this affects go-openapi validate, when it comes to run the json-schema test suite. The jsonschema test suite embeds a testcase with such verification (jsonschema_suite/refRemote.json).

Similarly to the "safe" version of methods I added to analysis with go-openapi/analysis#23,
I propose a similar SafeExpanded() which would really implement the "ContinueOnErrors" option
by calling a callback function and return only an expanded spec (possibly empty).

This would allow go-openapi/validate to fullfil its validation when its own ContinueOnErrors option is set.

Benefit: validate would report the full list of unresolved references instead of just the first found

Any recommandation/thought on that?

While performing some intensive testing on go-openapi/validate I stumbled on test timeout issues...

Profiling the test case with pprof, it came out that about half the validation time is spent in expander.go, even though $ref expansion accounts but for a small amount of the overall test suite.

That's a tremendous-looking ratio, don't you think?

$ go get github.com/go-openapi/[email protected]

go: downloading github.com/go-openapi/spec v0.20.5
go: github.com/go-openapi/[email protected]: verifying module: checksum mismatch
        downloaded: h1:7VP8zufGZEIV+zfSwHGTWgbMINz/WhW3oFsA47Rhjsk=
        sum.golang.org: h1:skHa8av4VnAtJU5zyAUXrrdK/NDiVX8lchbG+BfcdrE=

SECURITY ERROR
This download does NOT match the one reported by the checksum server.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.

It looks like you have might have rewritten the 0.20.5 tag to point to 0.20.6:
https://github.com/go-openapi/spec/tags
v0.20.6 [1005cfb]
v0.20.5 [1005cfb]

This breaks any package using v0.20.5, e.g. github.com/swaggo/[email protected] and all later versions so far

PathItem reference can be used with local definition, but OpenAPI 2.0 spec allows only external references for this object.

For example:

{
  "consumes": [
    "application/json"
  ],
  "produces": [
    "application/json"
  ],
  "swagger": "2.0",
  "info": {
    "description": "Test def with param Ref",
    "title": "ref",
    "version": "0.1.0"
  },
  "basePath": "/data/ref/beta1",
  "paths": {
    "/": {
      "$ref": "#/definitions/pi"
    }
  },
  "definitions": {
    "error": {
      "type": "object",
      "required": [
        "error"
      ],
      "properties": {
        "error": {
          "description": "Error object",
          "type": "object"
        }
      }
    },
    "pi": {
      "get": {
        "responses": {
          "200": {
            "description": "Generic response",
            "schema": {
              "$ref": "#/definitions/resp"
            }
          },
          "default": {
            "description": "Generic error response",
            "schema": {
              "$ref": "#/definitions/error"
            }
          }
        }
      }
    },
    "resp": {
      "type": "array",
      "items": {
        "type": "object"
      }
    }
  },
  "securityDefinitions": {
    "auth": {
      "type": "oauth2",
      "flow": "password",
      "tokenUrl": "/auth/token",
      "scopes": {
        "data.test": "default scope"
      }
    }
  },
  "security": [
    {
      "auth": [
        "data.test"
      ]
    }
  ]
}

This spec will be loaded and expanded but it shouldn't.

Hi,

Our legal department require us to point to Notice file in this repository.
Could you please add simple one.

Best Regards

Line 59: ii ij is string Value

reflect.ValueOf(ii).Int() panic: reflect: call of Int on string Value

Here's my swagger json files.

api_v2.0.json

{
  "swagger": "2.0",
  "info": {
    "version": "2016-01-01",
    "title": "Test",
    "license": {
      "name": "The Apache License (Version 2)",
      "url": "https://www.apache.org/licenses/LICENSE-2.0"
    }
  },
  "paths": {
    "/TestAction0": {
      "$ref": "./action.json#/TestAction0"
    },
    "/TestAction1": {
      "get": {
        "operationId": "TestAction1",
        "summary": "TestAction1",
        "responses": {
          "200": {
            "description": "OK",
            "schema": {
              "type": "object",
              "properties": {
                "action": {
                  "type": "string"
                },
                "ret_code": {
                  "type": "integer"
                },
                "data_set": {
                  "type": "array",
                  "items": {
                    "$ref": "#/definitions/data"
                  }
                }
              }
            }
          }
        }
      }
    }
  },
  "definitions": {
    "data": {
      "type": "object",
      "properties": {
        "data_id": {
          "type": "string"
        },
        "data_name": {
          "type": "string"
        },
        "description": {
          "type": "string"
        }
      }
    }
  }
}

action.json

{
  "TestAction0": {
    "get": {
      "tags": [
        "Action"
      ],
      "operationId": "TestAction0",
      "summary": "TestAction0",
      "responses": {
        "200": {
          "description": "OK",
          "schema": {
            "type": "object",
            "properties": {
              "action": {
                "type": "string"
              },
              "ret_code": {
                "type": "integer"
              },
              "total_count": {
                "type": "integer"
              },
              "data_set": {
                "type": "array",
                "items": {
                  "$ref": "#/definitions/data"
                }
              }
            }
          }
        }
      }
    }
  }
}

I have also tried '"$ref": "http://127.0.0.1:9292/action.json#/TestAction0"', but same error.

From go-swagger/go-swagger#1429 (see test case there)
Test case on $ref for full response
When response $ref points to another file (./responses.yaml#responses/BadRequest) where a new indirection is declared locally (to #/definitions/Error).

Fails to resolve ./responses.yaml/#definitions/Error

Declaring explicitly './responses.yaml#/definitions/Error' in responses.yaml works around the problem.

Any reason Bearer Auth Scheme is not added?
https://swagger.io/docs/specification/authentication/bearer-authentication/

STEPS TO REPRODUCE

Run application from e.g. standard Windows path like "C:\Program Files (x86)\AppName".

ACTUAL RESULT

panic: Invalid schema provided to SchemaValidator: open c:\program files (x86)\AppName\.root: The system cannot find the file specified.
goroutine 6 [running]:
github.com/go-openapi/validate.NewSchemaValidator(0xc000422000, 0xb86960, 0xc0000ca900, 0xc000405d10, 0x4, 0xcfbba0, 0xc00015f1d0, 0x0, 0x0, 0x0, ...)
        AppName/pkg/mod/github.com/go-openapi/[email protected]/schema.go:72 +0x100a
github.com/go-openapi/runtime/middleware.newUntypedParamBinder(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
       AppName/pkg/mod/github.com/go-openapi/[email protected]/middleware/parameter.go:47 +0x198
github.com/go-openapi/runtime/middleware.NewUntypedRequestBinder(0xc0004ab0b0, 0xc0000ca900, 0xcfbba0, 0xc00015f1d0, 0xc0004ab110)
        AppName/pkg/mod/github.com/go-openapi/[email protected]/middleware/request.go:40 +0x14f
github.com/go-openapi/runtime/middleware.(*defaultRouteBuilder).AddRoute(0xc000079c78, 0xbf2c66, 0x4, 0xc00049dee0, 0x6, 0xc0000d5180)
       AppName/pkg/mod/github.com/go-openapi/[email protected]/middleware/router.go:442 +0x600
github.com/go-openapi/runtime/middleware.DefaultRouter(0xc00004bc70, 0xcfdca0, 0xc0003ca4e0, 0xb69580, 0xc0004a4dc0)
        AppName/pkg/mod/github.com/go-openapi/[email protected]/middleware/router.go:141 +0x15b
github.com/go-openapi/runtime/middleware.NewRouter(0xc0004aa900, 0xce90c0, 0xc0004a4dc0, 0xc0004a4dc0, 0xc2e890)
        AppName/pkg/mod/github.com/go-openapi/[email protected]/middleware/router.go:73 +0xe9
github.com/go-openapi/runtime/middleware.(*Context).RoutesHandler(0xc0004aa900, 0xc2da48, 0xc2e860, 0xc2e858)
        AppName/pkg/mod/github.com/go-openapi/[email protected]/middleware/context.go:621 +0x84
github.com/go-openapi/runtime/middleware.(*Context).APIHandler(0xc0004aa900, 0x0, 0x0, 0x0)
        AppName/pkg/mod/github.com/go-openapi/[email protected]/middleware/context.go:612 +0x102
..........
AppName code

EXPECTED RESULT

You can place GO application in paths with parentheses or blanks.

ROOT CAUSE

Parentheses in paths are encoded in schemaLoader cache but they are compared with not encoded paths in the "func (r *schemaLoader) load(refURL *url.URL)". Possible fix is:

func (r *schemaLoader) load(refURL *url.URL) (interface{}, url.URL, bool, error) {
	debugLog("loading schema from url: %s", refURL)
	toFetch := *refURL
	toFetch.Fragment = ""

	var err error
	pth := toFetch.String()
	normalized := normalizeBase(pth)
	debugLog("loading doc from: %s", normalized)

+	unescaped, err := url.PathUnescape(normalized)
+	if err != nil {
+		return nil, url.URL{}, false, err
+	}

+	u := url.URL{Path: unescaped}

-	data, fromCache := r.cache.Get(normalized)
+	data, fromCache := r.cache.Get(u.RequestURI())
	if fromCache {
		return data, toFetch, fromCache, nil
	}

Let's make things official about this limitation:

• whenever $ref form a cycle remotely (i.e. the cycle is out of the root document)

Then the expanded result leaves some $ref items to the remote document with the cycle.
This means the expanded result is not always self-contained.

At this moment, this is not blocking, and libraries such as go-openapi/validate or go-openapi/analysis still work with this.

Hey, I am using golang 1.12.4 and this is how I have defined my structs.

type ApiResponse struct {
    InfoIDs []string                   `json:"information IDs" description:"This list contains Information IDs"`
    Identifier string                  `json:"id" description:"Identifier"`
    MoreInfo map[string][]Information  `json:"moreInfo" description:"More info of some data"`
}

type Information struct {
  InfoA string    `json:"infoA" description:"InfoA value"`
  InfoB string    `json:"infoB" description:"InfoB value"`
  InfoC int       `json:"infoC" description:"InfoC value"`
}

When I generate a spec file, I see the definition as below:

...
"filename.ApiResponse": {
    "required": [
        "information IDs",
         "id",
        "moreInfo"
    ],
    "properties": {
        "information IDs": {
            "description": "This list contains Information IDs",
            "type": "array",
            "items": {
                  "type": "string"
             }
          },
          "id": {
                 "description": "Identifier",
                 "type": "string"
          },
          "moreInfo": {
                "description": "More info of some data",
                "type": "object",
                "additionalProperties": {
                      "$ref": "#/definitions/filename.ApiResponse.moreInfo"
                 }
           }
    }
}
...

However, the correct definition should be like below:

....
"moreInfo": {
   "description": "More info of some data",
   "type": "object",
   "additionalProperties": {
       "type": "array",
       "items": {
           "$ref": "#/definitions/filename.ApiResponse.moreInfo"
       }
   }
}
...

Why is it that map[string][]slice doesn't get generated properly? Am I missing sth here?

Hello

While working to improve the validate package, I noticed some loops in that package calling recursively Ref.GetPointer().Get(s.spec.Spec()) where actually (supposedly) dead code.

I assume that GetPointer().Get() expands the full tree of $refs in the first call, so it shouldn't be needed to call it again.

Could you please confirm this behavior in current master and that it is supposed to be a stable assumption about $ref resolution by the spec package?

If confirmed, it would allow to throw out quite a lot of safeguards in the validate package.

Cheers,

Fred

Per the documentation: https://swagger.io/docs/specification/data-models/dictionaries/. The following two schemas are the same:

case: empty schema

{
  "additionalProperties": {}
}

case: true

{
  "additionalProperties": true
}

Also, from JSON Schema: https://json-schema.org/understanding-json-schema/reference/object.html#additional-properties:

Setting the additionalProperties schema to false means no additional properties will be allowed.

Which means following schema is different than case: empty schema:

case: false

{
  "additionalProperties": false
}

However, both case: empty schema and case: false are now parsed into the same value:

~/tmp/swagger via 🐹 v1.16.5 took 6s 
💤 cat a.json
{
    "definitions": {
        "empty": {
            "additionalProperties": {}
        },
        "false": {
            "additionalProperties": false
        }
    }
}


~/tmp/swagger via 🐹 v1.16.5 
💤 swagger expand a.json
{
  "paths": null,
  "definitions": {
    "empty": {
      "additionalProperties": false
    },
    "false": {
      "additionalProperties": false
    }
  }
}

We are using YAML/Swagger to define our APIs, and react-jsonschema-form to present them in a UI.

I get an expanded/resolved version of the JSON Schema via:

document, err := loads.spec (myYamlPath)
if err == nil {
expanded := document.Expanded()
}

This works great, with one major exception: the order of each object's properties, as defined in the original YAML file, is lost. The problem is in the SchemaProps struct in spec/schema.go:

Properties           map[string]*Schema `json:"properties,omitempty"`

Because this field is a map, Go's range() iterates over the keys (property names) in random order. I understand that this is a design goal of Go, and that it cannot be hacked around in a Go map.

So I have been trying to retain the order of the properties by adding another field to SchemaProps:

ProeprtyNames      []string           `json:"x-UiOrder,omitempty"`

Then wherever I see a value being set in the Properties field, I append its name to my list. I do this in the schema's SetProperty() method, which I changed expandSchema() to use. This results in pretty much what I want in the final JSON document; for example:

              "schema" : {
                 "description" : "An error",
                 **"x-UiOrder" : [
                    "message",
                    "code",
                    "fields"
                 ],**

Unfortunately, these property names are still in random order, due to the range in expandSchema():

for k := range target.Properties {

This is a Go range over a map, and thus it is random. I can't seem to find where target.Properties gets populated here.

Can somebody tell me where I should look to fix this? Or explain to me why my goal is impossible to achieve?

Thanks.

The ExpandSchema() method calls ioutil.TempFile(), which creates a temporary file, and the file is not deleted before program termination. As specified in the golang documentation, it is the caller's responsibility to delete temporary files created by ioutil.TempFile()

As a specific use case, ExpandSchema() is called by "swagger validate" to validate swagger spec files. Every invocation of "swagger validate" may end up creating multiple temporary files in the temp folder (e.g. /tmp for Linux). A workaround would be to delete the /tmp/root* files after execution, but this can be difficult in a CICD environment where many builds are invoked concurrently.

Here is a stack trace of running swagger validate under dlv. A /tmp/root491780128 temporary file is created, and the file is not deleted before program termination.

It looks like there is a missing call to defer os.Remove(file.Name()) in the ExpandSchema() method.

(dlv) bt
 0  0x00000000004c2388 in os.OpenFile
    at /usr/local/go/src/os/file_unix.go:147
 1  0x000000000057445a in io/ioutil.TempFile
    at /usr/local/go/src/io/ioutil/tempfile.go:55
 2  0x0000000000869953 in github.com/go-swagger/go-swagger/vendor/github.com/go-openapi/spec.ExpandSchema
    at ./vendor/github.com/go-openapi/spec/expander.go:534
 3  0x0000000000a8c25b in github.com/go-swagger/go-swagger/vendor/github.com/go-openapi/validate.NewSchemaValidator
    at ./vendor/github.com/go-openapi/validate/schema.go:53
 4  0x0000000000a9271a in github.com/go-swagger/go-swagger/vendor/github.com/go-openapi/validate.(*SpecValidator).Validate
    at ./vendor/github.com/go-openapi/validate/spec.go:103
 5  0x0000000000a92342 in github.com/go-swagger/go-swagger/vendor/github.com/go-openapi/validate.Spec
    at ./vendor/github.com/go-openapi/validate/spec.go:51
 6  0x0000000000ca6e77 in github.com/go-swagger/go-swagger/cmd/swagger/commands.(*ValidateSpec).Execute
    at ./cmd/swagger/commands/validate.go:46
 7  0x00000000009562a7 in github.com/go-swagger/go-swagger/vendor/github.com/jessevdk/go-flags.(*Parser).ParseArgs
    at ./vendor/github.com/jessevdk/go-flags/parser.go:316
 8  0x0000000000955481 in github.com/go-swagger/go-swagger/vendor/github.com/jessevdk/go-flags.(*Parser).Parse
    at ./vendor/github.com/jessevdk/go-flags/parser.go:186
 9  0x0000000000ca9f41 in main.main
    at ./cmd/swagger/swagger.go:125
10  0x00000000004310e4 in runtime.main
    at /usr/local/go/src/runtime/proc.go:195
(dlv) print name
"/tmp/root491780128"

I'm running into an issue where the Ref property on a Schema object is empty after expanding the root document. This makes it extremely hard to find a Schemas parent, or the URI by which that schema was referenced, which is useful for code generation purposes. Am I missing something? perhaps there's a better way to about this.

Here's an example:

ref := schema.Ref.String()
log.Println(ref) // output: "./some-schema.json"

opts := &opi.ExpandOptions{
	RelativeBase: "/tmp/swagger",
}
if err := opi.ExpandSchemaWithBasePath(schema, parent, nil, opts); err != nil {
	panic(fmt.Errorf("Error expanding shema ref %s -> %s", ref, err))
}

ref := schema.Ref.String()
log.Println(ref) // output: ""

This example is somewhat contrived, but in more complex examples, like when this code is called recursively with an already-expanded schema as parent, it's impossible to know the original path from which the nested schema was referenced. Maybe this is intended behavior. If that's the case, what's the recommended approach to preserve these reference URIs?

Most of this lib complexity lies with the resolver and expander features, but these ones bear almost no specifics about swagger specs.

Factoring out this complex but battle-tested feature could simplify the move to OAI 3.

IMHO, spec expansion should be refactored along the following lines:

• moves the core functionality to a separate package (e.g go-openapi/spec/expander)
• make individual spec objects Resolvable and Expandable
• expose additional tooling like normalization, etc which could be factorized with go-openapi/flatten which now overlaps with this logics quite a bit
• further factorization with analysis to single out common $ref functionality used by spec flattening: complete flattening logics (like name resolution, etc) would remain swagger-specific in analysis, but building blocks such as rebasing refs, replacing refs or schemas could be added to the new common set of tools

e.g. something like equipping spec objects with:

type Resolvable interface {
  ResolveWithRoot(root interface{}, opts ...*ExpandOptions, caches ...ResolutionCache) error
  ResolveWithBase(root interface{}, ref Refable, opts ...*ExpandOptions, caches ...ResolutionCache)  error
}

type Expandable interface {
  ExpandWithRoot(root interface{}, ref Refable, opts ...*ExpandOptions, cache ...ResolutionCache) 
  ExpandWithBase(root interface{}, ref Refable, opts ...*ExpandOptions, cache ...ResolutionCache) 
}

We could provide backward compatibility by leaving methods a the package level.

We are using go-openapi for validation of Custom Resources in Kubernetes (kubernetes/kubernetes#47263).

While using the spec.NewRef function, an empty string i.e spec.NewRef("") results in a Ref with a # - not an empty Ref value. Due to this, we incorporated an additional check.

Looking at the code in https://github.com/go-openapi/jsonreference, it looks like an empty Ref should be produced but I couldn't really figure out why that does not occur.

cc @sttts