microsoft / go-crypto-openssl Goto Github PK

Go crypto backend for Linux using OpenSSL

License: MIT License

Go 80.08% C 18.97% Shell 0.85% Makefile 0.10%

go-crypto-openssl's Introduction

go-crypto-openssl

The openssl package implements Go crypto primitives using OpenSSL shared libraries and cgo. When configured correctly, OpenSSL can be executed in FIPS mode, making the openssl package FIPS compliant.

The openssl package is designed to be used as a drop-in replacement for the boring package in order to facilitate integrating openssl inside a forked Go toolchain.

Disclaimer

A program directly or indirectly using this package in FIPS mode can claim it is using a FIPS-certified cryptographic module (OpenSSL), but it can't claim the program as a whole is FIPS certified without passing the certification process, nor claim it is FIPS compliant without ensuring all crypto APIs and workflows are implemented in a FIPS-compliant manner.

Background

FIPS 140-2 is a U.S. government computer security standard used to approve cryptographic modules. FIPS compliance may come up when working with U.S. government and other regulated industries.

Go FIPS compliance

The Go crypto package is not FIPS certified, and the Go team has stated that it won't be, e.g. in golang/go/issues/21734 Adam Langley says:

The status of FIPS 140 for Go itself remains "no plans, basically zero chance".

On the other hand, Google maintains a branch that uses cgo and BoringSSL to implement various crypto primitives: https://github.com/golang/go/blob/dev.boringcrypto/README.boringcrypto.md. As BoringSSL is FIPS 140-2 certified, an application using that branch is more likely to be FIPS 140-2 compliant, yet Google does not provide any liability about the suitability of this code in relation to the FIPS 140-2 standard.

Features

Multiple OpenSSL versions supported

The openssl package has support for multiple OpenSSL versions, namely 1.0.2, 1.1.0, 1.1.1 and 3.0.2.

All supported OpenSSL versions passes an small set of automatic tests that ensure they can be built and that there are no major regressions. These tests do not validate the cryptographic correctness of the openssl package.

On top of that, the Microsoft CI builds and tests a subset of the supported OpenSSL versions as part of the Microsoft Go fork release process. These tests are much more exhaustive and validate a specific OpenSSL version can produce working applications. Currently only OpenSSL 1.1.1 goes through this process.

Versions not listed above are not supported at all.

Dynamic OpenSSL loading

The OpenSSL shared library libcrypto is loaded at runtime using dlopen when calling openssl.Init. Therefore, dlopen's shared library search conventions also apply here.

The libcrypto shared library file name varies among different platforms, so a best effort is done to find and load the right file:

The base name is always libcrypto.so.
Well-known version strings are appended to the base name, until the file is found, in the following order: 3 -> 1.1 -> 11 -> 111 -> 1.0.2 -> 1.0.0.

This algorithm can be overridden by setting the environment variable GO_OPENSSL_VERSION_OVERRIDE to the desired version string. For example, GO_OPENSSL_VERSION_OVERRIDE="1.1.1k-fips" makes the runtime look for the shared library libcrypto.so.1.1.1k-fips before running the checks for well-known versions.

Building without OpenSSL headers

The openssl package does not use any symbol from the OpenSSL headers. There is no need that have them installed to build an application which imports this library.

Microsoft CI verifies that all the functions and constants defined in our headers match the ones in the OpenSSL headers for every supported OpenSSL version.

Portable OpenSSL

The OpenSSL bindings are implemented in such a way that the OpenSSL version used when building a program does not have to match with the OpenSSL version used when running it.

This feature does not require any additional configuration, but it only works with OpenSSL versions known and supported by the Go toolchain.

Limitations

OpenSSL is used for a given build only in limited circumstances:

The platform must be GOOS=linux.
The build must have cgo enabled.
The android build tag must not be specified.

Acknowledgements

The work done to support FIPS compatibility mode leverages code and ideas from other open-source projects:

All crypto stubs are a mirror of Google's dev.boringcrypto branch and the release branch ports of that branch.
The mapping between BoringSSL and OpenSSL APIs is taken from Fedora's Go fork.
The portable OpenSSL implementation is ported from Microsoft's .NET runtime cryptography module.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

go-crypto-openssl's People

Contributors

Stargazers

Watchers

Forkers

kudzip chaobingya hongruixing test-mass-forker-org-1 bosdhill luminolt nozomi-slibero xnox tobhe parkerha1 corhere fabianponce bonurgeyik kastenhq

go-crypto-openssl's Issues

Difficulty building the library

I am trying to use the library in my project, the folder structure looks roughly like this:

crypto/
├── main.go
├── go-crypto-openssl
   ├── ....h
   ├── ....c 
   ├── ....go
   └── ...._test.go

When trying to run tests from inside the library package, everything works fine, but when i try to build from inside my project and execute it, it fails with a SIGSEG.

I tried to see what's going on with gdb, it looks like i am missing the cgo types definitions for some reason:

(r1=<optimized out>) at _cgo_gotypes.go:1018
1018    _cgo_gotypes.go: No such file or directory.

Any idea on how to resolve this? (i am not that familiar with the workings of cgo)

Add API support for SHA3-384 and friends

openssl provides sha3-384 implementation.

when available please expose sha3-384 implementation.

This is to gain access to a FIPS certified SHA3 implementation instead of golang.org/x/crypto/sha3

ideally as a drop in replacement for golang.org/x/crypto/sha3

Panic when using the library

My code panic when I try to use the library but I don't know how to fix it. Can someone help?

Go ENV

$  go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN="/home/thanhpp/go/bin"
GOCACHE="/home/thanhpp/.cache/go-build"
GOENV="/home/thanhpp/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/thanhpp/go/pkg/mod"
GONOPROXY="github.com/KyberNetwork,github.com/nri4nudge"
GONOSUMDB="github.com/KyberNetwork,github.com/nri4nudge"
GOOS="linux"
GOPATH="/home/thanhpp/go"
GOPROXY="proxy.golang.org,sum.golang.org,index.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.20.5"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build820668886=/tmp/go-build -gno-record-gcc-switches"

OpenSSL Version

 $  openssl version
OpenSSL 3.1.1 30 May 2023 (Library: OpenSSL 3.1.1 30 May 2023)

Example code

package main

import (
	"fmt"

	"github.com/microsoft/go-crypto-openssl/openssl"
)

func main() {
	fmt.Println(openssl.SHA1([]byte("abc")))
}

Error log

$  go run .
fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x0]

runtime stack:
runtime.throw({0x4ad78a?, 0xffffffffffffffb8?})
	/usr/local/go/src/runtime/panic.go:1047 +0x5d fp=0x7ffdd49b7500 sp=0x7ffdd49b74d0 pc=0x437f7d
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:825 +0x3e9 fp=0x7ffdd49b7560 sp=0x7ffdd49b7500 pc=0x44ca69

goroutine 1 [syscall]:
runtime.cgocall(0x48a910, 0xc000060ed0)
	/usr/local/go/src/runtime/cgocall.go:157 +0x5c fp=0xc000060ea8 sp=0xc000060e70 pc=0x40a15c
github.com/microsoft/go-crypto-openssl/openssl._Cfunc_go_openssl_EVP_sha1()
	_cgo_gotypes.go:1519 +0x49 fp=0xc000060ed0 sp=0xc000060ea8 pc=0x4891c9
github.com/microsoft/go-crypto-openssl/openssl.SHA1({0xc000060f49, 0x3, 0x3})
	/home/thanhpp/go/pkg/mod/github.com/microsoft/[email protected]/openssl/sha.go:36 +0x38 fp=0xc000060f18 sp=0xc000060ed0 pc=0x489598
main.main()
	/home/thanhpp/go/src/test-microsoft-openssl/main.go:10 +0x36 fp=0xc000060f80 sp=0xc000060f18 pc=0x489836
runtime.main()
	/usr/local/go/src/runtime/proc.go:250 +0x207 fp=0xc000060fe0 sp=0xc000060f80 pc=0x43a8a7
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000060fe8 sp=0xc000060fe0 pc=0x465141

goroutine 2 [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000052fb0 sp=0xc000052f90 pc=0x43acd6
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:387
runtime.forcegchelper()
	/usr/local/go/src/runtime/proc.go:305 +0xb0 fp=0xc000052fe0 sp=0xc000052fb0 pc=0x43ab10
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000052fe8 sp=0xc000052fe0 pc=0x465141
created by runtime.init.6
	/usr/local/go/src/runtime/proc.go:293 +0x25

goroutine 18 [GC sweep wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00004e780 sp=0xc00004e760 pc=0x43acd6
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:387
runtime.bgsweep(0x0?)
	/usr/local/go/src/runtime/mgcsweep.go:278 +0x8e fp=0xc00004e7c8 sp=0xc00004e780 pc=0x42796e
runtime.gcenable.func1()
	/usr/local/go/src/runtime/mgc.go:178 +0x26 fp=0xc00004e7e0 sp=0xc00004e7c8 pc=0x41ce26
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00004e7e8 sp=0xc00004e7e0 pc=0x465141
created by runtime.gcenable
	/usr/local/go/src/runtime/mgc.go:178 +0x6b

goroutine 19 [GC scavenge wait]:
runtime.gopark(0xc000096000?, 0x4c3fd0?, 0x1?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00004ef70 sp=0xc00004ef50 pc=0x43acd6
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:387
runtime.(*scavengerState).park(0x542bc0)
	/usr/local/go/src/runtime/mgcscavenge.go:400 +0x53 fp=0xc00004efa0 sp=0xc00004ef70 pc=0x425893
runtime.bgscavenge(0x0?)
	/usr/local/go/src/runtime/mgcscavenge.go:628 +0x45 fp=0xc00004efc8 sp=0xc00004efa0 pc=0x425e65
runtime.gcenable.func2()
	/usr/local/go/src/runtime/mgc.go:179 +0x26 fp=0xc00004efe0 sp=0xc00004efc8 pc=0x41cdc6
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00004efe8 sp=0xc00004efe0 pc=0x465141
created by runtime.gcenable
	/usr/local/go/src/runtime/mgc.go:179 +0xaa

goroutine 20 [finalizer wait]:
runtime.gopark(0x1a0?, 0x543000?, 0x20?, 0xa8?, 0xc000052770?)
	/usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000052628 sp=0xc000052608 pc=0x43acd6
runtime.runfinq()
	/usr/local/go/src/runtime/mfinal.go:193 +0x107 fp=0xc0000527e0 sp=0xc000052628 pc=0x41be67
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000527e8 sp=0xc0000527e0 pc=0x465141
created by runtime.createfing
	/usr/local/go/src/runtime/mfinal.go:163 +0x45
exit status 2

CGO failed processing error

Hi, I have encountered this error: "go list failed to return CompiledGoFiles. This may indicate failure to perform cgo processing; try building at the command line. See https://golang.org/issue/38990." after I installed the package locally.

This is the relevant go env

      set GO111MODULE=on
      set GOENV=C:\Users\USERNAME\AppData\Roaming\go\env
      set GOPATH=C:\Users\USERNAME\go
      set GOPROXY=https://proxy.golang.org,direct
      set GOROOT=C:\Program Files\Go
      set GOVERSION=go1.19.2
      set GCCGO=gccgo
      set CC=gcc
      set CXX=g++
      set CGO_ENABLED=1
      set GOMOD=C:\Users\USERNAME\go\pkg\mod\github.com\microsoft\[email protected]\go.mod  
      set CGO_CFLAGS=-g -O2
      set CGO_CPPFLAGS=
      set CGO_CXXFLAGS=-g -O2
      set CGO_FFLAGS=-g -O2
      set CGO_LDFLAGS=-g -O2
      set PKG_CONFIG=pkg-config
      set GOGCCFLAGS=-m64 -mthreads -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-

When I run go list -json -compiled -e

    {
            "Dir": "C:\\Users\\USERNAME\\go\\pkg\\mod\\github.com\\microsoft\\[email protected]\\openssl",
            "ImportPath": "github.com/microsoft/go-crypto-openssl/openssl",
            "Name": "openssl",
            "Root": "C:\\Users\\USERNAME\\go\\pkg\\mod\\github.com\\microsoft\\[email protected]",
            "Module": {
                    "Path": "github.com/microsoft/go-crypto-openssl",
                    "Main": true,
                    "Dir": "C:\\Users\\USERNAME\\go\\pkg\\mod\\github.com\\microsoft\\[email protected]",
                    "GoMod": "C:\\Users\\USERNAME\\go\\pkg\\mod\\github.com\\microsoft\\[email protected]\\go.mod",        
                    "GoVersion": "1.16"
            },
            "Match": [
                    "."
            ],
            "Stale": true,
            "StaleReason": "not installed but available in build cache",
            "GoFiles": [
                    "big.go"
            ],
            "CompiledGoFiles": [
                    "big.go"
            ],
            "IgnoredGoFiles": [
                    "aes.go",
                    "aes_test.go",
                    "ecdh.go",
                    "ecdh_test.go",
                    "ecdsa.go",
                    "ecdsa_test.go",
                    "evpkey.go",
                    "openssl_funcs.h"
            ]
    }

My gopls settings

    {
      // "go.buildFlags": ["-tags=linux,!android"],
      "gopls": {
          "build.experimentalWorkspaceModule": true,
          "build.allowModfileModifications": true
      }
   }

It seems to ignore those files with C imported, so basically the only file that is successfully compiled is big.go, and this made my code that uses this package not able to recognize functions/identifiers in those cgo files. For example, it can only understand openssl.BigInt because this is the only thing that is compiled. All other requests, such as openssl.NewAESCipher, will have error similar to this shown: "undefined: openssl.NewAESCipher" with yellow squiggly lines "NewAESCipher not declared by package openssl (compile)"

Is there something wrong with my gopls settings, or is it an issue with go version (go.mod indicates 1.16 yet go version is 1.19)? Thanks!

Please add information in "About"

Branches with `v` prefix

Right now, the repo has a v0.1 branch, and v0.1.0 + v0.1.1 tags.

In my experience, I've only seen v prefixes before tags, and it's sometimes been a useful way to differentiate tags from branches. It's certainly valid to have v0.1 as a branch, just not something I've seen before.

In .NET land, the norm is e.g. release/6.0 and v6.0.1. I've also seen a bare version: 0.1 with v0.1.1 tag.

I'm curious if this naming style has history behind it, maybe in the Go ecosystem in particular?

(Figured we might as well discuss this on a GitHub issue for posterity. 😄)

@qmuntal @microsoft/golang-compiler

Incorrect error message "decrypt/encrypt" in multiple functions

The "EVP_PKEY_decrypt/encrypt failed" error is returned on error in cryptEVP and verifyEVP functions. The operation is not encrypt/decrypt, but signature verification.

Similarly, the cryptEVP function returns the same error when called by the evpSign function and fails. The failure is in signature generation, not in encrypt/decrypt.

File: openssl/evpkey.go