open-policy-agent / contrib Goto Github PK

Github actions currently fails building the following two sub-modules:

• data_filter_mongodb (due to missing REPOSITORY, possibly more problems)
• data_filter_example (python tests seems to fail intermittently - uses python 2 which is EOL soon)

Fix these problems and remove the current ignore condition in the build script.

There needs to be an example or documentation on securing opa while still allowing pam_authz to function properly.

contrib/data_filter_elasticsearch/cmd/opa-es-filtering have these link to import
"github.com/open-policy-agent/contrib/data_filter_elasticsearch/internal/api"
"github.com/open-policy-agent/contrib/data_filter_elasticsearch/internal/es"
however these links are changed to:
"github.com/open-policy-agent/contrib/tree/master/data_filter_elasticsearch/internal/api"
"github.com/open-policy-agent/contrib/tree/master/data_filter_elasticsearch/internal/es"

We (@adaptant-labs) have prepared a Dart-based example of HTTP API authorization with OPA as a proof of concept based off of @tsandall's OPA-Python example, which we would be happy to contribute - though it's not clear where exactly we should submit a PR for this, if at all.

In any case, the repository can be found at https://github.com/adaptant-labs/opa-api-authz-dart

We are planning on further generalizing the OPA support in Dart and further tying this into our on-going work on leveraging the Dart runtime with OpenFaaS, where we will also be using OPA.

pam_authz.so is not a very good name for sysadmins. Its unclear what the module does when installed on a system by someone that didn't build it from source. Please consider renaming the module pam_opa.so

We expect contrib directories to come with a Makefile that includes standard targets to build, test, publish Docker images, etc.

We should have a top-level Makefile or script that can run each sub-directory accordingly.

We can use Travis CI for CI as we already do this for OPA proper.

The config endpoint was added after the Open API specs (see open_api directory), and should be added accordingly: https://www.openpolicyagent.org/docs/latest/rest-api/#config-api

I am trying this tool and I followed the provided readme[1]. When I start the server from ./opa-es-filtering an error is occurred as follows. If you can provide a solution it will be grateful.

panic: elastic: Error 400 (Bad Request): The mapping definition cannot be nested under a type [_doc] unless include_type_name is set to true. [type=illegal_argument_exception] goroutine 1 [running]: main.main() /opa-elastic/contrib/data_filter_elasticsearch/cmd/opa-es-filtering/main.go:37 +0x38c

[1] https://github.com/open-policy-agent/contrib/blob/master/data_filter_elasticsearch/README.md

Thanks

One of the changes from Python 2 to Python 3 was PEP 3105, which changes the convention of print to be a called function, which requires the use of parenthesis. These have not yet been changed across the whole codebase.

Is it still available for java 8?
Or I have to switch into java 11?

When I tried to add dependency in POM.xml for the spring boot project. It doesn't resolve the dependency and throw following error.

Cannot resolve org.openpolicyagent:voter:1.0-SNAPSHOT

Team,

The rocks file at https://luarocks.org/modules/wada-ama/kong-plugin-opa is from 3 years ago. Could anyone please let me know where I can get the latest rocks, which would include the fix #146 ?

Thanks!

As subject. I found pam_opa works well with Ubuntu but not Centos 7.
the part of configuration of pam.d/sshd is as follows:
auth required /lib/security/pam_opa.so url=http://192.168.1.1:8181 authz_endpoint=/v1/data/sshd/authz display_endpoint=/v1/data/display pull_endpoint=/v1/data/pull log_level=debug

the configuration above works on Unbuntu.
Can you help me to find the root cause?
Thank you very much for your help

Chunan

The below examples won't work because the method verb isn't capitalized, but it is in the rego. Should be a simple fix to change get to GET. Will submit PR if time later.

When i make a POST request, the vote function in OPAVoter class is not functioning properly. The path fetched from the filter is always '//error'.

How to add OTP verification while using the pam_opa module

Has anyone attempted to integrate OPA and GraphQL for policies against Graph Nodes or other query / graph structures?

wada-ama/kong-plugin-opa#8

wada-ama/kong-plugin-opa#8 (comment)

@zahiyo Hi, will you fix this issues here?

In Elastic search data filtering example. Link to elastic search installation doesn't work

Change it to the new link i.e
https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html
This link has modes of elastic search installtions

The problem correctly flagged in #49 was only partly resolved by updating the docker-compose file, but leaving the policy syntax the same:

$ git clone https://github.com/open-policy-agent/contrib.git
Cloning into 'contrib'...
[..]
$ cd contrib/api_authz
$ make up-token
docker-compose -f docker/docker-compose-token.yaml up
Starting docker_api_server_1 ... done
Starting docker_opa_1        ... done
Attaching to docker_opa_1, docker_api_server_1
opa_1         | error: compile error: 3 errors occurred:
opa_1         | api_authz_token.rego:14: rego_unsafe_var_error: var _ is unsafe
opa_1         | api_authz_token.rego:14: rego_unsafe_var_error: var _ is unsafe
opa_1         | api_authz_token.rego:14: rego_unsafe_var_error: var payload is unsafe
docker_opa_1 exited with code 1

Fixing the syntax in the io.jwt.decode call resolves the problem

$ git diff docker/policy/api_authz_token.rego
diff --git a/api_authz/docker/policy/api_authz_token.rego b/api_authz/docker/policy/api_authz_token.rego
index 57facb3..abeab38 100644
--- a/api_authz/docker/policy/api_authz_token.rego
+++ b/api_authz/docker/policy/api_authz_token.rego
@@ -11,7 +11,7 @@ import input as http_api
 # io.jwt.decode takes one argument (the encoded token) and has three outputs:
 # the decoded header, payload and signature, in that order. Our policy only
 # cares about the payload, so we ignore the others.
-token = {"payload": payload} { io.jwt.decode(http_api.token, _, payload, _) }
+token = {"payload": payload} { io.jwt.decode(http_api.token, [_, payload, _]) }

 # Ensure that the token was issued to the user supplying it.
 user_owns_token { http_api.user = token.payload.azp }

$ make up-token
docker-compose -f docker/docker-compose-token.yaml up
Starting docker_api_server_1 ... done
Recreating docker_opa_1      ... done
Attaching to docker_api_server_1, docker_opa_1
opa_1         | time="2019-08-16T12:45:04Z" level=info msg="First line of log stream." addrs="[:8181]" insecure_addr=
api_server_1  | INFO:werkzeug: * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)

Kafka 2.4.0 came with a new Java authorizer interface, meaning that JSON structure currently supported by the Kafka plugin here is no longer in use since December 2019. Bisnode just released a new version of their plugin which supports this. We should either do the same here or remove the Kafka integration from contrib in favor of theirs.

In the data_filter_example, there is an issue which affects portability between different implementations of SQL. In particular, strings can be single quoted, others double quoted, or either depending upon the implementation. There should be a flag to control which kind of quotation is used.

Hi, I've attempted to run the example policy in the api_authz example, but I get the following errors when launching the docker container:

opa_1         | error: compile error: 3 errors occurred:
opa_1         | /policy/api_authz_token.rego:7: rego_unsafe_var_error: var _ is unsafe
opa_1         | /policy/api_authz_token.rego:7: rego_unsafe_var_error: var payload is unsafe
opa_1         | /policy/api_authz_token.rego:7: rego_unsafe_var_error: var _ is unsafe

Line 7 indicates this is an issue with the results from the io.jwt.decode call. I cannot find any documentation regarding unsafe variables and how to avoid this issue. Is there any further information I could use to solve this issue?

Hi, im trying to work on pam_opa.so module on our RHEL 7 server but i keep getting this error

error: PAM: User account has expired for [someuser] from [someip]

im following this guide https://github.com/open-policy-agent/contrib/tree/master/pam_opa/pam#configuration

i notice that when i build it in our ubuntu environment it works right away but for some reason building and configuring the pam module on rhel is not so straight forward. anyone tried to compile and configure pam_opa on any RHEL/CENTOS env here? can someone share how they do it?

Thanks

Hi there,

I am following below url in setting up OPA & Envoy on my Apple M1 Macbook. After kubelct apply on below url, Init container proxy_init is failing to come up and throwing below error.

Could you please help me on this.

Error : iptables v1.6.0: can't initialize iptables table `nat': iptables who? (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

https://raw.githubusercontent.com/open-policy-agent/opa-envoy-plugin/main/quick_start.yaml

We should migrate the contrib repo to run on top of GitHub actions as travis-ci.org is being shutdown in December.

Use Case scenario: we want OPA to make different authorization decisions based on the Accept (eg json vs xml) or other headers present in the request.

Currently the plugin provides only:

• token (jwt token)
• method (GET, POST, etc)
• path

We want to add headers. If to keep backward compatibility we don't want it set by default, we want a property configuration for the plugin to enable the option as in this old plugin: https://github.com/ninjaneers-team/kong-opa

We do NOT want to send the body too

Hello, I am new to OPA, I am trying to integrate OPA with linux PAM, for sudo and ssh authorization, but the tutorial covers only for docker version, but i am trying to install on standalone server. I have followed the instructions from https://github.com/open-policy-agent/contrib/blob/master/pam_opa/pam/README.md, but there is problem with the jansson library module (i can't find the module to copy it to PAM module) and its very difficult to follow, can anyone help me with installing the OPA-PAM for standalone, UBUNTU server. Thanks

At Arroyo, we're working with PostgreSQL, and the generated SQL queries relying on "INNER JOIN" are both invalid and highly sub-optimal when tweaked. Rather, the qualifiers should be using a 'WHERE' clause. In the mean time, we'll probably just be using an obvious internal hack, but this should be handled.

Currently there are no checks on the formatting of the pam_authz golang code. A make target should be added to run go fmt and check for unformatted code. The build should fail is this result is non-empty.

Hi I am new to OPA, I want to add policy that nodeslector should exists in pod.
I have made the below code not able to warap my head around the issue with this as not able to get desired output.

package kubernetes.admission

deny[reason] {
input.request.kind.kind == "Pod"
input.request.operation == "CREATE"
input.request.object.spec.nodeSelector
not count(input.request.object.spec.nodeSelector) > 0
reason := "pod with nodeselector not allowed at the specified location"

Output:-
{
"deny": []
}

}

The docker-compose file for kakfa authz demo uses a pre-built demo-kafka image. It is being used on this line: https://github.com/open-policy-agent/contrib/blob/master/kafka_authorizer/docker-compose.yml#L18 . Is the actual Dockerfile for this available somewhere? We'd like to see if it is using Confluent Kafka just like the zookeeper is the Confluent image.

Whilst trying to prove out OPA with Elasticsearch for my team I ran into several issues with the Elasticsearch data filtering example. I personally have not been able to successfully get past step 2 in the README.

Asking for help to ensure the example can run against newer versions of Elasticsearch and also support communicating to containerized Elasticsearch.

Elasticsearch version related error

• #115
• https://stackoverflow.com/questions/62803392/the-mapping-definition-cannot-be-nested-under-a-type-doc-unless-include-type
• https://www.elastic.co/guide/en/elasticsearch/reference/7.16/removal-of-types.html#_schedule_for_removal_of_mapping_types

Containerized Elasticsearch related error

• https://stackoverflow.com/questions/61455272/cannot-connect-to-elastic-search-no-active-connection-found-no-elasticsearch
• olivere/elastic#312

OPA is using go modules as a go dependency management tool.
Why shouldn't we use the same with ES as well?

I see ES was developed in 2018 back then go modules was not so feature rich.

WDYT @ashutosh-narkar ?

I can take this up.

hi, im having issue with my build pam_opa module. seems it cant reach the opa-server i've check many times to ensure that pam.d sshd file calling the correct url and i can even hit the server via curl manually. please the logs below and the version i used to build the pam_opa module

PAM File

auth required /lib/security/pam_opa.so url=http://:8181 authz_endpoint=/v1/data/sshd/authz display_endpoint=/v1/data/display pull_endpoint=/v1/data/pull log_level=debug
session [success=ok ignore=ignore module_unknown=ignore default=bad] pam_selinux.so close
session required pam_loginuid.so
session required pam_env.so # [1]
session required pam_env.so user_readenv=1 envfile=/etc/default/locale
session [success=ok ignore=ignore module_unknown=ignore default=bad] pam_selinux.so open
session required pam_namespace.so
session optional pam_keyinit.so force revoke
session include password-auth
session include postlogin
-session optional pam_reauthorize.so prepare

Logs

Jan 20 14:17:19 OPA-PAM[36249]: Defaulted to log level info
Jan 20 14:17:19 OPA-PAM[36249]: Parsing arg: url=http://:8181
Jan 20 14:17:19 OPA-PAM[36249]: Parsing arg: authz_endpoint=/v1/data/sshd/authz
Jan 20 14:17:19 OPA-PAM[36249]: Parsing arg: display_endpoint=/v1/data/display
Jan 20 14:17:19 OPA-PAM[36249]: Parsing arg: pull_endpoint=/v1/data/pull
Jan 20 14:17:19 OPA-PAM[36249]: Parsing arg: log_level=debug
Jan 20 14:17:19 OPA-PAM[36249]: Session log level is set to debug
Jan 20 14:17:19 OPA-PAM[36249]: Commencing display cycle.
Jan 20 14:17:19 OPA-PAM[36249]: Initializing HTTP request GET /v1/data/display
Jan 20 14:17:19 OPA-PAM[36249]: HTTP request body: (null)
Jan 20 14:17:19 OPA-PAM[36249]: HTTP request failed with error: Couldn't connect to server
Jan 20 14:17:19 OPA-PAM[36249]: Commencing pull cycle.
Jan 20 14:17:19 OPA-PAM[36249]: Initializing HTTP request GET /v1/data/pull
Jan 20 14:17:19 OPA-PAM[36249]: HTTP request body: (null)
Jan 20 14:17:19 OPA-PAM[36249]: HTTP request failed with error: Couldn't connect to server
Jan 20 14:17:19 OPA-PAM[36249]: Collecting system information.
Jan 20 14:17:19 OPA-PAM[36249]: Loaded sysinfo pam_username: opadmin
Jan 20 14:17:19 OPA-PAM[36249]: Loaded sysinfo pam_service: sshd
Jan 20 14:17:19 OPA-PAM[36249]: Loaded sysinfo pam_req_username:
Jan 20 14:17:19 OPA-PAM[36249]: Loaded sysinfo pam_req_hostname:
Jan 20 14:17:19 OPA-PAM[36249]: Commencing authz cycle.
Jan 20 14:17:19 OPA-PAM[36249]: Initializing HTTP request POST /v1/data/sshd/authz
Jan 20 14:17:19 OPA-PAM[36249]: HTTP request body: {"input":{"display_responses":{},"pull_responses":{"files":{},"env_vars":{}},"sysinfo":{"pam_username":"opadmin","pam_service":"sshd","pam_req_username":"","pam_req_hostname":""}}}
Jan 20 14:17:19 OPA-PAM[36249]: HTTP request failed with error: Couldn't connect to server
Jan 20 14:17:19 OPA-PAM[36249]: Freeing allocated data.

Version used to compile
https://github.com/linux-pam/linux-pam/releases/download/v1.3.1/Linux-PAM-1.3.1.tar.xz
http://www.digip.org/jansson/releases/jansson-2.12.tar.gz
https://curl.haxx.se/download/curl-7.68.0.tar.gz

HI,

Reading the documentation, exist a section to define bundles:
https://www.openpolicyagent.org/docs/latest/configuration/

bundles:
  authz:
    service: acmecorp
    resource: bundles/http/example/authz.tar.gz
    polling:
      min_delay_seconds: 60
      max_delay_seconds: 120
    signing:
      keyid: global_key
      scope: write

https://www.openpolicyagent.org/docs/v0.12.2/bundles/

$ tar tzf bundle.tar.gz
.manifest
roles
roles/bindings
roles/bindings/data.json
roles/permissions
roles/permissions/data.json
http
http/example
http/example/authz
http/example/authz/authz.rego

I would like to know if exist an example generating a bundle.tar.gz on fly to be consumed by Open Policy Agent

See

If this module is implemented with an account module instead of an auth module, it always returns success, regardless of what OPA says it should.

Instead, it should return the value on line 66, not discard it.

I've created an updated README.md for the existing spring_authz example to include some hints on using gradle, as well as the additional command line flag to turn on debugging (which makes it easier to see the data going back and forth). I have the updated README ready to go. I assume I can create a pull request after the issue is accepted?

The SQL and Elastic examples are referenced in a few places. We should make sure all of the tests are healthy. Concretely, we should add/update a Makefile in each directory to include a 'build' target that executes the integration tests in each. Note, the SQL example needs to start an instance of OPA. Perhaps the tests can run OPA in a Docker container if-present. This should work for most development environments as well as Travis. If Docker is not installed, just print a warning and skip the tests.

It looks like the image builds, but the tests fail to run. Possibly some API in k8s has changed in an incompatible way.

./test.sh
Waiting for OPA pod to come up
OPA pod is up - awaiting condition=Ready
OPA pod ready. Running tests.
=============================
Expected Error from server (InternalError): an error on the server ("Internal Server Error: \"/api/v1/namespaces/kube-system/pods?limit=500\": Post \"https://10.96.167.0/v0/data/k8s/authz/decision?timeout=30s\": dial tcp 10.96.167.0:443: connect: connection refused") has prevented the request from succeeding (get pods) == *OPA: denied access to namespace kube-system
make[1]: *** [Makefile:7: test] Error 1
make[1]: Leaving directory '/home/runner/work/contrib/contrib/k8s_authorization'
make: *** [Makefile:7: build] Error 2
Error: Process completed with exit code 2.

Hello @VineethReddy02

Thank you for making the data_filter_mongo contribution to OPA.

Can you provide the current status of this contribution?

When I attempt to run the employees/john example that you describe in https://github.com/open-policy-agent/contrib/blob/main/data_filter_mongodb/README.md, I get an empty 200 response (see my command line below).

Note that since I'm running on an M1 Mac and I could not find a M1 arch published version of vineeth97/opa-mongo, I needed t build from source. Also, i noticed that the code as provided reads a request body on the GET endpoint which, since I'm using curl to send requests, meant that I needed to add a POST endpoint that uses the same api.handleGetReq handler. I've tried to trace down through the debugger to pinpoint where things go wrong but this is complicated by the extremely nested nature of the evaluation. The best I've been able to determine so far is that the "defined" boolean at https://github.com/open-policy-agent/opa/blob/main/topdown/eval.go#L417 never gets set to true for any evaluated step executed by evalStep method so the method returns nil.

The only output I see is in the following, one-line log output message from the server log:

{"level":"info","ts":1681043557.8948538,"caller":"opa/opa.go:62","msg":"received request","request":{"method":"POST","path":["employees","john"],"user":"danerys"}}

Upgrading the opa dependency from 0.43.1 to latest 0.51.0 does not help matters.

Any suggestions would be appreciated.

Here is my command line I'm using to transmit the request to the server:

@curl -X POST http://localhost:9095/employees/john -d @- <<EOF

{
"input": {
"method": "GET",
"path": ["employees", "john"],
"user": "danerys"
}
}
EOF

Hi anyone, implemented token Authorization with ssh+sudo. having trouble on how to put the "-H Authorization: Bearer XXXXXXXXX" on the pam files sudo/sshd. TIA!

The pam_opa image does not build due to this error:

--2022-04-04 21:18:15--  https://www.digip.org/jansson/releases/jansson-2.11.tar.gz
Resolving www.digip.org (www.digip.org)... 91.232.155.81, 2001:67c:1be8:1337::443
Connecting to www.digip.org (www.digip.org)|91.232.155.81|:443... connected.
ERROR: The certificate of 'www.digip.org' is not trusted.
ERROR: The certificate of 'www.digip.org' has expired.
Removing intermediate container 604d4960237d
The command '/bin/sh -c wget https://www.digip.org/jansson/releases/jansson-2.11.tar.gz &&     tar -xvf jansson-2.11.tar.gz &&     cd jansson-2.11 &&     ./configure --prefix=/usr &&     make &&     make check &&     make install' returned a non-zero code: 5
make[1]: *** [Makefile:12: image] Error 5
make: *** [Makefile:7: build] Error 2

However, I am able to run wget https://www.digip.org/jansson/releases/jansson-2.11.tar.gz without issue on my M1 Mac (macOS Monterey 12.1, build 21C52). Presumably, the GH Actions runner either has a clock or a certificate store issue that is preventing this from working.

One possible fix is to use --no-check-certificate and then check the sha265 sum of the downloaded file against a known value.

See also: #168

I am using go module for opa-iptables plugin. For successfully build my project I need go module support which added to go 1.11. But current Travis CI is using go 1.10 which doesn't support go module.

It appears that this is a casualty of the recent changes to how modules are implemented breaking a transitive dependency of mtail. I found a potential workaround and discussed it here.

See also: #167 #166

The kong_api_authz module build appears to be broken on unit tests.

Steps to reproduce issue:

# from project root
cd kong_api_authz
make

Output:

[==========] Running tests from scanned files.
[----------] Global test environment setup.
[----------] Running tests from spec/kong/plugins/opa/access_spec.lua
[ RUN      ] spec/kong/plugins/opa/access_spec.lua @ 52: opa:access allow access
./src/kong/plugins/opa/access.lua:77: attempt to index field 'req' (a nil value)

stack traceback:
        ./src/kong/plugins/opa/access.lua:77: in function 'execute'
        spec/kong/plugins/opa/access_spec.lua:53: in function <spec/kong/plugins/opa/access_spec.lua:52>

[  ERROR   ] spec/kong/plugins/opa/access_spec.lua @ 52: opa:access allow access (2.54 ms)
[ RUN      ] spec/kong/plugins/opa/access_spec.lua @ 57: opa:access returns 403 when request is forbidden
./src/kong/plugins/opa/access.lua:77: attempt to index field 'req' (a nil value)

stack traceback:
        ./src/kong/plugins/opa/access.lua:77: in function 'execute'
        spec/kong/plugins/opa/access_spec.lua:60: in function <spec/kong/plugins/opa/access_spec.lua:57>

[  ERROR   ] spec/kong/plugins/opa/access_spec.lua @ 57: opa:access returns 403 when request is forbidden (1.97 ms)
[ RUN      ] spec/kong/plugins/opa/access_spec.lua @ 64: opa:access returns 500 on OPA server error or when not reachable
./src/kong/plugins/opa/access.lua:77: attempt to index field 'req' (a nil value)

stack traceback:
        ./src/kong/plugins/opa/access.lua:77: in function 'execute'
        spec/kong/plugins/opa/access_spec.lua:67: in function <spec/kong/plugins/opa/access_spec.lua:64>

[  ERROR   ] spec/kong/plugins/opa/access_spec.lua @ 64: opa:access returns 500 on OPA server error or when not reachable (2.30 ms)
[ RUN      ] spec/kong/plugins/opa/access_spec.lua @ 71: opa:access sends a request to the server defined in the configuration
./src/kong/plugins/opa/access.lua:77: attempt to index field 'req' (a nil value)

stack traceback:
        ./src/kong/plugins/opa/access.lua:77: in function 'execute'
        spec/kong/plugins/opa/access_spec.lua:78: in function <spec/kong/plugins/opa/access_spec.lua:71>

[  ERROR   ] spec/kong/plugins/opa/access_spec.lua @ 71: opa:access sends a request to the server defined in the configuration (2.24 ms)
[----------] 4 tests from spec/kong/plugins/opa/access_spec.lua (64.22 ms total)

[----------] Global test environment teardown.
[==========] 4 tests from 1 test file ran. (67.34 ms total)
[  PASSED  ] 0 tests.
[  ERROR   ] 4 errors, listed below:
[  ERROR   ] spec/kong/plugins/opa/access_spec.lua @ 52: opa:access allow access
[  ERROR   ] spec/kong/plugins/opa/access_spec.lua @ 57: opa:access returns 403 when request is forbidden
[  ERROR   ] spec/kong/plugins/opa/access_spec.lua @ 64: opa:access returns 500 on OPA server error or when not reachable
[  ERROR   ] spec/kong/plugins/opa/access_spec.lua @ 71: opa:access sends a request to the server defined in the configuration

 4 ERRORS

Error: test suite failed.

I will submit a pull request to fix this issue shortly.