MeshOPA

Kong Mesh integrates the Open Policy Agent (OPA) to provide access control for your services.

The agent is included in the data plane proxy sidecar, instead of the more common deployment as a separate sidecar.

When the MeshOPA policy is applied, the control plane configures the following:

• The embedded policy agent, with the specified policy
• Envoy, to use External Authorization that points to the embedded policy agent

To apply a policy with MeshOPA, you must do the following:

• Specify the group of data plane proxies to apply the policy to with the targetRef property.
• Provide a policy with the appendPolicies property. Policies are defined in the Rego language.
• Optionally provide custom configuration for the policy agent.

Inline

apiVersion: kuma.io/v1alpha1
kind: MeshOPA
metadata:
  name: mopa-1
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default # optional, defaults to `default` if unset
spec:
  targetRef:
    kind: Mesh
  default:
    agentConfig: # optional
      inlineString: | # one of: inlineString, secret
        decision_logs:
          console: true
    appendPolicies:
      - ignoreDecision: false # optional, defaults to 'false'
        rego:
          inlineString: | # one of: inlineString, secret
            package envoy.authz

            import input.attributes.request.http as http_request

            default allow = false

            token = {"valid": valid, "payload": payload} {
                [_, encoded] := split(http_request.headers.authorization, " ")
                [valid, _, payload] := io.jwt.decode_verify(encoded, {"secret": "secret"})
            }

            allow {
                is_token_valid
                action_allowed
            }

            is_token_valid {
              token.valid
              now := time.now_ns() / 1000000000
              token.payload.nbf <= now
              now < token.payload.exp
            }

            action_allowed {
              http_request.method == "GET"
              token.payload.role == "admin"
            }

type: MeshOPA
mesh: default
name: mopa-1
spec:
  targetRef:
    kind: Mesh
  default:
    agentConfig: # optional
      inlineString: | # one of: inlineString, secret
        decision_logs:
          console: true
    appendPolicies: # optional
      - ignoreDecision: false # optional, defaults to 'false'
        rego:
          inlineString: | # one of: inlineString, secret
            package envoy.authz

            import input.attributes.request.http as http_request

            default allow = false

            token = {"valid": valid, "payload": payload} {
                [_, encoded] := split(http_request.headers.authorization, " ")
                [valid, _, payload] := io.jwt.decode_verify(encoded, {"secret": "secret"})
            }

            allow {
                is_token_valid
                action_allowed
            }

            is_token_valid {
              token.valid
              now := time.now_ns() / 1000000000
              token.payload.nbf <= now
              now < token.payload.exp
            }

            action_allowed {
              http_request.method == "GET"
              token.payload.role == "admin"
            }

With secrets

Encoding the policy in a Secret provides some security for policies that contain sensitive data.

Kong Mesh defines a default configuration for OPA, but you can adjust the configuration to meet your environment’s requirements.

The following environment variables are available:

apiVersion: v1
kind: ConfigMap
metadata:
  name: kong-mesh-control-plane-config
  namespace: kong-mesh-system
data:
  config.yaml: |
    runtime:
      kubernetes:
        injector:
          sidecarContainer:
            envVars:
              KMESH_OPA_ENABLED: "false"
              KMESH_OPA_ADDR: ":8888"
              KMESH_OPA_CONFIG_OVERRIDES: "config1:x,config2:y"

kuma:
  controlPlane:
    config: |
      runtime:
        kubernetes:
          injector:
            sidecarContainer:
              envVars:
                KMESH_OPA_ENABLED: "false"
                KMESH_OPA_ADDR: ":8888"
                KMESH_OPA_CONFIG_OVERRIDES: "config1:x,config2:y"

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  ...
  template:
    metadata:
      ...
      annotations:
        # indicate to Kong Mesh that this Pod doesn't need a sidecar
        kuma.io/sidecar-env-vars: "KMESH_OPA_ENABLED=false;KMESH_OPA_ADDR=:8888;KMESH_OPA_CONFIG_OVERRIDES=config1:x,config2:y"

--opa-addr
--opa-config-path
--opa-diagnostic-addr
--opa-enabled                    
--opa-ext-authz-addr
--opa-set strings

You can configure the external authorization filter by adjusting the authConfig section.

apiVersion: kuma.io/v1alpha1
kind: MeshOPA
metadata:
  name: mopa-1
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default # optional, defaults to `default` if unset
spec:
  targetRef:
    kind: Mesh
  default:
    authConfig: # optional
      statusOnError: 413 # optional: defaults to 403.
      onAgentFailure: allow # optional: one of 'allow' or 'deny', defaults to 'deny' defines the behavior when communication with the agent fails or the policy execution fails.
      requestBody: # optional
          maxSize: 1024 # the max number of bytes to send to the agent, if we exceed this, the request to the agent will have: `x-envoy-auth-partial-body: true`.
          sendRawBody: true # use when the body is not plaintext. The agent request will have `raw_body` instead of `body`
...

type: MeshOPA
mesh: default
name: mopa-1
spec:
  targetRef:
    kind: Mesh
  default:
    authConfig: # optional
      statusOnError: 413 # optional: defaults to 403. http statusCode to use when the connection to the agent failed.
      onAgentFailure: allow # optional: one of 'allow' or 'deny', defaults to 'deny'. defines the behavior when communication with the agent fails or the policy execution fails.
      requestBody: # optional
        maxSize: 1024 # the maximum number of bytes to send to the agent, if we exceed this, the request to the agent will have: `x-envoy-auth-partial-body: true`.
        sendRawBody: true # use when the body is not plaintext. The agent request will have `raw_body` instead of `body`
...

By default, the body will not be sent to the agent. To send it, set authConfig.requestBody.maxSize to the maximum size of your body. If the request body is larger than this parameter, it will be truncated and the header x-envoy-auth-partial-body will be set to true.

The agentConfig field lets you define a custom configuration that points to an external management server:

apiVersion: kuma.io/v1alpha1
kind: MeshOPA
metadata:
  name: mopa-1
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default
spec:
  targetRef:
    kind: Mesh
  default:
    agentConfig:
      inlineString: |
        services:
          acmecorp:
            url: https://example.com/control-plane-api/v1
            credentials:
              bearer:
                token: "bGFza2RqZmxha3NkamZsa2Fqc2Rsa2ZqYWtsc2RqZmtramRmYWxkc2tm"

        discovery:
          name: example
          resource: /configuration/example/discovery

type: MeshOPA
mesh: default
name: mopa-1
spec:
  targetRef:
    kind: Mesh
  default:
    agentConfig:
      inlineString: | # one of: inlineString, secret
        services:
          acmecorp:
            url: https://example.com/control-plane-api/v1
            credentials:
              bearer:
                token: "bGFza2RqZmxha3NkamZsa2Fqc2Rsa2ZqYWtsc2RqZmtramRmYWxkc2tm"
        discovery:
          name: example
          resource: /configuration/example/discovery

In your organization, the mesh operator may want to set a policy for subset of proxies in the mesh. At the same time, service owners may want to exercise additional policies.

For example, the mesh operator may want to enable JWT token validation for all proxies in the mesh

apiVersion: kuma.io/v1alpha1
kind: MeshOPA
metadata:
  name: mopa-mesh-operator
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default
spec:
  targetRef:
    kind: Mesh
  default:
    appendPolicies:
      - rego:
          inlineString: |
            package operator
            
            import input.attributes.request.http as http_request
            
            default allow = false
            
            token = {"valid": valid, "payload": payload} {
                [_, encoded] := split(http_request.headers.authorization, " ")
                [valid, _, payload] := io.jwt.decode_verify(encoded, {"secret": "secret"})
            }
            
            allow {
                is_token_valid
                action_allowed
            }
            
            is_token_valid {
              token.valid
              now := time.now_ns() / 1000000000
              token.payload.nbf <= now
              now < token.payload.exp
            }
            
            action_allowed {
              http_request.method == "GET"
              token.payload.role == "admin"
            }

Service owner wants to block all requests on path /blocked:

apiVersion: kuma.io/v1alpha1
kind: MeshOPA
metadata:
  name: mopa-service-owner
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default
spec:
  targetRef:
    kind: MeshService
    name: test-server_kuma-demo_svc_80
  default:
    appendPolicies:
      - rego:
          inlineString: |
            package serviceowner
            
            default allow = true
            
            deny {
              input.parsed_path == ["blocked"]
            }

appendPolicies is a list you can append, therefore in the case of the data plane proxy test-server_kuma-demo_svc_80 service, both policies will be applied.

Kong Mesh will autogenerate an additional OPA decision policy:

package implicitkmesh
import data.operator
import data.serviceowner

allow {
  data.operator.allow
  not data.operator.deny
  data.serviceowner.allow
  not data.serviceowner.deny
}

It also configures the OPA agent decision path (plugins.envoy_ext_authz_grpc.path) to implicitkmesh/allow.

You can also add a rego policy which is not part of the decision. Set a appendPolicies[*].ignoreDecision to true so the rego policy won’t be added to autogenerated decision policy. This way, the mesh operator can expose utility functions to service owner.

The following example shows how to deploy and test a sample MeshOPA policy on Kubernetes, using the kuma-demo application.

1. Deploy the example application:

   kubectl apply -f https://bit.ly/demokuma
   

2. Make a request from the frontend to the backend:

   kubectl exec -i -t $(kubectl get pod -l "app=kuma-demo-frontend" -o jsonpath='{.items[0].metadata.name}' -n kuma-demo) -n kuma-demo -c kuma-fe -- curl backend:3001 -v
   

   The output looks like:

   Defaulting container name to kuma-fe.
   Use 'kubectl describe pod/kuma-demo-app-6787b4f7f5-m428c -n kuma-demo' to see all of the containers in this pod.
   *   Trying 10.111.108.218:3001...
   * TCP_NODELAY set
   * Connected to backend (10.111.108.218) port 3001 (#0)
   > GET / HTTP/1.1
   > Host: backend:3001
   > User-Agent: curl/7.67.0
   > Accept: */*
   >
   * Mark bundle as not supporting multiuse
   < HTTP/1.1 200 OK
   < x-powered-by: Express
   < cache-control: no-store, no-cache, must-revalidate, private
   < access-control-allow-origin: *
   < access-control-allow-methods: PUT, GET, POST, DELETE, OPTIONS
   < access-control-allow-headers: *
   < host: backend:3001
   < user-agent: curl/7.67.0
   < accept: */*
   < x-forwarded-proto: http
   < x-request-id: 1717af9c-2587-43b9-897f-f8061bba5ad4
   < content-length: 90
   < content-type: text/html; charset=utf-8
   < date: Tue, 16 Mar 2021 15:33:18 GMT
   < x-envoy-upstream-service-time: 1521
   < server: envoy
   <
   * Connection #0 to host backend left intact
   Hello World! Marketplace with sales and reviews made with <3 by the OCTO team at Kong Inc.
   

3. Apply a MeshOPA policy that requires a valid JWT token:

   echo "
   apiVersion: kuma.io/v1alpha1
   kind: MeshOPA
   metadata:
     namespace: kong-mesh-system
     name: mopa-1
     labels:
       kuma.io/mesh: default
   spec:
     targetRef:
       kind: Mesh
     default:
       appendPolicies:
         - rego:
             inlineString: |
               package envoy.authz
   
               import input.attributes.request.http as http_request
   
               default allow = false
   
               token = {\"valid\": valid, \"payload\": payload} {
                   [_, encoded] := split(http_request.headers.authorization, \" \")
                   [valid, _, payload] := io.jwt.decode_verify(encoded, {\"secret\": \"secret\"})
               }
   
               allow {
                   is_token_valid
                   action_allowed
               }
   
               is_token_valid {
                 token.valid
                 now := time.now_ns() / 1000000000
                 token.payload.nbf <= now
                 now < token.payload.exp
               }
   
               action_allowed {
                 http_request.method == \"GET\"
                 token.payload.role == \"admin\"
               }
   " | kubectl apply -f -
   

4. Make an invalid request from the frontend to the backend:

   kubectl exec -i -t $(kubectl get pod -l "app=kuma-demo-frontend" -o jsonpath='{.items[0].metadata.name}' -n kuma-demo) -n kuma-demo -c kuma-fe -- curl backend:3001 -v
   

   The output looks like:

   Defaulting container name to kuma-fe.
   Use 'kubectl describe pod/kuma-demo-app-6787b4f7f5-bwvnb -n kuma-demo' to see all of the containers in this pod.
   *   Trying 10.105.146.164:3001...
   * TCP_NODELAY set
   * Connected to backend (10.105.146.164) port 3001 (#0)
   > GET / HTTP/1.1
   > Host: backend:3001
   > User-Agent: curl/7.67.0
   > Accept: */*
   >
   * Mark bundle as not supporting multiuse
   < HTTP/1.1 403 Forbidden
   < date: Tue, 09 Mar 2021 16:50:40 GMT
   < server: envoy
   < x-envoy-upstream-service-time: 2
   < content-length: 0
   <
   * Connection #0 to host backend left intact
   

   Note the HTTP/1.1 403 Forbidden message. The application doesn’t allow a request without a valid token.

   The policy can take up to 30 seconds to propagate, so if this request succeeds the first time, wait and then try again.

5. Make a valid request from the frontend to the backend.

   Export the token into an environment variable:

   export ADMIN_TOKEN="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiYWRtaW4iLCJzdWIiOiJZbTlpIiwibmJmIjoxNTE0ODUxMTM5LCJleHAiOjI1MjQ2MDgwMDB9.H0-42LYzoWyQ_4MXAcED30u6lA5JE087eECV2nxDfXo"
   

   Make the request:

   kubectl exec -i -t $(kubectl get pod -l "app=kuma-demo-frontend" -o jsonpath='{.items[0].metadata.name}' -n kuma-demo) -n kuma-demo -c kuma-fe -- curl -H "Authorization: Bearer $ADMIN_TOKEN" backend:3001
   

   The output looks like:

   Defaulting container name to kuma-fe.
   Use 'kubectl describe pod/kuma-demo-app-6787b4f7f5-m428c -n kuma-demo' to see all of the containers in this pod.
   *   Trying 10.111.108.218:3001...
   * TCP_NODELAY set
   * Connected to backend (10.111.108.218) port 3001 (#0)
   > GET / HTTP/1.1
   > Host: backend:3001
   > User-Agent: curl/7.67.0
   > Accept: */*
   >
   * Mark bundle as not supporting multiuse
   < HTTP/1.1 200 OK
   < x-powered-by: Express
   < cache-control: no-store, no-cache, must-revalidate, private
   < access-control-allow-origin: *
   < access-control-allow-methods: PUT, GET, POST, DELETE, OPTIONS
   < access-control-allow-headers: *
   < host: backend:3001
   < user-agent: curl/7.67.0
   < accept: */*
   < x-forwarded-proto: http
   < x-request-id: 8fd7b398-1ba2-4c2e-b229-5159d04d782e
   < content-length: 90
   < content-type: text/html; charset=utf-8
   < date: Tue, 16 Mar 2021 17:26:00 GMT
   < x-envoy-upstream-service-time: 261
   < server: envoy
   <
   * Connection #0 to host backend left intact
   Hello World! Marketplace with sales and reviews made with <3 by the OCTO team at Kong Inc.
   

   The request is valid again because the token is signed with the secret private key, its payload includes the admin role, and it is not expired.