Ory Oathkeeper: Identity and Access Proxy Server¶

Ory has a great ecosystem of products when it comes to authentication and authorization. Ory Oathkeeper is an stateless Identity and Access Proxy server.

It is capable of acting as a reverse-proxy as well as a decision maker and policy enforcer for other proxy servers.

In today's application development world, if you're operating on HTTP layer, Ory Oathkeeper has a lot to offer to you.

Stick around to find out how.

What is Ory Oathkeeper?¶

Chances are, your application needs protection from unauthorized access, whether deployed into the internet and exposed publicly, or gated behind private network and only accessible to a certain privileged users.

That is what Ory Oathkeeper is good at, making sure that requests won't make it to the upstream server unless they are explicitly allowed.

It enforces protective measures to ensure unauthorized requests are denied. It does that by sitting at the frontier of your infrastructure, receiving traffics as they come in, inspecting its content, and making decisions based on the rules you've previously defined and instructed it to.

In this blog post, we will explore what Ory Oathkeeper can do, deploy and configure it in a way that will protect our upstream server.

This scenario is quite common, and you've probably come across it or developed a custom solution for your application before.

Hold your breath till the end to find out how to leverage this opensource solution to your advantage so that you won't ever have to reinvent the wheel again.

Why Ory Oathkeeper?¶

There are numerous reasons why Oathkeeper is a good fit at what it does. Here are some of the highlights you should be aware of:

• Proxy Server: One of the superpowers of Oathkeeper is its ability to sit at the forefront of your infrastructure and denying unauthorized requests.
• Decision Maker: Another mode of running Ory Oathkeeper is to use it as a policy enforcer, making decisions on whether or not a request should be granted access based on the defined rules.
• Open Source: Ory Oathkeeper is opensource with a permissive license, meaning you can inspect the source code, contribute to it, and even fork it if you want to.
• Stateless: Ory Oathkeeper is stateless, meaning it doesn't store any session data. This is a good thing because it makes it horizontally scalable and easy to deploy in a distributed environment.
• Pluggable: Ory products are adhering to plugin architecture; you can use all of them, some of them, or only one of them. This allows a lot of flexibility when migrating from a current solution or integrating with a third party service. This is the key feature that makes the entire suite very appealing to me.
• Full Featured: It comes with batteries included, providing experimental support for gRPC middleware¹ (if you're into Golang), and also stable support for WebSockets².
• Community: Ory has a great community of developers and users. If you ever get stuck, you can always ask for help in the community Slack channel³.

In short and more accurately put, Ory Oathkeeper is an Identity and Access Proxy (IAP for short)⁴. You will see later in this post how that comes into play.

How Does Ory Oathkeeper Work?¶

There are two modes of operation for Ory Oathkeeper:

1. Reverse Proxy Mode: Accepting raw traffics from the client and forwarding it to the upstream server.
2. Decision Maker Mode: Making decisions whether or not a request should be granted access. The frontier proxy server will query the decision API of Oathkeeper to grant or deny a request⁵.

Both of these modes rely solely on the API access rules written in human-readable YAML format⁶.(1) You can pass multiple rules to be applied for multiple upstream servers or backends.

1. Some will disagree with YAML files being human-readable . Though we are not in the business of picking sides, we are here to provide a technical guide.

After this rather long introductory, let's get our hands dirty and roll up our system administration sleeves.

Pre-requisites¶

This guide will be built upon our earlier infrastructure setup. You are more than welcome to stick to your own technology stack, however, since we had to deploy and make sure everything works perfectly, we picked our preferred stack as you see below:

• Kubernetes: Though you don't have to, this guide is built on top of Kubernetes, heavily relying on the operator pattern and the CRDs used to deploy our infrastructure as well as the Oathkeeper rules.
• cert-manager: We will need internet access to our cluster with TLS certificate from a trusted CA. That is where cert-manager and Gateway API are lending a generous hand. Take a look at cert-manager: All-in-One Kubernetes TLS Certificate Manager if you don't have it set up yet.
• External Secrets: For fetching the TLS certificates, we need access to our DNS provider (Cloudflare in this case). That's where we need the ESO to provide the API token for the cert-manager. Our earlier guide is a perfect place to set it up if you haven't already: External Secrets Operator: Fetching AWS SSM Parameters into Azure AKS.
• Ory Kratos: The main and most important part of this guide is the integration of Oathkeeper with Kratos. We will use Kratos to authenticate the users. Ultimately the requests reaching the upstream server will be authenticated and granted access by two of Ory's products. If you need help setting up Kratos, refer to Ory Kratos: Headless Authentication, Identity and User Management
• FluxCD: This is our technology of choice when it comes to Kubernetes deployments. You are free to pick simpler tools such as Helm CLI. FluxCD is a great tool that has a bit of learning curve. Check out our guide on GitOps Demystified: Introduction to FluxCD for Kubernetes if you need a starting point or GitOps Continuous Deployment: FluxCD Advanced CRDs if you are an advanced user.
• Jaeger: For the observation of traces between Oathkeeper and Kratos we get to configure both to send their traces to a custom endpoint. That's where Jaeger comes into play.
• Azure account: Optionally, an Azure VM having system assigned identity attached to it. We have covered this specific case two weeks ago in How to Access AWS From Azure VM Using OpenID Connect. We use this in the last part of this guide to send authenticated requests to Oathkeeper from an Azure VM.

Deploying Ory Oathkeeper¶

Being an stateless application by nature, Oathkeeper is a perfect fit for Kubernetes Deployment. This allows us to horizontally scale it on demand.

Let's write the server configuration and then deploy it to our Kubernetes cluster using FluxCD.

Oathkeeper Server Configuration¶

As with every other of Ory products, Oathkeeper relies heavily on its configuration file. This is, as usual, written in YAML format, making it easy to read and maintain as the complexity grows.

The following configuration is fetched in its entirety from the configuration reference⁷ and customized to fit our need.

An important thing to note is that we are only using the authenticators and authorizers that are being used in this blog post. The rest and the complete reference can be found in the official documentation⁷.

access_rules:
  matching_strategy: regexp
  repositories:
    - file:///etc/rules/access-rules.json
authenticators:
  cookie_session:
    config:
      check_session_url: http://kratos-public.auth/sessions/whoami
      preserve_path: true
      preserve_query: true
    enabled: true
  jwt:
    config:
      jwks_urls:
        - https://login.windows.net/common/discovery/keys
    enabled: true
  anonymous:
    config:
      subject: guest
    enabled: true
errors:
  handlers:
    redirect:
      config:
        to: https://ory.developer-friendly.blog/login
        when:
          - error:
              - unauthorized
        code: 301
      enabled: true
    json:
      config:
        verbose: true
      enabled: true
    www_authenticate:
      enabled: true
  fallback:
    - json
authorizers:
  allow:
    enabled: true
mutators:
  header:
    config:
      headers:
        x-user-id: "{{ print .Subject }}"
    enabled: true
log:
  format: json
  leak_sensitive_values: true
  level: info
tracing:
  service_name: oathkeeper
  providers:
    jaeger:
      sampling:
        trace_id_ratio: 1.0
        server_url: http://jaeger.monitoring:5778/sampling
      local_agent_address: jaeger.monitoring:6831
  provider: jaeger
serve:
  proxy:
    host: 0.0.0.0
    cors:
      allowed_origins:
        - https://*.developer-friendly.blog
      allow_credentials: true
      enabled: true
    port: 4455
  prometheus:
    host: 0.0.0.0
    port: 9000
  api:
    host: 0.0.0.0
    port: 4456

There are a lot to unpack. We may not be able to cover all, but let's explain some of the highlights.

Allowed Authentication Methods¶

Oathkeeper accepts in its configuration, the methods allowed for authentication and authorization. If you wish to use OAuth2 authentication, before using it in a Oathkeeper rule, you have to enable it in the configuration.

authenticators:
  cookie_session:
    config:
      check_session_url: http://kratos-public.auth/sessions/whoami
      preserve_path: true
      preserve_query: true
    enabled: true
  jwt:
    config:
      jwks_urls:
        - https://login.windows.net/common/discovery/keys
    enabled: true
  anonymous:
    config:
      subject: guest
    enabled: true

You can customize each method further with specific values. However, we will leave the customization to the Oathkeeper rule later in this blog. The URLs, however, are a required field and must be specified at the configuration level.

There will be more blog posts including Ory Oathkeeper and other authenticators and authorizers in the future. Stay tuned for more.

Tracing Endpoints¶

In all of the Ory products, you can specify where to ship your traces to.

That is possible through the same configuration over all the (currently) four products as below:

tracing:
  service_name: oathkeeper
  providers:
    jaeger:
      sampling:
        trace_id_ratio: 1.0
        server_url: http://jaeger.monitoring:5778/sampling
      local_agent_address: jaeger.monitoring:6831
  provider: jaeger

CORS Configuration¶

If you're accessing the Oathkeeper from the browser, you have to set the allowed origin addresses in the configurations.

Those "allowed" URLs are the hostnames that is in the address bar of a browser. If you specify a wildcard, Oathkeeper will intelligently allow the concrete value coming from the Origin header of the request.

For example, if *.developer-friendly.blog is in the allowed origins and the browser sends the request with Origin: example.developer-friendly.blog, the Oathkeeper will respond with Access-Control-Allow-Origin: example.developer-friendly.blog. Same goes with other subdomains.

The allow_credentials: true is perhaps the second most important part of this configuration. Without it your browser will not forward the cookies to the Oathkeeper server and you will always get a 401 Unauthorized response.

That part that makes it possible is in these configuration lines:

    cors:
      allowed_origins:
        - https://*.developer-friendly.blog
      allow_credentials: true
      enabled: true

Kubernetes Deployment Resources¶

There are different ways to deploy Oathkeeper⁸ and it highly depends on your infrastructure more than anything else.

In this blog post, we will refrain from using Docker Compose as that is something publicly available in the corresponding repository⁹ and example repository¹⁰.

Instead, we will share how to deploy Ory Oathkeeper in a Kubernetes cluster using FluxCD. We have in-depth guide on both topics in our archive if you're new to them.

As of writing this blog post, with the help of Oathkeeper Maester¹¹, Ory Oathkeeper has native Kubernetes support for rules, i.e., you can create Kubernetes resources to have Oathkeeper rules.

However, this will require proper Kubernetes RBAC as you see below.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: oathkeeper-maester-role
rules:
  - apiGroups:
      - oathkeeper.ory.sh
    resources:
      - rules
    verbs:
      - "*"
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - get
      - list
      - watch
      - create
      - patch
      - update

apiVersion: v1
kind: ServiceAccount
metadata:
  name: oathkeeper-maester

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: oathkeeper-maester-role-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: oathkeeper-maester-role
subjects:
  - kind: ServiceAccount
    name: oathkeeper-maester

The following two Deployment resources are the core of our Kustomization stack.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: oathkeeper-maester
spec:
  selector:
    matchLabels:
      app: oathkeeper-maester
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: oathkeeper-maester
    spec:
      automountServiceAccountToken: true
      containers:
        - args:
            - --metrics-addr=0.0.0.0:8080
            - controller
            - --rulesConfigmapName=oathkeeper-rules
            - --rulesConfigmapNamespace=FILLED_BY_KUSTOMIZATION
          command:
            - /manager
          image: oryd/oathkeeper-maester
          name: oathkeeper-maester
          ports:
            - containerPort: 8080
              name: metrics
              protocol: TCP
          securityContext:
            capabilities:
              drop:
                - ALL
            readOnlyRootFilesystem: true
            runAsNonRoot: true
            seccompProfile:
              type: RuntimeDefault
      serviceAccountName: oathkeeper-maester

apiVersion: apps/v1
kind: Deployment
metadata:
  name: oathkeeper
spec:
  selector:
    matchLabels:
      app: oathkeeper
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: oathkeeper
    spec:
      containers:
        - args:
            - serve
            - --config
            - /etc/oathkeeper/config.yml
          command:
            - oathkeeper
          image: oryd/oathkeeper
          livenessProbe:
            failureThreshold: 5
            httpGet:
              httpHeaders:
                - name: Host
                  value: 127.0.0.1
              path: /health/alive
              port: http-api
              scheme: HTTP
            initialDelaySeconds: 1
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          name: oathkeeper
          ports:
            - containerPort: 4456
              name: http-api
              protocol: TCP
            - containerPort: 4455
              name: http-proxy
              protocol: TCP
            - containerPort: 9000
              name: http-metrics
              protocol: TCP
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /health/alive
              port: http-api
              scheme: HTTP
            initialDelaySeconds: 1
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          securityContext:
            capabilities:
              drop:
                - ALL
            readOnlyRootFilesystem: true
            runAsNonRoot: true
          volumeMounts:
            - mountPath: /etc/oathkeeper
              name: oathkeeper-config
              readOnly: true
            - mountPath: /etc/rules
              name: oathkeeper-rules
              readOnly: true
      volumes:
        - configMap:
            defaultMode: 0444
            name: oathkeeper-config
          name: oathkeeper-config
        - configMap:
            defaultMode: 0444
            name: oathkeeper-rules
          name: oathkeeper-rules

We won't need to expose Oathkeeper Maester, but we require the Oathkeeper to be accessible to the cluster. Hence the Services below.

apiVersion: v1
kind: Service
metadata:
  name: oathkeeper-api
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: http-api
  selector:
    app: oathkeeper
  type: ClusterIP

apiVersion: v1
kind: Service
metadata:
  name: oathkeeper-proxy
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: http-proxy
  selector:
    app: oathkeeper
  type: ClusterIP

Now let's put this all together into a Kustomization file.

configMapGenerator:
  - name: oathkeeper-config
    files:
      - config.yml=oathkeeper-server-config.yml

images:
  - name: oryd/oathkeeper
    newTag: v0.40.7
  - name: oryd/oathkeeper-maester
    newTag: v0.1.10-arm64

replacements:
  - source:
      kind: Deployment
      name: oathkeeper
      fieldPath: metadata.namespace
    targets:
      - select:
          kind: Deployment
          name: oathkeeper-maester
        fieldPaths:
          - spec.template.spec.containers.[name=oathkeeper-maester].args.3
        options:
          delimiter: "="
          index: 1
          create: false
      - select:
          kind: ClusterRoleBinding
          name: oathkeeper-maester-role-binding
        fieldPaths:
          - subjects.[kind=ServiceAccount].namespace
        options:
          create: true

resources:
  - https://github.com/ory/oathkeeper-maester//config/crd?timeout=30s&ref=v0.1.10
  - clusterrole.yml
  - clusterrolebinding.yml
  - service-oathkeeper-api.yml
  - service-oathkeeper-metrics.yml
  - service-oathkeeper-proxy.yml
  - serviceaccount-maester.yml
  - deployment-oathkeeper-maester.yml
  - deployment-oathkeeper.yml

Notice the CRD installation in the Kustomization file. Without it, this stack won't be deployed properly.

resources:
  - https://github.com/ory/oathkeeper-maester//config/crd?timeout=30s&ref=v0.1.10

FluxCD Deployment Kustomization¶

To deploy the Oathkeeper, we need one last YAML file.

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: oathkeeper
  namespace: flux-system
spec:
  force: false
  healthChecks:
    - kind: Deployment
      name: oathkeeper
    - kind: Deployment
      name: oathkeeper-maester
  interval: 5m
  path: ./oathkeeper
  prune: true
  sourceRef:
    kind: GitRepository
    name: flux-system
  targetNamespace: auth
  timeout: 1m
  wait: true

Now let's deploy this into our cluster.

kubectl apply -f oathkeeper/kustomize.yml

This will take a few moments to pull the images and prepare everything. In the end, you should have the stack ready as you see in the screenshot below.

That is the VSCode Extension by the way. It can give a pretty good visual at needed times¹².

Is There a Simpler Way?¶

Of course there is.

There is almost always a way to less complexity, especially those complexities that are accidental!

The following Helm commands do all the things we worked so hard to achieve in the previous sections¹³.

helm repo add ory https://k8s.ory.sh/helm/charts
helm install oathkeeper ory/oathkeeper \
    --set oathkeeper.managedAccessRules=false \
    --version 0.43.x

Pick whichever you like. The decision between extensive customization and simplicity is yours to make.

Upstream Server Deployment¶

Whichever method you have employed to deploy Oathkeeper, you should have the server up and running.

At this point we will start using the Oathkeeper by creating rules and testing the authentication of our upstream server.

To make sure the test is realistic and with a concrete example, we will deploy our previous echo server example, an opensource project that echoes back the request it receives¹⁴.

To save space for the actual content, we will only provide the Deployment definition.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: echo-server
  template:
    metadata:
      labels:
        app: echo-server
    spec:
      containers:
        - envFrom:
            - configMapRef:
                name: echo-server
          image: ealen/echo-server
          name: echo-server
          ports:
            - containerPort: 80
              name: http
          resources:
            limits:
              cpu: 100m
              memory: 128Mi
            requests:
              cpu: 50m
              memory: 64Mi
          securityContext:
            capabilities:
              add:
                - NET_BIND_SERVICE
              drop:
                - ALL
            readOnlyRootFilesystem: true
      securityContext:
        runAsGroup: 1000
        runAsUser: 1000
        seccompProfile:
          type: RuntimeDefault

If you're interested to see the entire resources, click below.

PORT=80
LOGS__IGNORE__PING=false
ENABLE__HOST=true
ENABLE__HTTP=true
ENABLE__REQUEST=true
ENABLE__COOKIES=true
ENABLE__HEADER=true
ENABLE__ENVIRONMENT=false
ENABLE__FILE=false

apiVersion: v1
kind: Service
metadata:
  name: echo-server
spec:
  ports:
    - name: http
      port: 80
      targetPort: http
  type: ClusterIP

configMapGenerator:
  - envs:
      - configs.env
    name: echo-server

images:
  - name: ealen/echo-server
    newTag: 0.9.2

replacements:
  - source:
      kind: Deployment
      name: echo-server
      fieldPath: spec.template.metadata.labels
    targets:
      - select:
          kind: Service
          name: echo-server
        fieldPaths:
          - spec.selector
        options:
          create: true
  - source:
      kind: ConfigMap
      name: echo-server
      fieldPath: data.PORT
    targets:
      - select:
          kind: Deployment
          name: echo-server
        fieldPaths:
          - spec.template.spec.containers.[name=echo-server].ports.[name=http].containerPort

resources:
  - deployment.yml
  - service.yml

namespace: default

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: echo-server
  namespace: flux-system
spec:
  force: false
  interval: 5m
  path: ./echo-server
  prune: true
  sourceRef:
    kind: GitRepository
    name: flux-system
  targetNamespace: default
  wait: true

kubectl apply -f echo-server/kustomize.yml

Oathkeeper Rules¶

The final section of this blog post is the main part. Everything we've built so far was a preparation for this moment.

Internet Accessible Endpoint¶

The first step is to route all the traffic targeting the upstream server to the Oathkeeper proxy endpoint. Based on different types of deployments, you may end up executing this step differently.

But, if you have followed along with the guide so far, we need two pieces to make this come to life:

1. A DNS record for the host we want to expose.
2. The Kubernetes HTTPRoute resource that will accept the incoming traffics.

We will skip the first part as that is something we have covered multiple times in this blog and it is tailored to your DNS provider.

The second part is as you see below.

Notice that this HTTPRoute has to be in the same namespace as the Oathkeeper. The reason is that the Gateway will only route the traffics to the same namespace as the HTTPRoute¹⁵.

In short, we send the internet traffics to the Oathkeeper, and if all looks OK (the request is authenticated and whatnot), it will forward the request to the upstream server.

Otherwise, the user will get the proper error message from Oathkeeper before even a single byte reaches the upstream server. That is the true power of Oathkeeper.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: echo-server
spec:
  hostnames:
    - echo.developer-friendly.blog
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: developer-friendly-blog
      namespace: cert-manager
      sectionName: https
  rules:
    - backendRefs:
        - kind: Service
          name: oathkeeper-proxy
          port: 80
      filters:
        - responseHeaderModifier:
            set:
              - name: Strict-Transport-Security
                value: max-age=31536000; includeSubDomains; preload
          type: ResponseHeaderModifier
      matches:
        - path:
            type: PathPrefix
            value: /

Let's deploy this as a Kustomization stack.

resources:
  - httproute.yml

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: echo-server-rule
  namespace: flux-system
spec:
  force: false
  interval: 5m
  path: ./echo-server-rule
  prune: true
  sourceRef:
    kind: GitRepository
    name: flux-system
  targetNamespace: auth
  wait: true

kubectl apply -f echo-server-rule/kustomize.yml

Play 1: Anonymous Access¶

The first rule we will create is to allow anonymous access to the upstream server. This is a common use-case where you want to allow everyone to access the server without any authentication.

apiVersion: oathkeeper.ory.sh/v1alpha1
kind: Rule
metadata:
  name: echo-server
spec:
  authenticators:
    - handler: anonymous
  authorizer:
    handler: allow
  errors:
    - handler: json
  match:
    methods:
      - GET
      - POST
      - PUT
      - PATCH
      - DELETE
      - OPTIONS
    url: http<s?>://echo.developer-friendly.blog</?.*>
  mutators:
    - handler: header
  upstream:
    preserveHost: true
    url: http://echo-server.default

The flow of the request is as follows¹⁶:

Is the request authenticated? Yes, it is anonymous.

  authenticators:
    - handler: anonymous

Is it authorized? Yes, the rule allows access to everyone.

  authorizer:
    handler: allow

Do we need to change anything in the request? Yes, add a single x-user-id header (guest for anonymous).

  mutators:
    - handler: header

What if error happens before reaching upstream? Return the error as JSON.

  errors:
    - handler: json

The flow you see above is the most important part of how Ory Oathkeeper works. If you master this flow, you can create any kind of rule you want.

Notice that in our Rule definition, we are specifying the upstream_url as the <service_name>.<namespace>. This is due to the fact that the Oathkeeper stack and our echo-server are in separate namespaces.

Also note that the match.url is using regex format. This is only possible if you have access_rules.matching_strategy: regexp in your Oathkeeper server configuration.

Let's apply this stack:

resources:
  - httproute.yml
  - rule.yml

Let's send an anonymous request to verify it worked.

curl https://echo.developer-friendly.blog

The response is as below.

{
  "host": {
    "hostname": "echo.developer-friendly.blog",
    "ip": "::ffff:20.0.0.195",
    "ips": []
  },
  "http": {
    "baseUrl": "",
    "method": "GET",
    "originalUrl": "/",
    "protocol": "http"
  },
  "request": {
    "body": {},
    "cookies": {},
    "headers": {
      "accept": "*/*",
      "accept-encoding": "gzip",
      "b3": "c2720bd50c24d2254f9e593737409cff-3214b34f361875d0-1",
      "host": "echo.developer-friendly.blog",
      "traceparent": "00-c2720bd50c24d2254f9e593737409cff-3214b34f361875d0-01",
      "uber-trace-id": "c2720bd50c24d2254f9e593737409cff:3214b34f361875d0:0:1",
      "user-agent": "curl/7.81.0",
      "x-b3-sampled": "1",
      "x-b3-spanid": "3214b34f361875d0",
      "x-b3-traceid": "c2720bd50c24d2254f9e593737409cff",
      "x-envoy-expected-rq-timeout-ms": "15000",
      "x-envoy-internal": "true",
      "x-request-id": "97d77949-6ef0-46b0-86d3-59eb32344fe3",
      "x-user-id": "guest"
    },
    "params": {
      "0": "/"
    },
    "query": {}
  }
}

The user ID is coming from the following Oathkeeper server configuration.

  anonymous:
    config:
      subject: guest
    enabled: true

Play 2: Authenticated by Ory Kratos¶

At this stage, we should be able to use our previously deployed Ory Kratos server.

Let's modify the same rule so that the authenticated users and the identities of Kratos can send their request to this upstream server¹⁷.

apiVersion: oathkeeper.ory.sh/v1alpha1
kind: Rule
metadata:
  name: echo-server
spec:
  authenticators:
    - config:
        check_session_url: http://kratos-public.auth/sessions/whoami
        extra_from: "@this"
        force_method: GET
        only:
          - ory_kratos_session
        subject_from: identity.id
      handler: cookie_session
    - handler: anonymous
  authorizer:
    handler: allow
  errors:
    - handler: json
  match:
    methods:
      - GET
      - POST
      - PUT
      - PATCH
      - DELETE
      - OPTIONS
    url: http<s?>://echo.developer-friendly.blog</?.*>
  mutators:
    - handler: header
  upstream:
    preserveHost: true
    url: http://echo-server.default

Note that this will only work for browser users of Kratos. For mobile native clients using the API flow of Kratos, you'd want to use the X-Session-Token header instead of the cookie¹⁷.

If we authenticate to Kratos first and send an HTTP request to the echo-server, this is what we get.

{
  "host": {
    "hostname": "echo.developer-friendly.blog",
    "ip": "::ffff:20.0.0.67",
    "ips": []
  },
  "http": {
    "baseUrl": "",
    "method": "GET",
    "originalUrl": "/",
    "protocol": "http"
  },
  "request": {
    "body": {},
    "cookies": {
      "csrf_token_1c7f8e9fce29a4477c061c763377284ec29475bad1819ea48f4f48bb479dd449": "0AKt9i0pcgopFl2DbnuNsqxMC1/l0sqwOjZQtupwGhs=",
      "ory_kratos_session": "MTcxNzkyNDUyOHxxcGpkbkhqOW4zTFZaQUZtVFRxd2pQZ3F2UGFxQUlOalAyd18xcUZjVTh6U1pvQ1ZGd2dBMmRtVFBGSnkzY3ZkeDJRRnZvR29KclZ0a2E3RHFxN0NBV2ktMDNzcm1lcldGUndWTFBCSExVM1VKV2tfbzhNMUVmZDdMZ2wwUW1JSmI0UW96MGpGLXc0YU5xc095ZlJ0SzRiVkhKTkdBcUR0ZTZoYVYxd1dHNy1YM01oTDluYURZU09ZRUI1d2lpejFhcGtNenI5VkEtcHRTQkRIQ2xOUGcySWs2R3RvMlpURHZrODVoZVlacVlpSzNBVnNRZDg4MDJnMGRXaVNBbDVNN2pHeVZCMFVVMTZzQ2p2Y1U3RDF8GAjcTd0iTmLRfV2tDWxCbgz9Wau3WGGVI_wSJcBUt54="
    },
    "headers": {
      "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
      "accept-encoding": "gzip, deflate, br, zstd",
      "accept-language": "en-US",
      "b3": "19f74b8e495e129aa2b70f16a840d060-4b191ecca0c43e9f-1",
      "cookie": "csrf_token_1c7f8e9fce29a4477c061c763377284ec29475bad1819ea48f4f48bb479dd449=0AKt9i0pcgopFl2DbnuNsqxMC1/l0sqwOjZQtupwGhs=; ory_kratos_session=MTcxNzkyNDUyOHxxcGpkbkhqOW4zTFZaQUZtVFRxd2pQZ3F2UGFxQUlOalAyd18xcUZjVTh6U1pvQ1ZGd2dBMmRtVFBGSnkzY3ZkeDJRRnZvR29KclZ0a2E3RHFxN0NBV2ktMDNzcm1lcldGUndWTFBCSExVM1VKV2tfbzhNMUVmZDdMZ2wwUW1JSmI0UW96MGpGLXc0YU5xc095ZlJ0SzRiVkhKTkdBcUR0ZTZoYVYxd1dHNy1YM01oTDluYURZU09ZRUI1d2lpejFhcGtNenI5VkEtcHRTQkRIQ2xOUGcySWs2R3RvMlpURHZrODVoZVlacVlpSzNBVnNRZDg4MDJnMGRXaVNBbDVNN2pHeVZCMFVVMTZzQ2p2Y1U3RDF8GAjcTd0iTmLRfV2tDWxCbgz9Wau3WGGVI_wSJcBUt54=",
      "dnt": "1",
      "host": "echo.developer-friendly.blog",
      "priority": "u=4",
      "sec-fetch-dest": "document",
      "sec-fetch-mode": "navigate",
      "sec-fetch-site": "cross-site",
      "sec-gpc": "1",
      "traceparent": "00-19f74b8e495e129aa2b70f16a840d060-4b191ecca0c43e9f-01",
      "uber-trace-id": "19f74b8e495e129aa2b70f16a840d060:4b191ecca0c43e9f:0:1",
      "upgrade-insecure-requests": "1",
      "user-agent": "Mozilla/5.0 (X11; Linux x86_64; rv:126.0) Gecko/20100101 Firefox/126.0",
      "x-b3-sampled": "1",
      "x-b3-spanid": "4b191ecca0c43e9f",
      "x-b3-traceid": "19f74b8e495e129aa2b70f16a840d060",
      "x-envoy-expected-rq-timeout-ms": "15000",
      "x-envoy-internal": "true",
      "x-request-id": "5b5bedfd-ac4b-4ae3-a87f-952cf2b55453",
      "x-user-id": "90b31539-08ba-45b1-a5fc-5de840f19d7e"
    },
    "params": {
      "0": "/"
    },
    "query": {}
  }
}

The x-user-id in this response is just the same as if we take the session information from the Kratos itself.

Play 3: Azure VM Access¶

The idea in this scenario is that the virtual machine in the Azure cloud with system assigned identity can send authenticated requests to the echo-server, while Oathkeeper verifies the authenticity of the request using the Azure AD JWKs endpoint.

apiVersion: oathkeeper.ory.sh/v1alpha1
kind: Rule
metadata:
  name: echo-server
spec:
  authenticators:
    - config:
        check_session_url: http://kratos-public.auth/sessions/whoami
        extra_from: "@this"
        force_method: GET
        only:
          - ory_kratos_session
        subject_from: identity.id
      handler: cookie_session
    - config:
        jwks_urls:
          - https://login.windows.net/common/discovery/keys
        target_audience:
          - https://management.azure.com
      handler: jwt
    - handler: anonymous
  authorizer:
    handler: allow
  errors:
    - handler: json
  match:
    methods:
      - GET
      - POST
      - PUT
      - PATCH
      - DELETE
      - OPTIONS
    url: http<s?>://echo.developer-friendly.blog</?.*>
  mutators:
    - handler: header
  upstream:
    preserveHost: true
    url: http://echo-server.default

Azure AD JWT Audience¶

The target_audience you see in this rule will be identical to the aud claim in the JWT token of the Azure VM. To make sure you get it right, you can fetch the access token from within the VM¹⁸, decode its content and check the aud claim.

Azure AD JWKs Endpoint¶

How to get the jwks_url one might ask!? The answer is simple. If you use the same technique to decode the said token, you will see its iss claim. Using that issuer, you can append /.well-known/openid-configuration to get the OpenID Connect configuration of the server, and in the JSON payload response, the JWKs URL will be present.

Using this Oathkeeper rule, if we spin up an Azure VM and enable its system identity, we can get a JWT token¹⁹ from Azure AD and send it to Oathkeeper.

# From inside the Azure VM
token=$(curl \
  'http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fmanagement.azure.com%2F' \
  -H Metadata:true -s)

# or
token=$(az account get-access-token \
  --resource https://management.azure.com \
  --query "accessToken" -o tsv)

curl https://echo.developer-friendly.blog -H "Authorization $token"

The response will be as you see below. The x-user-id is the subject or the identity ID that the Identity Provider (Azure AD) knows the VM by.

{
  "host": {
    "hostname": "echo.developer-friendly.blog",
    "ip": "::ffff:20.0.0.67",
    "ips": []
  },
  "http": {
    "baseUrl": "",
    "method": "GET",
    "originalUrl": "/",
    "protocol": "http"
  },
  "request": {
    "body": {},
    "cookies": {},
    "headers": {
      "accept": "*/*",
      "accept-encoding": "gzip, deflate",
      "authorization": "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJhNGNkZGNmMC1kMjAwLTQ3M2ItYjdkMi1lMjQ1ZTgyOWRiMjkifQ.KSOICXTqr9TF7ltpaKloAOnQZIPXK-wz3mKoHI4zF7k",
      "b3": "a26738ea41facd800a9a99dc2d193fe5-1bc4f24219265111-1",
      "host": "echo.developer-friendly.blog",
      "traceparent": "00-a26738ea41facd800a9a99dc2d193fe5-1bc4f24219265111-01",
      "uber-trace-id": "a26738ea41facd800a9a99dc2d193fe5:1bc4f24219265111:0:1",
      "user-agent": "curl/7.81.0",
      "x-b3-sampled": "1",
      "x-b3-spanid": "1bc4f24219265111",
      "x-b3-traceid": "a26738ea41facd800a9a99dc2d193fe5",
      "x-envoy-expected-rq-timeout-ms": "15000",
      "x-envoy-internal": "true",
      "x-request-id": "ffae2311-da88-4a3c-9b33-0b29a873bc3c",
      "x-user-id": "a4cddcf0-d200-473b-b7d2-e245e829db29"
    },
    "params": {
      "0": "/"
    },
    "query": {}
  }
}

Other Types of Authenticators¶

As you can imagine, these are just a few examples of the capabilities of Oathkeeper and what it can bring to your table.

There are other types of authenticators, as well as authorizers, that can be used to gate an application behind a proxy server, granting access only to those trusted parties that you have explicitly defined.

As an example, if we remove the anonymous access from our rule, the request will be denied with a 401 Unauthorized status code.

{
  "error": {
    "code": 401,
    "message": "Access credentials are invalid",
    "request": "d1b7eadf-1383-4951-a162-f35717e3ee1b",
    "status": "Unauthorized"
  }
}

That is to say, the order in which Oathkeeper processes the rules is important. It will start from the top, and any authenticator that can handle the authentication process will be the only one ever consulted and the rest of the authenticators are comletely ignored, even if the matched authentication denies the request! Yes, even if any of the subsequent authenticators would have allowed the request. That's something to keep in mind when designing and creating your rules.

From the official documentation²⁰:

authenticators: A list of authentication handlers that authenticate the provided credentials. Authenticators are checked iteratively from index 0 to n and the first authenticator to return a positive result will be the one used. If you want the rule to first check a specific authenticator before "falling back" to others, have that authenticator as the first item in the array.

Observing the Traces¶

Remember we mentioned how use you can ship your traces with Ory products into a backend? Among different available solutions, Jaeger is one of the easy ones to set up and use.

When we engage both Oathkeeper and Kratos in one HTTP request, this is what we get in the Jaeger UI.

Pretty neat! Wouldn't you say?

Wrapping Up¶

All in all, Oathkeeper design and secure implementation will help you sleep tight at night, knowing that your application is safely guarded by a production grade and robust proxy server, consulting the proper authentication server before handing it to the upstream backend.

These days, I use the Oathkeeper even for my admin pages; even the ones not publicly accessible and only exposed to the private network. This helps secure the backend from unauthorized access.

There are other types of examples we can provide here, but with the ones you see here, you should have a good idea on what's possible and what you can do more with Ory Oathkeeper.

Regardless, we will have more examples of this topic in the future for other practical and production use-cases.

Conclusion¶

Based on my production experience over the years managing different types of applications and backends in various industries, there is the same pattern and approach for a desired authentication layer one might want to have.

It usually includes some sort of cunsultation with the Identity Provider, making sure the identity is coming from a trusted source, and then tightening it further by making an API call to the authorization server, making sure the identity is indeed allowed and granted access to such resource.

The plugin architecture of Ory makes this back and forth quite straightforward. There is little you can't do with the Ory suite and with the right configuration and architectural mindset, you can secure many of the knowingly hard-to-protect applications.

I can't recommend their products highly enough, being a happy customer and whatnot. But, even more so, knowing that it's easy to fall into the trap of thinking that one's security and authentication needs are beyond the common pattern happening around the industry

It's tempting to think that customization and in-house development is in order. That is wrong, in my humble opinion. You will lose countless engineering hours making something not nearly as secure as what is already available as an off-the-shelf and opensource solution.

Before investing many of your engineering efforts building something from scratch, I highly recommend trying Ory's products in the tenth of that time.

Ory is only one of the many solutions out there, but at the very least, you should have a basic understanding of what is already available to you around the industry before going all in on a custom solution.

Make your decisions wisely, and do the right things before doing things right.

Happy hacking and until next time , ciao.

If you enjoyed this blog post, consider sharing it with these buttons . Please leave a comment for us at the end, we read & love 'em all.

Share on Share on Share on Share on

Comments