Simple Dockerized gRPC Application with Envoy ext_authz Example

Envoy Proxy and gRPC are two immensely popular and useful open-source technologies with wide adoption, including by major enterprises.

So, when I set out to spin up a simple, locally runnable gRPC application where requests were mediated and authenticated via Envoy, I was mostly expecting to, you know, just copy and paste various example files into a few directories and docker-compose up with a self-satisfied smile.

For whatever reason, the exact combination of technologies I was exploring in this case didn’t show up in any easily discoverable tutorials or articles…until now!

This writeup will detail how to run the following: a gRPC client that makes a request to a dockerized golang server exposing a gRPC service. The request is proxied by Envoy and, by using a combination of HTTP and gRPC ext_authz filters, authenticated and authorized before arriving at the upstream server.

Prerequisites:

• Go (getting started guide)
• Protocol Buffer Compiler (installation guide)

Define Your RPC Service

In your project directory (/envoy-example in my case), make a directory for your proto definitions.

makedir protos && cd protos
touch hello.proto

Our service is going to be very straightforward: we will create a Hello service. In your hello.proto file, you should copy the following:

Protos provide concise definitions for structured data that can be used to generate code in a variety of languages. Today, we’re going to stick with golang. As a result, you may need to install the following Go plugins before continuing:

export GO111MODULE=on  # Enable module mode
go get google.golang.org/protobuf/cmd/protoc-gen-go \
         google.golang.org/grpc/cmd/protoc-gen-go-grpc
export PATH="$PATH:$(go env GOPATH)/bin"

Okay, now you’re ready to let protoc go to work:

protoc --go_out=. --go_opt=paths=source_relative \
    --go-grpc_out=. --go-grpc_opt=paths=source_relative \
    protos/hello.proto

What this does is generate the golang *and* the gRPC code to create a simple Hello service. Your directory should now look like:

/protos
  hello.proto
  hello.pb.go
  hello_grpc.pb.go

gRPC Server and Client

Before bringing Envoy and docker into the picture, let’s just get a simple request to succeed.

First of all, you will need to make your go module so that imports can be found:

// in your project root
go mod init envoy_example

Next, create a file /client/main.go

And /server/main.go

Now you should have the following directory structure:

go.mod
go.sum
/protos
  hello.proto
  hello.pb.go
  hello_grpc.pb.go
/server
  main.go
/client
  main.go

If you run the server and then the client, you should see something like:

Dockerize and Add Envoy

Let’s containerize this project, which gives us a good opportunity to add Envoy as well.

First, we’ll add the Dockerfile.server file to spin up a container for the gRPC backend:

Next, we can define our Dockerfile.envoy

Of course, a docker-compose.yaml to orchestrate:

And, finally, an envoy.yaml file:

Ok, that was a lot of files thrown at you! Let’s take a step back and make a few notes about what’s going on here.

First, though: remember to update the ./client/main.go file to dial the server at 1337 instead of the original port, because you are now no longer hitting the gRPC server directly—instead, we’re proxying our request via Envoy! A simple docker-compose up should confirm that everything is working: you’ll see the logs from envoy, and if you run your client code again, you should see the request logged by your Envoy container before seeing the response!

So, we have a client making a call to the dockerized Envoy sidecar. A few lines in the yaml defining our Envoy are worth noting:

listeners:
  - name: listener_0
    address:
      socket_address: { address: 0.0.0.0, port_value: 1337 }

We can see here that Envoy is listening on port 1337. That’s why our docker-compose.yaml file exposes container port 1337 to outside processes — and why our client now hits that port instead.

- match:
    prefix: "/" 
    grpc: {}
  route:  
    cluster: backend

This simply matches gRPC traffic and tells Envoy to route it to the backend cluster, which is defined in the clusters section at the bottom of the yaml.

If you were to, say, change the match prefix to something that specifically does NOT match your request (which is likely /hello.HelloService/Hello ), you’d find that Envoy returns a gRPC error: UNIMPLEMENTED—essentially the gRPC equivalent of 404 Not Found!

Introducing ext_authz

Envoy is more than just a proxy server—you can introduce filters that enable some pretty cool features, from rate limiting to logging to authentication and authorization.

In our case, we’ll be doing the latter with an ext_authz filter. In fact, we’re going to set up two of them, so that you can see how filters work together, and how to set up both gRPC and HTTP ext_authz filter services.

First, we’ll add two new services to our project.

gRPC ext_authz Service

Let’s go ahead and add another gRPC server that implements an ext_authz method Check (overview here). Start by adding another directory called extauth. Then, add yet another main.go file:

Yep, just another go server with a gRPC service that exposes Check. Now, the interesting thing to note here is that the request to Check is not the original request! If you were to examine the metadata context of the request, it’s a bunch of Envoy-related data.

The original request is wrapped in the req.GetAttributes().GetRequest() object. That’s where we look to get the metadata keys we originally set on the client context:

ctx := metadata.AppendToOutgoingContext(
  context.Background(), 
  "Authorization", "Bearer foo", 
  "Bar", "baz"
 )

Authorization and Bar will be set as HTTP headers on that embedded request. Kinda weird, but it works!

Here, we’re just checking the Authorization header for validity (is it three characters long? Very secure). Then, we attach a credential header with “permissions” to the request and return an OK to Envoy, which sees that and spirits along the request. Otherwise, we return a 401, which Envoy knows to transform into the appropriate gRPC error: UNAUTHENTICATED.

Now, we’ll also need to add and update some other files:

In our envoy.yaml, add the following cluster definition:

- name: extauth
  connect_timeout: 5s
  type: STRICT_DNS
  http2_protocol_options: {}
  lb_policy: round_robin
  hosts:
  - socket_address:
      address: extauth
      port_value: 4040

Also, your http_filters section should now look like this:

http_filters:
- name: envoy.ext_authz
  config:
    grpc_service:
      envoy_grpc:
        cluster_name: extauth
- name: envoy.router

The docker-compose.yaml file needs a new section as well:

extauth:
  build:
    context: .
    dockerfile: Dockerfile.extauth
  networks:
    - envoymesh
  expose:
    - "4040"
  ports:
    - "4040:4040"

And, as you may have guessed, a new Dockerfile.extauth:

That should do it! Go ahead and docker-compose up and give it a try. Update the Authorization header in your client call, and see how denials work. You’ll note plenty of useful logging in the envoy logs, as well!

So, now our little setup has grown a bit:

Now, what if you have multiple auth steps or more than one service that acts as part of your auth flow? One option would be consolidating your authentication and authorization logic in a single service. Another might be having the first service called by the ext_authz filter proxy any further calls itself.

Let’s go with an alternative, and simply introduce a final filter: one more ext_authz filter that will call a different service, so we can demo how filters are chained, as well as how to configure an HTTP service for ext_authz calls in addition to the RPC service we already implemented.

HTTP ext_authz Service

Alright, so we want to add a second server — let’s go ahead and add a new main.go file to a folder called extauth_http:

So here we have a simple HTTP server. The only handler performs another check: this time, rather than looking for an Authorization header, we can assume it’s already there and been authenticated correctly—otherwise, we wouldn’t have made it this far in the request flow!

Instead, we’ll look at the header added in the previous filter: we’ll look for my-credential-header and see if some arbitrarily designated “required” permission was included.

If so, congrats! This request will be forwarded on ahead to the upstream host. Otherwise, we’ll once again return a forbidden response, this time with a custom body that we defined to show how envoy will convert that to a gRPC message.

Now we’ll need to add the usual suspects again, including a new Dockerfile:

We’ll also add a new stanza to the docker-compose.yaml file:

httpauth:
  build:
    context: .
    dockerfile: Dockerfile.extauth_http
  networks:
    - envoymesh
  expose:
    - "10003"
  ports:
    - "10003:10003"

And, finally, a couple new envoy.yaml sections. The first is the filter config, which should go between the envoy.ext_authz filter and the envoy.router filter:

- name: envoy.filters.http.ext_authz
  typed_config:
    "@type": type.googleapis.com/envoy.config.filter.http.ext_authz.v2.ExtAuthz
    http_service:
      authorization_request: 
        allowed_headers: 
          patterns: 
          - prefix: "my-credential-header"
        server_uri:
          uri: 127.0.0.1:10003
          cluster: extauth_http
          timeout: 0.25s
     failure_mode_allow: false
     include_peer_certificate: true
  

One thing to note here is that we have to explicitly allow the header we’re setting in the first filter to be passed to the second. If we didn’t, only the original headers from the gRPC request’s MetadataContext would be provided to our HTTP server’s handler.

And lastly a stanza for the cluster definition:

- name: extauth_http
  connect_timeout: 0.25s
  type: STRICT_DNS
  lb_policy: round_robin
  hosts:
  - socket_address:
      address: host.docker.internal
      port_value: 10003

You can find the full envoy.yaml file here :)

Your final directory structure should look like:

go.mod
go.sum
envoy.yaml
docker-compose.yaml
Dockerfile.envoy
Dockerfile.extauth_http
Dockerfile.server
Dockerfile.extauth
/extauth
  main.go
/extauth_http
  main.go
/protos
  hello.proto
  hello.pb.go
  hello_grpc.pb.go
/server
  main.go
/client
  main.go

Putting it all together

Let’s go ahead and docker-compose up --build to build fresh containers for all this good stuff.

Barring any terrifying logs and failures that are surely not the fault of this impeccable guide, you should have the following setup running, indicated by voluminous logs:

You can play around with the headers you set in the client file and see how things get denied.

You can also mess around with how the ext_authz filters interact. Try and set a different “permission” in the gRPC ext_authz server, for example. This is a silly exercise in some ways, but imagine that that server was instead taking the auth header and looking up the permissions for that identity before adding them to the request. Similarly, the second filter you can imagine would be checking for specific permissions that apply to the upstream host it’s proxying for.

There’s a lot of ways you can go with all this. Luckily, now you have a full on sandbox environment to play with and experiment.

Have fun!