Skip to main content

Set up the Teleport Event Handler Plugin with the Teleport Kubernetes Operator

Report an Issue

In this guide, we will set up the Teleport Event Handler plugin with credentials to authenticate to the Teleport cluster and access the events API.

We will deploy the Teleport Event Handler plugin using the teleport-plugin-event-handler Helm chart and configure it to use the Teleport Kubernetes Operator to generate the credentials.

If your Teleport cluster does not have the Teleport Kubernetes Operator deployed, follow the guide to set up the Teleport Kubernetes Operator . If you wish to deploy the Teleport Event Handler Plugin in Kubernetes without the Teleport Kubernetes Operator, see Set up the Event Handler with tctl.

How it works​

The Event Handler plugin is a binary that runs independently of your Teleport cluster. It authenticates to your Teleport cluster using mutual TLS to begin forwarding events.

The Teleport Kubernetes Operator is a Teleport client that you install using the teleport-operator Helm chart. It is responsible for managing the Teleport resources to set up Machine & Workload Identity, tbot, for the teleport-plugin-event-handler Helm chart. This will generate the credentials to establish the mutual TLS connection between the Event Handler and the Teleport cluster.

Prerequisites​

  • A running Teleport cluster. If you want to get started with Teleport, sign up for a free trial or set up a demo environment.

  • Docker version v20.10.7.

  • A Kubernetes cluster to run the Teleport Event Handler plugin.

  • Teleport Kubernetes Operator running in the Kubernetes cluster where you will install the plugin.

  • Helm

  • kubectl

  • On your workstation, create a folder called event-handler, to hold configuration files and plugin state:

    mkdir -p event-handler
    cd event-handler

Step 1/2. Install the Event Handler plugin​

To allow Helm to install charts that are hosted in the Teleport Helm repository, use helm repo add:

helm repo add teleport https://charts.releases.teleport.dev

To update the cache of charts from the remote repository, run helm repo update:

helm repo update

Step 2/2. Set up the Event Handler plugin​

In this section, you will set up the Teleport Event Handler plugin and generate credentials that the plugin will use for authentication.

Generate a starter config file​

Generate a configuration file with placeholder values for the Teleport Event Handler plugin.

Run the configure command to generate a sample configuration. Assign teleport.example.com:443 to the DNS name and port of your Teleport Auth Service or Proxy Service. Assign fluentd.fluentd.svc.cluster.local to the DNS name of your log forwarder.

docker run -v `pwd`:/opt/teleport-plugin -w /opt/teleport-plugin public.ecr.aws/gravitational/teleport-plugin-event-handler:19.0.0-dev configure . teleport.example.com:443 --dns-names=fluentd.fluentd.svc.cluster.local

In order to export audit events, you'll need to have the root certificate and the client credentials available as a secret. Use the following command to create that secret in Kubernetes:

kubectl create secret generic teleport-event-handler-client-tls --from-file=ca.crt=ca.crt,client.crt=client.crt,client.key=client.key

This will pack the content of ca.crt, client.crt, and client.key into the secret so the Helm chart can mount them to their appropriate path.

The plugin generates several setup files:

File(s)Purpose
ca.crt and ca.keySelf-signed CA certificate and private key for Fluentd
server.crt and server.keyFluentd server certificate and key
client.crt and client.keyFluentd client certificate and key, all signed by the generated CA
teleport-event-handler-role.yamluser and role resource definitions for Teleport's event handler
teleport-event-handler.tomlExample event handler configuration
fluent.confFluentd plugin configuration
Running the Event Handler separately from the log forwarder

This guide assumes that you are running the Event Handler on the same host or Kubernetes pod as your log forwarder. If you are not, you will need to instruct the Event Handler to generate mTLS certificates for subjects besides localhost. To do this, use the --dns-names flag of the teleport-event-handler configure command.

For example, if your log forwarder is addressable at fluentd.example.com, you would run the following configure command:

teleport-event-handler configure --dns-names=fluentd.example.com

The --dns-names flag accepts a comma-separated list of DNS names. It will append subject alternative names (SANs) to the server certificate (the one you will provide to your log forwarder) for each DNS name in the list.

If you have an existing Fluentd setup with TLS, issue a client certificate and key from the same certificate authority for the Teleport Event Handler to use.

Configure tbot for the Event Handler plugin​

In the generated configuration file teleport-plugin-event-handler-values.yaml, replace the teleport section with the crd and tbot sections. Assign operator-namespace to the namespace where the Teleport Kubernetes Operator is running in. Assign teleport.example.com:443 to the DNS name and port of your Teleport Proxy Service:

# values.yaml
crd:
  create: true
  namespace: operator-namespace

tbot:
  enabled: true
  clusterName: teleport.example.com
  teleportProxyAddress: teleport.example.com:443
warning

Ensure the teleport section is removed, since tbot is performing the authentication to the cluster.

This will generate the Event Handler role and bot user for the Machine & Workload Identity bot, tbot, to authenticate as when connecting to your Teleport cluster.

Specify a join token for tbot​

The tbot deployment authenticates to the Teleport cluster using a join token created by the crd section. Depending on where you deploy the Event Handler plugin relative to your Teleport cluster, you may need to modify the join token specification for tbot to properly authenticate.

When running teleport-plugin-event-handler in the same Kubernetes cluster as the Teleport cluster (eg. via the teleport-cluster Helm chart), we can use the default join token specification: kubernetes in_cluster join method.

Example configuration file​

Your helm configuration file teleport-plugin-event-handler-values.yaml should now contain settings similar to the following. We will configure the fluentd section to your log management solution of choice in the following guide.

eventHandler:
  storagePath: "./storage"
  timeout: "10s"
  batch: 20
  # concurrency is the number of concurrent sessions to process. By default, this is set to 5.
  concurrency: 5
  # The window size configures the duration of the time window for the event handler
  # to request events from Teleport. By default, this is set to 24 hours.
  # Reduce the window size if the events backend cannot manage the event volume
  # for the default window size.
  # The window size should be specified as a duration string, parsed by Go's time.ParseDuration.
  windowSize: "24h"
  # types is a list of event types to search when forwarding audit
  # events. For example, to limit forwarded events to user logins
  # and new Access Requests, you can assign this field to:
  # ["user.login", "access_request.create"]
  types: []
  # skipEventTypes lists types of audit events to skip. For example, to forward all
  # audit events except for new app deletion events, you can assign this to:
  # ["app.delete"]
  skipEventTypes: []
  # skipSessionTypes lists types of session recording events to skip. For example,
  # to forward all session events except for malformed SQL packet events,
  # you can assign this to:
  # ["db.session.malformed_packet"]
  skipSessionTypes: []

crd:
  create: true
  namespace: operator-namespace

tbot:
  enabled: true
  clusterName: teleport.example.com
  teleportProxyAddress: teleport.example.com:443

fluentd:
  url: "https://fluentd.fluentd.svc.cluster.local/events.log"
  sessionUrl: "https://fluentd.fluentd.svc.cluster.local/session.log"
  certificate:
    secretName: "teleport-event-handler-client-tls"
    caPath: "ca.crt"
    certPath: "client.crt"
    keyPath: "client.key"

persistentVolumeClaim:
  enabled: true

Consult the Helm chart reference guide for more information on the crd and tbot sections.

Next steps​

You have now configured the Teleport Event Handler plugin with credentials to access the Teleport events API.

If you are new to exporting audit events with Teleport, read Forwarding Events with Fluentd to learn the basics of how our Event Handler plugin works. While this guide focuses on Fluentd, the Event Handler plugin can export audit events to any endpoint that ingests JSON messages via HTTP.

Next, read our guides to setting up the Event Handler plugin to export audit events to your solution of choice: