Set up the Teleport Event Handler Plugin
In this guide, we will set up the Teleport Event Handler plugin with credentials to authenticate to the Teleport Auth Service and access the events API.
How it works
The Event Handler plugin is a binary that runs independently of your Teleport cluster. It authenticates to your Teleport cluster using mutual TLS to begin forwarding events.
Prerequisites
-
A running Teleport cluster. If you want to get started with Teleport, sign up for a free trial or set up a demo environment.
-
The
tctlandtshclients.Installing
tctlandtshclients-
Determine the version of your Teleport cluster. The
tctlandtshclients must be at most one major version behind your Teleport cluster version. Send a GET request to the Proxy Service at/v1/webapi/findand use a JSON query tool to obtain your cluster version. Replace teleport.example.com:443 with the web address of your Teleport Proxy Service:TELEPORT_DOMAIN=teleport.example.com:443TELEPORT_VERSION="$(curl -s https://$TELEPORT_DOMAIN/v1/webapi/find | jq -r '.server_version')" -
Follow the instructions for your platform to install
tctlandtshclients:- Mac
- Windows - Powershell
- Linux
Download the signed macOS .pkg installer for Teleport, which includes the
tctlandtshclients:curl -O https://cdn.teleport.dev/teleport-${TELEPORT_VERSION?}.pkgIn Finder double-click the
pkgfile to begin installation.dangerUsing Homebrew to install Teleport is not supported. The Teleport package in Homebrew is not maintained by Teleport and we can't guarantee its reliability or security.
curl.exe -O https://cdn.teleport.dev/teleport-v${TELEPORT_VERSION?}-windows-amd64-bin.zipUnzip the archive and move the `tctl` and `tsh` clients to your %PATH%
NOTE: Do not place the `tctl` and `tsh` clients in the System32 directory, as this can cause issues when using WinSCP.
Use %SystemRoot% (C:\Windows) or %USERPROFILE% (C:\Users\<username>) instead.
All of the Teleport binaries in Linux installations include the
tctlandtshclients. For more options (including RPM/DEB packages and downloads for i386/ARM/ARM64) see our installation page.curl -O https://cdn.teleport.dev/teleport-v${TELEPORT_VERSION?}-linux-amd64-bin.tar.gztar -xzf teleport-v${TELEPORT_VERSION?}-linux-amd64-bin.tar.gzcd teleportsudo ./installTeleport binaries have been copied to /usr/local/bin
-
Recommended: Configure Machine & Workload Identity to provide short-lived
Teleport credentials to the plugin. Before following this guide, follow a
Machine & Workload Identity deployment guide
to run the tbot binary on your infrastructure.
-
A server, virtual machine, Kubernetes cluster, or Docker environment to run the Teleport Event Handler plugin.
-
To check that you can connect to your Teleport cluster, sign in with
tsh login, then verify that you can runtctlcommands using your current credentials.For example, run the following command, assigning teleport.example.com to the domain name of the Teleport Proxy Service in your cluster and email@example.com to your Teleport username:
tsh login --proxy=teleport.example.com --user=email@example.comtctl statusCluster teleport.example.com
Version 19.0.0-dev
CA pin sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678
If you can connect to the cluster and run the
tctl statuscommand, you can use your current credentials to run subsequenttctlcommands from your workstation. If you host your own Teleport cluster, you can also runtctlcommands on the computer that hosts the Teleport Auth Service for full permissions. -
On your workstation, create a folder called
event-handler, to hold configuration files and plugin state:mkdir -p event-handlercd event-handler
Step 1/2. Install the Event Handler plugin
- Linux
- macOS
- Docker
- Helm
- Build via Go
The Event Handler plugin is provided in amd64 and arm64 binaries for downloading.
Replace ARCH with your required version.
curl -L -O https://cdn.teleport.dev/teleport-event-handler-v19.0.0-dev-linux-ARCH-bin.tar.gztar -zxvf teleport-event-handler-v19.0.0-dev-linux-ARCH-bin.tar.gzsudo ./teleport-event-handler/install
The Event Handler plugin is provided in amd64 and arm64 binaries for downloading.
Replace ARCH with your required version.
curl -L -O https://cdn.teleport.dev/teleport-event-handler-v19.0.0-dev-darwin-ARCH-bin.tar.gztar -zxvf teleport-event-handler-v19.0.0-dev-darwin-ARCH-bin.tar.gzsudo ./teleport-event-handler/install
Ensure that you have Docker installed and running.
docker pull public.ecr.aws/gravitational/teleport-plugin-event-handler:19.0.0-dev
To allow Helm to install charts that are hosted in the Teleport Helm repository, use helm repo add:
helm repo add teleport https://charts.releases.teleport.dev
To update the cache of charts from the remote repository, run helm repo update:
helm repo update
You will need Go >= 1.25.8 installed.
Run the following commands:
git clone https://github.com/gravitational/teleport.git --depth 1 -b branch/v19cd teleport/integrations/event-handlermake build
The resulting executable will have the name event-handler. To follow the
rest of this guide, rename this file to teleport-event-handler and move it
to /usr/local/bin.
Step 2/2. Set up the Event Handler plugin
In this section, you will set up the Teleport Event Handler plugin and generate credentials that the plugin will use for authentication.
Generate a starter config file
Generate a configuration file with placeholder values for the Teleport Event Handler plugin. Later in this guide, we will edit the configuration file for your environment.
- Cloud-Hosted
- Self-Hosted
- Helm Chart
- Local Docker test
Run the configure command to generate a sample configuration. Assign
teleport.example.com:443 to the DNS name and port of your Teleport Enterprise Cloud
account.
Assign localhost to the DNS name of your log forwarder.
teleport-event-handler configure . teleport.example.com:443 --dns-names=localhost
Run the configure command to generate a sample configuration. Assign
teleport.example.com:443 to the DNS name and port of your Teleport Auth
Service or Proxy Service.
Assign localhost to the DNS name of your log forwarder.
teleport-event-handler configure . teleport.example.com:443 --dns-names=localhost
Run the configure command to generate a sample configuration. Assign
teleport.example.com:443 to the DNS name and port of your Teleport Auth
Service or Proxy Service.
Assign fluentd.fluentd.svc.cluster.local to the DNS name of your log forwarder.
docker run -v `pwd`:/opt/teleport-plugin -w /opt/teleport-plugin public.ecr.aws/gravitational/teleport-plugin-event-handler:19.0.0-dev configure . teleport.example.com:443 --dns-names=fluentd.fluentd.svc.cluster.local
In order to export audit events, you'll need to have the root certificate and the client credentials available as a secret. Use the following command to create that secret in Kubernetes:
kubectl create secret generic teleport-event-handler-client-tls --from-file=ca.crt=ca.crt,client.crt=client.crt,client.key=client.key
This will pack the content of ca.crt, client.crt, and client.key into the
secret so the Helm chart can mount them to their appropriate path.
Run the configure command to generate a sample configuration. Assign
teleport.example.com:443 to the DNS name and port of your Teleport Auth
Service or Proxy Service.
Assign localhost to the DNS name of your log forwarder.
docker run -v `pwd`:/opt/teleport-plugin -w /opt/teleport-plugin public.ecr.aws/gravitational/teleport-plugin-event-handler:19.0.0-dev configure . teleport.example.com:443 --dns-names=localhost
The plugin generates several setup files:
| File(s) | Purpose |
|---|---|
ca.crt and ca.key | Self-signed CA certificate and private key for Fluentd |
server.crt and server.key | Fluentd server certificate and key |
client.crt and client.key | Fluentd client certificate and key, all signed by the generated CA |
teleport-event-handler-role.yaml | user and role resource definitions for Teleport's event handler |
teleport-event-handler.toml | Example event handler configuration |
fluent.conf | Fluentd plugin configuration |
Running the Event Handler separately from the log forwarder
This guide assumes that you are running the Event Handler on the same host or
Kubernetes pod as your log forwarder. If you are not, you will need to instruct
the Event Handler to generate mTLS certificates for subjects besides
localhost. To do this, use the --dns-names flag of the
teleport-event-handler configure command.
For example, if your log forwarder is addressable at fluentd.example.com,
you would run the following configure command:
teleport-event-handler configure --dns-names=fluentd.example.com
The --dns-names flag accepts a comma-separated list of DNS names. It will
append subject alternative names (SANs) to the server certificate (the one you
will provide to your log forwarder) for each DNS name in the list.
If you have an existing Fluentd setup with TLS, issue a client certificate and key from the same certificate authority for the Teleport Event Handler to use.
Define the Event Handler role
The teleport-event-handler configure command generated a file called
teleport-event-handler-role.yaml. This file defines a teleport-event-handler
role and a user with read-only access to the event API:
kind: role
metadata:
name: teleport-event-handler
spec:
allow:
rules:
- resources: ['event', 'session']
verbs: ['list','read']
version: v5
---
kind: user
metadata:
name: teleport-event-handler
spec:
roles: ['teleport-event-handler']
version: v2
Move this file to your workstation (or recreate it by pasting the snippet above)
and use tctl on your workstation to create the role and the user:
tctl create -f teleport-event-handler-role.yamluser "teleport-event-handler" has been created
role 'teleport-event-handler' has been created
You can also create and edit roles using the Web UI. Go to Access -> Roles and click Create New Role or pick an existing role to edit.
Enable issuing of credentials for the Event Handler role
- Machine & Workload Identity
- Long-lived identity files
With the role created, you now need to allow the Machine & Workload Identity bot to produce credentials for this role.
This can be done with tctl, replacing my-bot with the name of your bot:
tctl bots update my-bot --add-roles teleport-event-handler
In order for the Event Handler plugin to forward events from your Teleport
cluster, it needs signed credentials from the cluster's certificate authority.
The teleport-event-handler user cannot request this itself, and requires
another user to impersonate this account in order to request credentials.
Create a role that enables your user to impersonate the teleport-event-handler
user. First, paste the following YAML document into a file called
teleport-event-handler-impersonator.yaml:
kind: role
version: v5
metadata:
name: teleport-event-handler-impersonator
spec:
options:
# max_session_ttl defines the TTL (time to live) of SSH certificates
# issued to the users with this role.
max_session_ttl: 10h
# This section declares a list of resource/verb combinations that are
# allowed for the users of this role. By default nothing is allowed.
allow:
impersonate:
users: ["teleport-event-handler"]
roles: ["teleport-event-handler"]
Next, create the role:
tctl create teleport-event-handler-impersonator.yaml
You can also create and edit roles using the Web UI. Go to Access -> Roles and click Create New Role or pick an existing role to edit.
Add this role to the user that generates signed credentials for the Event Handler:
Assign the teleport-event-handler-impersonator role to your Teleport user by running the appropriate
commands for your authentication provider:
- Local User
- GitHub
- SAML
- OIDC
-
Retrieve your local user's roles as a comma-separated list:
ROLES=$(tsh status -f json | jq -r '.active.roles | join(",")') -
Edit your local user to add the new role:
tctl users update $(tsh status -f json | jq -r '.active.username') \ --set-roles "${ROLES?},teleport-event-handler-impersonator" -
Sign out of the Teleport cluster and sign in again to assume the new role.
-
Open your
githubauthentication connector in a text editor:tctl edit github/github -
Edit the
githubconnector, addingteleport-event-handler-impersonatorto theteams_to_rolessection.The team you should map to this role depends on how you have designed your organization's role-based access controls (RBAC). However, the team must include your user account and should be the smallest team possible within your organization.
Here is an example:
teams_to_roles: - organization: octocats team: admins roles: - access + - teleport-event-handler-impersonator -
Apply your changes by saving closing the file in your editor.
-
Sign out of the Teleport cluster and sign in again to assume the new role.
-
Retrieve your
samlconfiguration resource:tctl get --with-secrets saml/mysaml > saml.yamlNote that the
--with-secretsflag adds the value ofspec.signing_key_pair.private_keyto thesaml.yamlfile. Because this key contains a sensitive value, you should remove the saml.yaml file immediately after updating the resource. -
Edit
saml.yaml, addingteleport-event-handler-impersonatorto theattributes_to_rolessection.The attribute you should map to this role depends on how you have designed your organization's role-based access controls (RBAC). However, the group must include your user account and should be the smallest group possible within your organization.
Here is an example:
attributes_to_roles: - name: "groups" value: "my-group" roles: - access + - teleport-event-handler-impersonator -
Apply your changes:
tctl create -f saml.yaml -
Sign out of the Teleport cluster and sign in again to assume the new role.
-
Retrieve your
oidcconfiguration resource:tctl get oidc/myoidc --with-secrets > oidc.yamlNote that the
--with-secretsflag adds the value ofspec.signing_key_pair.private_keyto theoidc.yamlfile. Because this key contains a sensitive value, you should remove the oidc.yaml file immediately after updating the resource. -
Edit
oidc.yaml, addingteleport-event-handler-impersonatorto theclaims_to_rolessection.The claim you should map to this role depends on how you have designed your organization's role-based access controls (RBAC). However, the group must include your user account and should be the smallest group possible within your organization.
Here is an example:
claims_to_roles: - name: "groups" value: "my-group" roles: - access + - teleport-event-handler-impersonator -
Apply your changes:
tctl create -f oidc.yaml -
Sign out of the Teleport cluster and sign in again to assume the new role.
Export an identity file for the Event Handler user
Give the plugin access to a Teleport identity file. We recommend using Machine
ID for this in order to produce short-lived identity files that are less
dangerous if exfiltrated, though in demo deployments, you can generate
longer-lived identity files with tctl:
- Machine & Workload Identity
- Long-lived identity files
Configure tbot with an output that will produce the credentials needed by
the plugin. As the plugin will be accessing the Teleport API, the correct
output type to use is identity.
For this guide, the directory destination will be used. This will write these
credentials to a specified directory on disk. Ensure that this directory can
be written to by the Linux user that tbot runs as, and that it can be read by
the Linux user that the plugin will run as.
Modify your tbot configuration to add an identity output.
If running tbot on a Linux server, use the directory output to write
identity files to the /opt/machine-id directory:
services:
- type: identity
destination:
type: directory
# For this guide, /opt/machine-id is used as the destination directory.
# You may wish to customize this. Multiple outputs cannot share the same
# destination.
path: /opt/machine-id
If running tbot on Kubernetes, write the identity file to Kubernetes secret
instead:
services:
- type: identity
destination:
type: kubernetes_secret
name: teleport-event-handler-identity
If operating tbot as a background service, restart it. If running tbot in
one-shot mode, execute it now.
You should now see an identity file under /opt/machine-id or a Kubernetes
secret named teleport-event-handler-identity. This contains the private key and signed
certificates needed by the plugin to authenticate with the Teleport Auth
Service.
Like all Teleport users, teleport-event-handler needs signed credentials in order to
connect to your Teleport cluster. You will use the tctl auth sign command to
request these credentials.
The following tctl auth sign command impersonates the teleport-event-handler user,
generates signed credentials, and writes an identity file to the local
directory:
tctl auth sign --user=teleport-event-handler --out=identity
The plugin connects to the Teleport Auth Service's gRPC endpoint over TLS.
The identity file, identity, includes both TLS and SSH credentials. The
plugin uses the SSH credentials to connect to the Proxy Service, which
establishes a reverse tunnel connection to the Auth Service. The plugin
uses this reverse tunnel, along with your TLS credentials, to connect to the
Auth Service's gRPC endpoint.
Certificate Lifetime
By default, tctl auth sign produces certificates with a relatively short
lifetime. For production deployments, we suggest using Machine
& Workload Identity to programmatically issue and renew
certificates for your plugin. See our Machine & Workload Identity getting started
guide to learn more.
Note that you cannot issue certificates that are valid longer than your existing credentials.
For example, to issue certificates with a 1000-hour TTL, you must be logged in with a session that is
valid for at least 1000 hours. This means your user must have a role allowing
a max_session_ttl of at least 1000 hours (60000 minutes), and you must specify a --ttl
when logging in:
tsh login --proxy=teleport.example.com --ttl=60060
If you are running the plugin on a Linux server, create a data directory to hold certificate files for the plugin:
sudo mkdir -p /var/lib/teleport/plugins/api-credentialssudo mv identity /var/lib/teleport/plugins/api-credentials
If you are running the plugin on Kubernetes, Create a Kubernetes secret that contains the Teleport identity file:
kubectl -n teleport create secret generic --from-file=identity teleport-event-handler-identity
Once the Teleport credentials expire, you will need to renew them by running the
tctl auth sign command again.
Next steps
You have now configured the Teleport Event Handler plugin with credentials to access the Teleport events API.
If you are new to exporting audit events with Teleport, read Forwarding Events with Fluentd to learn the basics of how our Event Handler plugin works. While this guide focuses on Fluentd, the Event Handler plugin can export audit events to any endpoint that ingests JSON messages via HTTP.
Next, read our guides to setting up the Event Handler plugin to export audit events to your solution of choice:
- Export Teleport Audit Events with the Elastic Stack: How to configure the Event Handler plugin to forward Teleport audit logs to Logstash for ingestion in Elasticsearch so you can explore them in Kibana.
- Export Teleport Audit Events with Panther: How to configure the Event Handler plugin to send logs to Panther via Fluentd so you can explore your audit events in Panther.
- Export Teleport Audit Events with Splunk: How to configure the Event Handler plugin to send logs to Splunk's Universal Forwarder so you can explore your audit events in Splunk.
- Export Teleport Audit Events with Datadog: How to configure the Event Handler plugin to export audit logs to Datadog via Fluentd.