Using HashiCorp Vault

Flyte integrates with HashiCorp Vault by translating secret requests into Vault Agent Sidecar Injector annotations. This allows Flyte tasks to securely access secrets stored in Vault without the Flyte engine needing direct access to the secret values.

Enabling Vault Secret Injection

To use Vault for secrets, you must configure the Flyte pod webhook to use the Vault secret manager type. This is done in the webhook section of the Flyte configuration.

webhook:
  secretManagerTypes:
    - Vault
  vaultSecretManager:
    role: "flyte-role" # The Vault role for authentication
    kvVersion: 2       # Default KV version (deprecated in favor of per-secret version)
    annotations:       # Optional: Extra annotations for the Vault Agent
      vault.hashicorp.com/auth-type: "kubernetes"

The VaultSecretManagerConfig struct in flyteplugins/go/tasks/pluginmachinery/secret/config/config.go defines these fields:

type VaultSecretManagerConfig struct {
	Role        string            `json:"role" pflag:",Specifies the vault role to use"`
	KVVersion   KVVersion         `json:"kvVersion" pflag:"-,DEPRECATED! Use the GroupVersion field of the Secret request instead."`
	Annotations map[string]string `json:"annotations" pflag:"-,Annotation to be added to user task pod."`
}

Defining Secrets in Flyte

When you request a secret in a Flyte task, the Group and Key fields are mapped to Vault paths and keys.

• Group: The path to the secret in Vault (e.g., secret/data/my-app-secrets).
• Key: The specific key within that Vault secret.
• GroupVersion: Specifies the Vault secret engine type (kv1, kv2, or db).

Example Secret Request

In your Flyte task, you define the secret requirement:

from flytekit import task, Secret

@task(secret_requests=[Secret(group="secret/data/helper", key="api_key", group_version="kv2")])
def my_task():
    # Secret will be available at /etc/flyte/secrets/secret/data/helper/api_key
    ...

How Injection Works

The VaultSecretManagerInjector in flyteplugins/go/tasks/pluginmachinery/secret/vault_secret_manager.go processes the secret request and adds the following annotations to the task pod:

1. vault.hashicorp.com/agent-inject: "true"
2. vault.hashicorp.com/role: The configured role (e.g., "flyte-role").
3. vault.hashicorp.com/secret-volume-path: "/etc/flyte/secrets"
4. vault.hashicorp.com/agent-inject-secret-<id>: The secret Group.
5. vault.hashicorp.com/agent-inject-file-<id>: The path <Group>/<Key>.
6. vault.hashicorp.com/agent-inject-template-<id>: A Consul template to extract the specific key.

Supported Secret Engines

The CreateVaultAnnotationsForSecret function in flyteplugins/go/tasks/pluginmachinery/secret/utils.go generates different templates based on the GroupVersion:

GroupVersion	Vault Engine	Template Logic
kv1	KV Version 1	Extracts from .Data
kv2	KV Version 2	Extracts from .Data.data
db	Database	No template (injects the whole secret)

For kv2, the generated template looks like this:

{{- with secret "secret/data/helper" -}}{{ .Data.data.api_key }}{{- end -}}

Secret Mount Paths

Secrets are always mounted as files. The VaultSecretManagerInjector sets environment variables in the task container to help Flytekit locate them:

• FLYTE_SECRETS_DEFAULT_DIR: /etc/flyte/secrets
• FLYTE_SECRETS_FILE_PREFIX: "" (empty string)

The full path to a secret file is: /etc/flyte/secrets/<SecretGroup>/<SecretKey>

Customizing Vault Annotations

If your Vault deployment requires specific annotations (e.g., for namespace support or custom auth), use the annotations field in the VaultSecretManagerConfig. These are merged with the default annotations generated by Flyte.

webhook:
  vaultSecretManager:
    annotations:
      vault.hashicorp.com/namespace: "team-a"
      vault.hashicorp.com/auth-config-type: "gcp"

Limitations and Gotchas

• Environment Variables: The Vault injector does not support ENV_VAR mount requirements. If a secret request specifies ENV_VAR, the injector will return an error.
• Vault Agent Requirement: This feature requires the HashiCorp Vault Agent Sidecar Injector to be pre-installed and configured in your Kubernetes cluster. Flyte only adds the annotations; it does not manage the Vault Agent itself.
• Fixed Mount Path: Secrets are always mounted under /etc/flyte/secrets. You cannot customize this path per secret.
• Unique IDs: Flyte generates a random UUID for each secret's annotation keys (e.g., agent-inject-secret-550e8400...) to prevent collisions when multiple secrets are requested for the same pod.