Skip to main content

Configuring the Secret Webhook

To configure the Flyte secret management system, you must define a webhook configuration section that specifies how the mutating webhook server behaves and which secret backends it should use to inject secrets into task pods.

Basic Webhook Configuration​

The following JSON example demonstrates a standard configuration for the secret webhook, including certificate paths, listening ports, and the selection of secret manager backends.

{
    "webhook": {
        "metrics-prefix": "flyte:",
        "certDir": "/etc/webhook/certs",
        "listenPort": 9443,
        "serviceName": "flyte-pod-webhook",
        "servicePort": 443,
        "secretName": "flyte-pod-webhook",
        "secretManagerTypes": ["K8s", "AWS"],
        "secretEnvVarPrefix": "_UNION_",
        "webhookTimeout": 30,
        "kubeClientConfig": {
            "qps": 100,
            "burst": 25,
            "timeout": "30s"
        }
    }
}

Configuring Secret Manager Backends​

Flyte supports multiple secret manager types defined by the SecretManagerType enum in flyteplugins/go/tasks/pluginmachinery/secret/config/config.go. You should use the secretManagerTypes (plural) field to provide a prioritized list of backends.

  • K8s: Injects volume mounts for native Kubernetes secrets.
  • AWS: Injects a sidecar container (amazon/aws-secrets-manager-secret-sidecar) to pull secrets from AWS Secrets Manager.
  • GCP: Injects a sidecar container to pull secrets from GCP Secret Manager.
  • Vault: Integrates with Hashicorp Vault using the Vault Agent Injector.
  • Azure: Injects a sidecar container to pull secrets from Azure Key Vault.
  • Embedded: Directly calls the provider's API from the Flyte engine without a sidecar (currently primarily supported for AWS).

Provider-Specific Settings​

Each provider has its own configuration block within the Config struct. For example, to configure the AWS sidecar resources:

{
    "webhook": {
        "awsSecretManager": {
            "sidecarImage": "docker.io/amazon/aws-secrets-manager-secret-sidecar:v0.1.4",
            "resources": {
                "requests": {
                    "cpu": "200m",
                    "memory": "500Mi"
                },
                "limits": {
                    "cpu": "200m",
                    "memory": "500Mi"
                }
            }
        }
    }
}

Tuning the Kubernetes Client​

The webhook uses an internal Kubernetes client to interact with the API server. You can tune its performance using the KubeClientConfig struct.

// From flyteplugins/go/tasks/pluginmachinery/secret/config/config.go
type KubeClientConfig struct {
	QPS     int32           `json:"qps"`     // Max QPS to the master. Defaults to 5.
	Burst   int             `json:"burst"`   // Max burst rate for throttle. Defaults to 10.
	Timeout config.Duration `json:"timeout"` // Max duration for KubeAPI requests.
}

If you use the Embedded secret manager with the K8s type, the system will automatically fall back to the global kubeClientConfig if specific settings are not provided for the secret fetcher, as implemented in ResolveKubeClientConfigs():

func (c *Config) ResolveKubeClientConfigs() {
	if c.EmbeddedSecretManagerConfig.K8sConfig.KubeClientConfig.QPS == 0 {
		c.EmbeddedSecretManagerConfig.K8sConfig.KubeClientConfig.QPS = c.KubeClientConfig.QPS
	}
    // ... similar logic for Burst and Timeout
}

Advanced Pod Mutation with ImageBuilder​

The ImageBuilderConfig allows you to control how the webhook modifies pod specifications based on container images and labels. This is useful for environments where registry hostnames need to be replaced dynamically.

{
    "webhook": {
        "imageBuilderConfig": {
            "enabled": true,
            "hostnameReplacement": {
                "existing": "old.registry.com",
                "replacement": "new.registry.com"
            },
            "labelSelector": {
                "matchLabels": {
                    "inject-secrets": "true"
                }
            },
            "excludedContainerNames": ["init-embedded-secret"]
        }
    }
}

Troubleshooting and Gotchas​

  • Deprecated Field: The secretManagerType (singular) field is deprecated. Always use secretManagerTypes (plural) to ensure your configuration is future-proof and supports multiple backends.
  • Vault KV Version: In VaultSecretManagerConfig, the kvVersion field is deprecated. Flyte now prefers using the GroupVersion field of the Secret request.
  • Webhook Creation: By default, Flyte attempts to create the MutatingWebhookConfiguration automatically. If your deployment environment manages this externally (e.g., via Helm or Terraform), set disableCreateMutatingWebhookConfig: true.
  • Certificate Directory: The certDir defaults to /etc/webhook/certs/. If you are running locally or in a non-standard environment, you can set localCert: true to write certificates to the local file system.
  • Environment Variable Prefix: Secrets injected as environment variables will be prefixed with the value of secretEnvVarPrefix (default: _UNION_). Ensure your application code expects this prefix.