Skip to main content

Customizing Task Logs

Flyte allows you to customize task execution log links by defining URI templates that are populated with metadata from the execution environment. This is primarily achieved through the TemplateLogPlugin, which implements the Plugin interface to generate dynamic URLs for various log providers like Cloudwatch, Stackdriver, or custom Kubernetes dashboards.

Configuring Custom Log Templates​

To add custom log links to your Flyte deployment, you configure the TemplateLogPlugin within the logs section of the Flyte configuration. Each plugin defines a display name and one or more URI templates.

import (
    "github.com/flyteorg/flyte/v2/flyteplugins/go/tasks/pluginmachinery/tasklog"
    "github.com/flyteorg/flyte/v2/gen/go/flyteidl2/core"
)

// Example of a Cloudwatch log template configuration
cloudwatchPlugin := tasklog.TemplateLogPlugin{
    DisplayName: "Cloudwatch Logs",
    TemplateURIs: []tasklog.TemplateURI{
        "https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logEventViewer:group=/flyte-production/kubernetes;stream=var.log.containers.{{.podName}}_{{.namespace}}_{{.containerName}}-{{.containerId}}.log",
    },
    MessageFormat: core.TaskLog_JSON,
}

When Flyte generates the log link, it replaces the placeholders (e.g., {{.podName}}) with actual values from the Input struct provided during task execution.

Available Template Variables​

The TemplateLogPlugin supports a wide range of variables derived from the Input struct and the TaskExecutionID. These are defined in flyteplugins/go/tasks/pluginmachinery/tasklog/template.go:

VariableDescription
{{ .podName }}The name of the Kubernetes pod.
{{ .namespace }}The Kubernetes namespace where the task is running.
{{ .containerID }}The ID of the container (automatically stripped of prefixes like docker://).
{{ .containerName }}The name of the container within the pod.
{{ .generatedName }}A DNS-1123 compatible unique name for the task execution.
{{ .taskID }}The name of the task.
{{ .executionName }}The name of the workflow execution.
{{ .nodeID }}The unique ID of the node in the workflow graph.
{{ .podRFC3339StartTime }}The start time of the pod in RFC3339 format.
{{ .podUnixStartTime }}The start time of the pod as a Unix timestamp.

Dynamic log links allow you to pull values from the task's configuration metadata using the {{ .taskConfig.<key> }} syntax. This is useful for tools like VSCode or interactive debuggers where the connection details (like a port) are specific to the task instance.

To use dynamic links, define them in the DynamicTemplateURIs field of the TemplateLogPlugin:

// Example for a VSCode interactive link
vscodePlugin := tasklog.TemplateLogPlugin{
    Name:                "vscode",
    DisplayName:         "VSCode Link",
    DynamicTemplateURIs: []tasklog.TemplateURI{"vscode://flyteinteractive:{{ .taskConfig.port }}/{{ .podName }}"},
    MessageFormat:       core.TaskLog_JSON,
}

For these links to appear, the task must include a matching link_type in its configuration. Flyte's InitializeLogPlugins function in flyteplugins/go/tasks/logs/logging_utils.go checks the TaskTemplate config for a link_type key:

// From flyteplugins/go/tasks/pluginmachinery/tasklog/template.go
config := input.TaskTemplate.GetConfig()
linkType := config["link_type"] // e.g., "vscode"

Injecting Logs via Task Metadata​

You can also define log links directly within a task's metadata. Flyte will automatically convert these into TemplateLogPlugin instances during initialization.

In flyteplugins/go/tasks/logs/logging_utils.go, the InitializeLogPlugins function iterates over metadata log links:

if taskTemplate != nil && taskTemplate.GetMetadata() != nil {
    for _, logLink := range taskTemplate.GetMetadata().GetLogLinks() {
        plugins = append(plugins, tasklog.TemplateLogPlugin{
            DisplayName:   logLink.GetName(),
            TemplateURIs:  []tasklog.TemplateURI{logLink.GetUri()},
            MessageFormat: logLink.GetMessageFormat(),
            LinkType:      core.TaskLog_DASHBOARD.String(),
            IconUri:       logLink.GetIconUri(),
        })
    }
}

Troubleshooting and Implementation Details​

Container ID Sanitization​

Flyte automatically strips the container engine prefix from containerID before substitution. If your Kubernetes environment uses cri-o://<id> or docker://<id>, the template variable {{ .containerId }} will only contain the <id> portion. This logic is handled in flyteplugins/go/tasks/pluginmachinery/tasklog/template.go:

containerID := input.ContainerID
stripDelimiter := "://"
if split := strings.Split(input.ContainerID, stripDelimiter); len(split) > 1 {
    containerID = split[1]
}

DNS-1123 Compatibility​

The {{ .generatedName }} variable is sanitized using utils.ConvertToDNS1123SubdomainCompatibleString. This ensures that if you use the generated name in a URL that requires DNS compatibility (like a subdomain), it will not contain invalid characters like underscores.

Visibility Controls​

You can control when log links are visible in the Flyte Console using the following fields in TemplateLogPlugin:

  • ShowWhilePending: If set to true, the link is visible as soon as the task is created, even before it starts running.
  • HideOnceFinished: If set to true, the link disappears once the task reaches a terminal state.