Configuring Cache Keys and Expiration

To control how Flyte identifies cached task results and when those results should expire, you must configure the catalogCache settings in the plugin machinery. This configuration determines how cache keys are generated, whether they are scoped to the execution project, and how long a cached entry remains valid.

Configuration Structure

The primary configuration for Flyte's catalog integration is defined in the Config struct within flyteplugins/go/tasks/pluginmachinery/catalog/config.go.

type Config struct {
	ReaderWorkqueueConfig workqueue.Config   `json:"reader" pflag:",Catalog reader workqueue config. Make sure the index cache must be big enough to accommodate the biggest array task allowed to run on the system."`
	WriterWorkqueueConfig workqueue.Config   `json:"writer" pflag:",Catalog writer workqueue config. Make sure the index cache must be big enough to accommodate the biggest array task allowed to run on the system."`
	CacheKey              CacheKeyConfig     `json:"cacheKey" pflag:",Cache key configuration."`
	MaxCacheAge           stdconfig.Duration `json:"maxCacheAge" pflag:",Cache entries past this age will incur cache miss. 0 means cache never expires."`
}

type CacheKeyConfig struct {
	EnforceExecutionProjectDomain bool `json:"enforceExecutionProjectDomain" pflag:", Use execution project domain when computing the cache key. This means that even if you reference tasks/launchplans from a different project, cache keys will be computed based on the execution project domain instead."`
}

Enforcing Execution Project and Domain

By default, Flyte generates cache keys based on the task's own project and domain. If you want to ensure that tasks shared across different projects do not share cache entries (e.g., for security or resource isolation), set EnforceExecutionProjectDomain to true.

When this is enabled, Flyte uses the project and domain of the current execution context rather than the task's definition when computing the cache key.

Setting Cache Expiration

You can set a global maximum age for cache entries using the MaxCacheAge parameter. This is enforced by the Flyte Catalog client during the Get operation. If a cache hit is found but its last_updated_at metadata exceeds the configured duration, the client treats it as a cache miss.

The enforcement logic in flyteplugins/go/tasks/pluginmachinery/catalog/cache_service/client.go looks like this:

if c.maxCacheAge > 0 {
    lastUpdatedAt := output.GetMetadata().GetLastUpdatedAt()
    if lastUpdatedAt == nil {
        return catalog.Entry{}, grpcstatus.Error(codes.Internal, "received cache metadata without last_updated_at")
    }
    if time.Since(lastUpdatedAt.AsTime()) > c.maxCacheAge {
        return catalog.Entry{}, grpcstatus.Error(codes.NotFound, "artifact over age limit")
    }
}

Tuning Workqueues for Large Tasks

Flyte uses asynchronous workqueues for reading from and writing to the catalog. For workflows involving large array tasks, you must ensure the IndexCacheMaxItems in both ReaderWorkqueueConfig and WriterWorkqueueConfig is sufficiently large.

If the index cache is too small, the system may experience performance degradation or failures when processing the numerous sub-tasks within an array task.

How Cache Keys are Generated

Flyte generates a unique string for each cache entry by hashing several components of the task execution. The buildCacheKey function in flyteplugins/go/tasks/pluginmachinery/catalog/cache_service/client.go demonstrates this process:

func buildCacheKey(ctx context.Context, key catalog.Key) (string, error) {
	// 1. Hash the task identifier (excluding the version)
	identifierHash, err := catalog.HashIdentifierExceptVersion(ctx, key.Identifier)
	if err != nil {
		return "", err
	}

	// 2. Hash the interface signature (input/output types)
	signatureHash, err := generateInterfaceSignatureHash(ctx, key.TypedInterface)
	if err != nil {
		return "", err
	}

	// 3. Hash the actual input values
	inputsHash, err := hashInputs(ctx, key)
	if err != nil {
		return "", err
	}

	// 4. Combine with the CacheVersion string defined in the task template
	return fmt.Sprintf("%s-%s-%s-%s", identifierHash, signatureHash, inputsHash, key.CacheVersion), nil
}

Troubleshooting Cache Misses

If you encounter unexpected cache misses, check the following:

1. Artifact Age: If MaxCacheAge is set, check if the cached artifact is older than the limit. The client will return a NotFound error with the message "artifact over age limit".
2. Input Hashing: Ensure that the inputs being passed to the task are identical. Even small changes in input values will result in a different inputsHash.
3. Cache Version: The CacheVersion string in the task template is a manual override. If this version changes, all previous cache entries for that task become invalid.
4. Project/Domain Mismatch: If EnforceExecutionProjectDomain is enabled, verify that the task is being executed in the expected project and domain, as this will change the identifierHash.