Architecture Overview

Flyte's Application Service is architected with a strict separation between the Control Plane and the Data Plane. This design allows the control plane to focus on high-performance request handling and caching, while the data plane manages the complexities of Kubernetes resource orchestration.

Control Plane: The Caching Proxy

The control plane is implemented by the AppService class in app/service/app_service.go. It acts as a gateway for all external RPC requests. Instead of interacting with Kubernetes directly, it proxies requests to an internal client and maintains an in-memory TTL cache to optimize performance for read-heavy workloads.

Caching Strategy

The AppService uses an LRU cache (from github.com/hashicorp/golang-lru/v2/expirable) to store application metadata. This significantly reduces the number of cross-plane RPC calls and Kubernetes API requests for Get operations.

• Cache Hits: If a Get request finds a valid entry in the cache, it returns immediately without calling the data plane.
• Cache Invalidation: To ensure consistency, any write operation (Create, Update, or Delete) triggers an immediate removal of the corresponding entry from the cache.
• Transitional State Protection: The service explicitly avoids caching apps in a "transitional" state. If an app's desired state is ACTIVE but its current status is STOPPED, the Get response is not cached. This prevents the UI from being stuck showing a "Stopped" status during the window when Kubernetes is still spinning up the underlying pods.

// From app/service/app_service.go
func (s *AppService) Get(
	ctx context.Context,
	req *connect.Request[flyteapp.GetRequest],
) (*connect.Response[flyteapp.GetResponse], error) {
	// ... cache lookup ...
	
	resp, err := s.internalClient.Get(ctx, req)
	if err != nil {
		return nil, err
	}
    
    // Only cache if the app is not in a transitional state (e.g., starting up)
	if ok && appID.AppId != nil && s.cache != nil && !isTransitionalState(resp.Msg.GetApp()) {
		s.cache.Add(cacheKey(appID.AppId), resp.Msg.GetApp())
	}
	return resp, nil
}

Data Plane: The Kubernetes Controller

The data plane is implemented by the InternalAppService in app/internal/service/internal_app_service.go. It is the authoritative service that manages the lifecycle of applications by interacting with the Kubernetes API.

Resource Management

The data plane uses an AppK8sClient to map Flyte application entities to Knative Service (KService) resources.

• Deployment: When Create or Update is called, the service invokes s.k8s.Deploy(ctx, app), which applies the necessary CRDs to the cluster.
• Status Tracking: The Get and List methods retrieve live status directly from Kubernetes, ensuring that the reported ingress URLs and deployment conditions are accurate.
• Scaling: The Update method handles scaling logic. For example, setting the DesiredState to STOPPED triggers a call to s.k8s.Stop(ctx, appID), which scales the Knative Service to zero while preserving its configuration.

Deployment Modes

Flyte supports two deployment modes for these services, configured during the initialization phase in app/setup.go.

Unified Mode

In unified mode, both the control plane and data plane run within the same process. To avoid routing collisions (since they share the same Protobuf service definition), the data plane is mounted on the shared HTTP mux with an /internal prefix.

The AppService is configured with an internalClient that points to this local prefix. This allows the proxy to route requests to the data plane without leaving the process, eliminating network latency.

// From app/internal/setup.go
// The data plane is mounted with a prefix
path, handler := appconnect.NewAppServiceHandler(internalAppSvc, ...)
sc.Mux.Handle("/internal"+path, http.StripPrefix("/internal", handler))

Split Mode

In split mode, the control plane and data plane are deployed as separate services. The control plane is configured with the apps.internalAppServiceUrl pointing to the remote data plane instance. This mode is useful for scaling the control plane independently or for architectures where only specific nodes have the permissions required to interact with the Kubernetes API.

Configuration

You can tune the behavior of the architecture using the following configuration parameters:

Parameter	Default	Description
apps.cacheTtl	30s	The TTL for the in-memory cache in the control plane. Set to 0 to disable caching.
apps.internalAppServiceUrl	http://localhost:8091	The URL of the data plane service used by the control plane proxy.
internalApps.enabled	false	Whether the data plane (Kubernetes controller) should be initialized in the current process.

Request Flow Example

When a user requests the status of an application (Get), the following flow occurs:

1. Client sends a Get request to the Control Plane (AppService).
2. AppService checks its local LRU Cache.
3. On Cache Hit: The cached application metadata is returned immediately.
4. On Cache Miss:
   • AppService proxies the request to the InternalAppService (via the /internal prefix or remote URL).
   • InternalAppService queries the Kubernetes API for the current state of the KService.
   • InternalAppService returns the live status to the AppService.
   • AppService verifies the app is not in a transitional state and updates its LRU Cache.
5. AppService returns the response to the Client.