Project and Domain Isolation

Flyte uses projects and domains to provide a multi-tenant environment for organizing workflows, tasks, and executions. Projects serve as the top-level logical container for resources, while domains represent distinct environments (such as development, staging, or production) within those projects.

The Project and Domain Relationship

In Flyte, projects are persistent entities stored in the database, whereas domains are globally configured at the service level. When you query for a project, Flyte dynamically associates the globally configured domains with that project in the response.

This separation allows you to define a standard set of environments across your entire Flyte deployment without needing to replicate domain definitions for every individual project.

Managing Projects

The ProjectService in the runs service handles the lifecycle of projects. It interacts with the ProjectRepo to persist project metadata such as identifiers, names, and labels.

Creating and Updating Projects

When you create a project, Flyte stores it using the Project model. The Identifier serves as the unique key.

// From runs/repository/models/project.go
type Project struct {
	Identifier  string    `db:"identifier"`
	Name        string    `db:"name"`
	Description string    `db:"description"`
	Labels      []byte    `db:"labels"` // Marshaled protobuf bytes
	State       *int32    `db:"state"`
	CreatedAt   time.Time `db:"created_at"`
	UpdatedAt   time.Time `db:"updated_at"`
}

The ProjectService implements the CreateProject and UpdateProject RPCs, which transform incoming protobuf messages into this database model. Note that project labels are stored as marshaled bytes in the database, requiring transformation during API calls.

Global Domain Injection

One unique aspect of Flyte's implementation is that domains are not stored in the project table. Instead, the ProjectService is initialized with a list of system-wide domains. When a project is requested via GetProject or ListProjects, the service injects these domains into the response using a transformer.

// From runs/service/project_service.go
func (s *ProjectService) GetProject(
	ctx context.Context,
	req *connect.Request[project.GetProjectRequest],
) (*connect.Response[project.GetProjectResponse], error) {
	// ... validation and retrieval ...
	model, err := s.projectRepo.GetProject(ctx, req.Msg.GetId())
	
    // Domains (s.domains) are injected here during transformation
	projectProto, err := transformers.ProjectModelToProject(model, s.domains)
	if err != nil {
		return nil, connect.NewError(connect.CodeInternal, err)
	}

	return connect.NewResponse(&project.GetProjectResponse{
		Project: projectProto,
	}), nil
}

This ensures that every project in Flyte consistently exposes the same set of domains as defined in the service configuration.

Project Lifecycle and Seeding

Flyte supports automatic project creation during service startup, often referred to as "seeding." This is useful for ensuring that default projects like flytesnacks exist immediately upon deployment.

The seeding logic in runs/setup.go iterates through a configured list of project IDs and creates them if they do not already exist:

// From runs/setup.go
func seedProjects(ctx context.Context, projectRepo interfaces.ProjectRepo, projects []string) error {
	for _, projectID := range projects {
		state := int32(projectpb.ProjectState_PROJECT_STATE_ACTIVE)
		projectModel := &models.Project{
			Identifier: projectID,
			Name:       projectID,
			State:      &state,
		}

		if err := projectRepo.CreateProject(ctx, projectModel); err != nil {
			if errors.Is(err, interfaces.ErrProjectAlreadyExists) {
				continue
			}
			return err
		}
	}
	return nil
}

Visibility and Archival

Projects in Flyte can be in different states, primarily ACTIVE or ARCHIVED. To keep the UI and CLI responses clean, the ProjectService automatically filters out archived projects from the ListProjects results unless an explicit filter is provided by the user.

// From runs/service/project_service.go
// If no filters are supplied, hide archived (deleted) projects.
if req.Msg.GetFilters() == "" {
    listInput.Filter = impl.NewNotEqualFilter("state", int32(project.ProjectState_PROJECT_STATE_ARCHIVED))
}

If you need to retrieve archived projects, you must provide a specific filter string in your ListProjectsRequest to override this default behavior.

Configuration

You can configure the available domains and the projects to be seeded via the runs service configuration:

• Domains: Set the runs.domains list (e.g., development, staging, production).
• Seed Projects: Set the runs.seedProjects list to a collection of project identifiers you want Flyte to create on startup.