Querying and Filtering Runs

Flyte provides a flexible filtering and sorting system for querying runs, tasks, and events. This system abstracts SQL generation through a set of Filter interfaces and a ListResourceInput structure that handles pagination and sorting.

Querying Runs with Filters

To search for specific runs, you construct a ListResourceInput containing your filter criteria and pass it to a repository method like ListActions.

// List all root actions (runs) for a specific project and domain
runs, err := actionRepo.ListActions(ctx, interfaces.ListResourceInput{
    Filter: NewIsRootActionFilter().
        And(NewEqualFilter("project", "my-project")).
        And(NewEqualFilter("domain", "development")),
    Limit: 50,
})

Core Filter Types

Flyte implements several filter types in runs/repository/impl/filters.go:

• basicFilter: Handles standard comparisons like =, !=, >, <, LIKE, and IN (via ANY(?)).
• nullFilter: Specifically handles IS NULL and IS NOT NULL checks.
• compositeFilter: Combines multiple filters using AND or OR logic.

Common Filter Helpers

Use these helper functions from runs/repository/impl/filters.go to create filters without manually instantiating the underlying structs:

Helper	Description
NewEqualFilter(field, value)	Creates a field = value filter.
NewNotEqualFilter(field, value)	Creates a field != value filter.
NewIsRootActionFilter()	Filters for root actions (where parent_action_name is NULL).
NewRunActionsFilter(runID)	Filters for all actions belonging to a specific run.
NewTaskNameFilter(taskName)	Filters tasks by project, domain, and name.

Combining Filters

The Filter interface provides And() and Or() methods for chaining multiple conditions. This creates a compositeFilter that recursively builds nested SQL expressions with proper parentheses.

// Filter for runs that are either in 'SUCCEEDED' phase OR belong to a specific user
filter := NewEqualFilter("phase", "SUCCEEDED").
    Or(NewEqualFilter("deployed_by", "admin@example.com"))

input := interfaces.ListResourceInput{
    Filter: filter,
    Limit:  20,
}

Pagination and Sorting

Flyte uses keyset pagination via the CursorToken field in ListResourceInput.

Keyset Pagination

The CursorToken is an RFC3339Nano encoded timestamp representing the created_at time of the last item in the previous page.

// Page 1: Request 2 items
runsPage1, _ := actionRepo.ListActions(ctx, interfaces.ListResourceInput{
    Filter: NewIsRootActionFilter(),
    Limit:  2,
})

// Detect if next page exists and get token
// Note: ListActions returns Limit+1 items to probe for the next page
if len(runsPage1) > 2 {
    lastItem := runsPage1[1] // Get the last item of the current page
    token := lastItem.CreatedAt.UTC().Format(time.RFC3339Nano)

    // Page 2: Use the token
    runsPage2, _ := actionRepo.ListActions(ctx, interfaces.ListResourceInput{
        Filter:      NewIsRootActionFilter(),
        Limit:       2,
        CursorToken: token,
    })
}

Default Sorting

If no SortParameters are provided in ListResourceInput, Flyte defaults to:

1. phase ASC
2. created_at DESC

Security and Validation

When converting external requests (e.g., from Protobuf) into database filters, Flyte enforces a whitelist of allowed columns to prevent SQL injection.

The ConvertProtoFilters function in runs/repository/impl/filters.go requires a sets.Set[string] of allowed columns:

allowedColumns := sets.NewString("project", "domain", "phase", "created_at")
combinedFilter, err := ConvertProtoFilters(protoRequest.GetFilters(), allowedColumns)
if err != nil {
    // Returns error if protoRequest contains a field not in allowedColumns
    return err
}

Troubleshooting

Result Set Size

Repository List methods (like ListActions or ListTasks) are designed to return Limit + 1 items. This is intentional behavior used to determine if a "Next Page" exists without performing a separate count query. You should always trim the result set to the requested Limit before returning it to the user.

Limit Constraints

The NewListResourceInputFromProto helper in runs/repository/impl/requests.go enforces specific constraints:

• Default Limit: 50 (if 0 is provided).
• Maximum Limit: 200.
• Negative Limits: Will return an error.

Case Sensitivity

For text searches, use FilterExpressionContainsCaseInsensitive (or the contains_case_insensitive proto function), which generates LOWER(column) LIKE LOWER(?) in the underlying SQL. Standard NewEqualFilter and NewNotEqualFilter are case-sensitive depending on the database collation.