This class implements DataStore to interact with a stow location store. It manages a base container and can dynamically load other containers if enabled, recording them in a map. It also includes metrics for tracking operations.
Methods
CreateContainer()
@classmethod
def CreateContainer(
ctx: context.Context,
container: string
) - > stow.Container, error
Creates a new container in the stow location store. This method is used to provision new storage containers.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, allowing for cancellation and timeouts. |
| container | string | The name of the container to create. |
Returns
| Type | Description |
|---|
stow.Container, error | The newly created stow container and an error if the operation failed. |
createContainer()
@classmethod
def createContainer(
ctx: context.Context,
locID: locationID,
container: string
) - > stow.Container, error
Creates a new container in the stow location store for a specific location ID. This method is used to provision new storage containers.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, allowing for cancellation and timeouts. |
| locID | locationID | The identifier of the location where the container should be created. |
| container | string | The name of the container to create. |
Returns
| Type | Description |
|---|
stow.Container, error | The newly created stow container and an error if the operation failed. |
LoadContainer()
@classmethod
def LoadContainer(
ctx: context.Context,
container: string,
createIfNotFound: bool
) - > stow.Container, error
Loads an existing container from the stow location store. If the container does not exist and createIfNotFound is true, it will create a new container.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, allowing for cancellation and timeouts. |
| container | string | The name of the container to load. |
| createIfNotFound | bool | A boolean indicating whether to create the container if it does not already exist. |
Returns
| Type | Description |
|---|
stow.Container, error | The loaded or newly created stow container and an error if the operation failed. |
loadContainer()
@classmethod
def loadContainer(
ctx: context.Context,
locID: locationID,
container: string,
createIfNotFound: bool
) - > stow.Container, error
Loads an existing container from the stow location store for a specific location ID. If the container does not exist and createIfNotFound is true, it will create a new container.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, allowing for cancellation and timeouts. |
| locID | locationID | The identifier of the location from which to load the container. |
| container | string | The name of the container to load. |
| createIfNotFound | bool | A boolean indicating whether to create the container if it does not already exist. |
Returns
| Type | Description |
|---|
stow.Container, error | The loaded or newly created stow container and an error if the operation failed. |
getContainer()
@classmethod
def getContainer(
ctx: context.Context,
locID: locationID,
container: string
) - > c stow.Container, err error
Retrieves a container from the stow location store for a specific location ID. This method is used to access an existing container.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, allowing for cancellation and timeouts. |
| locID | locationID | The identifier of the location from which to get the container. |
| container | string | The name of the container to retrieve. |
Returns
| Type | Description |
|---|
c stow.Container, err error | The retrieved stow container and an error if the container could not be found or accessed. |
Head()
@classmethod
def Head(
ctx: context.Context,
reference: [DataReference](datareference.md?sid=flytestdlib_storage_datareference)
) - > Metadata, error
Fetches metadata for a data reference without retrieving the actual data. This is useful for checking existence or properties of an item.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, allowing for cancellation and timeouts. |
| reference | [DataReference](datareference.md?sid=flytestdlib_storage_datareference) | A reference to the data item for which to retrieve metadata. |
Returns
| Type | Description |
|---|
Metadata, error | The metadata associated with the data reference and an error if the item does not exist or cannot be accessed. |
List()
@classmethod
def List(
ctx: context.Context,
reference: [DataReference](datareference.md?sid=flytestdlib_storage_datareference),
maxItems: int,
cursor: Cursor
) - > [][DataReference](datareference.md?sid=flytestdlib_storage_datareference), Cursor, error
Lists data references within a specified container or prefix, with pagination support. This allows callers to browse the contents of a storage location.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, allowing for cancellation and timeouts. |
| reference | [DataReference](datareference.md?sid=flytestdlib_storage_datareference) | A reference to the container or prefix to list items from. |
| maxItems | int | The maximum number of items to return in a single listing operation. |
| cursor | Cursor | A cursor token to continue listing from a previous point, used for pagination. |
Returns
| Type | Description |
|---|
[][DataReference](datareference.md?sid=flytestdlib_storage_datareference), Cursor, error | A slice of data references, a cursor for the next page of results, and an error if the listing failed. |
ReadRaw()
@classmethod
def ReadRaw(
ctx: context.Context,
reference: [DataReference](datareference.md?sid=flytestdlib_storage_datareference)
) - > io.ReadCloser, error
Reads the raw content of a data item. This method provides direct access to the stored data.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, allowing for cancellation and timeouts. |
| reference | [DataReference](datareference.md?sid=flytestdlib_storage_datareference) | A reference to the data item whose raw content is to be read. |
Returns
| Type | Description |
|---|
io.ReadCloser, error | A ReadCloser for the item's content and an error if the item cannot be read or found. The caller is responsible for closing the ReadCloser. |
WriteRaw()
@classmethod
def WriteRaw(
ctx: context.Context,
reference: [DataReference](datareference.md?sid=flytestdlib_storage_datareference),
size: int64,
opts: [Options](../../flyteplugins/go/tasks/pluginmachinery/k8s/options.md?sid=flyteplugins_go_tasks_pluginmachinery_k8s_options),
raw: io.Reader
) - > error
Writes raw content to a data item, creating or overwriting it. This method is used to store data in the stow location.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, allowing for cancellation and timeouts. |
| reference | [DataReference](datareference.md?sid=flytestdlib_storage_datareference) | A reference to the data item where the raw content will be written. |
| size | int64 | The size of the raw content in bytes. |
| opts | [Options](../../flyteplugins/go/tasks/pluginmachinery/k8s/options.md?sid=flyteplugins_go_tasks_pluginmachinery_k8s_options) | Optional parameters for the write operation, such as content type or metadata. |
| raw | io.Reader | The raw content to be written to the data item. |
Returns
| Type | Description |
|---|
error | An error if the write operation failed. |
Delete()
@classmethod
def Delete(
ctx: context.Context,
reference: [DataReference](datareference.md?sid=flytestdlib_storage_datareference)
) - > error
Deletes a data item from the stow location store. This method removes the specified item permanently.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, allowing for cancellation and timeouts. |
| reference | [DataReference](datareference.md?sid=flytestdlib_storage_datareference) | A reference to the data item to be deleted. |
Returns
| Type | Description |
|---|
error | An error if the delete operation failed. |
GetBaseContainerFQN()
@classmethod
def GetBaseContainerFQN(
ctx: context.Context
) - > [DataReference](datareference.md?sid=flytestdlib_storage_datareference)
Retrieves the fully qualified name of the base container configured for this StowStore. This provides information about the default storage location.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, allowing for cancellation and timeouts. |
Returns
| Type | Description |
|---|
[DataReference](datareference.md?sid=flytestdlib_storage_datareference) | The DataReference representing the fully qualified name of the base container. |
CreateSignedURL()
@classmethod
def CreateSignedURL(
ctx: context.Context,
reference: [DataReference](datareference.md?sid=flytestdlib_storage_datareference),
properties: [SignedURLProperties](signedurlproperties.md?sid=flytestdlib_storage_signedurlproperties)
) - > [SignedURLResponse](signedurlresponse.md?sid=flytestdlib_storage_signedurlresponse), error
Generates a pre-signed URL for a data item, allowing temporary access without requiring direct authentication. This is useful for sharing private content securely.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, allowing for cancellation and timeouts. |
| reference | [DataReference](datareference.md?sid=flytestdlib_storage_datareference) | A reference to the data item for which to create a signed URL. |
| properties | [SignedURLProperties](signedurlproperties.md?sid=flytestdlib_storage_signedurlproperties) | Properties defining the signed URL, such as expiration time and permissions. |
Returns
| Type | Description |
|---|
[SignedURLResponse](signedurlresponse.md?sid=flytestdlib_storage_signedurlresponse), error | A SignedURLResponse containing the generated URL and an error if the operation failed. |
getLocation()
@classmethod
def getLocation(
id: locationID
) - > stow.Location
Retrieves a stow location by its identifier. This internal method is used to access specific storage locations.
Parameters
| Name | Type | Description |
|---|
| id | locationID | The identifier of the location to retrieve. |
Returns
| Type | Description |
|---|
stow.Location | The stow location corresponding to the provided ID. |