AsyncPlugin defines the interface for plugins that call Async Web APIs.
Methods
GetConfig()
@classmethod
def GetConfig() - > [PluginConfig](pluginconfig.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_pluginconfig)
GetConfig gets the loaded plugin config. This will be used to control the interactions with the remote service.
Returns
| Type | Description |
|---|
[PluginConfig](pluginconfig.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_pluginconfig) | The loaded plugin configuration. |
ResourceRequirements()
@classmethod
def ResourceRequirements(
ctx: context.Context,
tCtx: [TaskExecutionContextReader](taskexecutioncontextreader.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_taskexecutioncontextreader)
) - > (pluginsCore.ResourceNamespace, pluginsCore.ResourceConstraintsSpec, error)
ResourceRequirements analyzes the task to execute and determines the ResourceNamespace to be used when allocating tokens.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, typically used for cancellation and deadlines. |
| tCtx | [TaskExecutionContextReader](taskexecutioncontextreader.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_taskexecutioncontextreader) | The task execution context reader, providing information about the task to be executed. |
Returns
| Type | Description |
|---|
(pluginsCore.ResourceNamespace, pluginsCore.ResourceConstraintsSpec, error) | A tuple containing the resource namespace, resource constraints specification, and an error if the operation fails. |
Create()
@classmethod
def Create(
ctx: context.Context,
tCtx: [TaskExecutionContextReader](taskexecutioncontextreader.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_taskexecutioncontextreader)
) - > ([ResourceMeta](resourcemeta.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_resourcemeta), [Resource](resource.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_resource), error)
Create a new resource using the TaskExecutionContext provided. Ideally, the remote service uses the name in the TaskExecutionMetadata to launch the resource in an idempotent fashion. This function will be on the critical path of the execution of a workflow and therefore it should not do expensive operations before making the webAPI call. Flyte will call this api at least once. It's important that the callee service is idempotent to ensure no resource leakage or duplicate requests. Flyte has an in-memory cache that does a best effort idempotency guarantee. It's required to return a resourceMeta object (that will be cached in memory). In case the remote API returns the actually created resource, it's advisable to also return that in optionalResource output parameter. Doing so will instruct the system to call Status() immediately after Create() and potentially terminate early if the resource has already been executed/failed. If the remote API failed due to a system error (network failure, timeout... etc.), the plugin should return a non-nil error. The system will automatically retry the operation based on the plugin config.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, typically used for cancellation and deadlines. |
| tCtx | [TaskExecutionContextReader](taskexecutioncontextreader.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_taskexecutioncontextreader) | The task execution context reader, providing information needed to create the resource. |
Returns
| Type | Description |
|---|
([ResourceMeta](resourcemeta.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_resourcemeta), [Resource](resource.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_resource), error) | A tuple containing the resource metadata, an optional resource object if returned by the remote API, and an error if the creation fails. |
Get()
@classmethod
def Get(
ctx: context.Context,
tCtx: [GetContext](getcontext.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_getcontext)
) - > ([Resource](resource.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_resource), error)
Get the resource that matches the keys. If the plugin hits any failure, it should stop and return the failure. This API will be called asynchronously and periodically to update the set of tasks currently in progress. It's acceptable if this API is blocking since it'll be called from a background go-routine. Best practices: 1) Instead of returning the entire response object retrieved from the WebAPI, construct a smaller object that has enough information to construct the status/phase, error and/or output. 2) This object will NOT be serialized/marshaled. It's, therefore, not a requirement to make it so. 3) There is already client-side throttling in place. If the WebAPI returns a throttling error, you should return it as is so that the appropriate metrics are updated and the system administrator can update throttling params accordingly.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, typically used for cancellation and deadlines. |
| tCtx | [GetContext](getcontext.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_getcontext) | The context for retrieving the resource, containing the necessary keys. |
Returns
| Type | Description |
|---|
([Resource](resource.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_resource), error) | A tuple containing the latest state of the resource and an error if the retrieval fails. |
Delete()
@classmethod
def Delete(
ctx: context.Context,
tCtx: [DeleteContext](deletecontext.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_deletecontext)
) - > error
Delete the object in the remote service using the resource key. Flyte will call this API at least once. If the resource has already been deleted, the API should not fail.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, typically used for cancellation and deadlines. |
| tCtx | [DeleteContext](deletecontext.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_deletecontext) | The context for deleting the resource, containing the resource key. |
Returns
| Type | Description |
|---|
error | An error if the deletion fails, otherwise nil. |
Status()
@classmethod
def Status(
ctx: context.Context,
tCtx: [StatusContext](statuscontext.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_statuscontext)
) - > (pluginsCore.PhaseInfo, error)
Status checks the status of a given resource and translates it to a Flyte-understandable PhaseInfo. This API should avoid making any network calls and should run very efficiently.
Parameters
| Name | Type | Description |
|---|
| ctx | context.Context | The context for the operation, typically used for cancellation and deadlines. |
| tCtx | [StatusContext](statuscontext.md?sid=flyteplugins_go_tasks_pluginmachinery_webapi_statuscontext) | The context for checking the status of the resource. |
Returns
| Type | Description |
|---|
(pluginsCore.PhaseInfo, error) | A tuple containing the phase information of the resource and an error if the status check fails. |