Downloader

No overview available.

Methods

---

handleBlob()

@classmethod
def handleBlob(
    ctx: context.Context,
    blob: *core.Blob,
    toPath: string
) - > interface{}, error

Handles a blob by writing its content to the specified path. Callers use this to persist binary data referenced by a blob.

Parameters

Name	Type	Description
ctx	context.Context	The context for the operation, used for cancellation and deadlines.
blob	*core.Blob	The blob object containing the data to be handled.
toPath	string	The file path where the blob's content should be written.

Returns

Type	Description
interface{}, error	A generic interface and an error if the operation fails.

---

handleSchema()

@classmethod
def handleSchema(
    ctx: context.Context,
    schema: *core.Schema,
    toFilePath: string
) - > interface{}, error

Handles a schema by writing its definition to the specified file path. This method is used to store schema definitions locally.

Parameters

Name	Type	Description
ctx	context.Context	The context for the operation, used for cancellation and deadlines.
schema	*core.Schema	The schema object to be handled.
toFilePath	string	The file path where the schema definition should be written.

Returns

Type	Description
interface{}, error	A generic interface and an error if the operation fails.

---

handleBinary()

@classmethod
def handleBinary(
    _: context.Context,
    b: *core.Binary,
    toFilePath: string,
    writeToFile: bool
) - > interface{}, error

Handles binary data, optionally writing it to a file. Callers use this to process or store raw binary content.

Parameters

Name	Type	Description
_	context.Context	The context for the operation, currently unused.
b	*core.Binary	The binary object containing the data to be handled.
toFilePath	string	The file path where the binary data should be written if 'writeToFile' is true.
writeToFile	bool	A boolean indicating whether the binary data should be written to 'toFilePath'.

Returns

Type	Description
interface{}, error	A generic interface and an error if the operation fails.

---

handleError()

@classmethod
def handleError(
    _: context.Context,
    b: *core.Error,
    toFilePath: string,
    writeToFile: bool
) - > interface{}, error

Handles an error object, optionally writing its details to a file. This is used to log or persist error information.

Parameters

Name	Type	Description
_	context.Context	The context for the operation, currently unused.
b	*core.Error	The error object to be handled.
toFilePath	string	The file path where the error details should be written if 'writeToFile' is true.
writeToFile	bool	A boolean indicating whether the error details should be written to 'toFilePath'.

Returns

Type	Description
interface{}, error	A generic interface and an error if the operation fails.

---

handleGeneric()

@classmethod
def handleGeneric(
    ctx: context.Context,
    b: *structpb.Struct,
    toFilePath: string,
    writeToFile: bool
) - > interface{}, error

Handles a generic struct, optionally writing its content to a file. This method provides a way to process arbitrary structured data.

Parameters

Name	Type	Description
ctx	context.Context	The context for the operation, used for cancellation and deadlines.
b	*structpb.Struct	The generic struct object to be handled.
toFilePath	string	The file path where the struct's content should be written if 'writeToFile' is true.
writeToFile	bool	A boolean indicating whether the struct's content should be written to 'toFilePath'.

Returns

Type	Description
interface{}, error	A generic interface and an error if the operation fails.

---

handlePrimitive()

@classmethod
def handlePrimitive(
    primitive: *core.Primitive,
    toFilePath: string,
    writeToFile: bool
) - > interface{}, error

Handles a primitive value, optionally writing it to a file. This method is used to process and store basic data types.

Parameters

Name	Type	Description
primitive	*core.Primitive	The primitive value to be handled.
toFilePath	string	The file path where the primitive value should be written if 'writeToFile' is true.
writeToFile	bool	A boolean indicating whether the primitive value should be written to 'toFilePath'.

Returns

Type	Description
interface{}, error	A generic interface and an error if the operation fails.

---

handleScalar()

@classmethod
def handleScalar(
    ctx: context.Context,
    scalar: *core.Scalar,
    toFilePath: string,
    writeToFile: bool
) - > interface{}, *core.Scalar, error

Handles a scalar value, optionally writing it to a file. This method processes and returns scalar data, potentially storing it.

Parameters

Name	Type	Description
ctx	context.Context	The context for the operation, used for cancellation and deadlines.
scalar	*core.Scalar	The scalar object to be handled.
toFilePath	string	The file path where the scalar value should be written if 'writeToFile' is true.
writeToFile	bool	A boolean indicating whether the scalar value should be written to 'toFilePath'.

Returns

Type	Description
interface{}, *core.Scalar, error	A generic interface, the processed scalar object, and an error if the operation fails.

---

handleLiteral()

@classmethod
def handleLiteral(
    ctx: context.Context,
    lit: *core.Literal,
    filePath: string,
    writeToFile: bool
) - > interface{}, *core.Literal, error

Handles a literal value, optionally writing it to a file. This method processes and returns literal data, potentially storing it.

Parameters

Name	Type	Description
ctx	context.Context	The context for the operation, used for cancellation and deadlines.
lit	*core.Literal	The literal object to be handled.
filePath	string	The file path where the literal value should be written if 'writeToFile' is true.
writeToFile	bool	A boolean indicating whether the literal value should be written to 'filePath'.

Returns

Type	Description
interface{}, *core.Literal, error	A generic interface, the processed literal object, and an error if the operation fails.

---

handleCollection()

@classmethod
def handleCollection(
    ctx: context.Context,
    c: *core.LiteralCollection,
    dir: string,
    writePrimitiveToFile: bool
) - > []interface{}, *core.LiteralCollection, error

Handles a collection of literals, potentially writing primitive elements to files within a specified directory. Callers use this to process and store collections of data.

Parameters

Name	Type	Description
ctx	context.Context	The context for the operation, used for cancellation and deadlines.
c	*core.LiteralCollection	The literal collection to be handled.
dir	string	The directory path where primitive elements of the collection should be written if 'writePrimitiveToFile' is true.
writePrimitiveToFile	bool	A boolean indicating whether primitive elements within the collection should be written to files in 'dir'.

Returns

Type	Description
[]interface{}, *core.LiteralCollection, error	A slice of generic interfaces representing the handled elements, the processed literal collection, and an error if the operation fails.

---

RecursiveDownload()

@classmethod
def RecursiveDownload(
    ctx: context.Context,
    inputs: *core.LiteralMap,
    dir: string,
    writePrimitiveToFile: bool
) - > [VarMap](varmap.md?sid=flytecopilot_data_varmap), *core.LiteralMap, error

Recursively downloads a map of literals to a specified directory, optionally writing primitive values to files. This method is used to fetch and store complex data structures.

Parameters

Name	Type	Description
ctx	context.Context	The context for the operation, used for cancellation and deadlines.
inputs	*core.LiteralMap	The literal map containing data to be downloaded.
dir	string	The directory path where downloaded files and primitive values should be stored.
writePrimitiveToFile	bool	A boolean indicating whether primitive values within the literal map should be written to individual files.

Returns

Type	Description
[VarMap](varmap.md?sid=flytecopilot_data_varmap), *core.LiteralMap, error	A map of variables, the processed literal map, and an error if the operation fails.

---

DownloadInputs()

@classmethod
def DownloadInputs(
    ctx: context.Context,
    inputRef: storage.DataReference,
    outputDir: string
) - > error

Downloads inputs referenced by a storage data reference to a specified output directory. Callers use this to retrieve and store input data for processing.

Parameters

Name	Type	Description
ctx	context.Context	The context for the operation, used for cancellation and deadlines.
inputRef	storage.DataReference	The data reference pointing to the inputs to be downloaded.
outputDir	string	The directory path where the downloaded inputs should be stored.

Returns

Type	Description
error	An error if the download operation fails.

---