ScriptTransformer (autotransform.transformer.script)

The implementation for the ScriptTransformer.

class autotransform.transformer.script.ScriptTransformer(*, args: List[str], script: str, timeout: int, chunk_size: int | None = None)

Bases: Transformer[NoneType]

A Transformer that makes changes using an invoked script. Sentinel values can be used in args to provide custom arguments for a run. The available sentinel values for args are:

<<KEY>>: A list of the Items for a Batch. If the per_item flag is set this will simply

be the key of an Item.

<<TARGET_PATH>>: A list of target_path extra_data field for a Batch. If the per_item flag

is set this will simply be the target_path of an Item.

<<EXTRA_DATA>>: A JSON encoded mapping from Item key to that Item’s extra_data. If the

per_item flag is set, this will simply be a JSON encoding of the Item’s extra_data. If extra_data is not present for an item, it is treated as an empty Dict.

<<METADATA>>: A JSON encoded version of the Batch’s metadata.

_FILE can be appended to any of these (i.e. <<KEY_FILE>>) and the arg will instead be replaced

with a path to a file containing the value.

Additionally, <<EXTRA_DATA/some_key>> can be used as a sentinel value to get a list of all the values from extra_data for each item with that key. Values will be converted to strings and returned as a list.

args

The arguments to supply to the script.

Type:

List[str]

script

The script to run.

Type:

str

timeout

The timeout to use for the script process.

Type:

int

chunk_size

The size of chunks to operate on. None indicates no chunking. Defaults to None.

Type:

Optional[int], optional

name

The name of the Component.

Type:

ClassVar[TransformerName]

args: List[str]

chunk_size: int | None

classmethod chunk_size_must_be_positive(v: int | None) → int | None

Validates that chunk

Parameters:

v (Optional[int]) – The chunk_size that was set.

Raises:

ValueError – Raised if chunk_size is not None and is less than 1.

Returns:

The validated chunk_size.

Return type:

Optional[int]

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: ClassVar[TransformerName] = 'script'

classmethod per_item_legacy_setting_validator(values: Dict[str, Any]) → Dict[str, Any]

Validates chunk size using legacy per_item setting.

Parameters:

values (Dict[str, Any]) – The values used to configure the ScriptTransformer.

Raises:

ValueError – Raised if per item is set with a chunk_size that is not 1.

Returns:

The fixed values.

Return type:

Mapping[str, Any]

script: str

timeout: int

transform(batch: Batch) → None

Executes a simple script to transform the given Batch. Sentinel values can be used in args to provide custom arguments for a run. The available sentinel values for args are:

<<KEY>>: A json encoded list of the Items for a Batch. <<EXTRA_DATA>>: A JSON encoded mapping from Item key to that Item’s extra_data.

If extra_data is not present for an item, it is treated as an empty Dict.

<<METADATA>>: A JSON encoded version of the Batch’s metadata.

_FILE can be appended to any of these (i.e. <<KEY_FILE>>) and the arg will instead be replaced with a path to a file containing the value.

Parameters:

batch (Batch) – The batch that will be transformed.