ChunkBatcher (autotransform.batcher.chunk)

The implementation for the ChunkBatcher.

class autotransform.batcher.chunk.ChunkBatcher(*, chunk_size: int, title: str, max_chunks: int | None = None, metadata: Dict[str, Any] | None = None)

Bases: Batcher

A batcher which chunks Items in to several Batches. A chunk size is supplied that determines the size of Batches. A maximum number of chunks can optionally be supplied. If the chunk size would result in more than the maximum number of chunks, the chunk size is increased to the minimum required to create only max_chunks. The title for a batch will have [X/N] prepended to it, where X represents the chunk number and N represents the total number of chunks.

chunk_size

The size of chunks.

Type:

int

title

The title to use for Batches.

Type:

str

max_chunks

The maximum number of chunks to create. Defaults to None.

Type:

Optional[int], optional

metadata

The metadata to associate with Batches. Defaults to None.

Type:

Optional[Dict[str, Any]], optional

name

The name of the Component.

Type:

ClassVar[BatcherName]

batch(items: Sequence[Item]) → List[Batch]

Take filtered Items and chunk them into Batches.

Parameters:

items (Sequence[Item]) – The filtered Items to separate.

Returns:

A list of Batches representing each chunk of the filtered

Items.

Return type:

List[Batch]

chunk_size: int

max_chunks: int | None

metadata: Dict[str, Any] | None

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: ClassVar[BatcherName] = 'chunk'

title: str