Transform

A transform modifies batches, splits, and assets. Transforms follow a scikit-learn-style API: optional fit() on the full dataset, then transform() applied to each shard. Transforms can be composed with Compose.

There are two execution modes:

• Offline transforms are applied via apply() and written to a new on-disk hash directory. Use these for expensive, dataset-wide operations (tokenization, graph construction, normalization).

• Live transforms are attached via live() and run at load time inside a benchmark loader. Use these for augmentations or featurization that should vary between epochs.

See Transforms for the full catalog.

class bioverse.transform.Compose(*transforms: Transform)

Bases: Transform

Apply a sequence of transforms in order.

Examples

from bioverse.transform import Compose
from bioverse.transforms import TokenizeResidues, KnnGraph

pipeline = Compose(TokenizeResidues(), KnnGraph(k=16))
batches, split, assets = pipeline(batches, split, assets)

class bioverse.transform.Transform

Bases: object

Modify batches, splits, and assets in a scikit-learn-style pipeline.

Transforms follow a fit → transform lifecycle. Offline transforms are applied via apply() and materialized to disk; live transforms run at load time via live().

Override transform_batch(), transform_split(), and/or transform_assets() to implement specific operations. Set filter to "scenes" to drop scenes marked by a scene_filter column.

Examples

Offline transform applied when building a dataset:

from bioverse.transforms import TokenizeResidues, KnnGraph

dataset.apply(TokenizeResidues(), KnnGraph(k=16))

Live augmentation attached to a benchmark:

from bioverse.transforms import Random2DRotate

benchmark.live(Random2DRotate())