PackableSample

PackableSample()

Base class for samples that can be serialized with msgpack.

This abstract base class provides automatic serialization/deserialization for dataclass-based samples. Fields annotated as NDArray or NDArray | None are automatically converted between numpy arrays and bytes during packing/unpacking.

Subclasses should be defined either by: 1. Direct inheritance with the @dataclass decorator 2. Using the @packable decorator (recommended)

Examples

>>> @packable
... class MyData:
...     name: str
...     embeddings: NDArray
...
>>> sample = MyData(name="test", embeddings=np.array([1.0, 2.0]))
>>> packed = sample.packed  # Serialize to bytes
>>> restored = MyData.from_bytes(packed)  # Deserialize

Attributes

Name	Description
as_wds	Serialize for writing to WebDataset (__key__ + msgpack).
packed	Serialize to msgpack bytes. NDArray fields are auto-converted.

Methods

Name	Description
from_bytes	Create an instance from raw msgpack bytes.
from_data	Create an instance from unpacked msgpack data.

from_bytes

PackableSample.from_bytes(bs)

Create an instance from raw msgpack bytes.

from_data

PackableSample.from_data(data)

Create an instance from unpacked msgpack data.