chunking module¶

Extensions for chunking records and mapped arrays.

col_idxs_mapper GroupIdxsMapper¶

Default instance of GroupIdxsMapper for per-column indices.

col_lens_mapper GroupLensMapper¶

Default instance of GroupLensMapper for per-column lengths.

fix_field_in_records function¶

fix_field_in_records(
    record_arrays,
    chunk_meta,
    ann_args=None,
    mapper=None,
    field='col'
)

Fix a field of the record array in each chunk.

merge_records function¶

merge_records(
    results,
    chunk_meta,
    ann_args=None,
    mapper=None
)

Merge chunks of record arrays.

Mapper is only applied on the column field.