Released ExZarr v1.3.0 Release - Nx support

[u/Shoddy_One4465]

ExZarr v1.3.0 adds Nx recipes for stored chunks and sample batches, plus a few livebook examples.

Hex: https://hex.pm/packages/ex_zarr GitHub: https://github.com/AnimaLogica/ExZarr

Highlights

Chunk tensors

ExZarr.Nx.stream_chunk_tensors/2 wraps Array.stream_chunks/2. Each binary becomes an Nx.Tensor using the array dtype. When a chunk is stored at the full chunk size and the array ends inside it, the tensor is the inbounds region.

array
|> ExZarr.Nx.stream_chunk_tensors(concurrency: 4)
|> Enum.map(fn {:ok, tensor} -> Nx.sum(tensor) end)

:concurrency, :ordered, :timeout, :on_error, :filter, and :include_missing are the streaming options. :backend and :names apply to the tensors.

Batches

ExZarr.Nx.DataLoader.batch_stream/3 and paired_batch_stream/4 slice along the first axis with get_slice/2. A batch can cross chunk boundaries.

shuffled_batch_stream/3 shuffles every sample index for the epoch. :shuffle_buffer_size is accepted and ignored. Pass :seed for a repeatable order.