fix: sampler docs

[ilan-gold]

No description provided.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.80%. Comparing base (c9a5971) to head (a4c94e9).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #125   +/-   ##
=======================================
  Coverage   90.80%   90.80%           
=======================================
  Files          10       10           
  Lines         805      805           
=======================================
  Hits          731      731           
  Misses         74       74           

Files with missing lines	Coverage Δ
src/annbatch/abc/sampler.py	100.00% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[felix0097]

I would definitely add an example here @selmanozleyen on how to actually implement this. One very simple example:

1. How do I write out my own sampler (one example class, e.g. weighted sampler, or fully random sampler)
2. How do I plug this into the rest of your code base e.g. how to actually use this

Moreover, did you check how this behaves on the full Tahoe dataset for e.g. fully random sampling. With the old code the memory usage blew up significantly. If this is still the case we should add a warning/caveats section

cc @ilan-gold

[felix0097]

@ilan-gold & @selmanozleyen I've added a detailed doc page on how to implement a custom sampler. Feel free to comment/edit

[felix0097]

ToDo: Update docs to reflect #127

[ilan-gold]

Maybe good to unalign the chunk boundaries to show that alignment with virtual chunk boundaries is not necessary?

[felix0097]

I've added a comment to clarify this 👍

[ilan-gold]

I think these should be unaligned still, which I am now more convinced of because it's a virtual concatenation of datasets, not zarr chunks (see below)

[ilan-gold]

Another reason I'd like the chunks to be a bit more complex - we order in-memory by on-disk dataset. So Chunk 0-99 definitely goes first. We could loosen things but that's how it works right now

[ilan-gold]

See https://annbatch--125.org.readthedocs.build/en/125/generated/annbatch.types.LoadRequest.html#annbatch.types.LoadRequest note at the bottom

[ilan-gold]

Still think these should be actually random indices

[ilan-gold]

Here again, would do Dataset 0 virtual-concatenation indices: 0-3