Skip to content

Zarr v3 #1104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jeromekelleher wants to merge 4 commits into from
Closed

Zarr v3 #1104

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a3169d4
Starting point for zarr v3 updates
jeromekelleher Mar 2, 2026
7419fe3
Detailed work for zarr v3
jeromekelleher Mar 2, 2026
133dfa5
Update uv lock for new bio2zarr
jeromekelleher Mar 3, 2026
9c0b417
Fix docs, hacking example data for zarr v3
jeromekelleher Mar 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/docs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ on:
merge_group:
jobs:
Docs:
uses: tskit-dev/.github/.github/workflows/docs.yml@v12
uses: tskit-dev/.github/.github/workflows/docs.yml@v13
29 changes: 0 additions & 29 deletions .github/workflows/tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,32 +38,3 @@ jobs:
matrix:
python: [ 3.11, 3.13 ]
os: [ macos-latest, ubuntu-24.04, windows-latest ]

zarr-v3-compatibility:
name: Test zarr v3 error
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4.2.2
with:
submodules: true

- name: Install uv and set the python version
uses: astral-sh/setup-uv@v6
with:
python-version: "3.12"
version: "0.8.15"

- name: Setup venv
run: |
uv venv

- name: Install zarr v3 and tsinfer
run: |
uv pip install -e .
uv pip install --upgrade zarr

- name: Test zarr v3 error
run: |
uv run --no-sync python -c "import tsinfer" 2>&1 | grep -q "zarr version.*is not supported"

8 changes: 4 additions & 4 deletions docs/_static/example_data.vcz/.zattrs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"contigs": [
"0"
],
"source": "sgkit-0.9.0"
"contigs": [
"0"
],
"source": "sgkit-0.9.0"
}
2 changes: 1 addition & 1 deletion docs/_static/example_data.vcz/.zgroup
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
{
"zarr_format": 2
"zarr_format": 2
}
244 changes: 1 addition & 243 deletions docs/_static/example_data.vcz/.zmetadata
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,243 +1 @@
{
"metadata": {
".zattrs": {
"contigs": [
"0"
],
"source": "sgkit-0.9.0"
},
".zgroup": {
"zarr_format": 2
},
"call_genotype/.zarray": {
"chunks": [
8,
3,
2
],
"compressor": {
"blocksize": 0,
"clevel": 5,
"cname": "lz4",
"id": "blosc",
"shuffle": 1
},
"dtype": "|i1",
"fill_value": null,
"filters": null,
"order": "C",
"shape": [
8,
3,
2
],
"zarr_format": 2
},
"call_genotype/.zattrs": {
"_ARRAY_DIMENSIONS": [
"variants",
"samples",
"ploidy"
],
"comment": "Call genotype. Encoded as allele values (0 for the reference, 1 for\nthe first allele, 2 for the second allele), -1 to indicate a\nmissing value, or -2 to indicate a non allele in mixed ploidy datasets.",
"mixed_ploidy": false
},
"call_genotype_mask/.zarray": {
"chunks": [
8,
3,
2
],
"compressor": {
"blocksize": 0,
"clevel": 5,
"cname": "lz4",
"id": "blosc",
"shuffle": 1
},
"dtype": "|i1",
"fill_value": null,
"filters": null,
"order": "C",
"shape": [
8,
3,
2
],
"zarr_format": 2
},
"call_genotype_mask/.zattrs": {
"_ARRAY_DIMENSIONS": [
"variants",
"samples",
"ploidy"
],
"comment": "A flag for each call indicating which values are missing.",
"dtype": "bool"
},
"call_genotype_phased/.zarray": {
"chunks": [
8,
3
],
"compressor": {
"blocksize": 0,
"clevel": 5,
"cname": "lz4",
"id": "blosc",
"shuffle": 1
},
"dtype": "|i1",
"fill_value": null,
"filters": null,
"order": "C",
"shape": [
8,
3
],
"zarr_format": 2
},
"call_genotype_phased/.zattrs": {
"_ARRAY_DIMENSIONS": [
"variants",
"samples"
],
"comment": "A flag for each call indicating if it is phased or not. If omitted\nall calls are unphased.",
"dtype": "bool"
},
"contig_id/.zarray": {
"chunks": [
1
],
"compressor": {
"blocksize": 0,
"clevel": 5,
"cname": "lz4",
"id": "blosc",
"shuffle": 1
},
"dtype": "<U1",
"fill_value": null,
"filters": null,
"order": "C",
"shape": [
1
],
"zarr_format": 2
},
"contig_id/.zattrs": {
"_ARRAY_DIMENSIONS": [
"contigs"
],
"comment": "Contig identifiers."
},
"sample_id/.zarray": {
"chunks": [
3
],
"compressor": {
"blocksize": 0,
"clevel": 5,
"cname": "lz4",
"id": "blosc",
"shuffle": 1
},
"dtype": "<U2",
"fill_value": null,
"filters": null,
"order": "C",
"shape": [
3
],
"zarr_format": 2
},
"sample_id/.zattrs": {
"_ARRAY_DIMENSIONS": [
"samples"
],
"comment": "The unique identifier of the sample."
},
"variant_allele/.zarray": {
"chunks": [
8,
2
],
"compressor": {
"blocksize": 0,
"clevel": 5,
"cname": "lz4",
"id": "blosc",
"shuffle": 1
},
"dtype": "|S1",
"fill_value": null,
"filters": null,
"order": "C",
"shape": [
8,
2
],
"zarr_format": 2
},
"variant_allele/.zattrs": {
"_ARRAY_DIMENSIONS": [
"variants",
"alleles"
],
"comment": "The possible alleles for the variant."
},
"variant_contig/.zarray": {
"chunks": [
8
],
"compressor": {
"blocksize": 0,
"clevel": 5,
"cname": "lz4",
"id": "blosc",
"shuffle": 1
},
"dtype": "<i8",
"fill_value": null,
"filters": null,
"order": "C",
"shape": [
8
],
"zarr_format": 2
},
"variant_contig/.zattrs": {
"_ARRAY_DIMENSIONS": [
"variants"
],
"comment": "Index corresponding to contig name for each variant. In some less common\nscenarios, this may also be equivalent to the contig names if the data\ngenerating process used contig names that were also integers."
},
"variant_position/.zarray": {
"chunks": [
8
],
"compressor": {
"blocksize": 0,
"clevel": 5,
"cname": "lz4",
"id": "blosc",
"shuffle": 1
},
"dtype": "<i8",
"fill_value": null,
"filters": null,
"order": "C",
"shape": [
8
],
"zarr_format": 2
},
"variant_position/.zattrs": {
"_ARRAY_DIMENSIONS": [
"variants"
],
"comment": "The reference position of the variant."
}
},
"zarr_consolidated_format": 1
}
{"metadata": {".zgroup": {"zarr_format": 2}, ".zattrs": {"contigs": ["0"], "source": "sgkit-0.9.0"}, "ancestral_state/.zattrs": {"_ARRAY_DIMENSIONS": ["variants"]}, "ancestral_state/.zarray": {"shape": [8], "chunks": [8], "dtype": "<U1", "fill_value": "", "order": "C", "filters": null, "dimension_separator": ".", "compressor": {"id": "blosc", "cname": "lz4", "clevel": 5, "shuffle": 1, "blocksize": 0}, "zarr_format": 2}, "call_genotype/.zattrs": {"_ARRAY_DIMENSIONS": ["variants", "samples", "ploidy"], "comment": "Call genotype. Encoded as allele values (0 for the reference, 1 for\nthe first allele, 2 for the second allele), -1 to indicate a\nmissing value, or -2 to indicate a non allele in mixed ploidy datasets.", "mixed_ploidy": false}, "call_genotype/.zarray": {"shape": [8, 3, 2], "chunks": [8, 3, 2], "dtype": "|i1", "fill_value": null, "order": "C", "filters": null, "dimension_separator": ".", "compressor": {"id": "blosc", "cname": "lz4", "clevel": 5, "shuffle": 1, "blocksize": 0}, "zarr_format": 2}, "call_genotype_mask/.zattrs": {"_ARRAY_DIMENSIONS": ["variants", "samples", "ploidy"], "comment": "A flag for each call indicating which values are missing.", "dtype": "bool"}, "call_genotype_mask/.zarray": {"shape": [8, 3, 2], "chunks": [8, 3, 2], "dtype": "|i1", "fill_value": null, "order": "C", "filters": null, "dimension_separator": ".", "compressor": {"id": "blosc", "cname": "lz4", "clevel": 5, "shuffle": 1, "blocksize": 0}, "zarr_format": 2}, "call_genotype_phased/.zattrs": {"_ARRAY_DIMENSIONS": ["variants", "samples"], "comment": "A flag for each call indicating if it is phased or not. If omitted\nall calls are unphased.", "dtype": "bool"}, "call_genotype_phased/.zarray": {"shape": [8, 3], "chunks": [8, 3], "dtype": "|i1", "fill_value": null, "order": "C", "filters": null, "dimension_separator": ".", "compressor": {"id": "blosc", "cname": "lz4", "clevel": 5, "shuffle": 1, "blocksize": 0}, "zarr_format": 2}, "contig_id/.zattrs": {"_ARRAY_DIMENSIONS": ["contigs"], "comment": "Contig identifiers."}, "contig_id/.zarray": {"shape": [1], "chunks": [1], "dtype": "|O", "fill_value": "", "order": "C", "filters": [{"id": "vlen-utf8"}], "dimension_separator": ".", "compressor": {"id": "blosc", "cname": "lz4", "clevel": 5, "shuffle": 1, "blocksize": 0}, "zarr_format": 2}, "sample_id/.zattrs": {"_ARRAY_DIMENSIONS": ["samples"], "comment": "The unique identifier of the sample."}, "sample_id/.zarray": {"shape": [3], "chunks": [3], "dtype": "<U2", "fill_value": null, "order": "C", "filters": null, "dimension_separator": ".", "compressor": {"id": "blosc", "cname": "lz4", "clevel": 5, "shuffle": 1, "blocksize": 0}, "zarr_format": 2}, "variant_allele/.zattrs": {"_ARRAY_DIMENSIONS": ["variants", "alleles"], "comment": "The possible alleles for the variant."}, "variant_allele/.zarray": {"shape": [8, 2], "chunks": [8, 2], "dtype": "|O", "fill_value": "", "order": "C", "filters": [{"id": "vlen-utf8"}], "dimension_separator": ".", "compressor": {"id": "blosc", "cname": "lz4", "clevel": 5, "shuffle": 1, "blocksize": 0}, "zarr_format": 2}, "variant_contig/.zattrs": {"_ARRAY_DIMENSIONS": ["variants"], "comment": "Index corresponding to contig name for each variant. In some less common\nscenarios, this may also be equivalent to the contig names if the data\ngenerating process used contig names that were also integers."}, "variant_contig/.zarray": {"shape": [8], "chunks": [8], "dtype": "<i8", "fill_value": null, "order": "C", "filters": null, "dimension_separator": ".", "compressor": {"id": "blosc", "cname": "lz4", "clevel": 5, "shuffle": 1, "blocksize": 0}, "zarr_format": 2}, "variant_position/.zattrs": {"_ARRAY_DIMENSIONS": ["variants"], "comment": "The reference position of the variant."}, "variant_position/.zarray": {"shape": [8], "chunks": [8], "dtype": "<i8", "fill_value": null, "order": "C", "filters": null, "dimension_separator": ".", "compressor": {"id": "blosc", "cname": "lz4", "clevel": 5, "shuffle": 1, "blocksize": 0}, "zarr_format": 2}}, "zarr_consolidated_format": 1}
41 changes: 23 additions & 18 deletions docs/_static/example_data.vcz/contig_id/.zarray
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,20 +1,25 @@
{
"chunks": [
1
],
"compressor": {
"blocksize": 0,
"clevel": 5,
"cname": "lz4",
"id": "blosc",
"shuffle": 1
},
"dtype": "<U4",
"fill_value": "",
"filters": null,
"order": "C",
"shape": [
1
],
"zarr_format": 2
"shape": [
1
],
"chunks": [
1
],
"dtype": "|O",
"fill_value": "",
"order": "C",
"filters": [
{
"id": "vlen-utf8"
}
],
"dimension_separator": ".",
"compressor": {
"id": "blosc",
"cname": "lz4",
"clevel": 5,
"shuffle": 1,
"blocksize": 0
},
"zarr_format": 2
}
8 changes: 4 additions & 4 deletions docs/_static/example_data.vcz/contig_id/.zattrs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"_ARRAY_DIMENSIONS": [
"contigs"
],
"comment": "Contig identifiers."
"_ARRAY_DIMENSIONS": [
"contigs"
],
"comment": "Contig identifiers."
}
Binary file modified docs/_static/example_data.vcz/contig_id/0
View file
Open in desktop
Binary file not shown.
Loading
Loading