# Zarr-Python 3.2.0 Store Triage

**Date:** 2026-04-29
**Scope:** `FsspecStore` regressions and adjacent bugs that affect 3.2.0 release readiness
**Last released version:** 3.1.6 (2026-03-19)
**Anchor PR:** [zarr-developers/zarr-python#3926](https://github.com/zarr-developers/zarr-python/pull/3926) "fix(storage): preserve leading slashes in FsspecStore.path"

## TL;DR

PR #3926 should merge before 3.2.0 ships. Without it, 3.2.0 introduces a new regression for any caller that passes an absolute filesystem path to `FsspecStore` (the titiler-xarray upstream test breakage is the canary). With #3926, the next release matches v3.1.6's path-handling contract on the affected join sites.

Two further changes belong in the same release window: extend the `_dereference_path` migration to the listing methods (closes a pre-existing `path="/"` bug invisible to current tests) and document the verbatim-path contract on `FsspecStore.__init__`. Both are inexpensive and stop the recurring bug cycle that produced #3924 in the first place.

Three pre-existing footguns are out of scope for 3.2.0 but should be tracked: `clear()` walking the filesystem root when `path="/"` and backend is `LocalFileSystem`; `MemoryFileSystem`'s leading-slash equality inconsistency; and `__eq__`/`__hash__` sensitivity to path surface form. None are new in 3.2.0; all are acceptable to defer if labeled.

The longer-term direction is composition + per-backend stores via `obstore`, with `FsspecStore` as the verbatim-path fallback. That is a 4.0 conversation, not a 3.2.0 conversation.

## Background

Path handling in `FsspecStore` has cycled through five PRs since 2024, each adding or removing a backend-agnostic path constraint that turned out to break some backend. The pattern is consistent enough that it is worth treating as the dominant risk in the store layer:

| PR | Change | Reverted by | Reason |
|---|---|---|---|
| [#2348](https://github.com/zarr-developers/zarr-python/pull/2348) | "raise error if path includes scheme" | [#3343](https://github.com/zarr-developers/zarr-python/pull/3343) | swift-fs needs scheme in path |
| [#3193](https://github.com/zarr-developers/zarr-python/pull/3193) | check on `auto_mkdir` | [#3193](https://github.com/zarr-developers/zarr-python/pull/3193) (same) | not all backends support `auto_mkdir` |
| [#3679](https://github.com/zarr-developers/zarr-python/pull/3679) | refactor `_dereference_path` to `_join_paths` | [#3926](https://github.com/zarr-developers/zarr-python/pull/3926) (in progress) | broke `path="/"` for ReferenceFileSystem ([#3922](https://github.com/zarr-developers/zarr-python/issues/3922)) |
| [#3924](https://github.com/zarr-developers/zarr-python/pull/3924) | apply `normalize_path` to constructor | [#3926](https://github.com/zarr-developers/zarr-python/pull/3926) (in progress) | stripped leading `/` from absolute LocalFileSystem paths |
| [#3926](https://github.com/zarr-developers/zarr-python/pull/3926) | restore verbatim path + `_dereference_path` | (open) | restores v3.1.6 contract |

The repeated lesson is that path semantics are backend-specific and a single backend-agnostic normalization step cannot exist without breaking some other backend. Verbatim storage with a backend-side join helper (the v3.1.6 contract) is the only configuration that has been stable across all observed backends.

## Pre-3.2.0 Triage

These items are blockers or near-blockers for the 3.2.0 release. Each is concrete and small.

### 1. Merge #3926 (REGRESSION FIX, BLOCKING)

**Severity:** High. **Status:** Approved by maxrjones, awaiting merge.

Without this, 3.2.0 ships a new regression vs v3.1.6: any `FsspecStore` constructed with an absolute filesystem path (whether passed directly, or returned by `from_url("file:///...")`, or extracted from `from_upath`) will silently turn into a CWD-relative path because `normalize_path("/home/foo")` returns `"home/foo"`. Confirmed downstream impact: `titiler-xarray` test-upstream job fails on every `dataset_3d.zarr` fixture access.

### 2. Extend `_dereference_path` to listing methods (PRE-EXISTING BUG, SHOULD-FIX)

**Severity:** Medium. **Status:** Not addressed by #3926 in its current form.

`list`, `list_dir`, and `list_prefix` in `_fsspec.py` still build paths via `f"{self.path}/{prefix}"` and `f"{self.path}/"`. With `self.path == "/"` this produces `//{prefix}`, the exact bug class that #3922 reported on the read path. The bug is pre-existing in v3.1.6, but it surfaces in the same `ReferenceFileSystem`-with-`path="/"` workflow that #3926 advertises as fixed. Users who encounter the listing variant will reasonably file it as "the #3926 fix is incomplete."

The fix is small:

async def list_dir(self, prefix: str) -> AsyncIterator[str]:
    full_prefix = _dereference_path(self.path, prefix.rstrip("/"))
    try:
        allfiles = await self.fs._ls(full_prefix, detail=False)
    except FileNotFoundError:
        return
    for onefile in allfiles:
        yield _relativize_path(path=onefile, prefix=self.path)

async def list_prefix(self, prefix: str) -> AsyncIterator[str]:
    full_prefix = _dereference_path(self.path, prefix)
    async for onefile in self.fs._find(full_prefix, detail=False, maxdepth=None, withdirs=False):
        yield _relativize_path(path=onefile, prefix=self.path)

async def list(self) -> AsyncIterator[str]:
    allfiles = await self.fs._find(self.path, detail=False, withdirs=False)
    for onefile in allfiles:
        yield _relativize_path(path=onefile, prefix=self.path)

Add parametrized listing tests against `ReferenceFileSystem` with `path="/"`, `path=""`, and `path="prefix"`. The fixture infrastructure is already in place from #3926.

**Recommendation:** roll into #3926 or open as #3927 with explicit "blocker for 3.2.0" label.

### 3. Document the verbatim-path contract (PROCESS, SHOULD-FIX)

**Severity:** Medium (process risk). **Status:** Not addressed.

The five-PR cycle in the table above happened because no contributor saw a written-down contract for what `FsspecStore.path` is supposed to mean. Each "let's normalize this" PR was written in good faith. Without a docstring committing to verbatim semantics, the cycle will continue. This is the cheapest single change with the highest leverage.

**Recommendation:** add the docstring note below to `FsspecStore.__init__`. Two lines:

> `path` is stored verbatim and combined with the zarr key via `_dereference_path` at I/O time. No normalization is applied at construction. Backend-specific path validation (rejecting `..`, normalizing backslashes, requiring leading `/`) is the responsibility of the caller or the underlying fsspec filesystem's `_strip_protocol`.

### 4. Test coverage for `from_url` and `from_upath` round-trips with absolute paths

**Severity:** Low-medium (CI coverage gap). **Status:** Not addressed.

The titiler-xarray regression was caught by a downstream CI job, not by zarr-python's own tests. Add tests that:

- Construct `FsspecStore.from_url("file:///tmp/test.zarr")` and verify `store.path == "/tmp/test.zarr"`.
- Construct `FsspecStore.from_upath(UPath("/tmp/test.zarr"))` and verify the same.
- Read a metadata document from each, exercising the join site against `LocalFileSystem`.

Without these, the next normalize-style refactor will reproduce #3924 with no zarr-python-side test signal.

### 5. Decide on listing test for `path=""` against multiple backends

**Severity:** Low. **Status:** Not addressed.

`list()` calls `self.fs._find(self.path, ...)` directly. With `self.path == ""`, behavior is backend-specific (S3 errors, LocalFileSystem lists CWD, MemoryFileSystem lists everything, ReferenceFileSystem enumerates all refs). This is not a bug being introduced; it is a divergence the test matrix does not currently document. Either add a parametrized test that pins each backend's actual behavior (so we notice if it changes) or accept the divergence and document it.

## Post-3.2.0 Triage

These items are real and non-trivial but should not gate 3.2.0. Each should be filed as an issue with the linked context.

### A. `clear()` catastrophic deletion when `path="/"` on `LocalFileSystem`

**Severity:** Catastrophic when triggered, low likelihood. **Status:** Pre-existing in v3.1.6; unchanged by #3926.

`FsspecStore(LocalFileSystem(asynchronous=True), path="/").clear()` calls `self.fs._find("/", withdirs=True)` and recursively `_rm`'s every result. The default `path="/"` makes this constructible with no explicit footgun. Possible mitigations, in order of intrusiveness:

1. Have `clear()` refuse to operate when `self.path` is `""` or `"/"` and the backend looks filesystem-shaped. Conservative; a small allow-list of "definitely safe" backends (memory, reference) opts in.
2. Refuse to construct `FsspecStore(LocalFileSystem, path="/")` at all and require an explicit path. Backwards-incompatible; many users have working code.
3. Change the default `path` from `"/"` to `""`. Backwards-incompatible for ReferenceFileSystem users who relied on the default sentinel.

**Recommendation:** option 1, behind a `_is_filesystem_root_unsafe(self.fs, self.path)` predicate, with a clear error message pointing at the predicate. Open as a labeled issue.

### B. `MemoryFileSystem` leading-slash equality inconsistency

**Severity:** Medium (silent incorrectness for cache/dedup), low likelihood. **Status:** Pre-existing; surfaced more visibly by the verbatim contract.

`MemoryFileSystem._strip_protocol` rewrites `"foo"` to `"/foo"` internally. So `FsspecStore(mem, path="foo")` and `FsspecStore(mem, path="/foo")` address the same bytes (I/O works) but `__eq__` returns False (cache and dedup misbehave). Fix is either documentation (acceptable) or a `MemoryFileSystem`-specific construction-time normalization (controversial because it reintroduces backend-specific normalization, which is the thing we just got out of).

**Recommendation:** document, do not fix. Track as a known quirk.

### C. `__eq__` and `__hash__` sensitivity to surface form

**Severity:** Low-medium. **Status:** Pre-existing.

`FsspecStore(fs, "foo")` and `FsspecStore(fs, "foo/")` are unequal under verbatim storage, even though they address the same bytes. Affects code that uses stores as dict keys or for cache deduplication. Probably rare in practice; should be called out in the class docstring.

**Recommendation:** docstring note. No code change.

### D. Composition-based store redesign

**Severity:** Architectural. **Status:** Out of scope for 3.2.0; tracked in [zarr-python-planning](https://github.com/d-v-b/zarr-python-planning).

The recurring path-handling bug pattern is a symptom of a broader issue: stores extend by inheritance rather than by composition of capability protocols, so backend-specific behavior gets baked into the generic class instead of into structurally-typed adapters. The zarr-python-planning README's `Stores` section describes the direction (capability protocols, `obstore`-backed stores for cloud, family-level fsspec subclasses for the long tail). This is 4.0 work, not 3.2.0 work.

### E. Custom fsspec backends have no validation surface

**Severity:** Low. **Status:** Pre-existing.

HuggingFace, Dropbox, WebDAV, and other community fsspec backends get the verbatim contract with no construction-time validation. Adding an optional `validate_path: Callable[[str], None] | None` parameter to `FsspecStore.__init__` (default `None`) gives users a place to plug in backend-specific checks without zarr-python having to ship a class for every backend. One-hour change, but the API decision (sync vs async, single-shot vs per-method) deserves discussion.

**Recommendation:** open as RFC issue, target 3.3.0 or 4.0.

### F. Symlink escape from `clear()` and `delete_dir`

**Severity:** Medium when triggered, low likelihood. **Status:** Pre-existing fsspec/POSIX semantics.

`LocalFileSystem._find` follows symlinks. A symlink inside the store root that points outside the store root means `clear()` can recursively delete files outside the store. Not a path-handling bug per se; an interaction between fsspec's traversal semantics and zarr's `clear()` implementation. Hardening would require either `_find` with `follow_symlinks=False` (not supported by all backends) or a post-traversal filter that rejects results outside `self.path`.

**Recommendation:** track but do not gate any release on it.

## Risk Register

| Issue | Severity | Likelihood | Status if 3.2.0 ships as-is | Status if 3.2.0 ships with #3926 + listing fix |
|---|---|---|---|---|
| Absolute LocalFileSystem path stripped (#3924 regression) | High | High (any abs-path user) | **Regression vs v3.1.6** | Fixed |
| ReferenceFileSystem `path="/"` `//key` on read | High | Medium (kerchunk users) | Fixed by #3924 side effect | Fixed by #3926 directly |
| ReferenceFileSystem `path="/"` `//key` on listing | Medium | Medium (ref-fs listing users) | **Pre-existing, still broken** | **Fixed (if listing migration included)** |
| `clear()` walks filesystem root with `path="/"` LocalFS | Catastrophic | Low | Pre-existing | Pre-existing |
| MemoryFileSystem leading-`/` equality | Medium | Low | Pre-existing | Pre-existing |
| `__eq__`/`__hash__` surface-form sensitivity | Low | Low | Pre-existing | Pre-existing |
| Symlink escape in `clear()` | Medium | Low | Pre-existing | Pre-existing |
| Custom backends have no validation hook | Low | Low | Pre-existing | Pre-existing |

**Net effect of merging #3926:** one regression fixed, no pre-existing bugs newly introduced.

**Net effect of merging #3926 + listing migration:** one regression fixed plus one pre-existing bug fixed.

## Recommended Sequencing

1. **This week:** merge #3926. Add the constructor docstring note. Either extend the listing methods in the same PR or land it as #3927 within the same release branch.
2. **Before 3.2.0 RC:** add the `from_url` and `from_upath` absolute-path round-trip tests. Confirm the titiler-xarray downstream job passes against the RC.
3. **Open as 3.3.0 / 4.0 issues, do not block 3.2.0:** `clear()` filesystem-root guard (item A), `validate_path` hook RFC (item E), symlink-escape hardening (item F), MemoryFileSystem and equality docstring updates (items B and C).
4. **Track in [zarr-python-planning](https://github.com/d-v-b/zarr-python-planning):** composition-based store redesign (item D). The README's `Stores` section is the right home for the design discussion; specific issues should land in the zarr-python repo when they have concrete proposals.

## Appendix: What Was a Bug When?

| Scenario | v3.1.6 (last release) | main without #3926 (would be 3.2.0) | main with #3926 (recommended 3.2.0) |
|---|---|---|---|
| `FsspecStore(ref_fs, path="/").get("zarr.json")` | Works (`_dereference_path` from `_common.py`) | Works (`normalize_path("/")` → `""`, `_join_paths` collapses) | Works (`_dereference_path` restored) |
| `FsspecStore(local_fs, path="/home/foo").get("zarr.json")` | Works | **BROKEN** (path becomes `home/foo`, relative to CWD) | Works |
| `FsspecStore(ref_fs, path="/").list_prefix("a")` | Broken (`f"//a"`) | Broken (`f"/a"` after normalize) | Broken (same as v3.1.6) unless listing migration also lands |
| `FsspecStore(local_fs, path="/").clear()` | Catastrophic | Less catastrophic (path becomes `""`) | Catastrophic (same as v3.1.6) |
| `FsspecStore(mem, path="foo") == FsspecStore(mem, path="/foo")` | False | True (both normalize to `foo`) | False (same as v3.1.6) |

The key cell: column 2 row 2 is the regression that gates the release. Column 3 row 3 is the pre-existing bug that the listing migration would close.