Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 89 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
[![codecov](https://codecov.io/gh/JaneliaSciComp/janelia_pathlib/branch/main/graph/badge.svg)](https://codecov.io/gh/JaneliaSciComp/janelia_pathlib)
[![PyPI](https://img.shields.io/pypi/v/janelia-pathlib.svg)](https://pypi.org/project/janelia-pathlib)

Translate Janelia file share paths to and from an unambiguous `janelia://` URI, so code can refer to a share without depending on OS-specific mount guesses.
Translate Janelia file share paths to and from an unambiguous `janelia:` URI, so code can refer to a share without depending on OS-specific mount guesses.

Janelia file shares are mounted at different paths on Linux, Mac, and Windows, and on macOS two different shares can even collide on the same guessed mount point (e.g. `/Volumes/<name>`). This package gives you two functions to move between a local path and a stable, host-qualified identity that's never ambiguous.
Janelia file shares are mounted at different paths on Linux, Mac, and Windows, and on macOS two different shares can even collide on the same guessed mount point (e.g. `/Volumes/<name>`). This package gives you two functions to move between a local path and a stable identity that's never ambiguous.

## Installation

Expand All @@ -27,42 +27,56 @@ Requires Python 3.11+. Zero runtime dependencies beyond `platformdirs`.
```python
from janelia_pathlib import to_janelia_uri, from_janelia_uri

# Canonicalize a local path to its unambiguous janelia:// identity
# Canonicalize a local path to its unambiguous janelia: identity
uri = to_janelia_uri("/labs/alpha/alphalab/sub/data")
# "janelia://fileserver.example.org/alphalab/sub/data"
# "janelia:/labs/alpha/alphalab/sub/data"

# Resolve a janelia:// URI back to a real, usable local path
# Resolve a janelia: URI back to a real, usable local path
path = from_janelia_uri(uri)
# "/Volumes/alphalab/sub/data" (on Mac), "/labs/alpha/alphalab/sub/data" (on Linux), ...
```

## Why `janelia://`?
## Why `janelia:`?

A share's local mount point differs by OS, and on macOS it isn't even guaranteed to be unique: two shares with the same share name on different servers (e.g. `host-a.example.org/data` and `host-b.example.org/data`) both nominally want `/Volumes/data` — macOS mounts the second at `/Volumes/data-1` instead. A bare path string like `/Volumes/data` can't tell you which share that actually is.

`janelia://host/share/subpath` encodes the SMB host and share name directly, which is always unique — so it's safe to store, log, or pass between machines, and always resolves back to the *correct* share regardless of what macOS happened to name the mountpoint.
`janelia:/<linux_path>` uses the share's Linux path as its identity. Linux mount points are fixed and admin-assigned, so they're unique and stable — unlike a macOS mountpoint name or a Windows drive letter, and unlike the SMB host, which changes if a share migrates to new hardware. The URI is safe to store, log, or pass between machines, and always resolves back to the *correct* share.

### Why one slash, not two?

`//` in a URI opens an [authority component](https://datatracker.ietf.org/doc/html/rfc3986#section-3.2) — the `host` in `https://host/path`. A Linux path has nothing to put there: a share's identity spans a variable number of leading segments (`/labs/alpha/home`, `/nrs/foxtrot`) and is only recoverable by prefix-matching the share table, so there's no fixed cut point. `janelia://labs/alpha/...` would silently parse `labs` — a storage tier — as the authority. The single-slash form says there is no authority, the same way `mailto:` and `urn:` do.

## API

### `to_janelia_uri(path)`

Translate a local path in the **current OS's own format** to its canonical `janelia://host/share/subpath` form. A path in a different OS's format (e.g. a Windows UNC path passed in while running on Linux) is not recognized — only `janelia://` URIs are portable across OSes; a plain path is assumed to describe the machine it's running on.
Translate a path to its canonical `janelia:/<linux_path>` form.

Three input formats are **self-describing** and resolve on any OS, regardless of the machine you're running on:

| Format | Example |
|---|---|
| Linux path | `/labs/alpha/alphalab/data` |
| Windows UNC | `\\fileserver.example.org\alphalab\data` |
| SMB URL | `smb://fileserver.example.org/alphalab/data` |

```python
from janelia_pathlib import to_janelia_uri

# On Linux:
# All three produce the same identity, on any OS:
to_janelia_uri("/labs/alpha/alphalab/data")
# "janelia://fileserver.example.org/alphalab/data"

# On Mac, once the share is mounted:
to_janelia_uri("/Volumes/alphalab/data")
# "janelia://fileserver.example.org/alphalab/data"
to_janelia_uri("smb://fileserver.example.org/alphalab/data")
# "janelia:/labs/alpha/alphalab/data"
```

**On macOS**, resolving a `/Volumes/...` path requires the share to be mounted already — `mac_path` alone can't be trusted as a guess (two shares can collide on the same mount name; see "Why `janelia://`?" above), so this checks live `mount` state rather than guessing. If the share isn't mounted, the path isn't recognized and passes through unchanged. Use `from_janelia_uri` first if you need to mount it.
Two formats are **inherently OS-local** and only resolve while running on that OS, with the share mounted:

| Format | Example | Why |
|---|---|---|
| macOS mountpoint | `/Volumes/alphalab/data` | The name is assigned from the share name alone, so two shares can collide (`/Volumes/data` vs `/Volumes/data-1`). Only the live `mount` table says which is which. |
| Windows drive letter | `Z:\data` | Per-user mapping state; the letter carries no information about the share. |

**Linux** paths always resolve without any live check, since Linux mount points are fixed and never ambiguous.
For these, the live mount table is consulted rather than guessing from `mac_path`/`windows_path` — a guess could be wrong in exactly the way the mount-name collision causes. If the share isn't mounted, the path passes through unchanged; use `from_janelia_uri` first if you need to mount it.

Paths that don't match any known share are returned unchanged:

Expand All @@ -71,23 +85,23 @@ to_janelia_uri("/tmp/local/scratch/file.txt")
# "/tmp/local/scratch/file.txt"
```

### `from_janelia_uri(path_str, *, timeout=30)`
### `from_janelia_uri(path_str, *, timeout=None)`

Translate a `janelia://host/share/subpath` string to a real, usable local path.
Translate a `janelia:/<linux_path>` string to a real, usable local path.

```python
from janelia_pathlib import from_janelia_uri

from_janelia_uri("janelia://fileserver.example.org/alphalab/data")
from_janelia_uri("janelia:/labs/alpha/alphalab/data")
# "/Volumes/alphalab/data" (on Mac)
# "/labs/alpha/alphalab/data" (on Linux)
```

On **macOS**, if the share isn't already mounted, this mounts it on demand (triggering the native SMB mount, which may prompt for credentials) and waits up to `timeout` seconds for the mount to appear, returning its *real* local mountpoint — not a guessed `/Volumes/<share>` path, which may not exist yet or may belong to a different, colliding share. Raises `RuntimeError` if the URI doesn't match a known share, or if mounting fails or times out.
On **macOS**, if the share isn't already mounted, this mounts it on demand (triggering the native SMB mount, which may prompt for credentials) and waits up to `timeout` seconds for the mount to appear, returning its *real* local mountpoint — not a guessed `/Volumes/<share>` path, which may not exist yet or may belong to a different, colliding share. `timeout` defaults to `JANELIA_PATHLIB_MOUNT_TIMEOUT` (30s). Raises `RuntimeError` if the URI doesn't match a known share, or if mounting fails or times out.

On **Linux** and **Windows**, no mounting step is needed — shares are already available at a fixed path (Linux) or accessible directly via UNC path (Windows).

A string that isn't a `janelia://` URI is returned unchanged:
A string that isn't a `janelia:` URI is returned unchanged:

```python
from_janelia_uri("/tmp/local/scratch/file.txt")
Expand All @@ -96,6 +110,31 @@ from_janelia_uri("/tmp/local/scratch/file.txt")

Both functions round-trip: `from_janelia_uri(to_janelia_uri(p))` resolves back to a valid local path for `p`'s share, and non-Janelia paths pass through both functions untouched.

### `janelia_path(path_str)`

An [`argparse`](https://docs.python.org/3/library/argparse.html) `type=` converter, for scripts that take a path on the command line:

```python
from janelia_pathlib import janelia_path

parser.add_argument("--data", type=janelia_path)
```

```console
$ my-script --data janelia:/labs/alpha/alphalab/data
```

The script receives `args.data` already converted to a local path, so it never has to care which OS it's on. Ordinary paths still work — anything that isn't a `janelia:` URI is passed through — so this is safe to add to an existing argument.

Failures become normal argparse errors rather than tracebacks:

```console
$ my-script --data janelia:/labs/nope/data
demo: error: argument --data: No known share for URI: janelia:/labs/nope/data
```

On macOS this mounts the share during `parse_args()`, which may block on a credential prompt. Set [`JANELIA_PATHLIB_NO_MOUNT`](#never-mounting) when nobody is there to answer.

## Share data

Share data is **not** bundled with the package — it's fetched from the [Fileglancer API](https://fileglancer.int.janelia.org/api/file-share-paths) and cached in your user cache directory:
Expand Down Expand Up @@ -138,6 +177,35 @@ for long; override it with:
export JANELIA_PATHLIB_FETCH_TIMEOUT=5
```

### Never mounting

On macOS, `from_janelia_uri` mounts a share that isn't mounted yet, which can
block on a credential prompt. To forbid that — so an unmounted share raises
instead — set:

```bash
export JANELIA_PATHLIB_NO_MOUNT=1
```

Already-mounted shares still resolve; this only stops new mounts. Use it in
cron jobs, CI, containers, or anywhere nobody is there to answer a prompt.

Whether a prompt can be answered is a property of the machine, not of the code
calling it, which is why this is an environment setting rather than a function
argument — the person running a script decides, not whoever wrote it.

### Mount timeout

When mounting *is* allowed, `from_janelia_uri` waits up to 30s for the mount to
appear. That wait covers a credential prompt, so it budgets for someone typing
a password — raise it for a slow authenticator:

```bash
export JANELIA_PATHLIB_MOUNT_TIMEOUT=90
```

Passing `timeout=` to `from_janelia_uri` overrides it per call.

## Development

This project uses [uv](https://docs.astral.sh/uv/) and
Expand Down
6 changes: 3 additions & 3 deletions src/janelia_pathlib/__init__.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
"""Translate Janelia file share paths to/from an unambiguous janelia:// URI."""

from importlib.metadata import PackageNotFoundError, version

from janelia_pathlib._path import from_janelia_uri, to_janelia_uri
from janelia_pathlib._argparse import janelia_path
from janelia_pathlib._translate import from_janelia_uri, to_janelia_uri

try:
__version__ = version("janelia-pathlib")
Expand All @@ -14,5 +13,6 @@
__all__ = [
"__version__",
"from_janelia_uri",
"janelia_path",
"to_janelia_uri",
]
41 changes: 41 additions & 0 deletions src/janelia_pathlib/_argparse.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
from __future__ import annotations

from argparse import ArgumentTypeError

from ._translate import from_janelia_uri


def janelia_path(path_str: str) -> str:
"""An ``argparse`` ``type=`` converter that turns a ``janelia:`` URI into a local path.

::

parser.add_argument("--data", type=janelia_path)

Anything that isn't a ``janelia:`` URI is passed through unchanged, so
an argument declared this way still accepts ordinary local paths.

On macOS this mounts the share if it isn't already, which may block on
a credential prompt during ``parse_args()``. Set
``JANELIA_PATHLIB_NO_MOUNT`` to turn that into an argument error
instead -- useful in headless or scripted contexts where nobody is
there to answer. Whether that applies is a property of the machine, not
of the script, so it's an environment setting.

Args:
path_str: The argument value, passed by ``argparse``.

Returns:
A local path in this OS's format, or ``path_str`` unchanged if it
isn't a ``janelia:`` URI.

Raises:
ArgumentTypeError: if the URI names no known share, or the share
can't be mounted.
"""
try:
return from_janelia_uri(path_str)
except RuntimeError as e:
# argparse only renders ArgumentTypeError/ValueError/TypeError as a
# clean "argument --x: ..." message; anything else is a traceback.
raise ArgumentTypeError(str(e)) from e
2 changes: 1 addition & 1 deletion src/janelia_pathlib/_fetch.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
The fetched shares.json is stored in the per-user cache directory so it
never lives in the source tree (which would leak internal Janelia paths
to a public repo). Invoked automatically on every use unless
JANELIA_PATHLIB_NO_FETCH is set (see ``_registry._load_shares``), and
JANELIA_PATHLIB_NO_FETCH is set (see ``_share.load_shares``), and
manually via the ``janelia-paths-fetch`` CLI.
"""

Expand Down
Loading
Loading