1.3 webdav abstraction

Author: aritchie

readme.md

@@ -146,7 +146,7 @@ is built on the one below and they compose in the same app.
 | **Mediator** | Shiny.Mediator handlers published as endpoints — requests as JSON, commands as a status code, stream requests as Server-Sent Events, all bound at compile time |
 | **DocumentDb** | A document type as a complete HTTP resource, with filtering, cursor paging, sparse fieldsets, ETag/If-Match, RFC 7396 merge-patch, a live SSE tail, and server-side scopes enforced on both sides of a write |
 | **gRPC** | Unary, client-streaming, server-streaming and bidirectional methods, deadlines, per-message compression and status in trailers — plus gRPC-Web for browsers and anything on HTTP/1.1. Marshalling is yours, so nothing reflects over your messages |
-| **WebDAV** | RFC 4918 classes 1 and 2 over a directory — `PROPFIND`, `PROPPATCH`, `MKCOL`, `COPY`, `MOVE`, `LOCK`/`UNLOCK`, the `If` header and dead properties — so an app's storage mounts as a drive with no client to write |
+| **WebDAV** | RFC 4918 classes 1 and 2 over a directory — or over anything an `IWebDavFileSystem` describes — `PROPFIND`, `PROPPATCH`, `MKCOL`, `COPY`, `MOVE`, `LOCK`/`UNLOCK`, the `If` header and dead properties — so an app's storage mounts as a drive with no client to write |
 | **Lifecycle** | Start, stop and restart at runtime, serialized and idempotent, with an observable state — an embedded server gets toggled, not just booted. It also follows the device: rebinding when the addresses change, and following the app between foreground and background |
 | **Resilience** | Every state change says *why* it happened and carries the exception behind it, and a server that stops serving never does it quietly. A listener that dies underneath a running server is logged, reported and rebound; a transient accept failure is retried with backoff; a restart or a rebind whose bind is refused keeps trying and, if it never succeeds, says so unmistakably rather than leaving a server that claims to be running with nothing behind it. On by default — an app that has to opt in to not-silently-dying will not have opted in |
 | **Discovery** | The other half of hosting on a phone. mDNS advertises the server as the device moves and withdraws it when the server stops — telling a restart apart from a stop, so peers are not made to re-resolve a service that never went away — and a publication the responder refuses is retried rather than leaving an app that runs perfectly and is found by nobody; the locator turns what is on the link into a base address a client can call |

skills/shiny-httpserver/SKILL.md

@@ -222,6 +222,10 @@ triggers:
 - WebDavMethods
 - WebDavHeaderNames
 - IWebDavPropertyStore
+- IWebDavFileSystem
+- PhysicalWebDavFileSystem
+- WebDavEntry
+- WebDavException
 - InMemoryWebDavPropertyStore
 - WebDavLock
 - WebDavLockScope
@@ -852,6 +856,14 @@ app.MapWebDav("/dav", o =>
   is the same page `shinyhttpserver` serves — say "open the mount URL in a browser" when a user asks
   for a UI over a directory; there is no other file-manager UI in this library.
 - The mount is excluded from the OpenAPI document — do not try to describe it.
+- **Not one directory?** Implement `IWebDavFileSystem` and set `o.FileSystem` instead of `RootPath`
+  — several roots side by side, platform-API folders, a photo library. Paths arrive relative,
+  `/`-separated, root `""`, already stripped of `..`/dotfiles/`Filter` misses. `WebDavEntry.Name` is
+  the URL segment and `DisplayName` what the client shows. Refuse with
+  `throw new WebDavException(403)`; `UnauthorizedAccessException`→403,
+  `FileNotFound`/`DirectoryNotFound`→404, other `IOException`→409. Let the 413 the write stream throws
+  propagate. `OpenReadAsync` returning a seekable stream wins over the entry's length — buffer
+  anything transcoded on the fly. Do not copy the data into a temp directory to use `RootPath`.
 
 ## Realtime
 

src/Shiny.Net.HttpServer.WebDav/IWebDavFileSystem.cs

@@ -0,0 +1,165 @@
+namespace Shiny.Net.HttpServer.WebDav;
+
+/// <summary>
+/// What a WebDAV mount is serving: a tree of collections and files, addressed by path.
+/// <para>
+/// A mount answers the protocol - hrefs, depths, locks, <c>If</c> headers, the multistatus
+/// XML - and this answers what is actually there. <see cref="PhysicalWebDavFileSystem"/> is a
+/// directory on disk and is what <see cref="WebDavOptions.RootPath"/> builds. Implement this when
+/// what a mount should show is not one directory: an app whose storage is a view over several
+/// roots, a folder a platform only lets it reach through its own API, or something that is not
+/// files at all.
+/// </para>
+/// <para>
+/// <b>Paths</b> are relative to the mount root, separated by <c>/</c>, with no leading slash, and
+/// the root itself is the empty string. By the time one gets here it has been percent-decoded,
+/// checked for <c>.</c> and <c>..</c> segments, for dotfiles (unless
+/// <see cref="WebDavOptions.ServeHiddenFiles"/>) and against <see cref="WebDavOptions.Filter"/> -
+/// so an implementation never sees a path that climbs. What it still owns is anything only it can
+/// know, such as a link that leads out of whatever it treats as its root.
+/// </para>
+/// <para>
+/// <b>Refusing.</b> Throw <see cref="WebDavException"/> with the status the client should get.
+/// The mount also answers <see cref="UnauthorizedAccessException"/> with 403,
+/// <see cref="FileNotFoundException"/> and <see cref="DirectoryNotFoundException"/> with 404, and
+/// any other <see cref="IOException"/> with 409 - the ones <c>System.IO</c> throws, so an
+/// implementation over the file system rarely needs to translate anything.
+/// </para>
+/// <para>
+/// <b>Why the reads are synchronous.</b> <see cref="GetEntry"/> and <see cref="GetChildren"/> are
+/// asked from inside the evaluation of an <c>If</c> header, which is synchronous, and a
+/// <c>PROPFIND</c> asks them once per resource it describes. They are metadata, which every file
+/// system answers without waiting on the bytes; the members that move bytes are asynchronous.
+/// </para>
+/// </summary>
+public interface IWebDavFileSystem
+{
+    /// <summary>
+    /// What is at <paramref name="path"/>, or null when nothing is. Must answer for the root
+    /// (<c>""</c>), which must be a collection.
+    /// </summary>
+    WebDavEntry? GetEntry(string path);
+
+    /// <summary>
+    /// The members of the collection at <paramref name="path"/>, in any order - the mount sorts
+    /// them. Each entry's <see cref="WebDavEntry.Name"/> is the segment that, appended to
+    /// <paramref name="path"/>, addresses it.
+    /// </summary>
+    IEnumerable<WebDavEntry> GetChildren(string path);
+
+    /// <summary>
+    /// Opens a file for reading. A seekable stream is what makes range requests work, and its
+    /// <see cref="Stream.Length"/> is served as the <c>Content-Length</c> in preference to
+    /// <see cref="WebDavEntry.Length"/> - so a file whose bytes are produced on demand, and whose
+    /// size is only known once they have been, is served correctly by handing back a buffered one.
+    /// </summary>
+    ValueTask<Stream> OpenReadAsync(string path, CancellationToken cancellationToken);
+
+    /// <summary>
+    /// Creates or replaces the file at <paramref name="path"/> with <paramref name="content"/>. The
+    /// parent collection is known to exist. Write somewhere else first and move into place where
+    /// that is possible: the stream is a request body, and a client that stops sending halfway
+    /// should not leave half a file where a whole one used to be.
+    /// <para>
+    /// The stream enforces <see cref="WebDavOptions.MaxUploadBytes"/> itself, by throwing a
+    /// <see cref="WebDavException"/> once it is passed - let that propagate.
+    /// </para>
+    /// </summary>
+    ValueTask WriteAsync(string path, Stream content, CancellationToken cancellationToken);
+
+    /// <summary>Creates a collection. The parent is known to exist and the path to be free.</summary>
+    ValueTask CreateDirectoryAsync(string path, CancellationToken cancellationToken);
+
+    /// <summary>Removes a file, or a collection and everything in it.</summary>
+    ValueTask DeleteAsync(string path, CancellationToken cancellationToken);
+
+    /// <summary>
+    /// Moves a file or collection. The destination's parent exists and the destination itself does
+    /// not - an overwriting <c>MOVE</c> has already deleted it.
+    /// </summary>
+    ValueTask MoveAsync(string source, string destination, CancellationToken cancellationToken);
+
+    /// <summary>
+    /// Copies a file or collection, on the same terms as <see cref="MoveAsync"/>.
+    /// </summary>
+    /// <param name="recursive">
+    /// False only for a collection copied with <c>Depth: 0</c>, which RFC 4918 §9.8.3 defines as
+    /// the collection without its members.
+    /// </param>
+    ValueTask CopyAsync(string source, string destination, bool recursive, CancellationToken cancellationToken);
+
+    /// <summary>
+    /// Room left and room used where <paramref name="path"/> lives, for RFC 4331 quota - or null
+    /// when there is no answer, and the properties are then left out.
+    /// </summary>
+    WebDavQuota? GetQuota(string path);
+}
+
+/// <summary>One file or collection, as a <see cref="IWebDavFileSystem"/> describes it.</summary>
+/// <param name="Name">
+/// The path segment this entry is addressed by. Also what a client is shown, unless
+/// <see cref="DisplayName"/> says otherwise.
+/// </param>
+/// <param name="IsCollection">A folder, rather than a file.</param>
+/// <param name="Length">Bytes, for a file. Ignored for a collection.</param>
+/// <param name="CreatedUtc">Reported as <c>creationdate</c>.</param>
+/// <param name="LastModifiedUtc">Reported as <c>getlastmodified</c>, and half of the default ETag.</param>
+public sealed record WebDavEntry(
+    string Name,
+    bool IsCollection,
+    long Length,
+    DateTimeOffset CreatedUtc,
+    DateTimeOffset LastModifiedUtc
+)
+{
+    /// <summary>
+    /// What to call it where that differs from the segment - the <c>displayname</c> property and
+    /// the browser listing. Null uses <see cref="Name"/>.
+    /// </summary>
+    public string? DisplayName { get; init; }
+
+    /// <summary>The content type, when the file system knows better than the extension does.</summary>
+    public string? ContentType { get; init; }
+
+    /// <summary>
+    /// A quoted entity tag. Null builds one from <see cref="LastModifiedUtc"/> and
+    /// <see cref="Length"/>, which is the shape the static file handler uses, so a file fetched over
+    /// one and written back over the other is recognised as the same entity.
+    /// </summary>
+    public string? ETag { get; init; }
+
+    /// <summary>
+    /// Hidden by the platform's own rules, beyond a leading dot. Treated as a dotfile is:
+    /// left out unless <see cref="WebDavOptions.ServeHiddenFiles"/> is on.
+    /// </summary>
+    public bool IsHidden { get; init; }
+
+    /// <summary>
+    /// A link to somewhere else in the tree. Listed like anything else, but an infinite-depth
+    /// <c>PROPFIND</c> does not walk into it - a link to one of its own ancestors would otherwise
+    /// be a walk that only the result cap ends.
+    /// </summary>
+    public bool IsLink { get; init; }
+}
+
+/// <summary>RFC 4331 quota: what is left, and what is used, in bytes.</summary>
+public readonly record struct WebDavQuota(long AvailableBytes, long UsedBytes);
+
+/// <summary>
+/// A refusal with the status the client should see, thrown from a <see cref="IWebDavFileSystem"/>.
+/// <para>
+/// The message is not sent: a WebDAV client shows the status as an error of its own, and a
+/// sentence written for a file system's log is not written for whoever is looking at Finder.
+/// </para>
+/// </summary>
+public sealed class WebDavException : Exception
+{
+    public WebDavException(int statusCode, string? message = null, Exception? innerException = null)
+        : base(message ?? $"WebDAV request refused with {statusCode}.", innerException)
+    {
+        this.StatusCode = statusCode;
+    }
+
+    /// <summary>The HTTP status to answer with.</summary>
+    public int StatusCode { get; }
+}

src/Shiny.Net.HttpServer.WebDav/Internal/UploadLimitStream.cs

@@ -0,0 +1,56 @@
+namespace Shiny.Net.HttpServer.WebDav.Internal;
+
+/// <summary>
+/// A request body that refuses to be read past <see cref="WebDavOptions.MaxUploadBytes"/>.
+/// <para>
+/// The limit used to be enforced by the loop that copied the body to disk. With the copy now the
+/// file system's, the body is what has to enforce it, and throwing is the only way a read can say
+/// "stop" to a copy it does not own: a short read would be taken for the end of the file, and the
+/// truncated upload would be moved into place as though it were whole.
+/// </para>
+/// </summary>
+sealed class UploadLimitStream(Stream inner, long limit) : Stream
+{
+    long total;
+
+    public override bool CanRead => true;
+    public override bool CanSeek => false;
+    public override bool CanWrite => false;
+    public override long Length => throw new NotSupportedException();
+
+    public override long Position
+    {
+        get => this.total;
+        set => throw new NotSupportedException();
+    }
+
+    public override int Read(byte[] buffer, int offset, int count)
+        => this.Count(inner.Read(buffer, offset, count));
+
+    public override int Read(Span<byte> buffer)
+        => this.Count(inner.Read(buffer));
+
+    public override async ValueTask<int> ReadAsync(Memory<byte> buffer, CancellationToken cancellationToken = default)
+        => this.Count(await inner.ReadAsync(buffer, cancellationToken).ConfigureAwait(false));
+
+    public override Task<int> ReadAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken)
+        => this.ReadAsync(buffer.AsMemory(offset, count), cancellationToken).AsTask();
+
+    int Count(int read)
+    {
+        this.total += read;
+
+        if (this.total > limit)
+            throw new WebDavException(StatusCodes.Status413PayloadTooLarge, "The upload is larger than this mount accepts.");
+
+        return read;
+    }
+
+    public override void Flush()
+    {
+    }
+
+    public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
+    public override void SetLength(long value) => throw new NotSupportedException();
+    public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException();
+}

src/Shiny.Net.HttpServer.WebDav/Internal/WebDavHandler.CopyMove.cs

@@ -37,14 +37,14 @@ async ValueTask CopyOrMoveAsync(HttpContext context, bool move)
             return;
         }
 
-        var isCollection = Directory.Exists(source.Full);
-
-        if (!isCollection && !File.Exists(source.Full))
+        if (this.Stat(source) is not { } sourceEntry)
         {
             await StatusAsync(context, StatusCodes.Status404NotFound).ConfigureAwait(false);
             return;
         }
 
+        var isCollection = sourceEntry.IsCollection;
+
         // Moving the root would take the directory the mount is defined by with it.
         if (move && source.IsRoot)
         {
@@ -122,9 +122,7 @@ async ValueTask CopyOrMoveAsync(HttpContext context, bool move)
             return;
         }
 
-        var destinationParent = Path.GetDirectoryName(destination.Full);
-
-        if (destinationParent is null || !Directory.Exists(destinationParent))
+        if (!this.IsCollection(destination.Parent))
         {
             await StatusAsync(context, StatusCodes.Status409Conflict).ConfigureAwait(false);
             return;
@@ -139,7 +137,7 @@ async ValueTask CopyOrMoveAsync(HttpContext context, bool move)
         if (move && !await this.CheckLockAsync(context, source, tokens, subtree: isCollection).ConfigureAwait(false))
             return;
 
-        var destinationExisted = File.Exists(destination.Full) || Directory.Exists(destination.Full);
+        var destinationExisted = this.Stat(destination) is not null;
 
         if (!await this.CheckLockAsync(context, destination, tokens, subtree: destinationExisted).ConfigureAwait(false))
             return;
@@ -154,10 +152,7 @@ async ValueTask CopyOrMoveAsync(HttpContext context, bool move)
 
             // RFC 4918 §9.8.4: an overwriting COPY behaves as if the destination had been DELETEd
             // first. Doing it literally is also the only way a collection's stale members go.
-            if (Directory.Exists(destination.Full))
-                Directory.Delete(destination.Full, recursive: true);
-            else
-                File.Delete(destination.Full);
+            await this.fileSystem.DeleteAsync(destination.Relative, context.RequestAborted).ConfigureAwait(false);
 
             this.locks.ReleaseTree(destination.Relative);
 
@@ -168,21 +163,16 @@ await this.properties
 
         if (move)
         {
-            if (isCollection)
-                Directory.Move(source.Full, destination.Full);
-            else
-                File.Move(source.Full, destination.Full);
+            await this.fileSystem.MoveAsync(source.Relative, destination.Relative, context.RequestAborted).ConfigureAwait(false);
 
             // A lock does not travel with the resource — RFC 4918 §9.9.1.
             this.locks.ReleaseTree(source.Relative);
         }
-        else if (isCollection)
-        {
-            CopyTree(source.Full, destination.Full, shallow);
-        }
         else
         {
-            File.Copy(source.Full, destination.Full, overwrite: false);
+            await this.fileSystem
+                .CopyAsync(source.Relative, destination.Relative, recursive: !(isCollection && shallow), context.RequestAborted)
+                .ConfigureAwait(false);
         }
 
         await this.properties
@@ -197,27 +187,6 @@ await StatusAsync(
         ).ConfigureAwait(false);
     }
 
-    static void CopyTree(string source, string destination, bool shallow)
-    {
-        Directory.CreateDirectory(destination);
-
-        if (shallow)
-            return;
-
-        foreach (var file in Directory.EnumerateFiles(source))
-            File.Copy(file, Path.Combine(destination, Path.GetFileName(file)), overwrite: true);
-
-        foreach (var directory in Directory.EnumerateDirectories(source))
-        {
-            // Not descending into a link keeps a cycle inside the root from turning a copy into an
-            // unbounded one.
-            if (new DirectoryInfo(directory).Attributes.HasFlag(FileAttributes.ReparsePoint))
-                continue;
-
-            CopyTree(directory, Path.Combine(destination, Path.GetFileName(directory)), shallow: false);
-        }
-    }
-
     /// <summary>Maps a <c>Destination</c> header onto a path in this mount.</summary>
     DestinationKind ResolveDestination(HttpContext context, string header, out DavPath path)
     {