feat: build and run on Windows#8
Conversation
There was a problem hiding this comment.
Pull request overview
This PR removes golang.org/x/sys/unix usage from core read/write paths and replaces raw-fd pread/pwrite syscalls with portable *os.File.ReadAt/WriteAt, enabling the library to compile (and the sorted builder + query/read paths to run) on Windows while keeping platform-specific preallocation behind OS build tags.
Changes:
- Replaced
unix.Pwrite/Preadcall sites with(*os.File).WriteAt/ReadAtand adjusted APIs to pass*os.File/io.ReaderAtinstead of raw fds. - Refactored
preallocFileto accept*os.File, usingftruncate-free fallback (os.File.Truncate) on!linux && !darwinand retainingfallocate/F_PREALLOCATEon Linux/macOS. - Updated platform documentation to reflect Windows compilation support and the unsorted builder limitation.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| platform_other.go | Removes unix dependency by using os.File.Truncate for the non-Linux/Darwin prealloc fallback (covers Windows). |
| platform_linux.go | Updates preallocFile to accept *os.File and uses Fallocate(int(f.Fd()), ...). |
| platform_darwin.go | Updates preallocFile to accept *os.File and uses FcntlFstore(f.Fd(), ...). |
| index_writer.go | Switches positioned writes from unix.Pwrite on cached fds to file.WriteAt. |
| builder_unsorted.go | Switches unsorted writer flush from unix.Pwrite to WriteAt and removes cached fd state. |
| builder_unsorted_finish.go | Switches partition reads from unix.Pread to io.ReaderAt.ReadAt. |
| CLAUDE.md | Updates platform support notes to reflect Windows compilation and unsorted builder limitation. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
7afc6c5 to
7c9a82c
Compare
7c9a82c to
2422b5b
Compare
2422b5b to
89b83c7
Compare
The core read/write paths used unix.Pwrite/Pread on raw fds, and platform_other.go imported x/sys/unix, so the library failed to compile on Windows — breaking downstream importers there. - Replace raw-fd syscalls with *os.File.WriteAt/ReadAt (pwrite/pread on Unix, OVERLAPPED on Windows; same lock-free concurrent-write property). - Route the !linux && !darwin prealloc fallback through os.File.Truncate (no unix import); only platform_linux/darwin keep x/sys/unix. - The unsorted builder's anonymous temp files use unlink-while-open on Unix (platform_unix.go) and FILE_FLAG_DELETE_ON_CLOSE on Windows (platform_windows.go); the temp dir is removed in cleanup. Build throughput unchanged within ~5% noise. Windows is compile-verified via cross-compile but not exercised in CI (ubuntu+macos only); the temp directory is best-effort cleaned and may leak on a Windows crash. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
89b83c7 to
0037986
Compare
|
nit: It doesn't look like the ci/cd doesn't actually check windows. I only see macos/ubuntu |
yes, this is a follow up task which has been de-prioritized but can be done later |
The core read/write paths used
unix.Pwrite/Preadon raw fds, so the library failed to compile on Windows — breaking downstream importers there.This swaps them for portable
*os.File.WriteAt/ReadAtand routes the!linux && !darwinprealloc fallback throughos.File.Truncate(nouniximport). The unsorted builder's anonymous temp files use unlink-while-open on Unix andFILE_FLAG_DELETE_ON_CLOSEon Windows, so the whole library builds and runs on Windows.FILE_FLAG_DELETE_ON_CLOSE)On Unix
WriteAtis the same pwrite syscall and keeps the lock-free concurrent-write property. Build throughput unchanged within ~5% noise (10M keys, ptrhash).Verified:
go test -race ./..., gofmt/vet/golangci-lint, andGOOS=windows/freebsdcross-compile.Not verified / known limitations (CI is ubuntu+macos only, so the Windows runtime path is unexercised):
cleanup(), so an abnormal exit leaks an emptystreamhash-parts-*dir. A handle held by AV/indexer at close time can also leave the dir briefly in delete-pending and skip removal.🤖 Generated with Claude Code