Arc

Open, SQL-native time-series database for telemetry you need to keep. Arc ingests 34M+ records/sec, stores data as standard Parquet on infrastructure you own, and lets you query recent and historical data together. InfluxDB Line Protocol and Telegraf compatible. Single binary. AGPL-3.0.

Prefer a UI? Arc Launchpad is a self-hosted web console for the Arc instances you run — SQL console, schema explorer, logs, monitoring, and management for tokens, retention, alerts, continuous queries, and MQTT ingestion. Deploy it alongside Arc with Docker Compose. Docs.

---

Telemetry is easy to collect and expensive to keep

Machines, services, vehicles, and devices produce data continuously. The operational problem is not only ingesting the latest readings — it is keeping the full-resolution history available for debugging, analysis, compliance, and the next question nobody has asked yet.

Teams evaluating a time-series database usually run into the same trade-offs:

• Retention cliffs: Older data is downsampled, exported, or deleted because storage costs grow too quickly.
• Split hot and cold paths: Recent data is queryable in one system while historical data waits in a warehouse or object store.
• Operational overhead: A simple workload turns into a PostgreSQL extension stack or a multi-service cluster.
• Migration friction: Existing agents, dashboards, and Line Protocol writers make changing databases risky.
• Vendor lock-in: Proprietary storage makes it difficult to use your data elsewhere or leave later.

Arc is built for teams that want to keep the data, query the whole history, and start with a small deployment. It combines high-throughput ingestion, automatic Parquet storage and compaction, analytical SQL, retention policies, and continuous queries in one binary.

Built for aerospace telemetry. Useful anywhere machines never stop producing data.

---

What Arc is (and isn't)

Arc is a complete time-series analytical database: ingestion pipeline, Parquet storage engine, compaction system, SQL query layer, retention policy manager, continuous query scheduler, and telemetry integrations — in one binary. It uses DuckDB as its query engine, while Arc adds the pieces needed to run a durable ingestion and analytics service: high-throughput writes with automatic Parquet flushing, background compaction, scheduled compute, data lifecycle management, authentication, backup and restore, and enterprise clustering.

Arc is not a wrapper. You don't bring your own ingestion, compaction, or retention policies. Arc provides the full stack.

Why teams evaluate Arc

• Keep full-resolution history instead of choosing between retention and cost.
• Use standard SQL with window functions, CTEs, joins, and analytical aggregations.
• Own the files: Arc stores data as open Apache Parquet on local disk, S3, Azure, or MinIO.
• Start small: run one binary on a laptop, edge box, or server before adding enterprise clustering.
• Migrate gradually: use InfluxDB Line Protocol and Telegraf-compatible ingestion to dual-write and validate before cutover.

When Arc may not be the right choice

• If your workload is primarily transactional relational data and already fits comfortably in PostgreSQL, start by evaluating TimescaleDB.
• If you need a mature metrics-only replacement for Prometheus and depend on PromQL, evaluate VictoriaMetrics.
• If your organization cannot approve AGPL-3.0 software, use Arc Enterprise's commercial license or choose an Apache-licensed alternative.
• If you need the largest established community and the lowest adoption risk, Arc is newer than TimescaleDB, VictoriaMetrics, and QuestDB.

-- Telemetry: hourly sensor summary across a full history
SELECT
  device_id,
  DATE_TRUNC('hour', timestamp) AS hour,
  AVG(value) AS average_value,
  MIN(value) AS minimum_value,
  MAX(value) AS maximum_value
FROM telemetry.sensor_readings
WHERE timestamp > NOW() - INTERVAL '30 days'
GROUP BY device_id, hour
ORDER BY hour DESC;

-- Telemetry: correlate readings with device metadata
SELECT
  d.site,
  r.device_id,
  AVG(r.value) AS average_value
FROM telemetry.sensor_readings AS r
JOIN telemetry.devices AS d ON d.device_id = r.device_id
WHERE r.timestamp > NOW() - INTERVAL '24 hours'
GROUP BY d.site, r.device_id;

Standard SQL. Window functions, CTEs, joins, aggregations. No proprietary query language.

Start with the workload you already have

Arc accepts InfluxDB Line Protocol directly, so existing Telegraf inputs can write to Arc without changing the collection layer. A low-risk migration usually looks like this:

1. Point a small slice of ingestion at Arc, or dual-write to both systems.
2. Compare the data and query results over an overlapping time window.
3. Move one dashboard or workload at a time.
4. Decommission the old database only after a full retention cycle has passed.

See the InfluxDB migration guide, or compare Arc with TimescaleDB, InfluxDB, ClickHouse, and Elasticsearch.

---

Live Demo

See Arc in action: https://basekick.net/demos

---

Performance

Benchmarked on Apple MacBook Pro M3 Max (14 cores, 36GB RAM, 1TB NVMe). Test config: 12 concurrent workers, 1000-record batches, columnar data.

Ingestion (August 2026)

Protocol	Throughput	p50 Latency	p99 Latency
MessagePack Columnar	34.0M rec/s	0.29ms	1.40ms
MessagePack + Zstd	24.9M rec/s	0.42ms	1.53ms
MessagePack + GZIP	24.6M rec/s	0.42ms	1.53ms
Line Protocol	4.7M rec/s	2.19ms	6.61ms

All rows measured over a 60-second sustained run. The MessagePack Columnar row is 2,043,451,000 records ingested in 60 seconds — and that's not rows streamed into a memory buffer: every record was received over HTTP, decoded, time-sorted, and durably written to disk as queryable Parquet, at 0.29ms median latency, on a laptop.

Measured on a development build; these improvements ship in 26.09.1: ingest no longer dictionary-encodes Parquet (compaction re-encodes files anyway) and the msgpack columnar path now decodes payloads directly into typed column arrays, eliminating per-value allocations — see the 26.09.1 release notes.

Compaction

Automatic background compaction merges small Parquet files into optimized larger files:

Metric	Before	After	Reduction
Files	43	1	97.7%
Size	372 MB	36 MB	90.4%

Benefits:

• 10x storage reduction via better compression and encoding
• Faster queries - scan 1 file vs 43 files
• Lower cloud costs - less storage, fewer API calls

Query (May 2026)

Arc speaks three wire formats from the same query engine. Arrow IPC is the throughput leader for analytical clients (Grafana, pyarrow, polars) that can take an Arrow dependency — zero-copy from the engine's internal columnar buffers. MessagePack (experimental, columnar) is the choice for clients that don't speak Arrow but want smaller bytes and faster decode than JSON — same envelope shape as JSON, native binary types for timestamps and binary columns. JSON stays the default for ergonomic compatibility.

Benchmark: 393.7M-row cpu measurement, 5 iterations per query, M3 Max. Latency is p50 in milliseconds. The five SELECT-LIMIT rows were measured back-to-back in the same session so the three columns are apples-to-apples; the DuckDB-bound rows (Time Bucket, Date Trunc, GROUP BY) are dominated by query execution and converge across wire formats.

Query	JSON (ms)	MessagePack (ms)	Arrow IPC (ms)	msgpack vs JSON	Arrow vs JSON
COUNT(*) — 393.7M rows	1.03	1.03	0.86	1.00x	1.20x
SELECT LIMIT 10K	18.4	16.6	14.7	1.11x	1.25x
SELECT LIMIT 100K	48.1	33.2	31.0	1.45x	1.55x
SELECT LIMIT 500K	173.2	81.1	61.1	2.14x	2.84x
SELECT LIMIT 1M	334.2	133.6	105.4	2.49x	3.17x
Time Range (7d) LIMIT 10K	15.0	15.5	15.5	0.97x	0.97x
Time Bucket (1h, 7d)	4.7	4.8	4.7	0.98x	1.00x
Date Trunc (day, 30d)	416	415	413	1.00x	1.01x
GROUP BY host	452	450	450	1.00x	1.00x
GROUP BY host + hour	645	660	672	0.98x	0.96x

Best throughput on LIMIT 1M (1M-row payload, single connection):

• Arrow IPC: 9.49M rows/sec (105.4ms)
• MessagePack: 7.49M rows/sec (133.6ms)
• JSON: 2.99M rows/sec (334.2ms)
• COUNT(*): ~382B rows/sec equivalent (393.7M rows in 1.03ms — parquet footer reads, not a row scan)

Notes on the table: the wire-format speedups manifest on response-heavy queries (≥100k rows) where encoding dominates the per-request wall time. For aggregations (Time Bucket, Date Trunc, GROUP BY) the response is tiny — a few rows — and DuckDB execution is 99%+ of the wall time; all three formats converge. The Arrow IPC win comes from a memcpy of the column buffer; the MessagePack endpoint walks each cell through a typed columnar encoder (one type-switch per column, not per row) and lands at ~78% of Arrow IPC's throughput while remaining decodable by any msgpack client without an Arrow dependency.

The MessagePack endpoint is experimental (gated behind the duckdb_arrow build tag, no operator-tunable row cap yet) — see the 26.06.1 release notes for the wire-format spec, operational constraints, and the columnar-redesign story.

---

Single binary. Zero dependencies.

Arc deploys as one statically-linked executable. No JVM, no Python environment, no PostgreSQL cluster to manage, no ZooKeeper ensemble to babysit. Run it on a laptop, a factory edge box, an on-premises server, or a Kubernetes cluster. Same binary, same config surface.

• Air-gap ready: No external services required at runtime. No license server, no cloud dependency.
• Edge to cloud: Deploy at the tactical edge, in a sovereign cloud, or on-premises.
• Minimal footprint: One process. Memory usage proportional to active workload, not fleet size.

---

Quick Start

# Build
make build

# Run
./arc

# Verify
curl http://localhost:8000/health

---

Installation

Docker

# Docker Hub
docker run -d \
  -p 8000:8000 \
  -v arc-data:/app/data \
  basekicklabs/arc:latest

# or GitHub Container Registry
docker run -d \
  -p 8000:8000 \
  -v arc-data:/app/data \
  ghcr.io/basekick-labs/arc:latest

Multi-arch images (linux/amd64 + linux/arm64) are published to both registries on every release.

macOS (Homebrew)

brew install basekick-labs/tap/arc

Apple Silicon. DuckDB is statically linked, so there are no runtime dependencies. (Use brew install --formula arc if you tap first, to disambiguate from the arc browser cask.)

Debian/Ubuntu

wget https://github.com/basekick-labs/arc/releases/download/v26.06.3/arc_26.06.3_amd64.deb
sudo dpkg -i arc_26.06.3_amd64.deb
sudo systemctl enable arc && sudo systemctl start arc

RHEL/Fedora

wget https://github.com/basekick-labs/arc/releases/download/v26.06.3/arc-26.06.3-1.x86_64.rpm
sudo rpm -i arc-26.06.3-1.x86_64.rpm
sudo systemctl enable arc && sudo systemctl start arc

Kubernetes (Helm)

helm install arc https://github.com/basekick-labs/arc/releases/download/v26.06.3/arc-26.06.3.tgz

Build from Source

# Prerequisites: Go 1.26+

# Clone and build
git clone https://github.com/basekick-labs/arc.git
cd arc
make build

# Or build directly with Go (the duckdb_arrow tag is required)
go build -tags=duckdb_arrow ./cmd/arc

# Run
./arc

FIPS 140-3 Build

For US defense/federal and other regulated environments, Arc ships an optional arc-fips build: the same source at the same version, compiled against the CMVP-certified Go Cryptographic Module and run in FIPS-only mode. Pick the -fips artifact instead of the standard one.

# Binary — download arc-fips-linux-amd64 (or -arm64) from the release
# Container — same repos, -fips tag suffix:
docker run -d -p 8000:8000 -v arc-data:/app/data ghcr.io/basekick-labs/arc:VERSION-fips
# or basekicklabs/arc:VERSION-fips

# Build from source:
make build-fips      # -> arc-fips (GOFIPS140=v1.0.0, -tags=duckdb_arrow,fips)

The FIPS build reports the same version as the standard build and logs "fips_mode":true at startup. Cutover note: existing bcrypt-hashed API tokens must be rotated when moving to the FIPS build (it stores new tokens with PBKDF2 and fails bcrypt verification closed). The Go Cryptographic Module is CMVP-certified; Arc itself is not a CMVP-listed module. See the FIPS 140-3 mode guide.

---

Ecosystem & Integrations

Tool	Description	Link
Arc Launchpad	Self-hosted web UI: SQL console, schema explorer, logs, monitoring, and management for tokens, retention, alerts, continuous queries, MQTT ingestion, and teams	GitHub · Docs
VS Code Extension	Browse databases, run queries, visualize results	Marketplace
Grafana Data Source	Native Grafana plugin for dashboards and alerting	GitHub
Telegraf Output Plugin	Ship data from 300+ Telegraf inputs directly to Arc	Docs
Python SDK	Query and ingest from Python applications	PyPI
Superset Dialect (JSON)	Apache Superset connector using JSON transport	GitHub
Superset Dialect (Arrow)	Apache Superset connector using Arrow transport	GitHub

---

Features

Core Capabilities

• Columnar storage: Parquet format with full analytical SQL engine

• Workloads: Industrial IoT, manufacturing, energy, fleet telemetry, aerospace, observability, and event analytics

• Ingestion: MessagePack columnar (fastest), InfluxDB Line Protocol, MQTT, TLE (satellite telemetry)

• Query: Full analytical SQL; JSON, columnar MessagePack (experimental), and Apache Arrow IPC responses

• Compaction: Tiered (hourly/daily) automatic Parquet file merging — 10x storage reduction

• Data Lifecycle: Retention policies, continuous queries, tiered storage (hot/cold)

• Durability: Optional write-ahead log (WAL), backup and restore

• Storage: Local filesystem, S3, MinIO

• Auth: Token-based authentication with in-memory caching

• Durability: Optional write-ahead log (WAL)

• Data Management: GDPR-compliant delete operations

• Observability: Prometheus metrics, structured logging, graceful shutdown

• Reliability: Circuit breakers, retry with exponential backoff

• Supply chain: SBOM (SPDX + CycloneDX), Trivy scans, cosign-signed releases, SLSA L3 provenance

• FIPS 140-3: Optional arc-fips build against the CMVP-certified Go Cryptographic Module — see Installation

• Edge Sync (coming 26.09.1): Spoke-to-hub data transport for disconnected operations

---

Configuration

Arc uses TOML configuration with environment variable overrides.

[server]
host = "0.0.0.0"
port = 8000

[storage]
backend = "local"        # local, s3, minio
local_path = "./data/arc"

[ingest]
flush_interval = "5s"
max_buffer_size = 50000

[auth]
enabled = true

Environment variables use ARC_ prefix:

export ARC_SERVER_PORT=8000
export ARC_STORAGE_BACKEND=s3
export ARC_AUTH_ENABLED=true

See arc.toml for complete configuration reference.

---

Project Structure

arc/
├── cmd/arc/              # Application entry point
├── internal/
│   ├── api/              # HTTP handlers (Fiber) — query, write, import, TLE, admin
│   ├── audit/            # Audit logging for API operations
│   ├── auth/             # Token authentication and RBAC
│   ├── backup/           # Backup and restore (data, metadata, config)
│   ├── circuitbreaker/   # Resilience patterns (retry, backoff)
│   ├── cluster/          # Raft consensus, node roles, WAL replication
│   ├── compaction/       # Tiered hourly/daily Parquet file merging
│   ├── config/           # TOML configuration with env var overrides
│   ├── database/         # Query engine and connection management
│   ├── governance/       # Per-token query quotas and rate limiting
│   ├── ingest/           # MessagePack, Line Protocol, TLE, Arrow writer
│   ├── license/          # License validation and feature gating
│   ├── logger/           # Structured logging (zerolog)
│   ├── metrics/          # Prometheus metrics
│   ├── mqtt/             # MQTT subscriber — topic-to-measurement ingestion
│   ├── pruning/          # Query-time partition pruning
│   ├── query/            # Parallel partition executor
│   ├── queryregistry/    # Active/completed query tracking
│   ├── scheduler/        # Continuous queries and retention policies
│   ├── shutdown/         # Graceful shutdown coordinator
│   ├── sql/              # SQL parsing utilities
│   ├── storage/          # Local, S3, Azure backends
│   ├── telemetry/        # Usage telemetry
│   ├── tiering/          # Hot/cold storage lifecycle management
│   └── wal/              # Write-ahead log
├── pkg/models/           # Shared data structures (Record, ColumnarRecord)
├── benchmarks/           # Performance benchmarking suites
├── deploy/               # Docker Compose and Kubernetes configs
├── helm/                 # Helm charts
├── scripts/              # Utility scripts (analysis, backfill, debugging)
├── arc.toml              # Configuration file
├── Makefile              # Build commands
└── go.mod

---

Development

make deps           # Install dependencies
make build          # Build binary
make run            # Run without building
make test           # Run tests
make test-coverage  # Run tests with coverage
make bench          # Run benchmarks
make lint           # Run linter
make fmt            # Format code
make clean          # Clean build artifacts

---

License

Arc is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0).

• Free to use, modify, and distribute
• If you modify Arc and run it as a service, you must share your changes under AGPL-3.0

For commercial licensing, contact: enterprise@basekick.net

---

Contributors

Thanks to everyone who has contributed code to Arc:

• @schotime (Adam Schroder) — Data-time partitioning, compaction API triggers, UTC fixes, bare-JOIN SQL rewriting fix
• @khalid244 — S3 partition pruning improvements, multi-line SQL query support
• @SAY-5 (Sai Asish Y) — MQTT nil-guard hardening (handlers + manager) with regression coverage

And a thank-you to community members whose bug reports drove fixes:

• @bjarneksat — reported the line-protocol null-field handling bug fixed in 26.03.1