Skip to content

About

A dashboard and control API for Celery background tasks — see, control, and pause single occurrences of periodic tasks. Works with Django, Flask, and FastAPI.

Topics

Resources

Contributing

Security policy

Stars

6 stars

Watchers

1 watching

Forks

Repository files navigation

celery-manager

A dashboard and control API for Celery background tasks — for any Django, Flask, or FastAPI project that already uses Celery.

PyPI CI Python 3.10–3.13 License: MIT Status: beta

Documentation · Quickstart · PyPI

Point it at your existing Celery app and you get a web UI and a REST API to see what your tasks are doing, act on them, and — the feature this project exists for — pause or cancel a single future occurrence of a periodic task without touching your beat schedule or restarting anything.

The celery-manager task list with filters, status badges, runtimes and start times
Task detail for a running import with a live progress bar A chord's workflow tree with one failed group member
Insights page with run volume, success rate and runtime charts Failures grouped by exception signature with retry-all
Workers page with live worker cards and runtime rate limits Queue depth per queue with purge controls
Schedules with one schedule paused until a set time Start a task sheet with prefilled arguments

New in the 0.3 release candidate: SQL-owned schedules, reviewed reconciliation, singleton leases, deeper alerts, failure review, revocable browser sessions, heartbeat history and a versioned status widget. The same release adds host-job observation and operational diagnostics: RunObserver for synchronous/async work without Celery, tenant-scoped read-only history, custom probes/outbox health, first-due scheduler expectations, correlated attempts, preflight, deployment notes and audited environment safeguards. These observe the host's existing execution; they do not replace it.

Production adoption: the 0.2 guide covers external schedulers, OIDC, dispatch policies, bounded logs/results and storage isolation. The requirements roadmap tracks requirements and follow-ups.

What you get

  • Task visibility — every task run (decorated or not) with status, args, worker, queue, timings, exceptions, and full tracebacks.
  • Full run history — browse past runs beyond the newest page, search by task name, task ID, or args/kwargs content, and filter by status and date range.
  • Live progress — call celery_manager.report_progress(current, total, message) inside a task body and watch the bar move, with elapsed time and a projected finish while the task is still running.
  • Log capture — logging output from inside a task body (and print() too, in a real worker — see why), attached to that run and readable in the dashboard.
  • Control — cancel/terminate, retry (optionally with edited args), bulk retry, bulk cancel, and start any registered task from a searchable picker with prefilled arguments. Retries keep their lineage.
  • Workflows — chains, groups and chords shown as a collapsible tree on every run that belongs to one.
  • Per-occurrence schedule control — skip or cancel one upcoming run of a periodic task, or pause a whole schedule indefinitely or until a set time; it resumes from the current time, with no catch-up.
  • Insights — runs, success rate and p50/p95 runtime over 1h–30d, a page per task, and failures grouped by exception signature with retry-all.
  • Built for agents — a Model Context Protocol server (celery-manager mcp, read-only by default, roles and audit apply), llms.txt for coding agents, and LLM-usage tracking (model, tokens, cost per task, with a spend alert) for the agent workloads your tasks run.
  • Workers and queues — live worker state, pool grow/shrink, queue consume/cancel, shutdown, runtime rate limits, broker queue depth and purge.
  • Shareable views — filters live in the URL, saved views, and CSV/JSON export of any filtered list.
  • Live updates — the dashboard follows changes from the events listener, workers and other API processes within about a second.
  • Schema migrations — Alembic migrations ship in the package and run at startup, or as a deploy step with celery-manager migrate.
  • Audit log — who did what, when, for every control action.
  • Light and dark themes — an explicit per-user choice, not just whatever the OS says.
  • Brandable — set the product name, accent colour, and logo from a settings screen; it applies for everyone on the deployment.
  • Built-in auth — bcrypt-hashed users and signed session cookies.
  • Roles — viewer (read-only), operator (day-to-day controls) and admin (purge, worker shutdown, settings), enforced on the server.
  • No Node toolchain required — the React dashboard ships prebuilt inside the Python package.

30-second quickstart

pip install --pre "celery-manager[standalone]"
celery-manager demo     # seeded SQLite dashboard on http://127.0.0.1:8765/

No broker, worker or database server needed. To wire it into your own app, see Install and Wire it up below, or run an example in examples/ (FastAPI, Flask, Django, Docker Compose, and examples/full_stack/, every process on one machine with one command).

Install

Try it first, with no broker, worker or database server:

pip install --pre "celery-manager[standalone]"
celery-manager demo     # seeded SQLite dashboard on http://127.0.0.1:8765/

See Is celery-manager the right tool? for how it compares with Flower, Sentry Crons and Grafana, and a 10-line quickstart per framework.

celery-manager is in beta (a pre-release on PyPI), so pass --pre to pip:

pip install --pre celery-manager

With extras:

pip install --pre "celery-manager[standalone]"   # adds uvicorn
pip install --pre "celery-manager[flask]"        # adds Flask and a2wsgi

Alternatively, install a tagged release straight from git:

pip install "git+https://github.com/Pythondeveloper6/celery-manager.git@v0.1.0a5"

(Extras with git: "celery-manager[standalone] @ git+https://github.com/Pythondeveloper6/celery-manager.git@v0.1.0a5".)

Optional extras:

  • standalone — pulls in uvicorn, to run the dashboard on its own rather than mounting it into an ASGI app you already serve.
  • flask — pulls in Flask and a2wsgi, to mount the dashboard in a Flask (WSGI) app; see Integrating with your app.

Wire it up

celery-manager runs in two places, and both need configuring — this is the step most easily missed:

from celery_manager.config import CeleryManagerConfig

config = CeleryManagerConfig(
    celery_app=your_celery_app,
    db_url="sqlite:///celery_manager.db",   # any SQLAlchemy URL
    secret_key="a-real-secret-in-production",
)

1. The dashboard process — build the ASGI app and serve or mount it:

from celery_manager.api.app import create_app

app, _ = create_app(config)   # then: uvicorn your_module:app

2. The Celery worker process — where @track-decorated tasks actually run:

import celery_manager

celery_manager.setup(config)   # at import time, in your tasks module

@app.task(bind=True)
@celery_manager.track(schedule_name="send-report-every-2-hours")
def send_report(self):
    ...

For per-occurrence skips, run beat with celery-manager's scheduler. It tags each firing with its scheduled time, so a skip hits exactly that run, however late it starts, and never a manual .delay():

celery -A your_module beat -S celery_manager.beat:Scheduler

Then run the events listener alongside your worker and beat, so the dashboard sees every task rather than only the decorated ones:

celery-manager events --celery-app your_module:your_celery_app \
                      --db-url sqlite:///celery_manager.db

The listener sees nothing unless workers emit events: set worker_send_task_events = True and task_send_sent_event = True (or run the worker with -E). See Required Celery settings.

Two things that stop people on a first run, both covered in the Quickstart: there is no default account, so you have to create one (celery-manager create-user admin --role admin --db-url ...); and on plain HTTP you need cookie_secure=False, or you will log in successfully and then immediately appear logged out.

Documentation

Full documentation is at https://pythondeveloper6.github.io/celery-manager/ (source in docs/, an MkDocs Material site — run mkdocs serve after pip install -e ".[docs]"):

Changes between versions — including what breaks for development builds — are in CHANGELOG.md.

A complete runnable Django example is in examples/django_demo/. The FastAPI example (examples/fastapi_demo/) also runs with no Redis (CELERY_MANAGER_EAGER=1) and is covered by a smoke test. A production-shaped Docker Compose deployment (dashboard, worker, beat, events listener, PostgreSQL, Redis) is in examples/docker/.

Security

The dashboard can cancel tasks, purge queues and edit schedules, so treat it as an admin surface: serve it over HTTPS, keep cookie_secure=True, set a long random secret_key from your secret store, create accounts explicitly (there is no default login) and use the viewer/operator/admin roles. See docs/security.md and SECURITY.md for the threat model and how to report a vulnerability privately.

Limits

  • Beta software: the surfaces in the stability policy are covered by a deprecation promise; everything else may change before 1.0.
  • Celery 5.x only; other task runners are observed, not controlled (details).
  • Retry, revoke, purge and worker controls need a reachable broker and live workers.
  • Per-occurrence skips need beat to run celery_manager.beat:Scheduler, and exactly one beat.
  • SQLite is fine for development; use PostgreSQL or MySQL for multi-host deployments.

Contributing

Issues and pull requests are welcome. Running the tests, linting and building the docs are covered in Working on celery-manager.

Roadmap

Shipped in 0.1.0a5: alerts, missed-run detection, Prometheus/OpenTelemetry metrics, sign-in with your app's users, user management in the dashboard, editing schedules from the UI, maintenance windows, a dead-letter view and data retention; and for Redis and RabbitMQ: broker health, queue wait time and depth history, delayed and stuck tasks, browsing waiting messages, and RabbitMQ management and dead-letter queues. Read the roadmap →

Requirements

Python 3.10+, Celery 5.x. Storage defaults to SQLite with zero configuration and swaps to Postgres/MySQL/anything SQLAlchemy supports via db_url.

Status

This is a beta, 0.3.0b1. APIs outside the stability policy may still change before 1.0; see the CHANGELOG for release history and docs/roadmap.md for planned follow-ups.

License

MIT — see LICENSE.

About

A dashboard and control API for Celery background tasks — see, control, and pause single occurrences of periodic tasks. Works with Django, Flask, and FastAPI.

Topics

Resources

Contributing

Security policy

Stars

6 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages