Skip to content

feat(roles): run the database dumps with low CPU and IO priority - #447

Merged
NavidSassan merged 8 commits into
mainfrom
feat/dump-low-priority
Oct 15, 2026
Merged

NavidSassan merged 8 commits into
mainfrom
feat/dump-low-priority

Conversation

@markuslf

Copy link
Copy Markdown
Member

Summary

  • The daily dump services of gitlab_ce, influxdb, mariadb_server, mongodb and postgresql_server run with Nice=19, IOSchedulingClass=best-effort, IOSchedulingPriority=7, CPUWeight=20 and IOWeight=20, so the dump no longer slows down the database. IOWeight only takes effect with the BFQ I/O scheduler. The READMEs got a section "How the Role Behaves".
  • MongoDB and MariaDB deliberately do not use the idle classes: the MongoDB dump holds fsyncLock, and mydumper holds LOCK TABLES ... READ LOCAL on non-InnoDB tables until each table is dumped.
  • mariadb-dump only passes the mydumper options the installed version knows (--dirty, --trx-tables=0, --threads=0), so the dump works with mydumper 0.10.x and 1.0.x.
  • The daemon-reload handlers already on main cover the changed unit files, so the tasks stay untouched.

Test plan

Before rebasing onto main (the task files were identical apart from a daemon_reload that the handlers now replace): mariadb, mongodb and influxdb on Rocky 8/9/10, Debian 12/13 and Ubuntu 22.04/24.04/26.04, PostgreSQL on RHEL 9/10, Debian 12/13 and Ubuntu 22.04/24.04/26.04. Where the role ran, the units carried the new values, the dump Result=success and the second run changed=0. CPUWeight is mapped to cpu.shares on Rocky 8 (cgroup v1).

Not testable: gitlab_ce (only systemd-analyze verify), MongoDB on Debian 13, Ubuntu 26.04 (no upstream packages), PostgreSQL on Rocky 8 (PGDG key), MariaDB dump on Debian 13 (no mydumper, see the mydumper repository PR).

markuslf and others added 7 commits October 14, 2026 16:58
Nice, IOSchedulingClass/Priority, CPUWeight and IOWeight keep the daily dump from slowing down the database. IOWeight only takes effect with the BFQ scheduler. Not 'idle' for MongoDB and MariaDB, as the dump holds a lock (fsyncLock, LOCK TABLES) that would starve other clients.
Nice, IOSchedulingClass/Priority, CPUWeight and IOWeight keep the daily dump from slowing down the database. IOWeight only takes effect with the BFQ scheduler. Not 'idle' for MongoDB and MariaDB, as the dump holds a lock (fsyncLock, LOCK TABLES) that would starve other clients.
Nice, IOSchedulingClass/Priority, CPUWeight and IOWeight keep the daily dump from slowing down the database. IOWeight only takes effect with the BFQ scheduler. Not 'idle' for MongoDB and MariaDB, as the dump holds a lock (fsyncLock, LOCK TABLES) that would starve other clients.
Nice, IOSchedulingClass/Priority, CPUWeight and IOWeight keep the daily dump from slowing down the database. IOWeight only takes effect with the BFQ scheduler. Not 'idle' for MongoDB and MariaDB, as the dump holds a lock (fsyncLock, LOCK TABLES) that would starve other clients.
Nice, IOSchedulingClass/Priority, CPUWeight and IOWeight keep the daily dump from slowing down the database. IOWeight only takes effect with the BFQ scheduler. Not 'idle' for MongoDB and MariaDB, as the dump holds a lock (fsyncLock, LOCK TABLES) that would starve other clients.
…ed version knows

Compare the mydumper version: --dirty from 0.15.2-8, --trx-tables=0 from 0.20.1-1, and use the number of CPUs instead of --threads=0 before 0.15.3-1, which hangs. Verified with mydumper 0.10.0 (Debian 12, Ubuntu 22.04, 24.04, 26.04) and 1.0.11 (Rocky 9).
@NavidSassan
NavidSassan force-pushed the feat/dump-low-priority branch from ee15b0c to 1b49710 Compare October 15, 2026 10:00
@NavidSassan
NavidSassan merged commit 0ce974b into main Oct 15, 2026
13 checks passed
@NavidSassan
NavidSassan deleted the feat/dump-low-priority branch October 15, 2026 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants