Skip to content

K8SPG-870 Add backup size#1672

Open
oksana-grishchenko wants to merge 8 commits into
mainfrom
K8SPG-870-backup-size
Open

K8SPG-870 Add backup size#1672
oksana-grishchenko wants to merge 8 commits into
mainfrom
K8SPG-870-backup-size

Conversation

@oksana-grishchenko

@oksana-grishchenko oksana-grishchenko commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

K8SPG-870 Add backup size

Problem:

Need to have status.size field for full, differential and incremental backups.
Snapshot backups are out of the scope.

Solution:
pgbackrest.GetInfo(...) runs pgbackrest info --output=json inside the DB pod, the output already contains the size:

bash-5.1$ pgbackrest info --output=json

[
  ...
  {
    "backup": [
      ...
      {
        ...
        "info": {
          "size": 23884028
        }
      }
    ]
  }
]

so we add the new field to the existing pgbackrest InfoBackup struct and then map it to PerconaPGBackupStatus

Related helm-chart PR:

CHECKLIST

Jira

  • Is the Jira ticket created and referenced properly?
  • Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
  • Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

  • Is an E2E test/test case added for the new feature/change?
  • Are unit tests added where appropriate?

Config/Logging/Testability

  • Are all needed new/changed options added to default YAML files?
  • Are all needed new/changed options added to the Helm Chart?
  • Did we add proper logging messages for operator actions?
  • Did we ensure compatibility with the previous version or cluster upgrade process?
  • Does the change support oldest and newest supported PG version?
  • Does the change support oldest and newest supported Kubernetes version?

@CLAassistant

CLAassistant commented Jul 3, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Comment thread e2e-tests/run-pr.csv Outdated
@@ -0,0 +1,25 @@
apiVersion: kuttl.dev/v1beta1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need a separate e2e test? Can't we just add extra assertions in the demand-backup test?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see we don't test diff and incr there though, but we can add that

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see we don't test diff and incr there though, but we can add that

this was the main reason I thought it might worth it to add a separate test. But I agree the test is too narrow, we could extend the demand-backup indeed. Let me update it, thanks

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed the new test, updated the ondemand-backup test by adding incr and diff backups creation and check if the status.size field is set. It required also to update the expected generation for pg for all the rest of the test, as well as the way we count backups to check retention (now we only need to check full backups, not all of them).

Test passes locally

@egegunes egegunes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but the test is failing

@egegunes egegunes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as far as I see we only update PerconaPGBackup object status but why do we need to increase PostgresCluster and PerconaPGCluster object generations in compare files?

@oksana-grishchenko

oksana-grishchenko commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

as far as I see we only update PerconaPGBackup object status but why do we need to increase PostgresCluster and PerconaPGCluster object generations in compare files?

Hi @egegunes! the increase is not this PR specific. Generation is increased each time we run a backup, because the startBackup function updates the cluster spec

Since we ran 2 more backups, generations now are increased by 2

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a status.size field to PerconaPGBackup so the operator can surface pgBackRest backup size (full/diff/incr) based on pgbackrest info --output=json.

Changes:

  • Extend pgBackRest info parsing to include backup.info.size and map it into PerconaPGBackupStatus.Size.
  • Update the PerconaPGBackup CRD/bundle schemas to expose the new status.size field.
  • Expand/adjust KUTTL E2E coverage for demand backups (full/diff/incr) and update related assertions.

Reviewed changes

Copilot reviewed 20 out of 21 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
pkg/apis/pgv2.percona.com/v2/perconapgbackup_types.go Adds status.size to the PerconaPGBackup API type.
percona/pgbackrest/pgbackrest.go Parses info.size from pgbackrest info JSON output.
percona/controller/pgbackup/controller.go Updates backup status with size from pgBackRest info and refactors backup lookup logic.
e2e-tests/tests/demand-backup/04-create-backup-repo1.yaml Adds diff/incr backup CRs for repo1.
e2e-tests/tests/demand-backup/04-assert.yaml Asserts status.size is present for full/diff/incr backups.
e2e-tests/tests/demand-backup/09-check-retention-repo1.yaml Updates retention counting logic for backups/jobs.
e2e-tests/tests/demand-backup/06-assert.yaml Updates expected generation/observedGeneration values.
e2e-tests/tests/demand-backup/08-assert.yaml Updates expected generation/observedGeneration values.
e2e-tests/tests/demand-backup/10-assert.yaml Updates expected generation/observedGeneration values.
e2e-tests/tests/demand-backup/11-assert.yaml Updates expected generation/observedGeneration values.
e2e-tests/tests/demand-backup/14-assert.yaml Updates expected generation/observedGeneration values.
e2e-tests/tests/demand-backup/15-assert.yaml Updates expected generation/observedGeneration values.
e2e-tests/tests/demand-backup/18-assert.yaml Updates expected generation/observedGeneration values.
e2e-tests/tests/demand-backup/19-assert.yaml Updates expected generation/observedGeneration values.
e2e-tests/tests/demand-backup/21-assert.yaml Updates expected generation/observedGeneration values.
e2e-tests/tests/demand-backup/22-assert.yaml Updates expected generation/observedGeneration values.
deploy/cw-bundle.yaml Updates published schema to include status.size.
deploy/crd.yaml Updates published schema to include status.size.
deploy/bundle.yaml Updates published schema to include status.size.
config/crd/bases/pgv2.percona.com_perconapgclusters.yaml Updates generated base CRD schema to include status.size (PerconaPGBackup CRD).
build/crd/percona/generated/pgv2.percona.com_perconapgbackups.yaml Updates generated CRD schema to include status.size.

Comment on lines 508 to 512
backupType := backup.Annotation[v2.PGBackrestAnnotationJobType]
if (backupType != pgBackup.Annotations[pNaming.AnnotationPGBackrestBackupJobType] ||
backupType != string(naming.BackupReplicaCreate)) && backup.Annotation[v2.PGBackrestAnnotationBackupName] != pgBackup.Name {
continue
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oksana-grishchenko should we address this comment?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR didn't change this logic, it just refactored the existing updatePGBackrestInfo func by extracting findBackupInPGBackrestInfo as a separate function.

I agree the condition looks odd, but I'm having a hard time understanding the reasoning behind it and why it was combined this way in the original PR #722. @pooknull do you happen to remember what was the goal of this condition?

So, answering the question:

should we address this comment?

I wouldn't change it unless I understand what was the original idea of this condition 😅

@gkech

gkech commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

please check copilot comments

@JNKPercona

Copy link
Copy Markdown
Collaborator
Test Name Result Time
backup-enable-disable passed 00:15:49
builtin-extensions passed 00:06:56
cert-manager-tls passed 00:12:17
custom-envs passed 00:24:41
custom-tls passed 00:06:10
database-init-sql passed 00:02:50
demand-backup passed 00:31:49
demand-backup-offline-snapshot passed 00:14:42
dynamic-configuration passed 00:03:31
finalizers passed 00:03:55
init-deploy passed 00:03:34
huge-pages passed 00:02:34
major-upgrade-14-to-15 passed 00:10:07
major-upgrade-15-to-16 passed 00:14:54
major-upgrade-16-to-17 passed 00:11:32
major-upgrade-17-to-18 passed 00:10:20
ldap passed 00:04:16
ldap-tls passed 00:05:56
monitoring failure 00:10:33
monitoring-pmm3 failure 00:10:35
one-pod passed 00:06:05
operator-self-healing passed 00:14:10
pitr passed 00:13:11
scaling passed 00:06:28
scheduled-backup failure 00:37:14
self-healing passed 00:10:27
sidecars passed 00:04:43
standby-pgbackrest passed 00:19:31
standby-streaming passed 00:14:00
start-from-backup passed 00:13:06
tablespaces passed 00:07:39
telemetry-transfer passed 00:04:53
upgrade-consistency passed 00:05:48
upgrade-minor passed 00:07:07
users passed 00:05:15
migration-from-crunchy-standby passed 00:19:46
migration-from-crunchy-pv passed 00:08:11
migration-from-crunchy-backup-restore passed 00:17:28
Summary Value
Tests Run 38/38
Job Duration 02:40:07
Total Test Time 07:02:22

commit: 100f793
image: perconalab/percona-postgresql-operator:PR-1672-100f793e8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants