K8SPG-870 Add backup size#1672
Conversation
| @@ -0,0 +1,25 @@ | |||
| apiVersion: kuttl.dev/v1beta1 | |||
There was a problem hiding this comment.
Do we need a separate e2e test? Can't we just add extra assertions in the demand-backup test?
There was a problem hiding this comment.
I see we don't test diff and incr there though, but we can add that
There was a problem hiding this comment.
I see we don't test diff and incr there though, but we can add that
this was the main reason I thought it might worth it to add a separate test. But I agree the test is too narrow, we could extend the demand-backup indeed. Let me update it, thanks
There was a problem hiding this comment.
removed the new test, updated the ondemand-backup test by adding incr and diff backups creation and check if the status.size field is set. It required also to update the expected generation for pg for all the rest of the test, as well as the way we count backups to check retention (now we only need to check full backups, not all of them).
Test passes locally
egegunes
left a comment
There was a problem hiding this comment.
LGTM but the test is failing
egegunes
left a comment
There was a problem hiding this comment.
as far as I see we only update PerconaPGBackup object status but why do we need to increase PostgresCluster and PerconaPGCluster object generations in compare files?
Hi @egegunes! the increase is not this PR specific. Generation is increased each time we run a backup, because the Since we ran 2 more backups, generations now are increased by 2 |
There was a problem hiding this comment.
Pull request overview
This PR adds a status.size field to PerconaPGBackup so the operator can surface pgBackRest backup size (full/diff/incr) based on pgbackrest info --output=json.
Changes:
- Extend pgBackRest info parsing to include
backup.info.sizeand map it intoPerconaPGBackupStatus.Size. - Update the PerconaPGBackup CRD/bundle schemas to expose the new
status.sizefield. - Expand/adjust KUTTL E2E coverage for demand backups (full/diff/incr) and update related assertions.
Reviewed changes
Copilot reviewed 20 out of 21 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| pkg/apis/pgv2.percona.com/v2/perconapgbackup_types.go | Adds status.size to the PerconaPGBackup API type. |
| percona/pgbackrest/pgbackrest.go | Parses info.size from pgbackrest info JSON output. |
| percona/controller/pgbackup/controller.go | Updates backup status with size from pgBackRest info and refactors backup lookup logic. |
| e2e-tests/tests/demand-backup/04-create-backup-repo1.yaml | Adds diff/incr backup CRs for repo1. |
| e2e-tests/tests/demand-backup/04-assert.yaml | Asserts status.size is present for full/diff/incr backups. |
| e2e-tests/tests/demand-backup/09-check-retention-repo1.yaml | Updates retention counting logic for backups/jobs. |
| e2e-tests/tests/demand-backup/06-assert.yaml | Updates expected generation/observedGeneration values. |
| e2e-tests/tests/demand-backup/08-assert.yaml | Updates expected generation/observedGeneration values. |
| e2e-tests/tests/demand-backup/10-assert.yaml | Updates expected generation/observedGeneration values. |
| e2e-tests/tests/demand-backup/11-assert.yaml | Updates expected generation/observedGeneration values. |
| e2e-tests/tests/demand-backup/14-assert.yaml | Updates expected generation/observedGeneration values. |
| e2e-tests/tests/demand-backup/15-assert.yaml | Updates expected generation/observedGeneration values. |
| e2e-tests/tests/demand-backup/18-assert.yaml | Updates expected generation/observedGeneration values. |
| e2e-tests/tests/demand-backup/19-assert.yaml | Updates expected generation/observedGeneration values. |
| e2e-tests/tests/demand-backup/21-assert.yaml | Updates expected generation/observedGeneration values. |
| e2e-tests/tests/demand-backup/22-assert.yaml | Updates expected generation/observedGeneration values. |
| deploy/cw-bundle.yaml | Updates published schema to include status.size. |
| deploy/crd.yaml | Updates published schema to include status.size. |
| deploy/bundle.yaml | Updates published schema to include status.size. |
| config/crd/bases/pgv2.percona.com_perconapgclusters.yaml | Updates generated base CRD schema to include status.size (PerconaPGBackup CRD). |
| build/crd/percona/generated/pgv2.percona.com_perconapgbackups.yaml | Updates generated CRD schema to include status.size. |
| backupType := backup.Annotation[v2.PGBackrestAnnotationJobType] | ||
| if (backupType != pgBackup.Annotations[pNaming.AnnotationPGBackrestBackupJobType] || | ||
| backupType != string(naming.BackupReplicaCreate)) && backup.Annotation[v2.PGBackrestAnnotationBackupName] != pgBackup.Name { | ||
| continue | ||
| } |
There was a problem hiding this comment.
@oksana-grishchenko should we address this comment?
There was a problem hiding this comment.
This PR didn't change this logic, it just refactored the existing updatePGBackrestInfo func by extracting findBackupInPGBackrestInfo as a separate function.
I agree the condition looks odd, but I'm having a hard time understanding the reasoning behind it and why it was combined this way in the original PR #722. @pooknull do you happen to remember what was the goal of this condition?
So, answering the question:
should we address this comment?
I wouldn't change it unless I understand what was the original idea of this condition 😅
|
please check copilot comments |
commit: 100f793 |
K8SPG-870 Add backup size
Problem:
Need to have status.size field for full, differential and incremental backups.
Snapshot backups are out of the scope.
Solution:
pgbackrest.GetInfo(...)runspgbackrest info --output=jsoninside the DB pod, the output already contains the size:so we add the new field to the existing pgbackrest
InfoBackupstruct and then map it toPerconaPGBackupStatusRelated helm-chart PR:
CHECKLIST
Jira
Needs Doc) and QA (Needs QA)?Tests
Config/Logging/Testability