Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove db backup cronjobs #5418

Merged
merged 1 commit into from
Jul 11, 2023
Merged

Conversation

naseberry
Copy link
Contributor

@naseberry naseberry commented Jun 27, 2023

What

Remove hourly db backup cronjobs

Remove the cron jobs involved in storing and clearing the db backups

Why

Amazon, in theory, allows a point-in-time recovery making the backup files a resource drain that we are unlikely to need

TO-DO

  • Manually run the backup cleaner to remove any legacy backups

Checklist

Before you ask people to review this PR:

  • Tests and rubocop should be passing: bundle exec rake
  • Github should not be reporting conflicts; you should have recently run git rebase main.
  • There should be no unnecessary whitespace changes. These make diffs harder to read and conflicts more likely.
  • The PR description should say what you changed and why, with a link to the JIRA story.
  • You should have looked at the diff against main and ensured that nothing unexpected is included in your changes.
  • You should have checked that the commit messages say why the change was made.

@naseberry naseberry force-pushed the ph/ap-4061-remove-db-hourly-backups branch from 848809b to d9dbdc8 Compare July 10, 2023 13:54
@naseberry naseberry marked this pull request as ready for review July 10, 2023 14:16
@naseberry naseberry requested a review from a team as a code owner July 10, 2023 14:16
@naseberry naseberry force-pushed the ph/ap-4061-remove-db-hourly-backups branch 2 times, most recently from 0ba623d to 299b4b8 Compare July 10, 2023 14:27
@kmahern
Copy link
Contributor

kmahern commented Jul 10, 2023

Do we need to first check with CP that we able to carry out point in time restores before merging this pr? The ticket says that AWS in theory allows point-in-time restores but my understanding is that this needs to be configured. I have looked in the CP user guide but this only mentions one overnight backup.

@sonarcloud
Copy link

sonarcloud bot commented Jul 10, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

Copy link
Contributor

@colinbruce colinbruce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Maybe we could schedule a point-in-time recovery, the same as the original DR test?

@colinbruce colinbruce added the approved Approved by code reviewers label Jul 11, 2023
@naseberry
Copy link
Contributor Author

Do we need to first check with CP that we able to carry out point in time restores before merging this pr? The ticket says that AWS in theory allows point-in-time restores but my understanding is that this needs to be configured. I have looked in the CP user guide but this only mentions one overnight backup.

Quoting CP - "Point-in-time recovery for RDS relies on snapshots, either automated or manual. You can do point-in-time recovery to any point in time within the backup retention period; which is (by default) 7 days."

So yes, PITR is possible with CP's automated snapshots (although we may require CP support)

@jsugarman
Copy link
Contributor

Do we need to first check with CP that we able to carry out point in time restores before merging this pr? The ticket says that AWS in theory allows point-in-time restores but my understanding is that this needs to be configured. I have looked in the CP user guide but this only mentions one overnight backup.

It does look and sound like a combination of an automated daily backup plus point in time recovery (PITR) can allow us to restore an instance of the DB from up to 7 days ago until up to ~5 minutes ago.

I have used the cli to confirm LatestRestorableTime is showing as ~5 minutes ago for Apply production RDS instance. Documentation confirms that a daily backup is taken so it sounds like we do not need to take a manual snapshot. But when was the last time a DR scenario was test?

@naseberry naseberry force-pushed the ph/ap-4061-remove-db-hourly-backups branch from 299b4b8 to 9e0e561 Compare July 11, 2023 13:37
@naseberry naseberry merged commit 968d812 into main Jul 11, 2023
5 checks passed
@naseberry naseberry deleted the ph/ap-4061-remove-db-hourly-backups branch July 11, 2023 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Approved by code reviewers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants