Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip checksum for diskless replication #1180

Conversation

Shivshankar-Reddy
Copy link
Contributor

fixes: #1129
Below is the background information from #1129 by @madolson

When we are not writing to disk (either on the primary or replica) there isn't a lot of sense in doing CRC64 checksumming during fullsync. We do checking summing at the TCP layer already. At AWS we've seen it can take up to 15% overhead for not much of a benefit. We should look into a mechanism so that a replica can indicate that it will load something directly into memory, since today we only indicate we support the EOF marker. The replica might still choose to save the RDB file to disk unless repl-diskless-load is enabled. With this mechanism in place, we should be able to skip the checksum for replication, while still using it for RDB.

Here is my understanding and proposal as a fix for the issue. Please correct me if my understanding is wrong or missing something!

image

  • Added 'replication-diskless-load' to sendCommand replica details in syncWithPrimary and FullSyncWithPrimary.
  • Added rdbFlag to skip checksum based on 'replica-diskless-load' configuaration from the replica.

@Shivshankar-Reddy
Copy link
Contributor Author

Abandoning this due to cyclic commit conflict and created #1181 for the same.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Skip CRC64 checksumming when doing diskless replication
1 participant