Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use linxGnu/grocksdb instead of gorocksdb #11683

Closed
2 of 4 tasks
robert-zaremba opened this issue Apr 19, 2022 · 8 comments
Closed
2 of 4 tasks

Use linxGnu/grocksdb instead of gorocksdb #11683

robert-zaremba opened this issue Apr 19, 2022 · 8 comments
Labels
C:Store T: Performance Performance improvements

Comments

@robert-zaremba
Copy link
Collaborator

Summary

We forked tecbot/gorocksdb (which doesn't have any activity any more) to add OptimisticTransactionDB::NewCheckpoint functionality.

There is https://github.com/linxGnu/grocksdb which is also a fork of tecbot/gorocksdb, is active and supports the latest Rocksdb (v7) and provides additional optimizations.

Objective

Update cosmos-sdk/db to support https://github.com/linxGnu/grocksdb or fork it into cosmos, add OptimisticTransactionDB::NewCheckpoint and use it instead of cosmos/gorocksdb.

Ref: osmosis-labs/osmosis#695

CC: @faddat , @roysc


For Admin Use

  • Not duplicate issue
  • Appropriate labels applied
  • Appropriate contributors tagged
  • Contributor assigned/self-assigned
@faddat
Copy link
Contributor

faddat commented Apr 19, 2022

Happy to work on this.

I should mention:

From every metric I can find (iavl benchmarks and state syncing osmo/gaia (tho both seem broke, both apply snapshots fastest with pebble)) PebbleDB is in fact preferable to RocksDB.

Trouble is, I think there is a bug in my implementation.

@robert-zaremba
Copy link
Collaborator Author

pebbledb is better event than linuxgnu/grocksdb?

@tac0turtle
Copy link
Member

can we not upstream to the cosmos fork? im wary of switching forks every few months.

@alexanderbez
Copy link
Contributor

can we not upstream to the cosmos fork? im wary of switching forks every few months.

This.

@tac0turtle
Copy link
Member

Lets upstream and create a new release. Happy to coordinate this

@faddat
Copy link
Contributor

faddat commented May 2, 2022

It is not upstreamable

They are separate git histories.

I mean we could just replace our whole repo.... but that doesn't seem like a great idea.

linxGnu/grocksdb is very well-maintained.

So maybe instead of creating our own fork of it, we should consider a donation to its author?

@tac0turtle
Copy link
Member

lets see what @roysc and @i-norden say. they are our db experts

@tac0turtle
Copy link
Member

this is part of cosmos-db

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C:Store T: Performance Performance improvements
Projects
None yet
Development

No branches or pull requests

4 participants