Performance review

[faddat]

goleveldb

https://youtu.be/Ho6DHkZax2c

badgerdb

https://youtu.be/tITLgGKDMDc

rocksdb

https://youtu.be/47PWHwkM0MY

All of notional's relay infrastructure has had to adopt rocksdb (osmosis, terra, gaia, juno) out of necessity. Given the overwhelming improvement, is there any reason not to work it into default workflows (slowly and in a non-jarring manner)

3xish is pretty good-- as is so far-- 100% compatibly.

Reasons not to rocksdb

• not all go makes compiles not super fun:

cosmos/gorocksdb#3

• pebbledb?
  https://github.com/cockroachdb/pebble

[faddat]

https://github.com/cockroachdb/pebble/blob/e2a8aef27e1b170d78f466fb97fda036a693abab/batch.go#L418

cockroachdb/pebble#1416

[boz]

3x on a drop-in-ish replacement is amazing @faddat!

I think that by defining a prefix extractor that partitions by store key prefixes, we may be able to get a big boost in query performance.

[tac0turtle]

Hey, it seems there is some increased performance in your video, but it would be good to get benchmarks and profiles to see the increased performance. Also I believe you would see similar performance across all dbs when they grow in size.

At the end, write performance is not a major issue in the sdk. We are working on changes in IAVL to improve read performance which will improve writes while state executing. This theory was put together through profiling.

[faddat]

@marbar3778 could we arrange a call or DM session to discuss relaying?

The issue is reads and I don't know how to properly quantify but this helps a great deal there, too.

I think that if you are following a hunch that IAVL changes will yeild bigger gains-- I think I agree. @ValarDragon and I have discussed this some, too.

[tac0turtle]

Happy to hop on a call!! this is helping push forward on things.

[faddat]

@boz -- @marbar3778 explained to me a bit more how we look for things in iavl, and I totally agree, those prefixes should help with performance a great deal.

[faddat]

This is superceded by #248