Skip to content

Commit

Permalink
Introduced a custom LevelDB impl with regions and better key format
Browse files Browse the repository at this point in the history
This new impl (which is not loadable by vanilla) is targeted at very large worlds, which experience significant I/O performance issues due to a variety of issues described in #6580.

Two main changes are made in RegionizedLevelDB:
- First, multiple LevelDBs are used, which cover a fixed NxN segment of terrain, similar to Anvil in Java. However, there's no constraint on these region sizes. Several experimental sizes are supported by default in WorldProviderManager.
- Second, bigEndianLong(morton2d(chunkX, chunkZ)) is used for chunk keys instead of littleEndianInt(chunkX).littleEndianInt(chunkZ). This new scheme has much better cache locality than Mojang's version, which reduces overlap and costly DB compactions.

The following new provider options are available as a result of this change:
- custom-leveldb-regions-32
- custom-leveldb-regions-64
- custom-leveldb-regions-128
- custom-leveldb-regions-256

Smaller sizes will likely be less space-efficient, but will also probably have better performance.
Once a sweet spot is found, a default will be introduced.

Note that the different variations of custom-leveldb-regions-* are not cross-compatible.
Conversion between the different formats is necessary if you want to change formats.
  • Loading branch information
dktapps committed Dec 21, 2024
1 parent 306623e commit 1224055
Show file tree
Hide file tree
Showing 4 changed files with 1,082 additions and 753 deletions.
18 changes: 17 additions & 1 deletion src/world/format/io/WorldProviderManager.php
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,11 @@

use pocketmine\utils\Utils;
use pocketmine\world\format\io\leveldb\LevelDB;
use pocketmine\world\format\io\leveldb\RegionizedLevelDB;
use pocketmine\world\format\io\region\Anvil;
use pocketmine\world\format\io\region\McRegion;
use pocketmine\world\format\io\region\PMAnvil;
use pocketmine\world\WorldCreationOptions;
use function strtolower;
use function trim;

Expand All @@ -41,10 +43,24 @@ final class WorldProviderManager{
private WritableWorldProviderManagerEntry $default;

public function __construct(){
$leveldb = new WritableWorldProviderManagerEntry(LevelDB::isValid(...), fn(string $path, \Logger $logger) => new LevelDB($path, $logger), LevelDB::generate(...));
$leveldb = new WritableWorldProviderManagerEntry(
LevelDB::isValid(...),
fn(string $path, \Logger $logger) => new LevelDB($path, $logger),
LevelDB::generate(...)
);
$this->default = $leveldb;
$this->addProvider($leveldb, "leveldb");

//any arbitrary size is supported, but powers of 2 are best
//these are the most likely to be useful
foreach([32, 64, 128, 256] as $regionLength){
$this->addProvider(new WritableWorldProviderManagerEntry(
fn(string $path) => RegionizedLevelDB::isValid($path, $regionLength),
fn(string $path, \Logger $logger) => new RegionizedLevelDB($path, $logger, $regionLength),
fn(string $path, string $name, WorldCreationOptions $options) => RegionizedLevelDB::generate($path, $name, $options, $regionLength)
), "custom-leveldb-regions-$regionLength");
}

$this->addProvider(new ReadOnlyWorldProviderManagerEntry(Anvil::isValid(...), fn(string $path, \Logger $logger) => new Anvil($path, $logger)), "anvil");
$this->addProvider(new ReadOnlyWorldProviderManagerEntry(McRegion::isValid(...), fn(string $path, \Logger $logger) => new McRegion($path, $logger)), "mcregion");
$this->addProvider(new ReadOnlyWorldProviderManagerEntry(PMAnvil::isValid(...), fn(string $path, \Logger $logger) => new PMAnvil($path, $logger)), "pmanvil");
Expand Down
Loading

0 comments on commit 1224055

Please sign in to comment.