Skip to content
This repository has been archived by the owner on Jun 11, 2024. It is now read-only.

Commit

Permalink
Merge pull request #148 from LiskHQ/147-fix-genesis-block-creation-logic
Browse files Browse the repository at this point in the history 
Fix genesis block creation logic
  • Loading branch information
@sameersubudhi
sameersubudhi authored Sep 7, 2023
2 parents 4656773 + 78741c5 commit 473426a
Show file tree
Hide file tree
Showing 7 changed files with 114 additions and 172 deletions.
11 changes: 4 additions & 7 deletions docs/migration.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ The Lisk Migrator v2 also allows users to download and start the Lisk Core v4.x

```
USAGE
$ lisk-migrator [-d <value>] [-m <value>] [-c <value>] [-o <value>] [-p <value>] [-p <value>] [--snapshot-time-gap <value>] [--auto-download-lisk-core-v4] [--auto-migrate-config] [--auto-start-lisk-core-v4] [--use-existing-snapshot]
$ lisk-migrator [-d <value>] [-m <value>] [-c <value>] [-o <value>] [-p <value>] [-p <value>] [--snapshot-time-gap <value>] [--auto-migrate-config] [--auto-start-lisk-core-v4] [--use-existing-snapshot]
FLAGS
-c, --config=config Custom configuration file path.
Expand All @@ -82,15 +82,14 @@ FLAGS
-o, --output=output File path to write the genesis block json. If not provided, it will default to cwd/genesis_block.json.
-s, --snapshot-height=snapshot-height (Required) The height at which the re-genesis block will be generated. Can be specified with the SNAPSHOT_HEIGHT as well.
-v, --version Shows the CLI version.
--auto-download-lisk-core-v4 Download lisk-core v4 automatically. Default to false.
--auto-migrate-config Migrate user configuration automatically. Default to false.
--auto-start-lisk-core-v4 Start lisk-core v4 automatically. Default to false.
--snapshot-time-gap=snapshot-time-gap The number of seconds elapsed between the block at height HEIGHT_SNAPSHOT and the snapshot block.
EXAMPLES
lisk-migrator --snapshot-height 20931763 --lisk-core-path /path/to/data-dir
lisk-migrator --snapshot-height 20931763 --lisk-core-path /path/to/data-dir --auto-download-lisk-core-v4 --auto-start-lisk-core-v4 --auto-migrate-config
lisk-migrator --snapshot-height 20931763 --lisk-core-path /path/to/data-dir --auto-start-lisk-core-v4 --auto-migrate-config
```

<!--
Expand All @@ -104,13 +103,13 @@ You can start the migration script by running the following command in the termi
**Mainnet**

```
lisk-migrator --snapshot-height [recommendedSnapshotHeight] --output ~/.lisk/lisk-core/config/mainnet --lisk-core-v3-data-path ~/lisk-main --auto-download-lisk-core-v4 --auto-migrate-config --auto-start-lisk-core-v4
lisk-migrator --snapshot-height [recommendedSnapshotHeight] --output ~/.lisk/lisk-core/config/mainnet --lisk-core-v3-data-path ~/lisk-main --auto-migrate-config --auto-start-lisk-core-v4
```

**Testnet**

```
lisk-migrator --snapshot-height [recommendedSnapshotHeight] --output ~/.lisk/lisk-core/config/testnet --lisk-core-v3-data-path ~/lisk-test --auto-download-lisk-core-v4 --auto-migrate-config --auto-start-lisk-core-v4
lisk-migrator --snapshot-height [recommendedSnapshotHeight] --output ~/.lisk/lisk-core/config/testnet --lisk-core-v3-data-path ~/lisk-test --auto-migrate-config --auto-start-lisk-core-v4
```

- `--snapshot-height`:
Expand All @@ -120,8 +119,6 @@ lisk-migrator --snapshot-height [recommendedSnapshotHeight] --output ~/.lisk/lis
The absolute path to the directory, where the newly generated genesis block should be saved.
- `--lisk-core-v3-data-path`:
The absolute path to the directory, where the Lisk Core v3.x node is located.
- `--auto-download-lisk-core-v4`:
Download Lisk Core v4.x automatically.
- `--auto-migrate-config`:
Migrate Lisk Core v3.x configuration to v4.x automatically.
- `--auto-start-lisk-core-v4`:
Expand Down
87 changes: 38 additions & 49 deletions src/index.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
import util from 'util';
import * as fs from 'fs-extra';
import { join } from 'path';
import { Application, ApplicationConfig, PartialApplicationConfig } from 'lisk-framework';
import { ApplicationConfig, PartialApplicationConfig } from 'lisk-framework';
import { Database } from '@liskhq/lisk-db';
import * as semver from 'semver';
import { Command, flags as flagsParser } from '@oclif/command';
Expand Down Expand Up @@ -45,13 +45,13 @@ import {
setTokenIDLskByNetID,
setHeightPrevSnapshotBlockByNetID,
} from './utils/chain';
import { createGenesisBlock, writeGenesisBlock } from './utils/genesis_block';
import { createGenesisBlock, writeGenesisAssets } from './utils/genesis_block';
import { CreateAsset } from './createAsset';
import { ApplicationConfigV3, NetworkConfigLocal, NodeInfo } from './types';
import { installLiskCore, startLiskCore } from './utils/node';
import { copyDir, resolveAbsolutePath } from './utils/fs';

let finalConfigCorev4: PartialApplicationConfig;
let configCoreV4: PartialApplicationConfig;
class LiskMigrator extends Command {
public static description = 'Migrate Lisk Core to latest version';

Expand Down Expand Up @@ -96,12 +96,6 @@ class LiskMigrator extends Command {
description: 'Migrate user configuration automatically. Default to false.',
default: false,
}),
'auto-download-lisk-core-v4': flagsParser.boolean({
required: false,
env: 'AUTO_DOWNLOAD_LISK_CORE',
description: 'Download lisk core v4 automatically. Default to false.',
default: false,
}),
'auto-start-lisk-core-v4': flagsParser.boolean({
required: false,
env: 'AUTO_START_LISK_CORE',
Expand All @@ -113,18 +107,17 @@ class LiskMigrator extends Command {
public async run(): Promise<void> {
try {
const { flags } = this.parse(LiskMigrator);
const liskCoreV3Path = resolveAbsolutePath(
const liskCoreV3DataPath = resolveAbsolutePath(
flags['lisk-core-v3-data-path'] ?? DEFAULT_LISK_CORE_PATH,
);
const outputPath = flags.output ?? join(__dirname, '..', 'output');
const snapshotHeight = flags['snapshot-height'];
const customConfigPath = flags.config;
const autoMigrateUserConfig = flags['auto-migrate-config'] ?? false;
const autoDownloadLiskCoreV4 = flags['auto-download-lisk-core-v4'];
const autoStartLiskCoreV4 = flags['auto-start-lisk-core-v4'];
const snapshotTimeGap = Number(flags['snapshot-time-gap'] ?? SNAPSHOT_TIME_GAP);

const client = await getAPIClient(liskCoreV3Path);
const client = await getAPIClient(liskCoreV3DataPath);
const nodeInfo = (await client.node.getNodeInfo()) as NodeInfo;
const { version: appVersion, networkIdentifier } = nodeInfo;

Expand All @@ -145,7 +138,7 @@ class LiskMigrator extends Command {
cli.action.stop('Snapshot height is valid');

const networkConstant = NETWORK_CONSTANT[networkIdentifier] as NetworkConfigLocal;
const networkDir = `${outputPath}/${networkIdentifier}`;
const outputDir = `${outputPath}/${networkIdentifier}`;

if (autoStartLiskCoreV4) {
if (!networkConstant) {
Expand All @@ -171,23 +164,23 @@ class LiskMigrator extends Command {

// User specified custom config file
const configV3: ApplicationConfigV3 = customConfigPath
? await getConfig(liskCoreV3Path, customConfigPath)
: await getConfig(liskCoreV3Path);
? await getConfig(liskCoreV3DataPath, customConfigPath)
: await getConfig(liskCoreV3DataPath);

await setTokenIDLskByNetID(networkIdentifier);
await setHeightPrevSnapshotBlockByNetID(networkIdentifier);

await observeChainHeight({
label: 'Waiting for snapshot height to be finalized',
liskCoreV3Path,
liskCoreV3DataPath,
height: snapshotHeight,
delay: 500,
isFinal: true,
});

// Create new DB instance based on the snapshot path
cli.action.start('Creating database instance');
const snapshotDirPath = join(liskCoreV3Path, SNAPSHOT_DIR);
const snapshotDirPath = join(liskCoreV3DataPath, SNAPSHOT_DIR);
const db = new Database(snapshotDirPath);
cli.action.stop();

Expand All @@ -200,24 +193,11 @@ class LiskMigrator extends Command {
cli.action.stop();

// Create an app instance for creating genesis block
const configFilePath = await resolveConfigPathByNetworkID(networkIdentifier);
const configV4 = await fs.readJSON(configFilePath);
const { app } = await Application.defaultApplication(configV4, true);

cli.action.start('Creating genesis block');
const blockAtSnapshotHeight = ((await client.block.getByHeight(
snapshotHeight,
)) as unknown) as Block;
const genesisBlock = await createGenesisBlock(
app,
genesisAssets,
blockAtSnapshotHeight,
snapshotTimeGap,
);
cli.action.stop();
const defaultConfigFilePath = await resolveConfigPathByNetworkID(networkIdentifier);
const defaultConfigV4 = await fs.readJSON(defaultConfigFilePath);

cli.action.start(`Exporting genesis block to the path ${networkDir}`);
await writeGenesisBlock(genesisBlock, genesisAssets, networkDir);
cli.action.start(`Exporting genesis block to the path ${outputDir}`);
await writeGenesisAssets(genesisAssets, outputDir);
cli.action.stop();

if (autoMigrateUserConfig) {
Expand All @@ -228,7 +208,7 @@ class LiskMigrator extends Command {
cli.action.start('Migrating user configuration');
const migratedConfigV4 = (await migrateUserConfig(
configV3,
configV4,
defaultConfigV4,
snapshotHeight,
)) as ApplicationConfig;
cli.action.stop();
Expand All @@ -239,26 +219,35 @@ class LiskMigrator extends Command {

if (!isValidConfig) throw new Error('Migrated user configuration is invalid.');

cli.action.start(`Exporting user configuration to the path: ${networkDir}`);
await writeConfig(migratedConfigV4, networkDir);
cli.action.start(`Exporting user configuration to the path: ${outputDir}`);
await writeConfig(migratedConfigV4, outputDir);
cli.action.stop();

// Set finalConfigCorev4 to the migrated Core config
finalConfigCorev4 = migratedConfigV4 as PartialApplicationConfig;
// Set configCoreV4 to the migrated Core config
configCoreV4 = migratedConfigV4 as PartialApplicationConfig;
}

if (autoDownloadLiskCoreV4) {
cli.action.start('Installing lisk-core v4');
await installLiskCore();
cli.action.stop();
}
cli.action.start('Installing lisk-core v4');
await installLiskCore();
cli.action.stop();

cli.action.start('Creating genesis block');
const blockAtSnapshotHeight = ((await client.block.getByHeight(
snapshotHeight,
)) as unknown) as Block;
await createGenesisBlock(
networkConstant.name,
defaultConfigFilePath,
outputDir,
blockAtSnapshotHeight,
snapshotTimeGap,
);
cli.action.stop();

if (autoStartLiskCoreV4) {
try {
// TODO: Verify and update the implementation
// If finalConfigCorev4 is not set to the migrated config use the default config
if (!autoMigrateUserConfig) {
finalConfigCorev4 = configV4;
configCoreV4 = defaultConfigV4;
}

cli.action.start(`Creating legacy.db at ${LEGACY_DB_PATH}`);
Expand All @@ -273,12 +262,12 @@ class LiskMigrator extends Command {
if (isLiskCoreV3Stopped) {
const isUserConfirmed = await cli.confirm(`
Start Lisk Core with the following configuration? [yes/no] \n
${util.inspect(finalConfigCorev4, false, 3)}`);
${util.inspect(configCoreV4, false, 3)}`);

if (isUserConfirmed) {
cli.action.start('Starting lisk-core v4');
const network = networkConstant.name as string;
await startLiskCore(this, finalConfigCorev4, appVersion, liskCoreV3Path, network);
await startLiskCore(this, liskCoreV3DataPath, configCoreV4, network, outputDir);
this.log('Started Lisk Core v4 at default data directory.');
cli.action.stop();
} else {
Expand Down
10 changes: 5 additions & 5 deletions src/utils/chain.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ let heightPreviousSnapshotBlock: number;
interface ObserveParams {
readonly label: string;
readonly height: number;
readonly liskCoreV3Path: string;
readonly liskCoreV3DataPath: string;
readonly delay: number;
readonly isFinal: boolean;
}
Expand Down Expand Up @@ -91,8 +91,8 @@ const getRemainingTime = (currentHeight: number, observedHeight: number): string
export const observeChainHeight = async (options: ObserveParams): Promise<number> => {
const observedHeight = options.height;
const startHeight = options.isFinal
? (await getNodeInfo(options.liskCoreV3Path)).finalizedHeight
: (await getNodeInfo(options.liskCoreV3Path)).height;
? (await getNodeInfo(options.liskCoreV3DataPath)).finalizedHeight
: (await getNodeInfo(options.liskCoreV3DataPath)).height;

if (startHeight >= observedHeight) {
return startHeight;
Expand Down Expand Up @@ -120,8 +120,8 @@ export const observeChainHeight = async (options: ObserveParams): Promise<number
let height!: number;
try {
height = options.isFinal
? (await getNodeInfo(options.liskCoreV3Path)).finalizedHeight
: (await getNodeInfo(options.liskCoreV3Path)).height;
? (await getNodeInfo(options.liskCoreV3DataPath)).finalizedHeight
: (await getNodeInfo(options.liskCoreV3DataPath)).height;
} catch (error) {
return reject(error);
}
Expand Down
8 changes: 4 additions & 4 deletions src/utils/config.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -264,10 +264,10 @@ export const validateConfig = async (config: ApplicationConfig): Promise<boolean
}
};

export const writeConfig = async (config: ApplicationConfig, outputPath: string): Promise<void> => {
if (!existsSync(outputPath)) {
mkdirSync(outputPath, { recursive: true });
export const writeConfig = async (config: ApplicationConfig, outputDir: string): Promise<void> => {
if (!existsSync(outputDir)) {
mkdirSync(outputDir, { recursive: true });
}

writeFileSync(resolve(outputPath, 'config.json'), JSON.stringify(config));
writeFileSync(resolve(outputDir, 'config.json'), JSON.stringify(config));
};
51 changes: 16 additions & 35 deletions src/utils/genesis_block.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,8 @@ import * as crypto from 'crypto';
import * as fs from 'fs-extra';
import path from 'path';
import { Block as BlockVersion3 } from '@liskhq/lisk-chain';
import { Block as BlockVersion4 } from 'lisk-framework';
import { codec, Schema } from '@liskhq/lisk-codec';
import { GenesisAssetEntry } from '../types';
import { SNAPSHOT_BLOCK_VERSION } from '../constants';
import { execAsync } from './process';

(BigInt.prototype as any).toJSON = function () {
return this.toString();
Expand Down Expand Up @@ -46,46 +44,29 @@ export const createChecksum = async (filePath: string): Promise<string> => {
};

export const createGenesisBlock = async (
// TODO: Update type any once GenesisBlockGenerateInput exported by SDK
app: any,
assets: GenesisAssetEntry[],
network: string,
configFilepath: string,
outputDir: string,
blockAtSnapshotHeight: BlockVersion3,
snapshotTimeGap: number,
): Promise<BlockVersion4> => {
const input = {
assets: assets.map((a: { module: string; schema: Schema; data: object }) => ({
module: a.module,
data: codec.fromJSON(a.schema, a.data),
schema: a.schema,
})),
chainID: Buffer.from(app.config.genesis.chainID, 'hex'),
timestamp: blockAtSnapshotHeight.header.timestamp + snapshotTimeGap,
height: blockAtSnapshotHeight.header.height + 1,
previousBlockID: blockAtSnapshotHeight.header.previousBlockID,
};
) => {
const height = blockAtSnapshotHeight.header.height + 1;
const timestamp = blockAtSnapshotHeight.header.timestamp + snapshotTimeGap;
const previousBlockID = blockAtSnapshotHeight.header.id.toString('hex');

const genesisBlock = await app.generateGenesisBlock(input);
genesisBlock.header.version = SNAPSHOT_BLOCK_VERSION;
return genesisBlock;
const genesisBlockCreateCommand = `lisk-core genesis-block:create --network ${network} --config=${configFilepath} --output=${outputDir} --assets-file=${outputDir}/genesis_assets.json --height=${height} --previous-block-id=${previousBlockID} --timestamp=${timestamp}`;

await execAsync(genesisBlockCreateCommand);
};

export const writeGenesisBlock = async (
genesisBlock: BlockVersion4,
export const writeGenesisAssets = async (
genesisAssets: GenesisAssetEntry[],
outputPath: string,
outputDir: string,
): Promise<void> => {
if (fs.existsSync(outputPath)) fs.rmdirSync(outputPath, { recursive: true });
fs.mkdirSync(outputPath, { recursive: true });

fs.writeFileSync(path.resolve(outputPath, 'genesis_block.blob'), genesisBlock.getBytes());

const genesisBlockJsonFilepath = path.resolve(outputPath, 'genesis_block.json');
fs.writeFileSync(genesisBlockJsonFilepath, JSON.stringify(genesisBlock, null, '\t'));

const genesisBlockHash = await createChecksum(genesisBlockJsonFilepath);
fs.writeFileSync(path.resolve(outputPath, 'genesis_block.json.SHA256'), genesisBlockHash);
if (fs.existsSync(outputDir)) fs.rmdirSync(outputDir, { recursive: true });
fs.mkdirSync(outputDir, { recursive: true });

const genesisAssetsJsonFilepath = path.resolve(outputPath, 'genesis_assets.json');
const genesisAssetsJsonFilepath = path.resolve(outputDir, 'genesis_assets.json');
fs.writeFileSync(
genesisAssetsJsonFilepath,
JSON.stringify({ assets: genesisAssets }, null, '\t'),
Expand Down
Loading

0 comments on commit 473426a

Please sign in to comment.