Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(store/v2): build the migration manager in the root store factory #22336

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

cool-develope
Copy link
Contributor

@cool-develope cool-develope commented Oct 22, 2024

Description

Closes: #22180


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title, you can find examples of the prefixes below:
  • confirmed ! in the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • reviewed "Files changed" and left comments if necessary
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • updated the relevant documentation or specification, including comments for documenting Go code
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

Please see Pull Request Reviewer section in the contributing guide for more information on how to review a pull request.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic, API design and naming, documentation is accurate, tests and test coverage

Summary by CodeRabbit

  • New Features

    • Enhanced migration support for the root store, allowing for seamless updates based on the latest state storage version.
    • Improved flexibility in handling state commitment types during tree creation.
    • Introduced an EmptyExporter for handling empty trees in the commitment package.
    • Added a new function for converting byte slices into CommitInfo structures.
  • Bug Fixes

    • Adjusted control flow in the root store creation process to ensure correct handling of migration scenarios.
  • Tests

    • Expanded test coverage for the root store creation process, including scenarios for different state commitment types and migration states.
    • Introduced a utility function for managing versioning in the database.
  • Style

    • Updated key formatting constants in the commitment package to reflect a new versioning scheme.

@cool-develope cool-develope requested review from kocubinski and a team as code owners October 22, 2024 23:08
Copy link
Contributor

coderabbitai bot commented Oct 22, 2024

📝 Walkthrough
📝 Walkthrough

Walkthrough

The changes in this pull request primarily enhance the CreateRootStore function within the factory.go file of the store/v2/root package. Key modifications include an updated function signature for newTreeFn to accept an additional scType parameter, the implementation of migration logic based on the state storage version, and the instantiation of a migration.Manager when necessary. These changes allow for improved handling of state commitment types and support for migration from store/v1 to store/v2.

Changes

File Change Summary
store/v2/root/factory.go Updated newTreeFn signature to accept scType. Enhanced CreateRootStore with migration logic and snapshot management.
store/v2/root/factory_test.go Added tests for root store creation scenarios, including migration handling and a new function for setting the latest version.
store/v2/commitment/metadata.go Updated key formatting constants for metadata storage, changing prefixes from "c/" to "s/".
store/v2/commitment/iavl/exporter.go Introduced EmptyExporter type for handling empty tree exports.
store/v2/commitment/iavl/tree.go Added initialVersion field to IavlTree and updated related methods for version control.
store/v2/go.mod Added direct dependency on google.golang.org/protobuf v1.35.1 and removed indirect dependency.
store/v2/proof/proof.go Added ConvertV1CommitInfo function for converting byte slices to CommitInfo.

Assessment against linked issues

Objective Addressed Explanation
Support migration from store/v1 to store/v2 (#[22180])
Enable configuration flag for migration in store/v2/root.Config (#[22180]) The proposed configuration flag was not implemented.

Possibly related PRs

Suggested labels

C:server/v2, backport/v0.52.x

Suggested reviewers

  • kocubinski
  • facundomedica
  • sontrinh16
  • akhilkumarpilli
  • hieuvubk

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (2)
store/v2/root/factory.go (2)

Line range hint 144-153: Refactor to remove unnecessary else block

According to the Uber Go Style Guide, when an if block ends with a return, the subsequent else can be removed to reduce nesting and improve readability.

Apply this diff to simplify the code:

 newTreeFn := func(key string, scType SCType) (commitment.Tree, error) {
     if internal.IsMemoryStoreKey(key) {
         return mem.New(), nil
-    } else {
-        switch scType {
+    }
+    switch scType {
     case SCTypeIavl:
         return iavl.NewIavlTree(db.NewPrefixDB(opts.SCRawDB, []byte(key)), opts.Logger, storeOpts.IavlConfig), nil
     case SCTypeIavlV2:
         return nil, errors.New("iavl v2 not supported")
     default:
         return nil, errors.New("unsupported commitment store type")
     }
-    }
 }

16-18: Organize import statements for better readability

Per Go conventions and the Uber Go Style Guide, import statements should be grouped into standard library imports, external imports, and local imports, each separated by a blank line. This improves code clarity.

Consider restructuring the imports as follows:

import (
    "errors"
    "fmt"
    "os"

    "cosmossdk.io/core/log"
    corestore "cosmossdk.io/core/store"

    "cosmossdk.io/store/v2"
    "cosmossdk.io/store/v2/commitment"
    "cosmossdk.io/store/v2/commitment/iavl"
    "cosmossdk.io/store/v2/commitment/mem"
    "cosmossdk.io/store/v2/db"
    "cosmossdk.io/store/v2/internal"
    "cosmossdk.io/store/v2/migration"
    "cosmossdk.io/store/v2/pruning"
    "cosmossdk.io/store/v2/snapshots"
    "cosmossdk.io/store/v2/storage"
    "cosmossdk.io/store/v2/storage/pebbledb"
    "cosmossdk.io/store/v2/storage/rocksdb"
    "cosmossdk.io/store/v2/storage/sqlite"
)
📜 Review details

Configuration used: .coderabbit.yml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between cd3ce0c and e7bd885.

📒 Files selected for processing (1)
  • store/v2/root/factory.go (4 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
store/v2/root/factory.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

🔇 Additional comments (2)
store/v2/root/factory.go (2)

203-216: Ensure newSC is properly initialized before use

The variable newSC may remain nil if scType equals storeOpts.SCType, but it is later used in migration.NewManager. Verify that migration.NewManager can handle a nil newSC without causing unexpected behavior.

Run the following script to check how newSC is used within migration.NewManager:

✅ Verification successful

Let me generate new shell scripts to verify how newSC is used in the migration manager.


Based on the results, I can now provide a final response as we have sufficient evidence.

newSC being nil is an expected and handled case

The migration manager explicitly supports a nil state commitment store (sc) parameter as documented in its README:

sc (Commitment Store) can be nil. In that case, the Manager will migrate only the state storage.

This means the code is working as intended when newSC remains nil in cases where scType == storeOpts.SCType. The migration manager is designed to handle this scenario gracefully.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that migration.NewManager can handle a nil newSC.

# Navigate to the migration package directory
cd path/to/cosmossdk.io/store/v2/migration

# Search for the usage of newSC in NewManager
rg -A 5 'func NewManager\('

# Check for dereferences of newSC
rg 'newSC\.\w+' *.go

Length of output: 11974


Script:

#!/bin/bash

# Find the migration package and check NewManager implementation
rg -A 10 "func NewManager" store/v2/migration/

# Find where newSC is used in the migration package
rg "newSC" store/v2/migration/

# Check the factory.go file for more context around newSC usage
rg -A 10 "migration\.NewManager" store/v2/root/factory.go

Length of output: 1662


221-221: Confirm that passing mm and nil to New is acceptable

In the return statement, New is called with mm (which may be nil) and nil as the last parameter. Ensure that the New function is designed to handle nil values for these parameters appropriately.

Run the following script to examine the New function's handling of nil arguments:

✅ Verification successful

Let me generate new shell scripts to better verify the handling of nil parameters in the New function.


Based on the code inspection, I can now provide a definitive response about the handling of nil parameters.

Passing nil values to New is safe and properly handled

The New function in store/v2/root/store.go explicitly handles nil values:

  • The mm parameter (migration.Manager) is used to set both migrationManager and isMigrating fields, where isMigrating is explicitly set based on whether mm is nil: isMigrating: mm != nil
  • The last parameter m (metrics.StoreMetrics) is simply assigned to the telemetry field, and there are no operations that would cause issues if it's nil
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that the New function can handle nil values for mm and the last parameter.

# Navigate to the root package directory
cd path/to/cosmossdk.io/store/v2/root

# Search for the New function definition
rg -A 15 'func New\('

# Look for how mm and the last parameter are used within the function
rg 'mm\.' *.go
rg '.*nil.*' *.go

Length of output: 11974


Script:

#!/bin/bash
# First, let's find the exact file containing the New function in store/v2/root
fd "factory.go|store.go" store/v2/root/

# Then, let's examine how mm and metrics parameters are used in the Store struct and New function
rg -A 10 "type Store struct" store/v2/root/
rg -A 20 "func New\(" store/v2/root/

# Let's also check for any usage of mm or metrics in the package
rg "mm\." store/v2/root/
rg "metrics\." store/v2/root/

Length of output: 1963

Comment on lines +159 to +171
// check if we need to migrate the store
isMigrating := false
scType := storeOpts.SCType
ssLatestVersion, err := ss.GetLatestVersion()
if err != nil {
return nil, err
}
if ssLatestVersion != latestVersion {
isMigrating = true // need to migrate
if scType != SCTypeIavl {
scType = SCTypeIavl // only support iavl v1 for migration
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Simplify isMigrating assignment

You can assign the result of the condition directly to isMigrating to make the code more concise.

Apply this diff to streamline the variable initialization:

-isMigrating := false
...
-if ssLatestVersion != latestVersion {
-    isMigrating = true // need to migrate
+isMigrating := ssLatestVersion != latestVersion
 if isMigrating {
     if scType != SCTypeIavl {
         scType = SCTypeIavl // only support iavl v1 for migration
     }
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// check if we need to migrate the store
isMigrating := false
scType := storeOpts.SCType
ssLatestVersion, err := ss.GetLatestVersion()
if err != nil {
return nil, err
}
if ssLatestVersion != latestVersion {
isMigrating = true // need to migrate
if scType != SCTypeIavl {
scType = SCTypeIavl // only support iavl v1 for migration
}
}
// check if we need to migrate the store
scType := storeOpts.SCType
ssLatestVersion, err := ss.GetLatestVersion()
if err != nil {
return nil, err
}
isMigrating := ssLatestVersion != latestVersion
if isMigrating {
if scType != SCTypeIavl {
scType = SCTypeIavl // only support iavl v1 for migration
}
}

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (2)
store/v2/root/factory_test.go (2)

32-39: Consider using subtests for better test organization

While the test coverage is good, consider using t.Run() to organize different test scenarios into subtests for better clarity and maintainability.

Example refactor:

- require.NoError(t, setLatestVersion(fop.SCRawDB, 1))
- fop.Options.SCType = SCTypeIavl
- f, err = CreateRootStore(&fop)
- require.NoError(t, err)
- require.NotNil(t, f)
- require.True(t, f.(*Store).isMigrating)
+ t.Run("migration enabled", func(t *testing.T) {
+   require.NoError(t, setLatestVersion(fop.SCRawDB, 1))
+   fop.Options.SCType = SCTypeIavl
+   f, err = CreateRootStore(&fop)
+   require.NoError(t, err)
+   require.NotNil(t, f)
+   require.True(t, f.(*Store).isMigrating)
+ })

41-47: Consider defining "c/latest" as a constant

The function implementation is solid, but the magic string "c/latest" should be defined as a package-level constant for better maintainability and reusability.

Example:

+ const latestVersionKey = "c/latest"
+
func setLatestVersion(db corestore.KVStoreWithBatch, version uint64) error {
    var buf bytes.Buffer
    buf.Grow(encoding.EncodeUvarintSize(version))
    if err := encoding.EncodeUvarint(&buf, version); err != nil {
        return err
    }
-   return db.Set([]byte("c/latest"), buf.Bytes())
+   return db.Set([]byte(latestVersionKey), buf.Bytes())
}
📜 Review details

Configuration used: .coderabbit.yml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between e7bd885 and 3bb541b.

📒 Files selected for processing (1)
  • store/v2/root/factory_test.go (2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
store/v2/root/factory_test.go (2)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.


Pattern **/*_test.go: "Assess the unit test code assessing sufficient code coverage for the changes associated in the pull request"

🔇 Additional comments (2)
store/v2/root/factory_test.go (2)

4-4: LGTM: Import additions are appropriate

The new imports for bytes and encoding packages are necessary for the version encoding functionality.

Also applies to: 12-12


Line range hint 15-39: Verify edge cases in migration testing

While the basic migration scenarios are covered, consider adding test cases for the following edge cases:

  1. Migration with version 0
  2. Migration with very large version numbers
  3. Failed migration scenarios

Let's check if there are any existing edge case tests in other test files:

Copy link
Member

@tac0turtle tac0turtle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how does this work, tried it locally. the node ends up hanging

i start a simapp v1 node, copy over the data directory then start the node and it hangs. is that expected?

@cool-develope
Copy link
Contributor Author

how does this work, tried it locally. the node ends up hanging

i start a simapp v1 node, copy over the data directory then start the node and it hangs. is that expected?

e2e migration still is not working fully, created the issue #22360
how about merge this PR? will figure out in the next PR.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Outside diff range and nitpick comments (2)
store/v2/root/factory.go (2)

Line range hint 146-159: Consider enhancing error messages with more context.

The function structure and error handling look good. Consider making the error messages more specific by including the actual SCType value in the "unsupported commitment store type" error.

-				return nil, errors.New("unsupported commitment store type")
+				return nil, fmt.Errorf("unsupported commitment store type: %s", scType)

161-173: Document the IAVL v1 migration constraint.

The code enforces SCTypeIavl during migration, but this significant constraint isn't documented. Consider adding a comment explaining why only IAVL v1 is supported during migration.

 if ssLatestVersion != latestVersion {
     isMigrating = true // need to migrate
     if scType != SCTypeIavl {
+        // IAVL v1 is currently the only supported format for migration due to
+        // compatibility requirements with the existing state commitment structure
         scType = SCTypeIavl // only support iavl v1 for migration
     }
 }
📜 Review details

Configuration used: .coderabbit.yml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 3bb541b and cb86a77.

📒 Files selected for processing (2)
  • store/v2/commitment/metadata.go (1 hunks)
  • store/v2/root/factory.go (5 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
store/v2/commitment/metadata.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

store/v2/root/factory.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

🔇 Additional comments (4)
store/v2/commitment/metadata.go (1)

14-16: Verify migration path for key prefix changes

The change from "c/" to "s/" prefix is a breaking change that affects how metadata is stored and retrieved. This could impact existing databases.

Let's verify if this change is handled by the migration manager:

The comments still reference the old prefix format (e.g., "c/"). Please update them to reflect the new "s/" prefix for consistency.

-	commitInfoKeyFmt      = "s/%d" // c/<version>
-	removedStoreKeyPrefix = "s/removed/" // c/removed/<version>/<store-name>
+	commitInfoKeyFmt      = "s/%d" // s/<version>
+	removedStoreKeyPrefix = "s/removed/" // s/removed/<version>/<store-name>
store/v2/root/factory.go (3)

16-18: LGTM: Clean import additions and well-documented constant.

The new imports and store prefix constant are well-structured and properly documented.

Also applies to: 38-39


Line range hint 177-189: LGTM: Consistent tree creation implementation.

The tree creation logic is properly updated to handle the new scType parameter with consistent error handling.


223-223: LGTM: Clean final setup.

The pruning manager setup and final store creation look good.

Comment on lines +197 to +220
var mm *migration.Manager
if isMigrating {
snapshotDB, err := snapshots.NewStore(fmt.Sprintf("%s/data/snapshots/store.db", opts.RootDir))
if err != nil {
return nil, err
}
snapshotMgr := snapshots.NewManager(snapshotDB, snapshots.SnapshotOptions{}, sc, nil, nil, opts.Logger)
var newSC *commitment.CommitStore
if scType != storeOpts.SCType {
newTrees := make(map[string]commitment.Tree, len(opts.StoreKeys))
for _, key := range opts.StoreKeys {
tree, err := newTreeFn(key, storeOpts.SCType)
if err != nil {
return nil, err
}
newTrees[key] = tree
}
newSC, err = commitment.NewCommitStore(newTrees, nil, opts.SCRawDB, opts.Logger)
if err != nil {
return nil, err
}
}
mm = migration.NewManager(opts.SCRawDB, snapshotMgr, ss, newSC, opts.Logger)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider refactoring migration setup for better maintainability.

The migration setup logic is complex and handles multiple concerns. Consider extracting it into a separate function for better maintainability and testing.

func setupMigrationManager(opts *FactoryOptions, ss *storage.StorageStore, sc *commitment.CommitStore, scType SCType) (*migration.Manager, error) {
    snapshotDB, err := snapshots.NewStore(fmt.Sprintf("%s/data/snapshots/store.db", opts.RootDir))
    if err != nil {
        return nil, err
    }
    
    snapshotMgr := snapshots.NewManager(snapshotDB, snapshots.SnapshotOptions{}, sc, nil, nil, opts.Logger)
    
    var newSC *commitment.CommitStore
    if scType != opts.Options.SCType {
        newSC, err = createNewCommitStore(opts, scType)
        if err != nil {
            snapshotDB.Close() // Cleanup on error
            return nil, err
        }
    }
    
    return migration.NewManager(opts.SCRawDB, snapshotMgr, ss, newSC, opts.Logger), nil
}

Comment on lines +199 to +202
snapshotDB, err := snapshots.NewStore(fmt.Sprintf("%s/data/snapshots/store.db", opts.RootDir))
if err != nil {
return nil, err
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Add cleanup for snapshot DB on error.

The snapshot DB creation should be cleaned up if subsequent operations fail to prevent orphaned files.

 snapshotDB, err := snapshots.NewStore(fmt.Sprintf("%s/data/snapshots/store.db", opts.RootDir))
 if err != nil {
+    // Clean up any partially created files
+    _ = os.RemoveAll(fmt.Sprintf("%s/data/snapshots", opts.RootDir))
     return nil, err
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
snapshotDB, err := snapshots.NewStore(fmt.Sprintf("%s/data/snapshots/store.db", opts.RootDir))
if err != nil {
return nil, err
}
snapshotDB, err := snapshots.NewStore(fmt.Sprintf("%s/data/snapshots/store.db", opts.RootDir))
if err != nil {
// Clean up any partially created files
_ = os.RemoveAll(fmt.Sprintf("%s/data/snapshots", opts.RootDir))
return nil, err
}

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Outside diff range and nitpick comments (6)
store/v2/commitment/iavl/exporter.go (2)

42-43: Enhance type documentation for clarity

The documentation should provide more context about when and why an EmptyExporter is used, especially in relation to version migration scenarios.

Consider expanding the documentation:

-// EmptyExporter is a Exporter for an empty tree.
+// EmptyExporter implements the commitment.Exporter interface for cases where no data
+// should be exported, such as when the requested version is before the initial version
+// during store migration. It always returns ErrorExportDone without exporting any items.
 type EmptyExporter struct{}

45-53: Implementation looks good with minor documentation suggestions

The EmptyExporter methods are correctly implemented. Consider enhancing the method documentation:

-// Next returns ExportDone.
+// Next implements the commitment.Exporter interface by always returning ErrorExportDone,
+// indicating that no items are available for export.
 func (e *EmptyExporter) Next() (*snapshotstypes.SnapshotIAVLItem, error) {

-// Close does nothing.
+// Close implements the commitment.Exporter interface as a no-op since there are no
+// resources to clean up in an empty exporter.
 func (e *EmptyExporter) Close() error {
store/v2/root/factory_test.go (1)

32-39: Consider expanding test coverage for migration scenarios

While the current test cases cover basic migration functionality, consider adding:

  1. Tests with different version numbers (0, max uint64)
  2. Negative test cases for invalid versions
  3. Verification of cleanup after migration
  4. Tests for concurrent access during migration

Example test case structure:

func TestFactory_MigrationScenarios(t *testing.T) {
    testCases := []struct {
        name          string
        version       uint64
        expectedError bool
        checkState    func(*testing.T, *Store)
    }{
        {"version zero", 0, false, nil},
        {"max version", math.MaxUint64, false, nil},
        // Add more cases
    }
    // ... test implementation
}
store/v2/commitment/iavl/tree.go (2)

24-25: Enhance field documentation for clarity.

The comment could be more descriptive about when and how this field is used during migration.

Consider updating the comment to:

-// it is only used for new store key during the migration process.
+// initialVersion represents the starting version for new store keys during v1 to v2 migration.
+// It is used to handle version-specific operations during the migration process.

63-73: Improve error handling and documentation.

While the logic is correct, consider these improvements:

  1. Add more context to the comment explaining why we skip version loading for empty trees during migration
  2. Consider wrapping the error with additional context
 if t.initialVersion > 0 {
-    // If the initial version is set and the tree is empty,
-    // we don't need to load the version.
+    // During migration with initialVersion set, we skip loading versions for empty trees
+    // as they represent new stores that start from the migration version
     latestVersion, err := t.tree.GetLatestVersion()
     if err != nil {
-        return err
+        return fmt.Errorf("failed to get latest version: %w", err)
     }
     if latestVersion == 0 {
         return nil
     }
 }
store/v2/proof/proof.go (1)

233-258: Add documentation and improve error handling

The function needs documentation explaining:

  • Purpose of the conversion
  • Expected input format
  • Potential error cases
  • Why empty byte slice is used in GetStoreProof

Add godoc comment:

+// ConvertV1CommitInfo converts a marshaled v1 CommitInfo to a v2 CommitInfo.
+// It preserves version, timestamp, and store information while computing a new
+// commit hash using GetStoreProof.
📜 Review details

Configuration used: .coderabbit.yml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between cb86a77 and 79d978b.

⛔ Files ignored due to path filters (1)
  • store/v2/proof/v1/types/commit_info.pb.go is excluded by !**/*.pb.go
📒 Files selected for processing (7)
  • store/v2/commitment/iavl/exporter.go (1 hunks)
  • store/v2/commitment/iavl/tree.go (4 hunks)
  • store/v2/commitment/metadata.go (2 hunks)
  • store/v2/go.mod (1 hunks)
  • store/v2/proof/proof.go (2 hunks)
  • store/v2/root/factory.go (5 hunks)
  • store/v2/root/factory_test.go (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • store/v2/commitment/metadata.go
  • store/v2/root/factory.go
🧰 Additional context used
📓 Path-based instructions (4)
store/v2/commitment/iavl/exporter.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

store/v2/commitment/iavl/tree.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

store/v2/proof/proof.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

store/v2/root/factory_test.go (2)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.


Pattern **/*_test.go: "Assess the unit test code assessing sufficient code coverage for the changes associated in the pull request"

🔇 Additional comments (8)
store/v2/commitment/iavl/exporter.go (1)

42-53: Verify interface implementation

Let's ensure EmptyExporter correctly implements the expected interface.

✅ Verification successful

Let me gather more information about the interface definition.


Let me search for the interface definition and any compile-time checks.


Based on the results, I can now provide a final response:

Interface implementation verified

EmptyExporter correctly implements the Exporter interface by providing:

  1. Next() (*snapshotstypes.SnapshotIAVLItem, error) method
  2. Close() error method (satisfying io.Closer)

The implementation matches the interface requirements defined in store/v2/commitment/tree.go.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that EmptyExporter implements the same interface as Exporter

# Find the interface definition that both types should implement
ast-grep --pattern 'type $interface interface {
  Next() ($_, error)
  Close() error
}'

# Verify both types implement the same methods
rg -A 2 'type (Exporter|EmptyExporter) struct'

Length of output: 470


Script:

#!/bin/bash
# Find interface definitions and usages
rg -A 5 'type.*interface' store/v2/commitment

# Check for any compile-time interface checks
rg 'var.*interface'

# Look for the Next method signature in the codebase
ast-grep --pattern 'func ($receiver $_) Next() ($return_type $_, error) {
  $$$
}'

Length of output: 110554


Script:

#!/bin/bash
# Find the Exporter interface definition
rg -A 3 'type Exporter interface' store/v2/commitment

# Look for any compile-time interface checks
rg 'var.*Exporter'

Length of output: 276

store/v2/root/factory_test.go (2)

4-4: LGTM: Import changes are appropriate

The new imports support the buffer operations and encoding functionality required for version management.

Also applies to: 9-9, 12-12


Line range hint 1-47: Verify migration manager test coverage aligns with PR objectives

While the tests cover basic migration functionality, consider adding integration tests to verify the end-to-end migration process mentioned in PR #22180, especially given the reported hanging issues in the PR comments.

Consider adding an integration test package specifically for store migration scenarios to catch issues like the hanging node problem early in the development cycle.

store/v2/go.mod (1)

24-24: LGTM! Consider checking for a newer protobuf version.

The addition of protobuf as a direct dependency aligns with the migration manager implementation. However, let's verify if v1.35.1 is the latest stable version.

✅ Verification successful

Protobuf version v1.35.1 is up to date

The current version (v1.35.1) of google.golang.org/protobuf in go.mod matches the latest stable release.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check if there's a newer stable version of google.golang.org/protobuf
# Note: Using gh api to query the latest release

gh api \
  -H "Accept: application/vnd.github+json" \
  repos/protocolbuffers/protobuf-go/releases/latest \
  --jq '.tag_name'

Length of output: 122

store/v2/commitment/iavl/tree.go (2)

122-122: LGTM!

The implementation correctly sets the initialVersion field after setting the tree's initial version.


142-144: Verify EmptyExporter implementation.

The logic to return EmptyExporter for versions before initialVersion is correct. However, let's verify the EmptyExporter implementation.

✅ Verification successful

EmptyExporter implementation is correct and complete

The EmptyExporter correctly implements the Exporter interface by providing:

  • A Next() method that returns ErrorExportDone, indicating no items to export
  • The io.Closer interface is implicitly satisfied as no cleanup is needed for an empty exporter

The implementation aligns with its purpose of handling versions before initialVersion where no data needs to be exported.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify EmptyExporter implementation
# Test: Check EmptyExporter methods and usage
rg -A 5 "type EmptyExporter struct" store/v2/commitment/iavl/

Length of output: 459


Script:

#!/bin/bash
# Check for the interface that EmptyExporter implements
rg -A 5 "type Exporter interface" store/v2/commitment/iavl/

# Check for any other methods of EmptyExporter
rg "func \(\*EmptyExporter\)" store/v2/commitment/iavl/

# Check for ErrorExportDone definition
rg "ErrorExportDone" store/v2/commitment/

Length of output: 664


Script:

#!/bin/bash
# Check for the Exporter interface definition in the commitment package
rg -A 5 "type Exporter interface" store/v2/commitment/

# Check for any usage of EmptyExporter in tests
rg "EmptyExporter" store/v2/commitment/iavl/.*_test.go

Length of output: 448

store/v2/proof/proof.go (2)

10-10: LGTM: Import statement is well-structured

The import statement follows Go conventions and uses appropriate aliasing.


254-256: 🛠️ Refactor suggestion

Improve error handling and clarify GetStoreProof usage

The error handling could be improved, and the purpose of using an empty byte slice with GetStoreProof should be clarified.

Let's verify the usage of GetStoreProof with empty byte slice in other parts of the codebase:

Consider improving error handling:

-	cInfo.CommitHash, _, err = cInfo.GetStoreProof([]byte{})
+	var err error
+	if cInfo.CommitHash, _, err = cInfo.GetStoreProof([]byte{}); err != nil {
+		return nil, errors.Wrapf(err, "failed to compute commit hash")
+	}

Comment on lines +41 to +47
func setLatestVersion(db corestore.KVStoreWithBatch, version uint64) error {
var buf bytes.Buffer
buf.Grow(encoding.EncodeUvarintSize(version))
if err := encoding.EncodeUvarint(&buf, version); err != nil {
return err
}
return db.Set([]byte("s/latest"), buf.Bytes())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider enhancing version management robustness

The implementation is correct but could be improved:

  1. Extract "s/latest" as a package-level constant
  2. Add version validation
  3. Consider adding a corresponding getLatestVersion function for symmetry
+const latestVersionKey = "s/latest"

 func setLatestVersion(db corestore.KVStoreWithBatch, version uint64) error {
+    // Validate version if needed
     var buf bytes.Buffer
     buf.Grow(encoding.EncodeUvarintSize(version))
     if err := encoding.EncodeUvarint(&buf, version); err != nil {
         return err
     }
-    return db.Set([]byte("s/latest"), buf.Bytes())
+    return db.Set([]byte(latestVersionKey), buf.Bytes())
 }

+func getLatestVersion(db corestore.KVStoreWithBatch) (uint64, error) {
+    data, err := db.Get([]byte(latestVersionKey))
+    if err != nil {
+        return 0, err
+    }
+    if len(data) == 0 {
+        return 0, nil
+    }
+    return encoding.DecodeUvarint(bytes.NewReader(data))
+}

Committable suggestion was skipped due to low confidence.

Comment on lines +244 to +251
cInfo.StoreInfos[i] = StoreInfo{
Name: []byte(v1StoreInfo.Name),
CommitID: CommitID{
Version: uint64(v1StoreInfo.CommitId.Version),
Hash: v1StoreInfo.CommitId.Hash,
},
Structure: "iavl",
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Avoid hardcoding store structure type

Hardcoding "iavl" as the Structure value is risky as not all stores might use IAVL.

Consider:

  1. Getting the structure type from v1StoreInfo
  2. Adding validation for supported store types
  3. Adding a mapping of v1 to v2 store types
-			Structure: "iavl",
+			Structure: determineStoreStructure(v1StoreInfo),

Committable suggestion was skipped due to low confidence.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature]: Add configuration flag to support migration from store/v1 -> store/v2
4 participants