Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/check metadata hash extension #642

Closed
wants to merge 57 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
b43bed6
draft: hotkey swap for senate
Jul 9, 2024
9a179bb
feature-gate hash
camfairchild Jul 12, 2024
5039e5b
use 1.10.0-rc2
camfairchild Jul 12, 2024
322752b
add feature
camfairchild Jul 12, 2024
9d1f6c9
feat: remove schedule coldkey swap
Jul 12, 2024
90512c2
chore: fix tests
Jul 12, 2024
dd87bca
Merge branch 'fix/hotkeyswap_senate' into feat/remove_schedule_coldke…
Jul 12, 2024
606b771
fixes for network resumption
Jul 12, 2024
8ebcf2d
chore: bump spec
Jul 12, 2024
7b84d3c
fix: add back arbitration check
Jul 12, 2024
2455082
fix: hotkey
Jul 12, 2024
f49bb55
fix: coldkey arb swap hotkey
Jul 12, 2024
884a179
Merge pull request #631 from opentensor/feat/remove_schedule_coldkey_…
distributedstatemachine Jul 12, 2024
76f20ce
swap delegate stake also
camfairchild Jul 13, 2024
4ccb879
add test for this
camfairchild Jul 13, 2024
73533b9
swap over stakinghotkeys map
camfairchild Jul 13, 2024
c370aad
check map first
camfairchild Jul 13, 2024
9b8c0e4
add staking hotkeys test
camfairchild Jul 13, 2024
baec581
no take
camfairchild Jul 13, 2024
a84ac6b
keep old stake
camfairchild Jul 13, 2024
221d320
add check to test
camfairchild Jul 13, 2024
f9164fc
fix some tests
camfairchild Jul 13, 2024
2a14df7
fix delegate test
camfairchild Jul 13, 2024
b970f50
update staking hotekys maps
camfairchild Jul 13, 2024
ad57e2b
init
camfairchild Jul 13, 2024
1cc7d3c
comment out
camfairchild Jul 13, 2024
6dcd0a1
add admin swap
camfairchild Jul 13, 2024
f474112
fix swap
camfairchild Jul 13, 2024
b708c1d
..
camfairchild Jul 13, 2024
5b30256
hotkey staking maps fix
Jul 13, 2024
fc245f8
remove staking hotkeys
Jul 13, 2024
494796e
remove commented code
Jul 13, 2024
f041ed9
cargo fmt
sam0x17 Jul 13, 2024
54ec750
cargo fix --workspace
sam0x17 Jul 13, 2024
ca51109
bump spec_version to 165
sam0x17 Jul 13, 2024
4dafd88
swap hotkey benchmark removed
Jul 15, 2024
d95a7df
add migration and tests for total coldkey hotfix
Jul 15, 2024
5e6fc7e
adds new test for missing hotkey value
Jul 15, 2024
3c61aa8
bump migration values
Jul 15, 2024
ff23b6f
fmt
Jul 15, 2024
78a71f9
lock file update
camfairchild Jul 15, 2024
fc0ccbd
add production just target
camfairchild Jul 15, 2024
b09b3e6
fmt
camfairchild Jul 15, 2024
0c2ad41
Merge pull request #636 from opentensor/total_coldkey_migration
unconst Jul 15, 2024
7364832
clippy
camfairchild Jul 16, 2024
64aca95
benchmarking fix
camfairchild Jul 16, 2024
b3fa487
feat: try runtime passing
Jul 16, 2024
8766c98
chore: remove commented code
Jul 16, 2024
92fc1f0
chore: make logs human readable
Jul 16, 2024
c255d92
chore: remove comments
Jul 16, 2024
0478596
Update pallets/subtensor/src/lib.rs
distributedstatemachine Jul 16, 2024
5adbbd9
Merge pull request #644 from opentensor/hotfix/storage_version_incons…
distributedstatemachine Jul 16, 2024
9ec6c4e
fmt
camfairchild Jul 16, 2024
893b1ec
use rc3 with new fix polkadot-sdk/pull/4117
camfairchild Jul 18, 2024
237634c
incl lock
camfairchild Jul 18, 2024
c80df3d
bump CI
sam0x17 Jul 18, 2024
4a4adfd
Merge branch 'main' into feat/check-metadata-hash-extension
camfairchild Jul 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
485 changes: 277 additions & 208 deletions Cargo.lock

Large diffs are not rendered by default.

139 changes: 70 additions & 69 deletions Cargo.toml

Large diffs are not rendered by default.

6 changes: 5 additions & 1 deletion justfile
Original file line number Diff line number Diff line change
Expand Up @@ -47,4 +47,8 @@ lint:
@echo "Running cargo clippy with automatic fixes on potentially dirty code..."
just clippy-fix
@echo "Running cargo clippy..."
just clippy
just clippy

production:
@echo "Running cargo build with metadata-hash generation..."
cargo +{{RUSTV}} build --profile production --features="runtime-benchmarks metadata-hash"
1 change: 1 addition & 0 deletions node/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ sp-io = { workspace = true }
sp-timestamp = { workspace = true }
sp-inherents = { workspace = true }
sp-keyring = { workspace = true }
frame-metadata-hash-extension = { workspace = true }
frame-system = { workspace = true }
pallet-transaction-payment = { workspace = true }
pallet-commitments = { path = "../pallets/commitments" }
Expand Down
2 changes: 2 additions & 0 deletions node/src/benchmarking.rs
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,7 @@ pub fn create_benchmark_extrinsic(
pallet_transaction_payment::ChargeTransactionPayment::<runtime::Runtime>::from(0),
pallet_subtensor::SubtensorSignedExtension::<runtime::Runtime>::new(),
pallet_commitments::CommitmentsSignedExtension::<runtime::Runtime>::new(),
frame_metadata_hash_extension::CheckMetadataHash::<runtime::Runtime>::new(true),
);

let raw_payload = runtime::SignedPayload::from_raw(
Expand All @@ -152,6 +153,7 @@ pub fn create_benchmark_extrinsic(
(),
(),
(),
None,
),
);
let signature = raw_payload.using_encoded(|e| sender.sign(e));
Expand Down
4 changes: 2 additions & 2 deletions pallets/admin-utils/tests/mock.rs
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ parameter_types! {
pub const InitialSubnetLimit: u16 = 10; // Max 10 subnets.
pub const InitialNetworkRateLimit: u64 = 0;
pub const InitialTargetStakesPerInterval: u16 = 1;
pub const InitialHotkeySwapCost: u64 = 1_000_000_000;
pub const InitialKeySwapCost: u64 = 1_000_000_000;
pub const InitialAlphaHigh: u16 = 58982; // Represents 0.9 as per the production default
pub const InitialAlphaLow: u16 = 45875; // Represents 0.7 as per the production default
pub const InitialLiquidAlphaOn: bool = false; // Default value for LiquidAlphaOn
Expand Down Expand Up @@ -166,7 +166,7 @@ impl pallet_subtensor::Config for Test {
type InitialSubnetLimit = InitialSubnetLimit;
type InitialNetworkRateLimit = InitialNetworkRateLimit;
type InitialTargetStakesPerInterval = InitialTargetStakesPerInterval;
type HotkeySwapCost = InitialHotkeySwapCost;
type KeySwapCost = InitialKeySwapCost;
type AlphaHigh = InitialAlphaHigh;
type AlphaLow = InitialAlphaLow;
type LiquidAlphaOn = InitialLiquidAlphaOn;
Expand Down
80 changes: 27 additions & 53 deletions pallets/subtensor/src/benchmarks.rs
Original file line number Diff line number Diff line change
Expand Up @@ -314,33 +314,33 @@ benchmarks! {
assert_ok!(Subtensor::<T>::register_network(RawOrigin::Signed(coldkey.clone()).into()));
}: dissolve_network(RawOrigin::Signed(coldkey), 1)

swap_hotkey {
let seed: u32 = 1;
let coldkey: T::AccountId = account("Alice", 0, seed);
let old_hotkey: T::AccountId = account("Bob", 0, seed);
let new_hotkey: T::AccountId = account("Charlie", 0, seed);

let netuid = 1u16;
Subtensor::<T>::init_new_network(netuid, 100);
Subtensor::<T>::set_min_burn(netuid, 1);
Subtensor::<T>::set_max_burn(netuid, 1);
Subtensor::<T>::set_target_registrations_per_interval(netuid, 256);
Subtensor::<T>::set_max_registrations_per_block(netuid, 256);

Subtensor::<T>::add_balance_to_coldkey_account(&coldkey.clone(), 10_000_000_000u64);
assert_ok!(Subtensor::<T>::burned_register(RawOrigin::Signed(coldkey.clone()).into(), netuid, old_hotkey.clone()));
assert_ok!(Subtensor::<T>::become_delegate(RawOrigin::Signed(coldkey.clone()).into(), old_hotkey.clone()));

let max_uids = Subtensor::<T>::get_max_allowed_uids(netuid) as u32;
for i in 0..max_uids - 1 {
let coldkey: T::AccountId = account("Axon", 0, i);
let hotkey: T::AccountId = account("Hotkey", 0, i);

Subtensor::<T>::add_balance_to_coldkey_account(&coldkey.clone(), 10_000_000_000u64);
assert_ok!(Subtensor::<T>::burned_register(RawOrigin::Signed(coldkey.clone()).into(), netuid, hotkey));
assert_ok!(Subtensor::<T>::add_stake(RawOrigin::Signed(coldkey).into(), old_hotkey.clone(), 1_000_000_000));
}
}: _(RawOrigin::Signed(coldkey), old_hotkey, new_hotkey)
// swap_hotkey {
// let seed: u32 = 1;
// let coldkey: T::AccountId = account("Alice", 0, seed);
// let old_hotkey: T::AccountId = account("Bob", 0, seed);
// let new_hotkey: T::AccountId = account("Charlie", 0, seed);

// let netuid = 1u16;
// Subtensor::<T>::init_new_network(netuid, 100);
// Subtensor::<T>::set_min_burn(netuid, 1);
// Subtensor::<T>::set_max_burn(netuid, 1);
// Subtensor::<T>::set_target_registrations_per_interval(netuid, 256);
// Subtensor::<T>::set_max_registrations_per_block(netuid, 256);

// Subtensor::<T>::add_balance_to_coldkey_account(&coldkey.clone(), 10_000_000_000u64);
// assert_ok!(Subtensor::<T>::burned_register(RawOrigin::Signed(coldkey.clone()).into(), netuid, old_hotkey.clone()));
// assert_ok!(Subtensor::<T>::become_delegate(RawOrigin::Signed(coldkey.clone()).into(), old_hotkey.clone()));

// let max_uids = Subtensor::<T>::get_max_allowed_uids(netuid) as u32;
// for i in 0..max_uids - 1 {
// let coldkey: T::AccountId = account("Axon", 0, i);
// let hotkey: T::AccountId = account("Hotkey", 0, i);

// Subtensor::<T>::add_balance_to_coldkey_account(&coldkey.clone(), 10_000_000_000u64);
// assert_ok!(Subtensor::<T>::burned_register(RawOrigin::Signed(coldkey.clone()).into(), netuid, hotkey));
// assert_ok!(Subtensor::<T>::add_stake(RawOrigin::Signed(coldkey).into(), old_hotkey.clone(), 1_000_000_000));
// }
// }: _(RawOrigin::Signed(coldkey), old_hotkey, new_hotkey)

commit_weights {
let tempo: u16 = 1;
Expand Down Expand Up @@ -429,30 +429,4 @@ reveal_weights {

}: reveal_weights(RawOrigin::Signed(hotkey.clone()), netuid, uids, weight_values, salt, version_key)

schedule_coldkey_swap {
let seed: u32 = 1;
let old_coldkey: T::AccountId = account("OldColdkey", 0, seed);
let new_coldkey: T::AccountId = account("NewColdkey", 0, seed + 1);
let hotkey: T::AccountId = account("Hotkey", 0, seed);

let netuid = 1u16;
let tempo = 1u16;
let block_number: u64 = Subtensor::<T>::get_current_block_as_u64();
let nonce = 0;

// Initialize the network
Subtensor::<T>::init_new_network(netuid, tempo);
Subtensor::<T>::set_network_registration_allowed(netuid, true);

// Add balance to the old coldkey account
let amount_to_be_staked: u64 = 1000000u32.into();
Subtensor::<T>::add_balance_to_coldkey_account(&old_coldkey.clone(), amount_to_be_staked+1000000000);
// Burned register the hotkey with the old coldkey
assert_ok!(Subtensor::<T>::burned_register(
RawOrigin::Signed(old_coldkey.clone()).into(),
netuid,
hotkey.clone()
));

}: schedule_coldkey_swap(RawOrigin::Signed(old_coldkey.clone()), new_coldkey.clone(), vec![], block_number, nonce)
}
46 changes: 31 additions & 15 deletions pallets/subtensor/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ pub mod pallet {

/// Tracks version for migrations. Should be monotonic with respect to the
/// order of migrations. (i.e. always increasing)
const STORAGE_VERSION: StorageVersion = StorageVersion::new(6);
const STORAGE_VERSION: StorageVersion = StorageVersion::new(7);

/// Minimum balance required to perform a coldkey swap
pub const MIN_BALANCE_TO_PERFORM_COLDKEY_SWAP: u64 = 100_000_000; // 0.1 TAO in RAO
Expand Down Expand Up @@ -247,7 +247,7 @@ pub mod pallet {
type InitialTargetStakesPerInterval: Get<u64>;
/// Cost of swapping a hotkey.
#[pallet::constant]
type HotkeySwapCost: Get<u64>;
type KeySwapCost: Get<u64>;
/// The upper bound for the alpha parameter. Used for Liquid Alpha.
#[pallet::constant]
type AlphaHigh: Get<u16>;
Expand Down Expand Up @@ -1140,6 +1140,9 @@ pub mod pallet {
DefaultBonds<T>,
>;

#[pallet::storage] // --- Storage for migration run status
pub type HasMigrationRun<T: Config> = StorageMap<_, Identity, Vec<u8>, bool, ValueQuery>;

/// ==================
/// ==== Genesis =====
/// ==================
Expand Down Expand Up @@ -1419,7 +1422,9 @@ pub mod pallet {
// Populate OwnedHotkeys map for coldkey swap. Doesn't update storage vesion.
.saturating_add(migration::migrate_populate_owned::<T>())
// Populate StakingHotkeys map for coldkey swap. Doesn't update storage vesion.
.saturating_add(migration::migrate_populate_staking_hotkeys::<T>());
.saturating_add(migration::migrate_populate_staking_hotkeys::<T>())
// Fix total coldkey stake.
.saturating_add(migration::migrate_fix_total_coldkey_stake::<T>());

weight
}
Expand Down Expand Up @@ -2061,17 +2066,17 @@ pub mod pallet {
}

/// The extrinsic for user to change its hotkey
#[pallet::call_index(70)]
#[pallet::weight((Weight::from_parts(1_940_000_000, 0)
.saturating_add(T::DbWeight::get().reads(272))
.saturating_add(T::DbWeight::get().writes(527)), DispatchClass::Operational, Pays::No))]
pub fn swap_hotkey(
origin: OriginFor<T>,
hotkey: T::AccountId,
new_hotkey: T::AccountId,
) -> DispatchResultWithPostInfo {
Self::do_swap_hotkey(origin, &hotkey, &new_hotkey)
}
///#[pallet::call_index(70)]
///#[pallet::weight((Weight::from_parts(1_940_000_000, 0)
///.saturating_add(T::DbWeight::get().reads(272))
///.saturating_add(T::DbWeight::get().writes(527)), DispatchClass::Operational, Pays::No))]
///pub fn swap_hotkey(
/// origin: OriginFor<T>,
/// hotkey: T::AccountId,
/// new_hotkey: T::AccountId,
///) -> DispatchResultWithPostInfo {
/// Self::do_swap_hotkey(origin, &hotkey, &new_hotkey)
///}

/// The extrinsic for user to change the coldkey associated with their account.
///
Expand All @@ -2098,7 +2103,6 @@ pub mod pallet {
) -> DispatchResultWithPostInfo {
Self::do_swap_coldkey(origin, &new_coldkey)
}

/// Unstakes all tokens associated with a hotkey and transfers them to a new coldkey.
///
/// # Arguments
Expand All @@ -2114,6 +2118,7 @@ pub mod pallet {
/// # Weight
///
/// Weight is calculated based on the number of database reads and writes.
#[cfg(test)]
#[pallet::call_index(72)]
#[pallet::weight((Weight::from_parts(21_000_000, 0)
.saturating_add(T::DbWeight::get().reads(3))
Expand Down Expand Up @@ -2253,6 +2258,17 @@ pub mod pallet {
pub fn dissolve_network(origin: OriginFor<T>, netuid: u16) -> DispatchResult {
Self::user_remove_network(origin, netuid)
}

/// Sets values for liquid alpha
#[pallet::call_index(64)]
#[pallet::weight((0, DispatchClass::Operational, Pays::No))]
pub fn sudo_hotfix_swap_coldkey_delegates(
_origin: OriginFor<T>,
_old_coldkey: T::AccountId,
_new_coldkey: T::AccountId,
) -> DispatchResult {
Ok(())
}
}

// ---- Subtensor helper functions.
Expand Down
80 changes: 80 additions & 0 deletions pallets/subtensor/src/migration.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
use super::*;
use alloc::string::String;
use frame_support::traits::DefensiveResult;
use frame_support::{
pallet_prelude::{Identity, OptionQuery},
Expand All @@ -24,6 +25,85 @@ pub mod deprecated_loaded_emission_format {
StorageMap<Pallet<T>, Identity, u16, Vec<(AccountIdOf<T>, u64)>, OptionQuery>;
}

/// Migrates and fixes the total coldkey stake.
///
/// This function iterates through all staking hotkeys, calculates the total stake for each coldkey,
/// and updates the `TotalColdkeyStake` storage accordingly. The migration is only performed if the
/// on-chain storage version is 6.
///
/// # Returns
/// The weight of the migration process.
pub fn do_migrate_fix_total_coldkey_stake<T: Config>() -> Weight {
// Initialize the weight with one read operation.
let mut weight = T::DbWeight::get().reads(1);

// Iterate through all staking hotkeys.
for (coldkey, hotkey_vec) in StakingHotkeys::<T>::iter() {
// Init the zero value.
let mut coldkey_stake_sum: u64 = 0;
weight = weight.saturating_add(T::DbWeight::get().reads(1));

// Calculate the total stake for the current coldkey.
for hotkey in hotkey_vec {
// Cant fail on retrieval.
coldkey_stake_sum =
coldkey_stake_sum.saturating_add(Stake::<T>::get(hotkey, coldkey.clone()));
weight = weight.saturating_add(T::DbWeight::get().reads(1));
}
// Update the `TotalColdkeyStake` storage with the calculated stake sum.
// Cant fail on insert.
TotalColdkeyStake::<T>::insert(coldkey.clone(), coldkey_stake_sum);
weight = weight.saturating_add(T::DbWeight::get().writes(1));
}
weight
}

/// Migrates and fixes the total coldkey stake.
///
/// This function checks if the migration has already run, and if not, it performs the migration
/// to fix the total coldkey stake. It also marks the migration as completed after running.
///
/// # Returns
/// The weight of the migration process.
pub fn migrate_fix_total_coldkey_stake<T: Config>() -> Weight {
let migration_name = b"fix_total_coldkey_stake_v7".to_vec();

// Initialize the weight with one read operation.
let mut weight = T::DbWeight::get().reads(1);

// Check if the migration has already run
if HasMigrationRun::<T>::get(&migration_name) {
log::info!(
"Migration '{:?}' has already run. Skipping.",
migration_name
);
return Weight::zero();
}

log::info!(
"Running migration '{}'",
String::from_utf8_lossy(&migration_name)
);

// Run the migration
weight = weight.saturating_add(do_migrate_fix_total_coldkey_stake::<T>());

// Mark the migration as completed
HasMigrationRun::<T>::insert(&migration_name, true);
weight = weight.saturating_add(T::DbWeight::get().writes(1));

// Set the storage version to 7
StorageVersion::new(7).put::<Pallet<T>>();
weight = weight.saturating_add(T::DbWeight::get().writes(1));

log::info!(
"Migration '{:?}' completed. Storage version set to 7.",
String::from_utf8_lossy(&migration_name)
);

// Return the migration weight.
weight
}
/// Performs migration to update the total issuance based on the sum of stakes and total balances.
/// This migration is applicable only if the current storage version is 5, after which it updates the storage version to 6.
///
Expand Down
Loading
Loading