Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Sharding Support to PSMDB Demand Backup Test #918

Open
wants to merge 16 commits into
base: main
Choose a base branch
from

Conversation

edithturn
Copy link

Demand backup test for PSMDB to include sharding:

  • Set up a sharded cluster with 2 shards and 3 nodes each.
  • Add a test collection and enable sharding on it.
  • Add an unsharded test collection (expected to land on the second shard).
  • Perform backup, drop data, restore, and verify data integrity.
  • Clean up by deleting backups, restores, and the cluster.

The reference script for this implementation was: demand-backup.e2e.ts

@edithturn edithturn requested a review from a team as a code owner December 10, 2024 10:48
ui/apps/everest/.e2e/utils/db-cmd-line.ts Outdated Show resolved Hide resolved
ui/apps/everest/.e2e/utils/db-cmd-line.ts Outdated Show resolved Hide resolved
ui/apps/everest/.e2e/utils/db-cmd-line.ts Outdated Show resolved Hide resolved
ui/apps/everest/.e2e/utils/db-cmd-line.ts Outdated Show resolved Hide resolved
ui/apps/everest/.e2e/utils/db-cmd-line.ts Outdated Show resolved Hide resolved
ui/apps/everest/.e2e/utils/db-cmd-line.ts Outdated Show resolved Hide resolved
ui/apps/everest/.e2e/utils/db-cmd-line.ts Outdated Show resolved Hide resolved
);
};

export const validateMongoDBSharding = async (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add parameter to this function for collection so you can just call it twice in the test with different collection parameter so that you don't duplicate the code for t1 and t2.

);

// Shard t1
await queryPSMDB(cluster, namespace, 'test', 'db.t1.createIndex({ a: 1 });');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move createIndex to prepareMongoDBTestDB function.

);

// Shard t2
await queryPSMDB(cluster, namespace, 'test', 'db.t2.createIndex({ b: 1 });');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move createIndex to prepareMongoDBTestDB function.

cluster,
namespace,
'test',
'db.t2.insertMany([{ b: 1 }, { b: 2 }, { b: 3 }]);'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please call the keys the same as in t1, so { a: 1 }, { a: 2 }, { a: 3 }. It will simplify things because we will have 1 function for configureMongoSharding which will just take parameter for collection.

);
};

export const configureMongoDBSharding = async (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add parameter for collection name to this function and call it twice so that there is no code duplication for t1 and t2.
enableSharding can be called in the test directly or it can be left here because I don't think it will have any effect if the sharding is already enabled on the collection.

queryTestDB,
} from '@e2e/utils/db-cmd-line';

export let isShardingEnabled = false;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems strange exporting this from the test and importing it into helper functions. Let's create a function psmdbShardingEnabled in db-cmd-line.ts and then use it inside the same file.

const isChecked = await shardingCheckbox?.isChecked();
if (!isChecked) {
if (shardingCheckbox) {
isShardingEnabled = true;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we create a new function in db-cmd-line.ts we don't need this here. Also we want to check if sharding is actually enabled in the backend (api call) and not did we just click on something in UI.

// await moveForward(page);
});

// Step to set number of shards
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove comment since test description below has the same.

Comment on lines +243 to +245
//if (db != 'psmdb') {
// expect(addedCluster?.spec.proxy.replicas).toBe(size);
//}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove

);
expect(addedCluster?.spec.engine.storage.size.toString()).toBe('1Gi');
expect(addedCluster?.spec.proxy.expose.type).toBe('internal');

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add something like:

expect(addedCluster?.spec.sharding.enabled).toBe(true)
expect(addedCluster?.spec.sharding.shards).toBe(2)
expect(addedCluster?.spec.sharding.configServer.replicas).toBe(3)
expect(addedCluster?.spec.proxy.replicas).toBe(3)

expect(storageClasses.length).toBeGreaterThan(0);

await page.goto('/databases');
await page.getByTestId('add-db-cluster-button').waitFor();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this part should be moved to utils to avoid duplication 🤔 we call the cluster creation button in a lot of places

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants