-
Notifications
You must be signed in to change notification settings - Fork 31
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add initial Cypress Rancher Integration test cases #502
Conversation
@TachunLin I tried to run these tests and after getting the setup running I got this error when trying to run the
|
@noahgildersleeve, appreciated for checking and remind me for the rancher bootstrap login. Currently we can utilize rancher first time login in the following environment
When using ipxe-example to setup environment, we can simply specify user
If the Rancher built on VM or bare machine, we need to enable root ssh login to rancher host environment and set the username password as the following:
I setup a brand new Ubuntu 20.04 VM with Rancher v2.6.8 to validate the rancher bootstrap login on regular VM Then rerun the single |
|
||
}); | ||
|
||
it.only('Rancher import Harvester', { baseUrl: constants.rancherUrl}, () => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This it.only
block causes none of the other tests in the spec to run.
@noahgildersleeve , thanks for checking the updated commit, it remind me to test the whole suite in command run mode In the beginning, I encounter the same failure as your attempts, cypress runner stop and prompt I tried using --browser chrome but still can't improve. Then I add the And update Cypress version from I found adding these setting make the execution less crash. And I notice there is an logic error while determine the RKE1 and RKE2/K3s status which cause the provisioning cluster failure if we start it from the scratch. Also I found we can't use the default But I still encounter error like Under further investigation, I found too frequently login could be possible been rejected by Rancher And clean up the |
ecb57e9
to
ffbfa6a
Compare
ffbfa6a
to
60667ab
Compare
Hi @noahgildersleeve, @WuJun2016 Verified can completely finish all RKE2 related test suite from scratch and also checked the K3s test suits Please help to check when you are available. Thank you. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Verified RKE2 test case, mainly failed in the class name selector error.
cypress/pageobjects/rancher.po.ts
Outdated
cy.contains(harvester_cluster_name).click(); | ||
}) | ||
|
||
cy.get(this.cloudCredential_page_confirmCreate).should('contain', 'Create').click(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cy.get(this.cloudCredential_page_confirmCreate).should('contain', 'Create').click(); | |
cy.wait(1000).get(this.cloudCredential_page_confirmCreate).should('contain', 'Create').click(); |
I failed in this case because the Kubeconfig was not ready, Recommend adding wait(1000)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the recommendation. Add wait(1000) to the line 324 in pageobjects/rancher.po.ts
private rke2Creation_page_k8s_rke2Latest = '#vs1__option-1'; | ||
private rke2Creation_page_k8s_rke2Stable = '#vs1__option-2'; | ||
|
||
private rke2Creation_page_k8s_k3sLatest = '#vs1__option-4'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the instruction, change the menu drop down list selection using LabeledSelectPo
in /utils/components/labeled-select.po
new LabeledSelectPo('.labeled-select', `:contains("Kubernetes Version")`).select({option: rke2_cluster_attributes.rke2_latest, selector: '.vs__dropdown-menu'})
cypress/pageobjects/rancher.po.ts
Outdated
cy.get(this.rke2Creation_page_networkNameCombo).click().then(($list) => { | ||
cy.get(this.rke2Creation_page_networkNameOption).should('contain', rke2_cluster_attributes.network_name).click(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cy.get(this.rke2Creation_page_networkNameCombo).click().then(($list) => { | |
cy.get(this.rke2Creation_page_networkNameOption).should('contain', rke2_cluster_attributes.network_name).click(); | |
const networkSelect = new LabeledSelectPo('.labeled-select', `:contains("Network Name")`); | |
networkSelect.select({option: rke2_cluster_attributes.network_name, selector: '.vs__dropdown-menu'}); |
Ditto, will be failed when there have other network options.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the instruction, change the menu drop down list selection using LabeledSelectPo in /utils/components/labeled-select.po
const networkSelect = new LabeledSelectPo('.labeled-select', `:contains("Network Name")`);
networkSelect.select({option: rke2_cluster_attributes.network_name, selector: '.vs__dropdown-menu'});
And also update the namespace and image drop down selection using the same manner
cypress/pageobjects/settings.po.ts
Outdated
openVlan(value: string) { | ||
const radio = new RadioButtonPo('.radio-group .radio-container', `:contains("Enabled")`); | ||
radio.input('Enabled'); | ||
new LabeledSelectPo('section .labeled-select.hoverable', `:contains("Default Network Interface ")`).select({option: value}) | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It was removed in v1.1.0.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove this function openVlan
in settings.po.ts
cypress/pageobjects/settings.po.ts
Outdated
enableVLAN(nic: string) { | ||
this.goTo(); | ||
this.checkIsCurrentPage(); | ||
// this.clickMenu('vlan', 'Edit Setting', 'vlan', HCI.CLUSTER_NETWORK); | ||
// this.openVlan(nic); | ||
// this.update('vlan', HCI.CLUSTER_NETWORK); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we can delete this function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove function enableVLAN
in settings.po.ts
// *********************************************** | ||
// This example commands.js shows you how to | ||
// create various custom commands and overwrite | ||
// existing commands. | ||
// | ||
// For more comprehensive examples of custom | ||
// commands please read more here: | ||
// https://on.cypress.io/custom-commands | ||
// *********************************************** | ||
// | ||
// | ||
// -- This is a parent command -- | ||
// Cypress.Commands.add('login', (email, password) => { ... }) | ||
// | ||
// | ||
// -- This is a child command -- | ||
// Cypress.Commands.add('drag', { prevSubject: 'element'}, (subject, options) => { ... }) | ||
// | ||
// | ||
// -- This is a dual command -- | ||
// Cypress.Commands.add('dismiss', { prevSubject: 'optional'}, (subject, options) => { ... }) | ||
// | ||
// | ||
// -- This will overwrite an existing command -- | ||
// Cypress.Commands.overwrite('visit', (originalFn, url, options) => { ... }) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this file should not be needed, it already exists here. https://github.com/harvester/tests/blob/main/cypress/support/commands.js
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After rebase, the command.js have been updated the latest content
// *********************************************************** | ||
// This example support/e2e.js is processed and | ||
// loaded automatically before your test files. | ||
// | ||
// This is a great place to put global configuration and | ||
// behavior that modifies Cypress. | ||
// | ||
// You can change the location of this file or turn off | ||
// automatically serving support files with the | ||
// 'supportFile' configuration option. | ||
// | ||
// You can read more here: | ||
// https://on.cypress.io/configuration | ||
// *********************************************************** | ||
|
||
// Import commands.js using ES2015 syntax: | ||
import './commands' | ||
|
||
// Alternatively you can use CommonJS syntax: | ||
// require('./commands') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this file should not be needed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After rebase, the e2e.js
file have already been removed
{ | ||
"name": "Using fixtures to represent data", | ||
"email": "[email protected]", | ||
"body": "Fixtures are a great way to mock data for responses to routes" | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
already exists.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After rebase, the example.json
remain unchanged.
const { defineConfig } = require("cypress"); | ||
|
||
module.exports = defineConfig({ | ||
e2e: { | ||
setupNodeEvents(on, config) { | ||
// implement node event listeners here | ||
}, | ||
}, | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After rebase, the cypress.config.js
remain unchanged.
cypress/constants/constants.ts
Outdated
public dashboardUrl = '/c/local/harvester/harvesterhci.io.dashboard'; | ||
public settingsUrl = '/c/local/harvester/harvesterhci.io.setting'; | ||
public uiSourceUrl = '/c/local/harvester/harvesterhci.io.setting/ui-source?mode=edit'; | ||
public hostsPage = '/c/local/harvester/harvesterhci.io.host'; | ||
public supportPage = '/c/local/harvester/support'; | ||
public vmPage = '/c/local/harvester/kubevirt.io.virtualmachine'; | ||
public settingBaseUrl = '/c/local/harvester/harvesterhci.io.setting'; | ||
public volumePage = '/c/local/harvester/harvesterhci.io.volume'; | ||
public imagePage = '/c/local/harvester/harvesterhci.io.virtualmachineimage'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This address should not be changed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After rebase, the constants.ts
can align with the harvester/test main
cypress/pageobjects/rancher.po.ts
Outdated
import LabeledInputPo from '@/utils/components/labeled-input.po'; | ||
import LabeledSelectPo from '@/utils/components/labeled-select.po'; | ||
import RadioButtonPo from '@/utils/components/radio-button.po' | ||
import { List } from 'cypress/types/lodash'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not used,
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove the unused import from rancher.po.ts
cypress/pageobjects/rancher.po.ts
Outdated
|
||
const constants = new Constants(); | ||
const settings = new SettingsPagePo(); | ||
var registrationURL; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ditto
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove the unused var registrationURL;
from rancher.po.ts
const IMAGE_NAME = 'focal-server-cloudimg-amd64.img'; | ||
const IMAGE_URL = 'https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Variable should be used.
e.g:
const IMAGE_NAME = imageEnv.name;
const IMAGE_URL = imageEnv.url;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update using environment variable in rancher_integration.spec.ts
const IMAGE_NAME = imageEnv.name;
const IMAGE_URL = imageEnv.url;
cypress/pageobjects/rancher.po.ts
Outdated
private home_page_mainMenu = '.menu'; | ||
private home_page_virtualManagement = ':nth-child(7) > .option > div'; | ||
|
||
private virtual_page_importButton = '.btn'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update the virtual_page_importButton to specific locator
virtual_page_importButton = '.btn.role-primary';
|
||
it('Harvester import Rancher', () => { | ||
cy.login(); | ||
rancher.registerRancher(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add checking current url after save the content
cy.url().should('contain', constants.settingsUrl);
|
||
rancher.rancherLogin(); | ||
|
||
rancher.createCloudCredential(rData.cloud_credential, rData.harvester_cluster_name); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The error may caused by the previous import Harvester failed or there is a harv-local cloud credential exists
ef87c84
to
aed24c9
Compare
Including the following test cases: 1. Prepare Harvester Image (On Harvester) 2. Prepare Harvester VLAN network (On Harvester) 4. Rancher import Harvester 5. Harvester import Rancher (On Harvester) 6. Check Harvester Cluster Status (On Rancher) 7. Create Cloud Credential 8. Provisiong RKE2 Cluster 9. Check RKE2 Cluster Status 10. Provisiong K3s Cluster 11. Check K3s Cluster Status 12. Verify RKE2 CSI driver 13. Verify RKE2 Cloud Provider 14. Verify RKE2 DHDP Load Balancer 15. Create Harvester VIP Pool 16. Create RKE2 Pool Load Balancer 17. Delete Active RKE2 Cluster 18. Delete Active K3s Cluster 19. Delete Cloud Credential 20. Delete imported Harveter cluster singed-off by [email protected]
By updating Cypress to 10.3 Signed-off-by: davidtclin <[email protected]>
2. Fix chronium render crash 3. Fix find element error of select guest cluster Signed-off-by: davidtclin <[email protected]>
Signed-off-by: davidtclin <[email protected]>
Signed-off-by: davidtclin <[email protected]>
Signed-off-by: davidtclin <[email protected]>
Harvester v1.1.1-rc Signed-off-by: davidtclin <[email protected]>
Signed-off-by: davidtclin <[email protected]>
1. Rebase and use the new Harvester function for prepare image and create vlan 2. Remove the unused import and function 3. Handle the popup confirm window of cluster registration in Harvester 4. Fix some stability 5. Use `LabeledSelectPo` to precisely handle dropdown menu selection
aed24c9
to
603da70
Compare
@WuJun2016 , @n313893254
And tested on Rancher Please help to check if I miss any, thank you. |
select kubernetes version 2. Update rke2 and k3s version attributes in rancher fixture file
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Modify the points we just discussed.
@khushboo-rancher @TachunLin This PR will be needed for the UI team to continue the cypress multi-tenancy testing development, please help to prioritize it after the v1.2.0 release, thanks. cc @WuJun2016 |
Close this PR since some of the test cases already been created in #907 |
Purpose
Develop Cypress test for Harvester Rancher integration related test cases.
Plan to fulfill the requirement to automated basic regression and KDM release test.
Development Environment
The development based on the following SUT version and UI
Prepare Test Environment
We can prepare the Rancher integration test environment in the following manner to provision Harvester with Rancher,
false
true
false
About Rancher First Time Login
Please be noted
In every test, while login Rancher, it will automatically check whether it's the first time login
The test function
rancher.rancherLogin()
support bootstrap login in the following scenarios.sshpass
on your local test runner environmentsshpass
on your local test runner environmentIf your environment did not fulfill above requirement, we would suggest you finishing the Rancher bootstrap on your own before running the rancher integration test
Cypress setup
npm install
npm ci
cypress.env.json.example
tocypress.env.json
file, update and add the following contentCheck test data
You can check and modify all related rancher integration test data in
../cypress/fixtures/rancher.json
The test data here is suitable for
Rancher v2.6.7
along with the current KDM release versionPlease be noted, on new KDM release, you may need to update these value accordingly.
And also need to check the image and ssh user name to fulfill what you used in the
Prepare Harvester Image
andPrepare Harvester VLAN network
test case, here I use focal-server and default vlan1I use ubuntu 20.04 cloud focal by settings the image name and url in
cypress.env.json
Execute Test case
Execute
npx cypress open
ornpx cypress run
for the testTest case list
Recommended group of tests
Validation before commit
Rancher import Harvester
Harvester import Rancher (On Harvester)
Check Harvester Cluster Status (On Rancher)
Create Cloud Credential
Provisiong RKE2 Cluster
Check RKE2 Cluster Status
Verify RKE2 CSI driver
Verify RKE2 Cloud Provider
Verify RKE2 DHDP Load Balancer
Create Harvester VIP Pool
Create RKE2 Pool Load Balancer
Delete Active RKE2 Cluster
Delete Cloud Credential
Delete imported Harvester cluster