Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[e2e] add initial integration test cases for network #681

Merged
merged 18 commits into from
Nov 24, 2023
Merged

[e2e] add initial integration test cases for network #681

merged 18 commits into from
Nov 24, 2023

Conversation

TachunLin
Copy link
Contributor

@TachunLin TachunLin commented Jan 6, 2023 •
edited
Loading

Add initial network backend test including:

  • 1. Test management network connectivity of VM

    • test_mgmt_network_connection

  • 2. Test vlan network connectivity of VM

    • test_vlan_network_connection

  • 3. Test VLAN network after reboot

    • test_reboot_vlan_connection

  • 4. Edit vm network, from VLAN to management

    • test_vlan_to_mgmt_connection

  • 5. Change management network to external VLAN

    • test_mgmt_to_vlan_connection

  • 6. Delete single network from multiple networks vm

    • test_delete_vlan_from_multiple

Prerequisite of environment

  1. At least two network interface, one for management and another for external vlan network
  2. The external vlan network have DHCP server configured to assign IP address to VM

Configuration setup

Edit config.yaml and provide the following attribute (take bare machine for example)

  1. vlan-id to create vlan network
  2. vlan-nic to create cluster network and config
  3. vlan-cidr to determine which IP subent range for vlan network test cases
# VLAN ID, will invoke the tests depended on external networking if not set to -1.
vlan-id: 1216

# Physical NIC for VLAN. Default is "harvester-mgmt"
vlan-nic: 'ens1f1'

# VLAN CIDR, indicate the vlan network subnet range"
vlan-cidr: '10.84.99.0/24'

Additional context

To test it on our local ipxe-example cluster, we can the following config

vlan-id: 1
vlan-nic: 'ens5'
vlan-cidr: '192.168.122.0/24'

@TachunLin TachunLin force-pushed the backend_network branch 2 times, most recently from 9b32b95 to 2bb27e4 Compare January 13, 2023 17:59
@TachunLin
Copy link
Contributor Author

  • Add test case of:

    • test_reboot_vlan_connection
    • test_mgmt_to_vlan_connection
    • test_vlan_to_mgmt_connection

  • Add delete cluster network

  • Add vlan-cidr in config.yaml

@TachunLin
Copy link
Contributor Author

Add test case of:

  • test_delete_vlan_from_multiple
    • Delete single network from multiple networks vm

@TachunLin TachunLin marked this pull request as ready for review February 2, 2023 14:18
@TachunLin TachunLin mentioned this pull request Feb 2, 2023
Closed
5 tasks
@TachunLin
Copy link
Contributor Author

Verified can pass all test suite on dolphin bare machine triggered from local terminal.

@TachunLin TachunLin force-pushed the backend_network branch 7 times, most recently from 52c99e1 to fd15b2c Compare April 13, 2023 10:51
@TachunLin TachunLin force-pushed the backend_network branch 10 times, most recently from da65f0d to d616a0f Compare April 26, 2023 13:58
@TachunLin TachunLin force-pushed the backend_network branch 5 times, most recently from a33498d to 527e565 Compare April 29, 2023 14:32
@TachunLin
Copy link
Contributor Author

Thanks for the review.

Verified the latest update can pass the TestBackendNetwork test suite on Jenkins runtest job 387
image

@bk201
Copy link
Member

bk201 commented Nov 23, 2023

@lanfon72 @khushboo-rancher I think David already addressed most comments and requests. Should we merge this PR to increase our test coverage first? We don't need the PR to be perfect.

If we see issues in daily tests, @TachunLin will continue improving it.

@lanfon72 lanfon72 self-requested a review November 23, 2023 06:30
@lanfon72
Copy link
Member

Fix conflict then we can continue

@TachunLin
Rebase to meet latest changes on master (23/11/23)
35ce4b3
@TachunLin
Update 01 according to review request:
70ff9c5 
1. Remove the vm.vm_instance function in apiclient/managers.py and use
   vms.vm_instance(unique_name) in every occurrence
2. Remove 'harvester_e2e_tests.fixtures.network' and 'harvester_e2e_tests.fixtures.vm'
3. Check the hard coded on functio parameter
4. Use the configure option to retrieve the password for all test cases
5. Use the image_info fixtures in fixtures/images.py to parse the image_url for create image from url
6. Remove all the wait_timeout = request.config.getoption('--wait-timeout') in every test case and set wait_timeout fixture instead.
7. Remove all related test case dependency on `mgmt_network_connection`
8. Update test skeleton test steps of test_vlan_network_connection
9. Replace the dependency of `mgmt_network_connection` to  `vlan_network_connection`
10. Update the test case test_reboot_vlan_connection with the corresponding manual test case reference:
11. Update all occurrence of  assertion message to `exceed the given timed out`
@TachunLin
Remove delete_vlan_network from models.py and use easilier manner to
06280ab 
implement deletion of vlan network interface from spec.network
@TachunLin
1. Remove the steps to retrieve IP addess in test_vlan_network
7b5c9ba 
2. Remove the fixutre vlan_subnet_prefix_store to store the vlan subnet
3. Remove the logic to check external vlan IP exists in subnet
4. Add checkpoint to validate unable to ssh to management network VM
   from external host on test case `test_mgmt_network_connection` and
`test_vlan_to_mgmt`
5. Add test steps into the related test skeleton section
@TachunLin
Update review comment of:
47b2166 
1. Move the add external vlan network steps prior to create VM to better align with the manual test steps
2. Updated all the ping, sshed command assert message to indicate on which type of NIC and drom where to run the command.
@TachunLin
Change the timing to snooze API in test_mgmt_to_vlan_connection test
8f3e06a 
case
@TachunLin
Add return code status check for vms.update, vms.get api call
8b80eae
@TachunLin
Add response content in delete multiple vlan ping assertion to check the
7cc2a8a 
Jenkins job execution failure
@TachunLin
Update according to review request 23/07/13
6aca88d
@TachunLin
Update according to review request (07/15)
9ef135c 
1. Use image_opensuse fixture in all test plan
2. Remove image_info fixture
@TachunLin
Update according to review request 23/07/18
d52de5f 
On test case test_mgmt_network_connection, test_vlan_network_connection
and test_reboot_vlan_connection

1. Check the number of interfaces
2. Use interfaces_data[0]['ipAddress'] to retrieve the only ip address
@TachunLin
Update according to review request 07/19
975640d 
1. Add network name and determine whether the interface name is also nic-1 after we started the VM to expose IP address.
2. Use image_opensuse.name to replace all image_name
@TachunLin
Remove unused test case dependency and uncomment the required ones in…
091ad2a 
… reboot_vlan_connection
@TachunLin
1. Set the added network name to nic-1 and validate it to confirm it's
3816136 
from the external vlan network for test case
`test_mgmt_to_vlan_connection`

2. Update the test skeleton steps in test case
   `test_vlan_network_connection` to align with manual test plan

3. Check only one NIC exists in the test case `
   test_mgmt_to_vlan_connection`
@TachunLin
1. Replaced the duplicate lines of code with the function `check_vm_r…
1e62812 
…unning` in the `test_delete_vlan_from_multiple` and `test_vlan_to_mgmt_connection` test cases

2. Removed the unnecessary line of code of `check_vm_running` and `check_vm_ip_exists` in the `test_mgmt_to_vlan_connection` test case

3. Change the judgement of specific ip address `10.52.x` to the name of the interface `default`

4. Update the subprocess command assertion in the relative test cases:
`test_mgmt_network_connection`, `test_vlan_to_mgmt_connection` ,`test_delete_vlan_from_multiple`
@TachunLin
1. Add cloud-init network data content in variable
c2e326d 
2. Set spec.network_data in test case `test_delete_vlan_from_multiple`

3. Remove using the function `check_vm_ip_exists` to determine two NICs
   case in `test_delete_vlan_from_multiple`

4. Add mechanism to check both the two IP address cases after startup
@TachunLin
Fix execution error on test case test_vlan_to_mgmt_connection to get
8873d8f 
vlan ip prior to the actual mgmt ip
@TachunLin
Add checking point to determine whether the interface ipAddress contains
cbf4425 
digit. To prevent the unexpected case when the interface can't retrieve
ip address from external vlan which cause test case
`test_vlan_to_mgmt_connection` failed.
@bk201 bk201 merged commit dcc6f30 into harvester:main Nov 24, 2023
2 of 3 checks passed
@TachunLin TachunLin deleted the backend_network branch June 14, 2024 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bk201 bk201 bk201 approved these changes

@lanfon72 lanfon72 lanfon72 approved these changes

@khushboo-rancher khushboo-rancher Awaiting requested review from khushboo-rancher

@irishgordo irishgordo Awaiting requested review from irishgordo

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@TachunLin @lanfon72 @bk201 @khushboo-rancher