Add testcases for GET /dsmetadata API #1199

shreyabiradar07 · 2024-05-27T12:13:23Z

Description

This PR adds below testcases for GET /dsmetadata API

List dsmetadata specifying a valid datasource
List dsmetadata for a datasource with parameters by specifying the following parameters:
- /dsmetadata?datasource=<datasource_name>&verbose=false
- /dsmetadata?datasource=<datasource_name>&verbose=true
- /dsmetadata?datasource=<datasource_name>&cluster_name=<cluster_name>&verbose=false
- /dsmetadata?datasource=<datasource_name>&cluster_name=<cluster_name>&verbose=true
- /dsmetadata?datasource=<datasource_name>&cluster_name=<cluster_name>&namespace=<namespace_name>&verbose=false
- /dsmetadata?datasource=<datasource_name>&cluster_name=<cluster_name>&namespace=<namespace_name>&verbose=true
List dsmetadata with invalid parameter values for datasource, cluster_name and namespace
- Non-existing datasource
- Non-existing cluster_name
- Non-existing namespace
List dsmetadata without specifying any parameters
List dsmetadata after creating a datasource but without importing metadata
List dsmetadata with datasource and namespace but without cluster_name
List the dsmetadata after deleting imported metadata
List the dsmetadata for a datasource that has been deleted

Fixes # 1129

Type of change

How has this been tested?

New Test X
Functional testsuite - Tested using local monitoring testsuite
Functional test output on Jenkins - 89 with above testcases

Test Configuration

Kubernetes clusters tested on: minikube

Checklist 🎯

Additional information

NA

chandrams · 2024-05-28T10:59:06Z

tests/scripts/helpers/kruize.py

+    print("Response status code = ", response.status_code)
+    print("\n************************************************************")
+    print(response.text)


Include a logging parameter to this function and enable this when it is true, this will help us enable/disable based on the test.

chandrams · 2024-05-30T05:30:36Z

tests/scripts/helpers/utils.py

@@ -50,6 +50,9 @@
 PERFORMANCE_RECOMMENDATIONS_AVAILABLE = "Performance Recommendations Available"
 CONTAINER_AND_EXPERIMENT_NAME = " for container : %s for experiment: %s.]"
 LIST_DATASOURCES_ERROR_MSG = "Given datasource name - \" %s \" either does not exist or is not valid"
+LIST_METADATA_DATASOURCE_NAME_ERROR_MSG = "Metadata for a given datasource name - \" %s \" either does not exist or is not valid"
+LIST_METADATA_ERROR_MSG = ("Metadata for a given datasource - \" %s \", cluster name - \" %s \", namespace - \"%s \" "


Have different error messages for each of the scenarios

chandrams · 2024-05-30T11:23:32Z

tests/scripts/local_monitoring_tests/rest_apis/test_list_metadata.py

+from helpers.list_metadata_json_cluster_name_without_verbose_schema import *
+from jinja2 import Environment, FileSystemLoader
+
+@pytest.mark.sanity


@shreyabiradar07 - Can you share the test results or link to the run

Jenkins test - https://ci.app-svc-perf.corp.redhat.com/job/ExternalTeams/job/Autotune/job/kruize_functional_tests/89/

Test results on minikube -
list_metadata_tests_local_monitoring.zip

chandrams · 2024-05-30T11:25:32Z

tests/scripts/local_monitoring_tests/rest_apis/test_list_metadata.py

+
+    list_metadata_json = response.json()
+    assert response.status_code == ERROR_STATUS_CODE
+    cluster_name = "null"


We don't need cluster name & namespace here as the list metadata is only with datasource

chandrams · 2024-05-30T11:26:07Z

tests/scripts/local_monitoring_tests/rest_apis/test_list_metadata.py

+@pytest.mark.parametrize("test_name, expected_status_code, datasource",
+                         [
+                             ("blank_datasource", 400, ""),
+                             ("null_datasource", 400, "null"),


null shouldn't be a string here, can you cross check with create experiment negative tests

test_type = {"blank": "", "null": "null", "invalid": "xyz"}
used in generate_test_data for generating negative scenarios in create experiment tests

Tried to strip the double quotes but still null is treated as a string - this might be because datasource here is an input query parameter
Also tried passing None but the parameter itself is getting omitted. Will further look for alternatives for null in Python

chandrams · 2024-05-30T11:27:35Z

tests/scripts/local_monitoring_tests/rest_apis/test_list_metadata.py

+    datasource = json_data['datasource_name']
+    # Currently only default cluster is supported by Kruize
+    cluster_name = "default"
+    namespace = "monitoring"


Namespace monitoring would only work on minikube. Has this been tested on openshift?

Tested the updated code on both minikube & openshift
PR1199_local_monitoring_tests_minikube.zip
PR1199_local_monitoring_tests_openshift.zip

In mandatory parameters missing test, there is a change required in the error message

Mandatory parameters missing [dataSourceName] - this should be datasource_name

delete metadata returns 405 instead of 200, can you please check this

Deleting the metadata... URL = http://192.168.49.2:31830/dsmetadata <Response [405]> Response status code = 405 delete metadata = 405

In the test, rest_apis/test_import_metadata.py::test_repeated_metadata_import, introduce some changes by creating new namespaces and deleting one of the namespace and validate if the second import output reflects these changes.

In mandatory parameters missing test, there is a change required in the error message

Mandatory parameters missing [dataSourceName] - this should be datasource_name

dataSourceName variable is referred from the DSMetadataAPIObject field during validation

delete metadata - 405 and current test failures - test_list_metadata_after_deleting_metadata & test_list_metadata_without_import_action will be resolved once PR 1178 is merged

In the test, rest_apis/test_import_metadata.py::test_repeated_metadata_import, introduce some changes by creating new namespaces and deleting one of the namespace and validate if the second import output reflects these changes.

Are these changes made?

Updated test rest_apis/test_import_metadata.py::test_repeated_metadata_import by dynamically creating and deleting namespaces to showcase the delete metadata API functionality

test log output test_repeated_metadata_import.txt

Updated test logs for rest_apis/test_import_metadata.py::test_repeated_metadata_import
updated_test_repeated_metadata_import.txt

chandrams · 2024-05-30T11:31:49Z

tests/scripts/local_monitoring_tests/rest_apis/test_list_metadata.py

+@pytest.mark.negative
+def test_list_metadata_without_import_action(cluster_type):
+    """
+    Test Description: This test validates GET /dsmetadata without importing metadata


Add expected behavior as well to the description

chandrams · 2024-06-03T11:31:47Z

tests/scripts/local_monitoring_tests/rest_apis/test_import_metadata.py

+
+    delete_namespace("repeated-metadata-import")
+    delete_namespace("local-monitoring-test")


EOF missing, fix wherever applicable

Updated missing EOF

chandrams

LGTM

chandrams

LGTM

Add testcases for list metadata API

3c2dc54

shreyabiradar07 self-assigned this May 27, 2024

shreyabiradar07 added test kruize-local Tag for mentioning all the PR's and issues raised which covers the kruize local monitoring usecase labels May 27, 2024

chandrams reviewed May 28, 2024

View reviewed changes

shreyabiradar07 added 3 commits May 28, 2024 18:35

Add json schemas for verbose parameter

8673217

Add parametrized tests with verbose

a921fcf

Update tests with delete metadata case and add logging

9b533df

shreyabiradar07 marked this pull request as ready for review May 28, 2024 13:53

Update test documentation

9d603a2

chandrams reviewed May 30, 2024

View reviewed changes

shreyabiradar07 added 3 commits May 31, 2024 13:19

Update response error messages and parameters

c3268d6

Update repeated metadata import testcase

2a4989d

Update rm requirements

7677066

chandrams reviewed Jun 3, 2024

View reviewed changes

shreyabiradar07 added 3 commits June 3, 2024 17:12

Add missing EOF

158461b

fix eof

c562b1c

format missing eof

3be5146

chandrams approved these changes Jun 3, 2024

View reviewed changes

Remove manual delete between repeated metadata imports

f785bed

chandrams approved these changes Jun 5, 2024

View reviewed changes

shreyabiradar07 added this to the Kruize 0.0.23_rm Release milestone Jun 18, 2024

Merge upstream 'mvp_demo' and fix conflicts

da34086

dinogun merged commit cfadfe0 into kruize:mvp_demo Jun 18, 2024
2 of 3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add testcases for GET /dsmetadata API #1199

Add testcases for GET /dsmetadata API #1199

shreyabiradar07 commented May 27, 2024 •

edited

Loading

chandrams May 28, 2024

shreyabiradar07 May 28, 2024

chandrams May 30, 2024

chandrams May 30, 2024

shreyabiradar07 May 31, 2024

chandrams May 30, 2024

chandrams May 30, 2024

shreyabiradar07 May 31, 2024

shreyabiradar07 May 31, 2024

chandrams May 30, 2024

shreyabiradar07 May 31, 2024

chandrams May 31, 2024

chandrams May 31, 2024

chandrams May 31, 2024

shreyabiradar07 May 31, 2024

shreyabiradar07 May 31, 2024

chandrams Jun 3, 2024

shreyabiradar07 Jun 3, 2024

shreyabiradar07 Jun 5, 2024

chandrams May 30, 2024

chandrams Jun 3, 2024

shreyabiradar07 Jun 3, 2024

chandrams left a comment

chandrams left a comment


		delete_namespace("repeated-metadata-import")
		delete_namespace("local-monitoring-test")

Add testcases for GET /dsmetadata API #1199

Add testcases for GET /dsmetadata API #1199

Conversation

shreyabiradar07 commented May 27, 2024 • edited Loading

Description

Type of change

How has this been tested?

Checklist 🎯

Additional information

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chandrams left a comment

Choose a reason for hiding this comment

chandrams left a comment

Choose a reason for hiding this comment

shreyabiradar07 commented May 27, 2024 •

edited

Loading