Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Datasource authentication updates #1289

Merged
merged 10 commits into from
Sep 17, 2024
Merged

Conversation

khansaad
Copy link
Contributor

@khansaad khansaad commented Sep 12, 2024

Description

This PR contains new feature addition of supporting different REST API authentication for Datasource, DB, cloudwatch and other services.

Fixes # (issue)

Type of change

  • Bug fix
  • New feature
  • Docs update
  • Breaking change (What changes might users need to make in their application due to this PR?)
  • Requires DB changes

How has this been tested?

Testing is in progress

Please describe the tests that were run to verify your changes and steps to reproduce. Please specify any test configuration required.

  • New Test X
  • Functional testsuite

Test Configuration

  • Kubernetes clusters tested on: Resource Hub cluster

Checklist 🎯

  • Followed coding guidelines
  • Comments added
  • Dependent changes merged
  • Documentation updated
  • Tests added or updated

Additional information

Tested using this custom image: quay.io/khansaad/autotune_operator:ds-auth

Screenshot from 2024-09-12 21-17-40

Shows the failure when wrong token path is passed :

Screenshot from 2024-09-12 22-43-21

@khansaad khansaad added enhancement New feature or request local_monitoring labels Sep 12, 2024
@khansaad khansaad added this to the Kruize 0.0.25_rm Release milestone Sep 12, 2024
@khansaad khansaad self-assigned this Sep 12, 2024
", credentials=" + credentials +
'}';
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EOF missing

@@ -38,8 +38,9 @@
*/
public class KruizeDeploymentInfo {
private static final Logger LOGGER = LoggerFactory.getLogger(KruizeDeploymentInfo.class);
public static String database_username;
public static String database_password;
public static String database_authentication_type;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this change required ?
this change will affect ROS

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stashed for now. Will be updating later on for common auth feature

@@ -75,10 +75,15 @@ data:
"adminUsername": "admin",
"hostname": "kruize-db-service",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this PR is limited to datasource auth , But not for DB ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted

@@ -92,7 +97,7 @@ data:
"savetodb": "true",
"dbdriver": "jdbc:postgresql://",
"plots": "true",
"local": "false",
"local": "true",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let default value set to false , Until ROS move into thanos

"logLevel": "INFO"
"authentication": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let make this PR limited scope .. Only datasource authentication

", headerName='" + headerName + '\'' +
'}';
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EOF missing

KruizeConstants.SupportedDatasources.PROMETHEUS
)
);
public JSONObject getJsonObjectForQuery(String url, String query, AuthenticationConfig authenticationConfig) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

notice that many functions are using authenticationConfig. This configuration should be set at the experiment level, similar to how experiment.setDatasource works. When retrieving the datasource using experiment.getDatasource, it should also return the associated authenticationConfig.

To achieve this, the authenticationConfig object should be linked to the datasource and accessible through it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated now.
Instead of passing authenticationObject in every function, refactored the methods to take the datasource object instead so that it will help in fetching both the datasource URL as well as authenticationConfig

@@ -73,7 +73,7 @@ public void append(LogEvent event) {
}

public static void configureLoggerForCloudWatchLog() {
if (cloudwatch_logs_access_key_id != null && !cloudwatch_logs_access_key_id.isEmpty() && cloudwatch_logs_secret_access_key != null && !cloudwatch_logs_secret_access_key.isEmpty() && cloudwatch_logs_region != null && !cloudwatch_logs_region.isEmpty()) {
if (cloudwatch_logs_authentication_credentials_access_key_id != null && !cloudwatch_logs_authentication_credentials_access_key_id.isEmpty() && cloudwatch_logs_authentication_credentials_secret_access_key != null && !cloudwatch_logs_authentication_credentials_secret_access_key.isEmpty() && cloudwatch_logs_region != null && !cloudwatch_logs_region.isEmpty()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets limit this change only related to only datasource

String queryResult = (String) ado.getValueForQuery(experimentTrial.getDatasourceInfoHashMap()
.get(containerMetric.getDatasource())
.getUrl().toString(), updatedContainerQuery);
.getUrl().toString(), updatedContainerQuery, noAuthConfig);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

authoconfig should get tagged to datasource obj and
datasource obj should get tagged to Experiment

so data pass using parameter should be just experiment Obj everywhere

…s passing the datasource URL and other review updates

Signed-off-by: Saad Khan <[email protected]>
Signed-off-by: Saad Khan <[email protected]>
@khansaad khansaad marked this pull request as ready for review September 12, 2024 15:55
Copy link
Contributor

@msvinaykumar msvinaykumar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Review Checklist

Common Review

  • Code formatting adheres to the project’s style guide.
  • No unnecessary commented-out code.
  • Appropriate use of comments and documentation.

Logical Flow Review

  • Code logic is clear and easy to follow.
  • All branches and conditions are covered and tested.
  • No redundant or duplicate code.

Demo Update

  • Demonstration examples are updated to reflect changes.
  • README and documentation include updated demo instructions.
  • Any necessary demo files are included and tested.

Naming Conventions

  • Variable and function names follow the project's naming conventions.
  • File names are descriptive and adhere to naming standards.
  • No ambiguous or misleading names.

Workflow

  • CI/CD pipelines are updated and passing.
  • Tests are included and all pass.
  • The workflow for deploying or integrating changes is clear and documented.

Does It Affect ROS

  • Changes are reviewed for compatibility with ROS.
  • ROS-specific configurations are updated if necessary.
  • Any ROS dependencies or integrations are tested and verified.

@chandrams
Copy link
Contributor

@khansaad - Can you resolve the conflicts.

@khansaad
Copy link
Contributor Author

@khansaad - Can you resolve the conflicts.

Done

@@ -134,7 +134,13 @@ data:
"provider": "prometheus",
"serviceName": "prometheus-k8s",
"namespace": "openshift-monitoring",
"url": ""
"url": "",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the other openshift yaml in BYODB

@rbadagandi1 rbadagandi1 linked an issue Sep 17, 2024 that may be closed by this pull request
28 tasks
Copy link
Contributor

@chandrams chandrams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chandrams
Copy link
Contributor

@khansaad - I see a merge conflict, can you please resolve this. Thanks.

@chandrams chandrams merged commit f2b3fbd into kruize:mvp_demo Sep 17, 2024
1 of 3 checks passed
"serviceName": "prometheus-k8s",
"namespace": "openshift-monitoring",
"url": "",
"authentication": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensure that this change, including the check for the authentication object, is applied only when local=true. The Kruize pods should not fail if the authentication object is absent, as sometimes datasource authentication is not required.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The datasource object is applicable only for local monitoring. Therefore, any code changes related to it have been made specifically within the local monitoring context and do not affect the ROS case.


public class AuthenticationStrategyFactory {

public static AuthenticationStrategy createAuthenticationStrategy(AuthenticationConfig authConfig) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also support for no auth stratgey

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request local_monitoring
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Test scenarios for Datasource authentication
3 participants