Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include more return values for SearchAnomalyDetectorsTool and SearchMonitorsTool #143

Merged
merged 3 commits into from
Jan 24, 2024

Conversation

ohltyler
Copy link
Member

@ohltyler ohltyler commented Jan 24, 2024

Description

This PR includes a handful of improvements to the alerting and AD tools:

  • includes more return fields to the anomaly detector and alerting monitor results
  • improves UT to cover these added return values
  • tunes descriptions of all 4 alerting & AD tools to list the individual return fields. This is necessary for an LLM to parse through the results better when a question may be asking about such fields (e.g., "When was detector 'A' last updated?", requiring to fetch detector 'A', then parse the 'lastUpdateTime' field).

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Tyler Ohlsen <[email protected]>
Copy link

codecov bot commented Jan 24, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (8074492) 80.51% compared to head (06e11cd) 80.73%.

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #143      +/-   ##
============================================
+ Coverage     80.51%   80.73%   +0.22%     
  Complexity      198      198              
============================================
  Files            13       13              
  Lines          1016     1028      +12     
  Branches        134      134              
============================================
+ Hits            818      830      +12     
  Misses          146      146              
  Partials         52       52              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -33,7 +33,7 @@
public class SearchAlertsTool implements Tool {
public static final String TYPE = "SearchAlertsTool";
private static final String DEFAULT_DESCRIPTION =
"This is a tool that finds alert trigger information. It takes 12 optional argument named sortOrder which defines the order of the results (options are asc or desc, and default is asc), and sortString which defines how to sort the results (default is monitor_name.keyword), and size which defines the size of the request to be returned (default is 20), and startIndex which defines the index to start from (default is 0), and searchString which defines the search string to use for searching a specific alert (default is an empty String), and severityLevel which defines the severity level to filter for (default is ALL), and alertState which defines the alert state to filter for (default is ALL), and monitorId which defines the monitor ID to filter for, and alertIndex which defines the alert index to search from (default is null), and monitorIds which defines the list of monitor IDs to filter for, and workflowIds which defines the list of workflow IDs to filter for(default is null), and alertIds which defines the list of alert IDs to filter for (default is null). The tool returns a list of alerts, and the total number of alerts.";
"This is a tool that finds alert trigger information. It takes 12 optional argument named sortOrder which defines the order of the results (options are asc or desc, and default is asc), and sortString which defines how to sort the results (default is monitor_name.keyword), and size which defines the size of the request to be returned (default is 20), and startIndex which defines the index to start from (default is 0), and searchString which defines the search string to use for searching a specific alert (default is an empty String), and severityLevel which defines the severity level to filter for (default is ALL), and alertState which defines the alert state to filter for (default is ALL), and monitorId which defines the monitor ID to filter for, and alertIndex which defines the alert index to search from (default is null), and monitorIds which defines the list of monitor IDs to filter for, and workflowIds which defines the list of workflow IDs to filter for(default is null), and alertIds which defines the list of alert IDs to filter for (default is null). The tool returns 2 values: a list of alerts (each containining id, version, schema version, monitor ID, workflow ID, workflow name, monitor name, monitor version, monitor user, trigger ID, trigger name, finding IDs, related doc IDs, state, start time, end time, last notifcation time, acknowledged time, error message, error history, severity, action execution results, aggregation result bucket, execution ID, associated alert IDs), and the total number of alerts.";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we see improvement where we defined what the alert object value contained? Do we have any information or metrics that showed doing this helps the tools as I see this is done for AD as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yet, that will be the next step. The main point here is that many questions are going to involve parsing responses to return fields within detectors/monitors/results/etc. (see example in description). For now, I'm starting out by at least exposing them, and tuning later as needed.

@ohltyler ohltyler merged commit 2028247 into opensearch-project:main Jan 24, 2024
13 checks passed
@ohltyler ohltyler deleted the tool-improvements branch January 24, 2024 20:00
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jan 24, 2024
…onitorsTool (#143)

Signed-off-by: Tyler Ohlsen <[email protected]>
(cherry picked from commit 2028247)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
ohltyler added a commit to ohltyler/skills that referenced this pull request Jan 24, 2024
ohltyler added a commit that referenced this pull request Jan 24, 2024
…ool and SearchMonitorsTool (#145)

* Include more return values for SearchAnomalyDetectorsTool and SearchMonitorsTool (#143)

Signed-off-by: Tyler Ohlsen <[email protected]>

* 2.x changes

Signed-off-by: Tyler Ohlsen <[email protected]>

---------

Signed-off-by: Tyler Ohlsen <[email protected]>
yuye-aws pushed a commit to yuye-aws/skills that referenced this pull request Apr 26, 2024
…ool and SearchMonitorsTool (opensearch-project#145)

* Include more return values for SearchAnomalyDetectorsTool and SearchMonitorsTool (opensearch-project#143)

Signed-off-by: Tyler Ohlsen <[email protected]>

* 2.x changes

Signed-off-by: Tyler Ohlsen <[email protected]>

---------

Signed-off-by: Tyler Ohlsen <[email protected]>
Signed-off-by: yuye-aws <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants