Add sample query to show top LLM users grouped by LLM model

README.md

@@ -60,6 +60,8 @@ To learn more about the variety of Google Cloud logs, how to enable and natively
 | 4.11| [Autoscaling usage per day in the past month](./src/4.11/4.11.md)| Audit Logs - Admin Activity| :white_check_mark:| | [T1496](https://attack.mitre.org/techniques/T1496/ "Resource Hijacking") |
 | 4.20| [Resource access by certain user identities in the past month](./src/4.20/4.20.md)| Audit Logs| :white_check_mark:| | [T1106](https://attack.mitre.org/techniques/T1106/ "Native API") |
 | 4.21| [Resource access by certain user identities in the past month (aggregated by day)](./src/4.21/4.21.md)| Audit Logs| :white_check_mark:| | [T1106](https://attack.mitre.org/techniques/T1106/ "Native API") |
+| 4.30| [Which users most frequently used LLM models?](./src/4.30/4.30.md)| Audit Logs - Data Access| :white_check_mark:| :white_check_mark:| [T1496](https://attack.mitre.org/techniques/T1496/ "Resource Hijacking"), [AML.T0051](https://atlas.mitre.org/techniques/AML.T0051 "LLM Prompt Injection"), [AML.T0057](https://atlas.mitre.org/techniques/AML.T0057 "LLM Data Leakage") |
+| 4.31| [Usage of LLM models over time](./src/4.31/4.31.md)| Audit Logs - Data Access| :white_check_mark:| :white_check_mark:| [T1496](https://attack.mitre.org/techniques/T1496/ "Resource Hijacking"), [AML.T0051](https://atlas.mitre.org/techniques/AML.T0051 "LLM Prompt Injection"), [AML.T0057](https://atlas.mitre.org/techniques/AML.T0057 "LLM Data Leakage") |
 | <div id="data-usage">5</div> | :droplet: **Data Usage**
 | 5.01| [Which users most frequently accessed data in the past week?](./src/5.01/5.01.md)| Audit Logs - Data Access| :white_check_mark:| | [T1530](https://attack.mitre.org/techniques/T1530/ "Data from Cloud Storage Object") |
 | 5.02| [Which users accessed most amount of data in the past week?](./src/5.02/5.02.md)| Audit Logs - Data Access| :white_check_mark:| | [T1530](https://attack.mitre.org/techniques/T1530/ "Data from Cloud Storage Object") |

backends/bigquery/sql/4_30_top_llm_users_by_model.sql

@@ -0,0 +1,33 @@
+/*
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+SELECT
+  protopayload_auditlog.authenticationInfo.principalEmail,
+  SUBSTR(protopayload_auditlog.resourceName, (STRPOS(protopayload_auditlog.resourceName, 'publishers/') + 11)) as modelName,
+  COUNT(*) as counter
+FROM
+  `[MY_PROJECT_ID].[MY_DATASET_ID].cloudaudit_googleapis_com_data_access`
+WHERE
+  -- in the past week
+  timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY) AND
+  protopayload_auditlog.serviceName = "aiplatform.googleapis.com" AND
+  SPLIT(protopayload_auditlog.methodName, '.')[SAFE_OFFSET(5)] IN ("Predict", "GenerateContent")
+GROUP BY
+  1, 2
+ORDER BY
+  counter DESC
+LIMIT 1000

backends/bigquery/sql/4_31_llm_usage_over_time.sql

@@ -0,0 +1,33 @@
+/*
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+SELECT
+  TIMESTAMP_TRUNC(timestamp, DAY) AS day,
+  protopayload_auditlog.resourceName,
+  COUNT(*) as counter
+FROM
+  `[MY_PROJECT_ID].[MY_DATASET_ID].cloudaudit_googleapis_com_data_access`
+WHERE
+  -- in the past week
+  timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY) AND
+  protopayload_auditlog.serviceName = "aiplatform.googleapis.com" AND
+  SPLIT(protopayload_auditlog.methodName, '.')[SAFE_OFFSET(5)] IN ("Predict", "GenerateContent")
+GROUP BY
+  1, 2
+ORDER BY
+  1 DESC, 3 DESC
+LIMIT 10000
+

backends/log_analytics/sql/4_30_top_llm_users_by_model.sql

@@ -0,0 +1,41 @@
+/*
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/*
+ * ||||||||||||||||||||||||||||||||||||||
+ *   Log Analytics chart configuration:
+ * ||||||||||||||||||||||||||||||||||||||
+ * - Chart type: Bar chart - Horizontal
+ * - Dimension (y-axis): principal_email
+ * - Measure (x-axis): counter
+ * - Breakdown: model_name 
+ */
+
+SELECT
+  proto_payload.audit_log.authentication_info.principal_email,
+  SUBSTR(proto_payload.audit_log.resource_name, (STRPOS(proto_payload.audit_log.resource_name, 'publishers/') + 11)) as model_name,
+  COUNT(*) as counter
+FROM
+  `[MY_PROJECT_ID].[MY_LOG_BUCKET_REGION].[MY_LOG_BUCKET_NAME]._AllLogs`
+WHERE
+  -- timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY) AND
+  proto_payload.audit_log.service_name = "aiplatform.googleapis.com" AND
+  SPLIT(proto_payload.audit_log.method_name, '.')[SAFE_OFFSET(5)] IN ("Predict", "GenerateContent")
+GROUP BY
+  1, 2
+ORDER BY
+  counter DESC
+LIMIT 1000

backends/log_analytics/sql/4_31_llm_usage_over_time.sql

@@ -0,0 +1,37 @@
+/*
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+ /*
+ * ||||||||||||||||||||||||||||||||||||||
+ *   Log Analytics chart configuration:
+ * ||||||||||||||||||||||||||||||||||||||
+ * - Chart type: Bar chart - Vertical
+ * - Dimension (x-axis): timestamp
+ * - Measure (y-axis): Count rows (default)
+ * - Breakdown: resource_name 
+ */
+
+SELECT
+  timestamp,
+  proto_payload.audit_log.resource_name,
+  proto_payload.audit_log.authentication_info.principal_email
+FROM
+  `[MY_PROJECT_ID].[MY_LOG_BUCKET_REGION].[MY_LOG_BUCKET_NAME]._AllLogs`
+WHERE
+  -- timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY) AND
+  proto_payload.audit_log.service_name = "aiplatform.googleapis.com" AND
+  SPLIT(proto_payload.audit_log.method_name, '.')[SAFE_OFFSET(5)] IN ("Predict", "GenerateContent")
+LIMIT 10000

bin/generate-docs.rb

@@ -62,7 +62,7 @@ def generate_detection_docs!(detection, output_doc_path)
 
     query_paths = detection['query_rel_paths'] || {}
 
-    template = ERB.new File.read("#{CSA_LIB_DIR}/doc_template.md.erb"), nil, "-"
+    template = ERB.new(File.read("#{CSA_LIB_DIR}/doc_template.md.erb"), trim_mode:"-")
     generated_doc = template.result(binding)
 
     print " => #{output_doc_path} => "

index.md

@@ -31,6 +31,8 @@
 | 4.11| [Autoscaling usage per day in the past month](./src/4.11/4.11.md)| Audit Logs - Admin Activity| :white_check_mark:| | [T1496](https://attack.mitre.org/techniques/T1496/ "Resource Hijacking") |
 | 4.20| [Resource access by certain user identities in the past month](./src/4.20/4.20.md)| Audit Logs| :white_check_mark:| | [T1106](https://attack.mitre.org/techniques/T1106/ "Native API") |
 | 4.21| [Resource access by certain user identities in the past month (aggregated by day)](./src/4.21/4.21.md)| Audit Logs| :white_check_mark:| | [T1106](https://attack.mitre.org/techniques/T1106/ "Native API") |
+| 4.30| [Which users most frequently used LLM models?](./src/4.30/4.30.md)| Audit Logs - Data Access| :white_check_mark:| :white_check_mark:| [T1496](https://attack.mitre.org/techniques/T1496/ "Resource Hijacking"), [AML.T0051](https://atlas.mitre.org/techniques/AML.T0051 "LLM Prompt Injection"), [AML.T0057](https://atlas.mitre.org/techniques/AML.T0057 "LLM Data Leakage") |
+| 4.31| [Usage of LLM models over time](./src/4.31/4.31.md)| Audit Logs - Data Access| :white_check_mark:| :white_check_mark:| [T1496](https://attack.mitre.org/techniques/T1496/ "Resource Hijacking"), [AML.T0051](https://atlas.mitre.org/techniques/AML.T0051 "LLM Prompt Injection"), [AML.T0057](https://atlas.mitre.org/techniques/AML.T0057 "LLM Data Leakage") |
 | <div id="data-usage">5</div> | :droplet: **Data Usage**
 | 5.01| [Which users most frequently accessed data in the past week?](./src/5.01/5.01.md)| Audit Logs - Data Access| :white_check_mark:| | [T1530](https://attack.mitre.org/techniques/T1530/ "Data from Cloud Storage Object") |
 | 5.02| [Which users accessed most amount of data in the past week?](./src/5.02/5.02.md)| Audit Logs - Data Access| :white_check_mark:| | [T1530](https://attack.mitre.org/techniques/T1530/ "Data from Cloud Storage Object") |

src/4.30/4.30.md

@@ -0,0 +1,140 @@
+# 4.30 - Which users most frequently used LLM models?
+Top users who used LLMs to generated content (including embeddings). Break down results by specific LLM model used.
+
+
+**Category:** Data Usage
+</br>
+**Use Cases:** Audit, Detect
+</br>
+**Data Sources:** Audit Logs - Data Access
+</br>
+
+
+
+## Queries or Rules
+BigQuery | Chronicle | Log Analytics
+--- | --- | ---
+[SQL](../../backends/bigquery/sql/4_30_top_llm_users_by_model.sql) | [Contribute rule](../../CONTRIBUTING.md) | [SQL](../../backends/log_analytics/sql/4_30_top_llm_users_by_model.sql)
+
+## Event Generation
+No event generation steps provided. [Contribute emulation test](../../CONTRIBUTING.md) to this use case.
+
+## Sample Event
+
+
+### google.cloud.aiplatform.v1.PredictionService.Predict
+```
+{
+  "protoPayload": {
+    "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
+    "authenticationInfo": {
+      "principalEmail": "[email protected]",
+      "principalSubject": "user:[email protected]"
+    },
+    "requestMetadata": {
+      "callerIp": "203.0.113.255",
+      "callerSuppliedUserAgent": "<redacted>",
+      "requestAttributes": {
+        "time": "2024-02-22T14:33:29.284843427Z",
+        "auth": {
+        }
+      },
+      "destinationAttributes": {
+      }
+    },
+    "serviceName": "aiplatform.googleapis.com",
+    "methodName": "google.cloud.aiplatform.v1.PredictionService.Predict",
+    "authorizationInfo": [
+      {
+        "resource": "projects/my-project/locations/us-central1/publishers/google/models/code-bison@002",
+        "permission": "aiplatform.endpoints.predict",
+        "granted": true,
+        "resourceAttributes": {
+        }
+      }
+    ],
+    "resourceName": "projects/my-project/locations/us-central1/publishers/google/models/code-bison@002",
+    "request": {
+      "endpoint": "projects/my-project/locations/us-central1/publishers/google/models/code-bison@002",
+      "@type": "type.googleapis.com/google.cloud.aiplatform.v1.PredictRequest"
+    },
+    "response": {
+      "@type": "type.googleapis.com/google.cloud.aiplatform.v1.PredictResponse"
+    }
+  },
+  "insertId": "1pbm5dbe1oqug",
+  "resource": {
+    "type": "audited_resource",
+    "labels": {
+      "service": "aiplatform.googleapis.com",
+      "project_id": "my-project",
+      "method": "google.cloud.aiplatform.v1.PredictionService.Predict"
+    }
+  },
+  "timestamp": "2024-02-22T14:33:29.284416639Z",
+  "severity": "INFO",
+  "logName": "projects/my-project/logs/cloudaudit.googleapis.com%2Fdata_access",
+  "receiveTimestamp": "2024-02-22T14:33:31.245712026Z"
+}
+```
+### google.cloud.aiplatform.v1beta1.PredictionService.GenerateContent
+```
+{
+  "protoPayload": {
+    "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
+    "authenticationInfo": {
+      "principalEmail": "[email protected]",
+      "principalSubject": "user:[email protected]"
+    },
+    "requestMetadata": {
+      "callerIp": "203.0.113.255",
+      "callerSuppliedUserAgent": "<redacted>",
+      "requestAttributes": {
+        "time": "2024-02-22T04:55:11.117617705Z",
+        "auth": {
+        }
+      },
+      "destinationAttributes": {
+      }
+    },
+    "serviceName": "aiplatform.googleapis.com",
+    "methodName": "google.cloud.aiplatform.v1beta1.PredictionService.GenerateContent",
+    "authorizationInfo": [
+      {
+        "resource": "projects/my-project/locations/us-central1/publishers/google/models/gemini-1.0-pro-001",
+        "permission": "aiplatform.endpoints.predict",
+        "granted": true,
+        "resourceAttributes": {
+        }
+      }
+    ],
+    "resourceName": "projects/my-project/locations/us-central1/publishers/google/models/gemini-1.0-pro-001",
+    "request": {
+      "@type": "type.googleapis.com/google.cloud.aiplatform.v1beta1.GenerateContentRequest",
+      "model": "projects/my-project/locations/us-central1/publishers/google/models/gemini-1.0-pro-001"
+    },
+    "response": {
+      "@type": "type.googleapis.com/google.cloud.aiplatform.v1beta1.GenerateContentResponse"
+    }
+  },
+  "insertId": "a9mqkae614f2",
+  "resource": {
+    "type": "audited_resource",
+    "labels": {
+      "service": "aiplatform.googleapis.com",
+      "method": "google.cloud.aiplatform.v1beta1.PredictionService.GenerateContent",
+      "project_id": "my-project"
+    }
+  },
+  "timestamp": "2024-02-22T04:55:11.116811865Z",
+  "severity": "INFO",
+  "logName": "projects/my-project/logs/cloudaudit.googleapis.com%2Fdata_access",
+  "receiveTimestamp": "2024-02-22T04:55:14.371010794Z"
+}
+```
+
+
+
+### References
+- https://cloud.google.com/vertex-ai/docs/general/audit-logging#audited_operations
+

src/4.30/4.30.yaml

@@ -0,0 +1,35 @@
+---
+id: '4.30'
+name: top_llm_users_by_model
+# Refer to corresponding queries: 4_30_top_llm_users_by_model.{sql|yaral|...}
+display_name: Which users most frequently used LLM models?
+category: Data Usage
+severity: Low
+attack_mapping:
+  - technique: T1496
+    title: Resource Hijacking
+    link: https://attack.mitre.org/techniques/T1496/
+  - technique: AML.T0051
+    title: LLM Prompt Injection
+    link: https://atlas.mitre.org/techniques/AML.T0051
+  - technique: AML.T0057
+    title: LLM Data Leakage
+    link: https://atlas.mitre.org/techniques/AML.T0057
+description: |
+  Top users who used LLMs to generated content (including embeddings). Break down results by specific LLM model used.
+
+sources:
+  - Audit Logs - Data Access
+
+use_cases:
+  - Audit
+  - Detect
+
+test:
+
+samples:
+  - google.cloud.aiplatform.v1.PredictionService.Predict
+  - google.cloud.aiplatform.v1beta1.PredictionService.GenerateContent
+
+references:
+  - https://cloud.google.com/vertex-ai/docs/general/audit-logging#audited_operations