Release 0.17.0 (2024-07-08)

### Features * Add a set wrapper around bitset for storing uint32 values * Add a thread safe wrapper around hash map * Add b&a e2e test env * Add data loading support for uint32 sets * Add health check to AWS mesh. * Add hook for running set query using uint32 sets as input * Add interestGroupNames to V1 API * Add latency metrics for cache uint32 sets functions * Add latency without custom code execution metric * Add option to use existing network on AWS. * Add padding to responses * Add request log context to request context * Add runsetqueryint udf hook * Add set operation functions for bitsets * Add support for int32_t sets to key value cache * Add support for reading and writing int sets to csv files * Add udf hook for running int sets set query (local lookup) * Allow pas request to pass consented debug config and log context * Implement sharded RunSetQueryInt rpc for lookup client * Implement uint32 sets sharded lookup support * Load consented debug token from server parameter * Pass LogContext and ConsentedDebugConfig to internal lookup server in sharded case * Plumb the safe path log context in the cache update execution path * Set verbosity level for PS_VLOG * Simplify thread safe hash map and use a single map for node storage * Support uint32 sets for query parsing and evaluation * Support uint32 sets in InternalLookup rpc * Switch absl log for PS_LOG and PS_VLOG for unsafe code path * Switch absl log to PS_LOG for safe code path * Switch absl vlog to PS_VLOG for safe code path * Update AWS coordinators public prod endpoint from GG to G3P ### Bug Fixes * Add missing include/library deps * Augment UDF loading info message * Correct copts build config. * Correct verbosity flag for gcp validator. * Effectively lock the key in the set map cleanup * Fix detached head of continuous e2e branch. * Properly initialize runSetQueryInt hook * Remove ignore interestGroupNames from envoy * Remove test filter to allow all unit tests run in the build * Simplify request context and pass it as shared pointer to the hooks * Upgrade common repo version * Use kms_binaries tar target from common repo * Use structured initializer for clarity ### Dependencies * **deps:** Upgrade build-system to 0.62.0 * **deps:** Upgrade data-plane-shared-libraries to 52239f15 2024-05-21 * **deps:** Upgrade pre-commit hooks ### GCP: Features * **GCP:** Switch to internal lb for the otlp collector * **GCP:** Switch to internal lb for the otlp collector with bug fixes ### Documentation * Add debugging playbook * Correct commands for sample_word2vec getting_started example * KV onboarding guide * Update to the ads retrieval explainer * Update word2vec example * Use aws_platform bazel config * Use local_{platform,instance} bazel configs Bug: N/A Change-Id: Iea31d2381ec9991dea2696be61d96ce861ed9ad9 GitOrigin-RevId: ad05723149ee0346c8322e1481fd1832e25ab752
privacysandbox · Jul 19, 2024 · 5d586e0 · 5d586e0
1 parent 9a60180
commit 5d586e0
Show file tree

Hide file tree

Showing 422 changed files with 12,351 additions and 3,367 deletions.
diff --git a/.bazelrc b/.bazelrc
@@ -5,7 +5,7 @@ build --output_filter='^//((?!(third_party):).)*$'`
 build --color=yes
 build --@io_bazel_rules_docker//transitions:enable=false
 build --workspace_status_command="bash tools/get_workspace_status"
-build --copt=-Werror=thread-safety-analysis
+build --copt=-Werror=thread-safety
 build --config=clang
 build --config=noexcept
 # Disable some ROMA error checking
@@ -118,7 +118,7 @@ build:gcp_platform --@google_privacysandbox_servers_common//:platform=gcp
 build:prod_mode --//:mode=prod
 build:prod_mode --@google_privacysandbox_servers_common//:build_flavor=prod
 
-# --config prod_mode: builds the service in prod mode
+# --config nonprod_mode: builds the service in nonprod mode
 build:nonprod_mode --//:mode=nonprod
 build:nonprod_mode --@google_privacysandbox_servers_common//:build_flavor=non_prod
 

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -27,7 +27,7 @@ exclude: (?x)^(
 fail_fast: true
 repos:
 - repo: https://github.com/pre-commit/pre-commit-hooks
-  rev: v4.5.0
+  rev: v4.6.0
   hooks:
   - id: end-of-file-fixer
   - id: fix-byte-order-marker
@@ -60,12 +60,12 @@ repos:
     exclude: ^(google_internal|builders/images)/.*$
 
 - repo: https://github.com/bufbuild/buf
-  rev: v1.29.0
+  rev: v1.31.0
   hooks:
   - id: buf-format
 
 - repo: https://github.com/pre-commit/mirrors-clang-format
-  rev: v17.0.6
+  rev: v18.1.4
   hooks:
   - id: clang-format
     types_or:
@@ -119,7 +119,7 @@ repos:
              )$
 
 - repo: https://github.com/DavidAnson/markdownlint-cli2
-  rev: v0.12.1
+  rev: v0.13.0
   hooks:
   - id: markdownlint-cli2
     name: lint markdown
@@ -154,7 +154,7 @@ repos:
     - --quiet
 
 - repo: https://github.com/psf/black
-  rev: 24.2.0
+  rev: 24.4.0
   hooks:
   - id: black
     name: black python formatter

diff --git a/BUILD.bazel b/BUILD.bazel
@@ -19,8 +19,9 @@ load("@io_bazel_rules_go//go:def.bzl", "nogo")
 package(default_visibility = ["//:__subpackages__"])
 
 # Config settings to determine which platform the system will be built to run on
+# Use --config=VALUE_platform specified in .bazelrc instead of using this flag directly.
 # Example:
-# bazel build components/... --//:platform=aws
+# bazel build components/... --config=aws_platform
 string_flag(
     name = "platform",
     build_setting_default = "aws",
@@ -64,6 +65,10 @@ config_setting(
     ],
 )
 
+# Config settings to determine which instance the system will be built to run on
+# Use --configVALUE_instance specified in .bazelrc instead of using this flag directly.
+# Example:
+# bazel build components/... --config=aws_instance
 string_flag(
     name = "instance",
     build_setting_default = "aws",

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,6 +2,84 @@
 
 All notable changes to this project will be documented in this file. See [commit-and-tag-version](https://github.com/absolute-version/commit-and-tag-version) for commit guidelines.
 
+## 0.17.0 (2024-07-08)
+
+
+### Features
+
+* Add a set wrapper around bitset for storing uint32 values
+* Add a thread safe wrapper around hash map
+* Add b&a e2e test env
+* Add data loading support for uint32 sets
+* Add health check to AWS mesh.
+* Add hook for running set query using uint32 sets as input
+* Add interestGroupNames to V1 API
+* Add latency metrics for cache uint32 sets functions
+* Add latency without custom code execution metric
+* Add option to use existing network on AWS.
+* Add padding to responses
+* Add request log context to request context
+* Add runsetqueryint udf hook
+* Add set operation functions for bitsets
+* Add support for int32_t sets to key value cache
+* Add support for reading and writing int sets to csv files
+* Add udf hook for running int sets set query (local lookup)
+* Allow pas request to pass consented debug config and log context
+* Implement sharded RunSetQueryInt rpc for lookup client
+* Implement uint32 sets sharded lookup support
+* Load consented debug token from server parameter
+* Pass LogContext and ConsentedDebugConfig to internal lookup server in sharded case
+* Plumb the safe path log context in the cache update execution path
+* Set verbosity level for PS_VLOG
+* Simplify thread safe hash map and use a single map for node storage
+* Support uint32 sets for query parsing and evaluation
+* Support uint32 sets in InternalLookup rpc
+* Switch absl log for PS_LOG and PS_VLOG for unsafe code path
+* Switch absl log to PS_LOG for safe code path
+* Switch absl vlog to PS_VLOG for safe code path
+* Update AWS coordinators public prod endpoint from GG to G3P
+
+
+### Bug Fixes
+
+* Add missing include/library deps
+* Augment UDF loading info message
+* Correct copts build config.
+* Correct verbosity flag for gcp validator.
+* Effectively lock the key in the set map cleanup
+* Fix detached head of continuous e2e branch.
+* Properly initialize runSetQueryInt hook
+* Remove ignore interestGroupNames from envoy
+* Remove test filter to allow all unit tests run in the build
+* Simplify request context and pass it as shared pointer to the hooks
+* Upgrade common repo version
+* Use kms_binaries tar target from common repo
+* Use structured initializer for clarity
+
+
+### Dependencies
+
+* **deps:** Upgrade build-system to 0.62.0
+* **deps:** Upgrade data-plane-shared-libraries to 52239f15 2024-05-21
+* **deps:** Upgrade pre-commit hooks
+
+
+### GCP: Features
+
+* **GCP:** Switch to internal lb for the otlp collector
+* **GCP:** Switch to internal lb for the otlp collector with bug fixes
+
+
+### Documentation
+
+* Add debugging playbook
+* Correct commands for sample_word2vec getting_started example
+* KV onboarding guide
+* Update to the ads retrieval explainer
+* Update word2vec example
+* Use aws_platform bazel config
+* Use local_{platform,instance} bazel configs
+
 ## 0.16.0 (2024-04-05)