Release 0.11.0 (2023-07-11)

### Features

* [Breaking change] Use UserDefinedFunctionsConfig instead of KVs for loading UDFs.
* [Sharding] Add hpke for s2s communication
* [Sharding] Allow for partial data lookups
* [Sharding] Making downstream requests in parallel
* Add bazel build flag --announce_rc
* Add bool parameter to allow routing V1 requests through V2.
* Add buf format pre-commit hook
* Add build time directive for reentrant parser.
* Add functions to retrieve instance information.
* Add internal run query client and server.
* Add JS hook for set query.
* Add lookup client and server for communication with shards
* Add MessageQueue for the request simulation system
* Add query grammar and interface for set queries.
* Add rate limiter for the request simulation system
* Add second map to store key value set and add set value update interfaces
* Add shard metadata for supporting sharded files
* Add simple microbenchmarks for key value cache
* Add UDF support for format data command.
* Add unit tests for query lexer.
* Adding cluster mappings manager
* Adding padding
* Apply custom lockings on the cache
* Connect InternalRunQuery to the parser
* Extend and simplify collect-logs to capture test outputs
* Extend use of scp deps via data-plane-shared repo
* Implement shard manager
* Move sharding function to public so it's available for file sharding
* Register a logging hook with the UDF.
* Register run query hook with udf framework.
* Sharding - realtime updates
* Sharding read flow fixes
* Simplify work done in set operations. Set operations can be passed by
* Snapshot files support UDF configs.
* Support reading and writing set queries to data files.
* Support reading and writing set values for csv files
* Support reading/writing DataRecords. Requires new DELTA format.
* Support writing sharded files
* Update data_loading.fb to support UDF code updates.
* Update pre-commit hook versions
* Update shard manager mappings continuously
* Upgrade build-system to release-0.28.0
* Upgrade build-system to v0.30.1
* Upgrade scp to 0.72.0
* Use Unix domain socket for internal lookup server.
* Utilize AWS deps via data-plane-shared repo

### Bug Fixes

* Add internal lookup client deadline.
* Catch error if insufficient args specified
* Fix aggregation logic for set values.
* Fix ASAN potential deadlock errors in key_value_cache_test
* Proper memory management of callback hook wrappers.
* Specify 2 workers for UDF execution.
* Upgrade pre-commit hooks
* Use shared pointer for UDF absl::Notification.

### Build Tools: Fixes

* **build:** Add scope-based sections in release notes

### Documentation

* Add docs for data loading capabilities
* Add explanation that access control is managed by IAM for writes.
* Point readme to a new sharding public explainer

Bug: 290798418
Change-Id: I691da695f5727a8517ed3e9f18a3a2d8c5b9e0bf
GitOrigin-RevId: 5958051464911b6da60c38bc2a83c3451adadf42

.bazelrc

@@ -1,3 +1,4 @@
+build --announce_rc
 build --verbose_failures
 build --client_env=CC=clang
 build --cxxopt=-std=c++17
@@ -61,3 +62,7 @@ build:ubsan --copt -O1
 build:ubsan --copt -fno-omit-frame-pointer
 build:ubsan --linkopt -fsanitize=undefined
 build:ubsan --linkopt -lubsan
+
+# --config local_instance: builds the service to run with the instance=local flag
+build:local_instance --//:instance=local
+build:local_instance --@google_privacysandbox_servers_common//:instance=local

.pre-commit-config.yaml

@@ -51,8 +51,13 @@ repos:
   - id: shellcheck
     exclude: '^(google_internal|builders/images)/.*$'
 
+- repo: https://github.com/bufbuild/buf
+  rev: v1.23.1
+  hooks:
+  - id: buf-format
+
 - repo: https://github.com/pre-commit/mirrors-clang-format
-  rev: v16.0.2
+  rev: v16.0.6
   hooks:
   - id: clang-format
     types_or:
@@ -65,7 +70,7 @@ repos:
     name: addlicense
     language: golang
     additional_dependencies:
-    - github.com/google/[email protected].0
+    - github.com/google/[email protected].1
     always_run: false
     pass_filenames: true
     entry: addlicense -v -ignore google_internal/third_party/**
@@ -76,7 +81,7 @@ repos:
     name: addlicense check
     language: golang
     additional_dependencies:
-    - github.com/google/[email protected].0
+    - github.com/google/[email protected].1
     always_run: false
     pass_filenames: true
     entry: addlicense -check -ignore google_internal/third_party/**
@@ -105,7 +110,7 @@ repos:
              )$
 
 - repo: https://github.com/DavidAnson/markdownlint-cli2
-  rev: v0.7.1
+  rev: v0.8.1
   hooks:
   - id: markdownlint-cli2
     name: lint markdown
@@ -117,7 +122,7 @@ repos:
     description: Format bazel WORKSPACE, BUILD and .bzl files with a standard convention.
     language: golang
     additional_dependencies:
-      - github.com/bazelbuild/buildtools/buildifier@5.1.0
+      - github.com/bazelbuild/buildtools/buildifier@6.1.1
     always_run: true
     pass_filenames: true
     types_or:

.versionrc.json

@@ -15,6 +15,36 @@
   ],
   "tagPrefix": "release-",
   "types": [
+    {
+      "section": "API: Features",
+      "type": "feat",
+      "scope": "api"
+    },
+    {
+      "section": "API: Fixes",
+      "type": "fix",
+      "scope": "api"
+    },
+    {
+      "section": "Terraform",
+      "type": "feat",
+      "scope": "terraform"
+    },
+    {
+      "section": "Terraform",
+      "type": "fix",
+      "scope": "terraform"
+    },
+    {
+      "section": "Build Tools: Features",
+      "type": "feat",
+      "scope": "build"
+    },
+    {
+      "section": "Build Tools: Fixes",
+      "type": "fix",
+      "scope": "build"
+    },
     {
       "section": "Features",
       "type": "feat"
@@ -27,10 +57,6 @@
       "section": "Documentation",
       "type": "docs"
     },
-    {
-      "section": "Terraform",
-      "type": "terraform"
-    },
     {
       "hidden": true,
       "type": "internal"

BUILD

@@ -113,3 +113,7 @@ EOF""",
     local = True,
     message = "copy bazel build and test logs",
 )
+
+exports_files([
+    "buf.yaml",
+])