Skip to content

Commit

Permalink
Document dynamic destination key should be unique over the pipeline (#…
Browse files Browse the repository at this point in the history
…32338)

* Document dynamic destination key should be unique over the pipeline

* address comments
  • Loading branch information
Abacn authored Aug 28, 2024
1 parent 8cc80ff commit 815a049
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -3995,7 +3995,12 @@ ValueProvider<TableReference> getTableWithDefaultProject(BigQueryOptions bqOptio
}
}

/** Clear the cached map of created tables. Used for testing. */
/**
* Clear the cached map of created tables. Used for testing only.
*
* <p>Should not be used in any PTransform as cache is a static member shared by different
* BigQueryIO.write.
*/
@VisibleForTesting
static void clearStaticCaches() throws ExecutionException, InterruptedException {
CreateTables.clearCreatedTables();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,10 @@ void setSideInputAccessorFromProcessContext(DoFn<?, ?>.ProcessContext context) {
/**
* Returns an object that represents at a high level which table is being written to. May not
* return null.
*
* <p>The method must return a unique object for different destination tables involved over all
* BigQueryIO write transforms in the same pipeline. See
* https://github.com/apache/beam/issues/32335 for details.
*/
public abstract DestinationT getDestination(@Nullable ValueInSingleWindow<T> element);

Expand Down

0 comments on commit 815a049

Please sign in to comment.