Release 16.4.0

[16.4.0] - 2021-08-24

Changed

• Various code changes to allow compilation and build on Java 11.

Circus Train 16.3.3 release

We are pleased to announce the release of Circus Train 16.3.3:

https://github.com/HotelsDotCom/circus-train/releases

with the binaries available in Maven Central here:

https://search.maven.org/#search%7Cga%7C1%7Ccircus-train

Changes:

• Issue where rename table operation would be incorrect if tables are in different databases.
• Added check in S3 delete operation so that it doesn't try to delete empty lists of keys.

We encourage everyone to upgrade to this new version and report any issues to us here:

https://github.com/HotelsDotCom/circus-train/issues

Release 16.3.2

[16.3.2] - 2020-10-27

Fixed

• Issue where external AVRO schemas generated lots of copy jobs. See #203.

Circus Train 16.3.1 release

We are pleased to announce the release of Circus Train 16.3.1:

https://github.com/HotelsDotCom/circus-train/releases

with the binaries available in Maven Central here:

https://search.maven.org/#search%7Cga%7C1%7Ccircus-train

The main change is the addition of the fields sourceTable and sourcePartitions to the CopierContext class.

We encourage everyone to upgrade to this new version and report any issues to us here:

https://github.com/HotelsDotCom/circus-train/issues

Circus Train 16.3.0 release

We are pleased to announce the release of Circus Train 16.3.0:

https://github.com/HotelsDotCom/circus-train/releases

with the binaries available in Maven Central here:

https://search.maven.org/#search%7Cga%7C1%7Ccircus-train

The main change is the addition of a method newInstance(CopierContext) (and the deprecation of existing newInstance() methods) to com.hotels.bdp.circustrain.api.copier.CopierFactory. This provides Copiers with more configuration information in a future proof manner (see #195).

We encourage everyone to upgrade to this new version and report any issues to us here:

https://github.com/HotelsDotCom/circus-train/issues

Circus Train 16.2.0 Release

We are pleased to announce the release of Circus Train 16.2.0: https://github.com/HotelsDotCom/circus-train/releases
with the binaries available in Maven Central here: https://search.maven.org/#search%7Cga%7C1%7Ccircus-train.

Notable new features in this release include:

• The version of Hive has been changed from 2.3.2 to 2.3.7, which allows Circus Train to be used on JDK >= 9.
• A new replication mode FULL_OVERWRITE which will overwrite a previously replicated table and delete its data. This is useful when incompatible schema changes are made, as you no longer need to manually clean up existing replicas before replicating.

We encourage everyone to upgrade to this new version and report any issue to us here: https://github.com/HotelsDotCom/circus-train/issues

Circus Train 16.0.0 release

We are pleased to announce the release of Circus Train 16.0.0:

https://github.com/HotelsDotCom/circus-train/releases

with the binaries available in Maven Central here:

https://search.maven.org/#search%7Cga%7C1%7Ccircus-train

The main backwards incompatible change is that we've moved the minimum supported Java version from 7 to 8. We have also increased the version numbers for the AWS SDK and Apache's HttpComponents HttpClient in use. 

We encourage everyone to upgrade to this new version and report any issues to us here:

https://github.com/HotelsDotCom/circus-train/issues

Circus Train 15.1.1 release

When replicating tables with large numbers of partitions, Replica.updateMetadata now calls add/alter partition in batches of 1000. See #166.

Circus Train 15.1.0 release

We are pleased to announce the release of Circus Train 15.1.0: https://github.com/HotelsDotCom/circus-train/releases

with the binaries available in Maven Central here: https://search.maven.org/#search%7Cga%7C1%7Ccircus-train

Circus Train 15.1.0 Is a new minor release with contains some more improvements related to the feature to replicate external AVRO schemas as well performance/bug fixes to the feature that generates partitions filters.

For more information on the changes, please see https://github.com/HotelsDotCom/circus-train/blob/master/CHANGELOG.md.
We encourage everyone to upgrade to this new version and report any issues to us here: https://github.com/HotelsDotCom/circus-train/issues

Circus Train 15.0.0 Release

[15.0.0] - 2019-11-12

We are pleased to announce the release of Circus Train 15.0.0:

https://github.com/HotelsDotCom/circus-train/releases

with the binaries available in Maven Central here:

https://search.maven.org/#search%7Cga%7C1%7Ccircus-train

Notable new features in this release include:

• Default avro-serde-options must now be included within transform-options. This is a backwards incompatible change.
• New copier-options.assume-role for the S3S3Copier. This allows one to configure the ARN of an IAM role to assume when writing S3 data to the target replica. This is useful when the target is in a different AWS account than CircusTrain is running in.

Bugs fixed in this release include:

• Table properties can now be added to default transformations

We encourage everyone to upgrade to this new version and report any issues to us here:

https://github.com/HotelsDotCom/circus-train/issues