chore: set initial host list provider according to the dialect (aws#567)

CHANGELOG.md

@@ -3,6 +3,14 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/#semantic-versioning-200).
 
+## [2.2.x] - ???
+### :crab: Changed
+- Dynamically sets the default host list provider based on the dialect used. User applications no longer need to manually set the AuroraHostListProvider when connecting to Aurora Postgres or Aurora MySQL databases.
+- Deprecated AuroraHostListConnectionPlugin. 
+  - As an enhancement, the wrapper is now able to automatically set the Aurora host list provider for connections to Aurora MySQL and Aurora PostgreSQL databases.
+    Aurora Host List Connection Plugin is deprecated. If you were using the `AuroraHostListConnectionPlugin`, you can simply remove the plugin from the `wrapperPlugins` parameter.
+    However, if you choose to, you can ensure the provider is used by specifying a topology-aware dialect, for more information, see [Database Dialects](docs/using-the-jdbc-driver/DatabaseDialects.md).
+
 ## [2.2.3] - 2023-07-28
 ### :magic_wand: Added
 - Developer plugin to help test various scenarios including events like network outages and database cluster failover. This plugin is NOT intended to be used in production environments and is only for testing ([PR #531](https://github.com/awslabs/aws-advanced-jdbc-wrapper/pull/531)).

docs/development-guide/LoadablePlugins.md

@@ -39,7 +39,7 @@ See the following classes for examples:
 - [ExecutionTimeConnectionPlugin](/wrapper/src/main/java/software/amazon/jdbc/plugin/ExecutionTimeConnectionPlugin.java)
     - The `ExecutionTimeConnectionPlugin` only overrides the `execute` method because it is only concerned with elapsed time during execution, it does not establish new connections or set up any host list provider.
 
-A `ConnectionPluginFactory` implementation is also required for the new custom plugin. This factory class is used to register and initialize custom plugins. See [AuroraHostListConnectionPluginFactory](/wrapper/src/main/java/software/amazon/jdbc/plugin/AuroraHostListConnectionPluginFactory.java) for a simple implementation example.
+A `ConnectionPluginFactory` implementation is also required for the new custom plugin. This factory class is used to register and initialize custom plugins. See [ExecutionTimeConnectionPluginFactory](/wrapper/src/main/java/software/amazon/jdbc/plugin/ExecutionTimeConnectionPluginFactory.java) for a simple implementation example.
 
 ### Subscribed Methods
 

docs/development-guide/Pipelines.md

@@ -67,7 +67,9 @@ it does not perform any additional work.
 
 For cases where keeping updated information on existing and available replicas is necessary,
 such as during the failover procedure, it is important to have a host list provider that can re-fetch information once in a while,
-like the [Aurora host list provider](/wrapper/src/main/java/software/amazon/jdbc/plugin/AuroraHostListConnectionPlugin.java).
+like the Aurora host list provider.
+The Aurora host list provider should be automatically used by the wrapper when the application is connecting to Aurora databases.
+However, you can ensure that the provider is used by specifying a topology-aware dialect, for more information, see [Database Dialects](../using-the-jdbc-driver/DatabaseDialects.md).
 
 ## Connection Changed Notification Pipeline
 
@@ -87,7 +89,7 @@ will be called whenever changes in the current node list are detected.
 
 Plugins should subscribe to this pipeline and the getHostSpecByStrategy pipeline if they implement a host selection strategy via the `getHostSpecByStrategy` method. In this case, plugins should override the `acceptsStrategy` and `getHostSpecByStrategy` methods to implement any desired logic. The `acceptsStrategy` method should return true for each selection strategy that the plugin supports.
 
-## getHostSpecByStrategy pipeline
+## Get HostSpec by Strategy pipeline
 
 Plugins should subscribe to this pipeline and the acceptsStrategy pipeline if they implement a host selection strategy. In this case, plugins should override both the `acceptsStrategy` method and the `getHostSpecByStrategy` method. The `acceptsStrategy` method should return true for each strategy that can be processed by the plugin in `getHostSpecByStrategy`. The `getHostSpecByStrategy` method should implement the desired logic for selecting a host using any plugin-accepted strategies. Host selection via a "random" strategy is supported by default.
 

docs/using-the-jdbc-driver/UsingTheJdbcDriver.md

@@ -108,7 +108,6 @@ The AWS JDBC Driver has several built-in plugins that are available to use. Plea
 |------------------------------------------------------------------------------------------------|---------------------------|------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | [Failover Connection Plugin](./using-plugins/UsingTheFailoverPlugin.md)                        | `failover`                | Aurora                 | Enables the failover functionality supported by Amazon Aurora clusters. Prevents opening a wrong connection to an old writer node dues to stale DNS after failover event. This plugin is enabled by default.                                                                                                                                                                                                                                                                                                           |
 | [Host Monitoring Connection Plugin](./using-plugins/UsingTheHostMonitoringPlugin.md)           | `efm`                     | Aurora                 | Enables enhanced host connection failure monitoring, allowing faster failure detection rates. This plugin is enabled by default.                                                                                                                                                                                                                                                                                                                                                                                       |
-| Aurora Host List Connection Plugin                                                             | `auroraHostList`          | Aurora                 | Retrieves Amazon Aurora clusters information. <br><br>**:warning:Note:** this plugin does not need to be explicitly loaded if the failover connection plugin is loaded.                                                                                                                                                                                                                                                                                                                                                |
 | Data Cache Connection Plugin                                                                   | `dataCache`               | Any database           | Caches results from SQL queries matching the regular expression specified in the  `dataCacheTriggerCondition` configuration parameter.                                                                                                                                                                                                                                                                                                                                                                                 |
 | Execution Time Connection Plugin                                                               | `executionTime`           | Any database           | Logs the time taken to execute any JDBC method.                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
 | Log Query Connection Plugin                                                                    | `logQuery`                | Any database           | Tracks and logs the SQL statements to be executed. Sometimes SQL statements are not passed directly to the JDBC method as a parameter, such as [executeBatch()](https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#executeBatch--). Users can set `enhancedLogQueryEnabled` to `true`, allowing the JDBC Wrapper to obtain SQL statements via Java Reflection. <br><br> :warning:**Note:** Enabling Java Reflection may cause a performance degradation.                                                |
@@ -120,6 +119,10 @@ The AWS JDBC Driver has several built-in plugins that are available to use. Plea
 | [Read Write Splitting Plugin](./using-plugins/UsingTheReadWriteSplittingPlugin.md)             | `readWriteSplitting`      | Aurora                 | Enables read write splitting functionality where users can switch between database reader and writer instances.                                                                                                                                                                                                                                                                                                                                                                                                        |
 | [Developer Plugin](./using-plugins/UsingTheDeveloperPlugin.md)                                 | `dev`                     | Any database           | Helps developers test various everyday scenarios including rare events like network outages and database cluster failover. The plugin allows injecting and raising an expected exception, then verifying how applications handle it.                                                                                                                                                                                                                                                                                   |
 
+:exclamation: **NOTE**: As an enhancement, the wrapper is now able to automatically set the Aurora host list provider for connections to Aurora MySQL and Aurora PostgreSQL databases.
+Aurora Host List Connection Plugin is deprecated. If you were using the Aurora Host List Connection Plugin, you can simply remove the plugin from the `wrapperPlugins` parameter.
+However, if you choose to, you can ensure the provider is used by specifying a topology-aware dialect, for more information, see [Database Dialects](../using-the-jdbc-driver/DatabaseDialects.md).
+
 :exclamation:**NOTE**: To see information logged by plugins such as `DataCacheConnectionPlugin` and `LogQueryConnectionPlugin`,
 > see the [Logging](#logging) section.
 

wrapper/src/main/java/software/amazon/jdbc/PluginServiceImpl.java

@@ -38,7 +38,7 @@
 import software.amazon.jdbc.dialect.Dialect;
 import software.amazon.jdbc.dialect.DialectManager;
 import software.amazon.jdbc.dialect.DialectProvider;
-import software.amazon.jdbc.dialect.TopologyAwareDatabaseCluster;
+import software.amazon.jdbc.dialect.HostListProviderSupplier;
 import software.amazon.jdbc.exceptions.ExceptionManager;
 import software.amazon.jdbc.hostlistprovider.StaticHostListProvider;
 import software.amazon.jdbc.util.CacheMap;
@@ -372,7 +372,7 @@ public void forceRefreshHostList(final Connection connection) throws SQLExceptio
   }
 
   void setNodeList(@Nullable final List<HostSpec> oldHosts,
-                   @Nullable final List<HostSpec> newHosts) {
+      @Nullable final List<HostSpec> newHosts) {
 
     final Map<String, HostSpec> oldHostMap = oldHosts == null
         ? new HashMap<>()
@@ -487,19 +487,22 @@ public Dialect getDialect() {
   }
 
   public void updateDialect(final @NonNull Connection connection) throws SQLException {
+    final Dialect originalDialect = this.dialect;
     this.dialect = this.dialectProvider.getDialect(
         this.originalUrl,
         this.initialConnectionHostSpec,
         connection);
+    if (originalDialect == this.dialect) {
+      return;
+    }
+
+    final HostListProviderSupplier supplier = this.dialect.getHostListProvider();
+    this.setHostListProvider(supplier.getProvider(props, this.originalUrl, this));
   }
 
   @Override
   public HostSpec identifyConnection(Connection connection) throws SQLException {
-    if (!(this.getDialect() instanceof TopologyAwareDatabaseCluster)) {
-      return null;
-    }
-
-    return this.hostListProvider.identifyConnection(connection);
+    return this.getHostListProvider().identifyConnection(connection);
   }
 
   @Override
@@ -509,7 +512,7 @@ public void fillAliases(Connection connection, HostSpec hostSpec) throws SQLExce
     }
 
     if (!hostSpec.getAliases().isEmpty()) {
-      LOGGER.finest(() -> Messages.get("PluginServiceImpl.nonEmptyAliases", new Object[] {hostSpec.getAliases()}));
+      LOGGER.finest(() -> Messages.get("PluginServiceImpl.nonEmptyAliases", new Object[]{hostSpec.getAliases()}));
       return;
     }
 
@@ -530,7 +533,7 @@ public void fillAliases(Connection connection, HostSpec hostSpec) throws SQLExce
     // Add the instance endpoint if the current connection is associated with a topology aware database cluster.
     final HostSpec host = this.identifyConnection(connection);
     if (host != null) {
-      hostSpec.addAlias(host.asAliases().toArray(new String[] {}));
+      hostSpec.addAlias(host.asAliases().toArray(new String[]{}));
     }
   }
 }

wrapper/src/main/java/software/amazon/jdbc/dialect/AuroraMysqlDialect.java

@@ -21,27 +21,19 @@
 import java.sql.SQLException;
 import java.sql.Statement;
 import java.util.List;
+import software.amazon.jdbc.hostlistprovider.AuroraHostListProvider;
 
-public class AuroraMysqlDialect extends MysqlDialect implements TopologyAwareDatabaseCluster {
+public class AuroraMysqlDialect extends MysqlDialect {
 
-  @Override
-  public String getTopologyQuery() {
-    return "SELECT SERVER_ID, CASE WHEN SESSION_ID = 'MASTER_SESSION_ID' THEN TRUE ELSE FALSE END, "
-        + "CPU, REPLICA_LAG_IN_MILLISECONDS, LAST_UPDATE_TIMESTAMP "
-        + "FROM information_schema.replica_host_status "
-        // filter out nodes that haven't been updated in the last 5 minutes
-        + "WHERE time_to_sec(timediff(now(), LAST_UPDATE_TIMESTAMP)) <= 300 OR SESSION_ID = 'MASTER_SESSION_ID' ";
-  }
-
-  @Override
-  public String getNodeIdQuery() {
-    return "SELECT @@aurora_server_id";
-  }
+  private static final String TOPOLOGY_QUERY =
+      "SELECT SERVER_ID, CASE WHEN SESSION_ID = 'MASTER_SESSION_ID' THEN TRUE ELSE FALSE END, "
+          + "CPU, REPLICA_LAG_IN_MILLISECONDS, LAST_UPDATE_TIMESTAMP "
+          + "FROM information_schema.replica_host_status "
+          // filter out nodes that haven't been updated in the last 5 minutes
+          + "WHERE time_to_sec(timediff(now(), LAST_UPDATE_TIMESTAMP)) <= 300 OR SESSION_ID = 'MASTER_SESSION_ID' ";
 
-  @Override
-  public String getIsReaderQuery() {
-    return "SELECT @@innodb_read_only";
-  }
+  private static final String NODE_ID_QUERY = "SELECT @@aurora_server_id";
+  private static final String IS_READER_QUERY = "SELECT @@innodb_read_only";
 
   @Override
   public boolean isDialect(final Connection connection) {
@@ -61,4 +53,15 @@ public boolean isDialect(final Connection connection) {
   public List</* dialect code */ String> getDialectUpdateCandidates() {
     return null;
   }
+
+  @Override
+  public HostListProviderSupplier getHostListProvider() {
+    return (properties, initialUrl, hostListProviderService) -> new AuroraHostListProvider(
+        properties,
+        initialUrl,
+        hostListProviderService,
+        TOPOLOGY_QUERY,
+        NODE_ID_QUERY,
+        IS_READER_QUERY);
+  }
 }

wrapper/src/main/java/software/amazon/jdbc/dialect/AuroraPgDialect.java

@@ -21,13 +21,13 @@
 import java.sql.SQLException;
 import java.sql.Statement;
 import java.util.logging.Logger;
+import software.amazon.jdbc.hostlistprovider.AuroraHostListProvider;
 
 /**
  * Suitable for the following AWS PG configurations.
  * - Regional Cluster
  */
-public class AuroraPgDialect extends PgDialect implements TopologyAwareDatabaseCluster {
-
+public class AuroraPgDialect extends PgDialect {
   private static final Logger LOGGER = Logger.getLogger(AuroraPgDialect.class.getName());
 
   private static final String extensionsSql =
@@ -37,24 +37,15 @@ public class AuroraPgDialect extends PgDialect implements TopologyAwareDatabaseC
 
   private static final String topologySql = "SELECT 1 FROM aurora_replica_status() LIMIT 1";
 
-  @Override
-  public String getTopologyQuery() {
-    return "SELECT SERVER_ID, CASE WHEN SESSION_ID = 'MASTER_SESSION_ID' THEN TRUE ELSE FALSE END, "
-        + "CPU, COALESCE(REPLICA_LAG_IN_MSEC, 0), LAST_UPDATE_TIMESTAMP "
-        + "FROM aurora_replica_status() "
-        // filter out nodes that haven't been updated in the last 5 minutes
-        + "WHERE EXTRACT(EPOCH FROM(NOW() - LAST_UPDATE_TIMESTAMP)) <= 300 OR SESSION_ID = 'MASTER_SESSION_ID' ";
-  }
+  private static final String TOPOLOGY_QUERY =
+      "SELECT SERVER_ID, CASE WHEN SESSION_ID = 'MASTER_SESSION_ID' THEN TRUE ELSE FALSE END, "
+          + "CPU, COALESCE(REPLICA_LAG_IN_MSEC, 0), LAST_UPDATE_TIMESTAMP "
+          + "FROM aurora_replica_status() "
+          // filter out nodes that haven't been updated in the last 5 minutes
+          + "WHERE EXTRACT(EPOCH FROM(NOW() - LAST_UPDATE_TIMESTAMP)) <= 300 OR SESSION_ID = 'MASTER_SESSION_ID' ";
 
-  @Override
-  public String getNodeIdQuery() {
-    return "SELECT aurora_db_instance_identifier()";
-  }
-
-  @Override
-  public String getIsReaderQuery() {
-    return "SELECT pg_is_in_recovery()";
-  }
+  private static final String NODE_ID_QUERY = "SELECT aurora_db_instance_identifier()";
+  private static final String IS_READER_QUERY = "SELECT pg_is_in_recovery()";
 
   @Override
   public boolean isDialect(final Connection connection) {
@@ -66,7 +57,7 @@ public boolean isDialect(final Connection connection) {
     boolean hasTopology = false;
     try {
       try (final Statement stmt = connection.createStatement();
-          final ResultSet rs = stmt.executeQuery(extensionsSql)) {
+           final ResultSet rs = stmt.executeQuery(extensionsSql)) {
         if (rs.next()) {
           final boolean auroraUtils = rs.getBoolean("aurora_stat_utils");
           LOGGER.finest(() -> String.format("auroraUtils: %b", auroraUtils));
@@ -77,7 +68,7 @@ public boolean isDialect(final Connection connection) {
       }
 
       try (final Statement stmt = connection.createStatement();
-          final ResultSet rs = stmt.executeQuery(topologySql)) {
+           final ResultSet rs = stmt.executeQuery(topologySql)) {
         if (rs.next()) {
           LOGGER.finest(() -> "hasTopology: true");
           hasTopology = true;
@@ -91,4 +82,15 @@ public boolean isDialect(final Connection connection) {
     }
     return false;
   }
+
+  @Override
+  public HostListProviderSupplier getHostListProvider() {
+    return (properties, initialUrl, hostListProviderService) -> new AuroraHostListProvider(
+        properties,
+        initialUrl,
+        hostListProviderService,
+        TOPOLOGY_QUERY,
+        NODE_ID_QUERY,
+        IS_READER_QUERY);
+  }
 }

wrapper/src/main/java/software/amazon/jdbc/dialect/Dialect.java

@@ -32,4 +32,6 @@ public interface Dialect {
   boolean isDialect(Connection connection);
 
   List</* dialect code */ String> getDialectUpdateCandidates();
+
+  HostListProviderSupplier getHostListProvider();
 }