Skip to content

Commit

Permalink
Add GraalVM Reachability Metadata and corresponding nativeTest for Hi…
Browse files Browse the repository at this point in the history 
…veServer2
  • Loading branch information
@linghengqian
linghengqian committed Jul 27, 2024
1 parent 2c967fa commit b7abc5b
Show file tree
Hide file tree
Showing 18 changed files with 832 additions and 371 deletions.
112 changes: 112 additions & 0 deletions ...ument/content/user-manual/shardingsphere-jdbc/graalvm-native-image/_index.cn.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,118 @@ Caused by: java.io.UnsupportedEncodingException: Codepage Cp1252 is not supporte

ClickHouse 不支持 ShardingSphere 集成级别的本地事务，XA 事务和 Seata AT 模式事务，更多讨论位于 https://github.com/ClickHouse/clickhouse-docs/issues/2300 。

7. 当需要通过 ShardingSphere JDBC 使用 Hive 方言时，受未关闭的 https://github.com/apache/logging-log4j2/issues/1539 影响，
用户应排除 HiveServer2 JDBC Driver 中的 apache/logging-log4j2 和 apache/logging-log4j1 相关的库，
并使用 qos-ch/slf4j 的其他实现来完成可能需要的日志记录。受 https://issues.apache.org/jira/browse/HIVE-28308 影响，
用户不应该使用 `classifier` 为 `standalone` 的 `org.apache.hive:hive-jdbc:4.0.0`，以避免依赖冲突。
若用户使用 qos-ch/logback 作为 qos-ch/slf4j 实现，则可能的配置例子如下，
```xml
<project>
<dependencies>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-jdbc</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-infra-database-hive</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-parser-sql-hive</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-jdbc</artifactId>
<version>4.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-service</artifactId>
<version>4.0.0</version>
<exclusions>
<exclusion>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk15on</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-reload4j</artifactId>
</exclusion>
<exclusion>
<groupId>org.antlr</groupId>
<artifactId>antlr4-runtime</artifactId>
</exclusion>
<exclusion>
<groupId>org.codehaus.janino</groupId>
<artifactId>commons-compiler</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.commons</groupId>
<artifactId>commons-dbcp2</artifactId>
</exclusion>
<exclusion>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
</exclusion>
<exclusion>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
</exclusion>
<exclusion>
<groupId>org.codehaus.janino</groupId>
<artifactId>janino</artifactId>
</exclusion>
<exclusion>
<groupId>com.google.code.findbugs</groupId>
<artifactId>jsr305</artifactId>
</exclusion>
<exclusion>
<groupId>com.fasterxml.woodstox</groupId>
<artifactId>woodstox-core</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client-api</artifactId>
<version>3.3.6</version>
</dependency>
</dependencies>
</project>
```
受 https://github.com/grpc/grpc-java/issues/10601 影响，用户如果在项目中引入了 `org.apache.hive:hive-service`，
则需要在项目的 classpath 的 `META-INF/native-image/io.grpc/grpc-netty-shaded` 文件夹下创建包含如下内容的文件 `native-image.properties`，
```properties
Args=--initialize-at-run-time=\
io.grpc.netty.shaded.io.netty.channel.ChannelHandlerMask,\
io.grpc.netty.shaded.io.netty.channel.nio.AbstractNioChannel,\
io.grpc.netty.shaded.io.netty.channel.socket.nio.SelectorProviderUtil,\
io.grpc.netty.shaded.io.netty.util.concurrent.DefaultPromise,\
io.grpc.netty.shaded.io.netty.util.internal.MacAddressUtil,\
io.grpc.netty.shaded.io.netty.util.internal.SystemPropertyUtil,\
io.grpc.netty.shaded.io.netty.util.NetUtilInitializations
```

## 贡献 GraalVM Reachability Metadata

ShardingSphere 对在 GraalVM Native Image 下的可用性的验证，是通过 GraalVM Native Build Tools 的 Maven Plugin 子项目来完成的。
Expand Down
114 changes: 114 additions & 0 deletions ...ument/content/user-manual/shardingsphere-jdbc/graalvm-native-image/_index.en.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,120 @@ Possible configuration examples are as follows,
ClickHouse does not support local transactions, XA transactions, and Seata AT mode transactions at the ShardingSphere integration level.
More discussion is at https://github.com/ClickHouse/clickhouse-docs/issues/2300 .

7. When using the Hive dialect via ShardingSphere JDBC, users are affected by the unresolved issue at https://github.com/apache/logging-log4j2/issues/1539,
necessitating the exclusion of libraries related to `apache/logging-log4j2` and `apache/logging-log4j1` from the HiveServer2 JDBC Driver.
Instead, they should employ alternative implementations from `qos-ch/slf4j` for any required logging functionalities.
Further, due to the impact of https://issues.apache.org/jira/browse/HIVE-28308,
users must refrain from using the `classifier` `standalone` version of `org.apache.hive:hive-jdbc:4.0.0` to circumvent dependency conflicts.
If opting for `qos-ch/logback` as the `qos-ch/slf4j` implementation, a possible configuration example could be structured as follows,
```xml
<project>
<dependencies>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-jdbc</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-infra-database-hive</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-parser-sql-hive</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-jdbc</artifactId>
<version>4.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-service</artifactId>
<version>4.0.0</version>
<exclusions>
<exclusion>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk15on</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-reload4j</artifactId>
</exclusion>
<exclusion>
<groupId>org.antlr</groupId>
<artifactId>antlr4-runtime</artifactId>
</exclusion>
<exclusion>
<groupId>org.codehaus.janino</groupId>
<artifactId>commons-compiler</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.commons</groupId>
<artifactId>commons-dbcp2</artifactId>
</exclusion>
<exclusion>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
</exclusion>
<exclusion>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
</exclusion>
<exclusion>
<groupId>org.codehaus.janino</groupId>
<artifactId>janino</artifactId>
</exclusion>
<exclusion>
<groupId>com.google.code.findbugs</groupId>
<artifactId>jsr305</artifactId>
</exclusion>
<exclusion>
<groupId>com.fasterxml.woodstox</groupId>
<artifactId>woodstox-core</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client-api</artifactId>
<version>3.3.6</version>
</dependency>
</dependencies>
</project>
```
Affected by https://github.com/grpc-java/issues/10601, should users incorporate `org.apache.hive:hive-service` into their project,
it is imperative to create a file named `native-image.properties` within the directory `META-INF/native-image/io.grpc/grpc-netty-shaded` of the classpath,
containing the following content,
```properties
Args=--initialize-at-run-time=\
io.grpc.netty.shaded.io.netty.channel.ChannelHandlerMask,\
io.grpc.netty.shaded.io.netty.channel.nio.AbstractNioChannel,\
io.grpc.netty.shaded.io.netty.channel.socket.nio.SelectorProviderUtil,\
io.grpc.netty.shaded.io.netty.util.concurrent.DefaultPromise,\
io.grpc.netty.shaded.io.netty.util.internal.MacAddressUtil,\
io.grpc.netty.shaded.io.netty.util.internal.SystemPropertyUtil,\
io.grpc.netty.shaded.io.netty.util.NetUtilInitializations
```

## Contribute GraalVM Reachability Metadata

The verification of ShardingSphere's availability under GraalVM Native Image is completed through the Maven Plugin subproject
Expand Down
96 changes: 0 additions & 96 deletions ...rg/apache/shardingsphere/infra/database/hive/metadata/data/loader/HiveMetaDataLoader.java
View file

This file was deleted.

6 changes: 5 additions & 1 deletion .../native-image/org.apache.shardingsphere/generated-reachability-metadata/proxy-config.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@
"interfaces":["io.seata.config.Configuration"]
},
{
"condition":{"typeReachable":"org.apache.shardingsphere.mode.repository.standalone.jdbc.JDBCRepository"},
"condition":{"typeReachable":"org.apache.shardingsphere.mode.manager.cluster.ClusterContextManagerBuilder"},
"interfaces":["java.sql.Connection"]
},
{
"condition":{"typeReachable":"org.apache.shardingsphere.mode.manager.standalone.StandaloneContextManagerBuilder"},
"interfaces":["org.apache.hive.service.rpc.thrift.TCLIService$Iface"]
}
]
Loading

0 comments on commit b7abc5b

Please sign in to comment.