Skip to content

Commit

Permalink
Support connecting to Postgres via DistSQL in Proxy Native
Browse files Browse the repository at this point in the history
  • Loading branch information
linghengqian committed Oct 5, 2024
1 parent 980228b commit a4342ca
Show file tree
Hide file tree
Showing 54 changed files with 1,565 additions and 671 deletions.
1 change: 1 addition & 0 deletions RELEASE-NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

1. SQL Parser: Support PostgreSQL, openGauss function table and update from segment parse - [#32994](https://github.com/apache/shardingsphere/pull/32994)
2. DistSQL: Support running DistSQL under Proxy Native in the form of GraalVM Native Image - [#33095](https://github.com/apache/shardingsphere/pull/33095)
2. DistSQL: Support connecting to Postgres via DistSQL in Proxy Native - [#33124](https://github.com/apache/shardingsphere/pull/33124)

### Bug Fix

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -451,6 +451,12 @@ HiveServer2 不支持 ShardingSphere 集成级别的本地事务,XA 事务和
8. 由于 https://github.com/oracle/graal/issues/7979 的影响,
对应 `com.oracle.database.jdbc:ojdbc8` Maven 模块的 Oracle JDBC Driver 无法在 GraalVM Native Image 下使用。

9. 由于 https://github.com/apache/doris/issues/9426 的影响,当通过 Shardinghere JDBC 连接至 Apache Doris FE,
用户需自行提供 `apache/doris` 集成模块相关的 GraalVM Reachability Metadata。

10. 由于 https://github.com/prestodb/presto/issues/23226 的影响,当通过 Shardinghere JDBC 连接至 Presto Server,
用户需自行提供 `com.facebook.presto:presto-jdbc` 和 `prestodb/presto` 集成模块相关的 GraalVM Reachability Metadata。

## 贡献 GraalVM Reachability Metadata

ShardingSphere 对在 GraalVM Native Image 下的可用性的验证,是通过 GraalVM Native Build Tools 的 Maven Plugin 子项目来完成的。
Expand All @@ -463,13 +469,13 @@ ShardingSphere 定义了 `shardingsphere-test-native` 的 Maven Module 用于为

ShardingSphere 定义了 `nativeTestInShardingSphere` 的 Maven Profile 用于为 `shardingsphere-test-native` 模块执行 nativeTest 。

贡献者必须安装 Docker Engine 以执行 `testcontainers-java` 相关的单元测试,以 https://java.testcontainers.org/supported_docker_environment/ 为准。

假设贡献者处于新的 Ubuntu 22.04.4 LTS 实例下,其可通过如下 bash 命令通过 SDKMAN! 管理 JDK 和工具链,
并为 `shardingsphere-test-native` 子模块执行 nativeTest。

你必须安装 Docker Engine 以执行 `testcontainers-java` 相关的单元测试。

```bash
sudo apt install unzip zip curl sed -y
sudo apt install unzip zip -y
curl -s "https://get.sdkman.io" | bash
source "$HOME/.sdkman/bin/sdkman-init.sh"
sdk install java 22.0.2-graalce
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -468,6 +468,12 @@ More discussion is available at https://cwiki.apache.org/confluence/display/Hive
8. Due to https://github.com/oracle/graal/issues/7979 ,
the Oracle JDBC Driver corresponding to the `com.oracle.database.jdbc:ojdbc8` Maven module cannot be used under GraalVM Native Image.

9. Due to https://github.com/apache/doris/issues/9426, when connecting to Apache Doris FE via Shardinghere JDBC,
users need to provide GraalVM Reachability Metadata related to the `apache/doris` integration module.

10. Due to https://github.com/prestodb/presto/issues/23226, when connecting to Presto Server via Shardinghere JDBC,
users need to provide GraalVM Reachability Metadata related to the `com.facebook.presto:presto-jdbc` and `prestodb/presto` integration module.

## Contribute GraalVM Reachability Metadata

The verification of ShardingSphere's availability under GraalVM Native Image is completed through the Maven Plugin subproject
Expand All @@ -481,13 +487,13 @@ This subset of unit tests avoids the use of third-party libraries such as Mockit

ShardingSphere defines the Maven Profile of `nativeTestInShardingSphere` for executing nativeTest for the `shardingsphere-test-native` module.

Contributors must install Docker Engine to execute unit tests related to `testcontainers-java`, as per https://java.testcontainers.org/supported_docker_environment/ .

Assuming that the contributor is under a new Ubuntu 22.04.4 LTS instance, Contributors can manage the JDK and tool chain through
`SDKMAN!` through the following bash command, and execute nativeTest for the `shardingsphere-test-native` submodule.

You must install Docker Engine to execute `testcontainers-java` related unit tests.

```bash
sudo apt install unzip zip curl sed -y
sudo apt install unzip zip -y
curl -s "https://get.sdkman.io" | bash
source "$HOME/.sdkman/bin/sdkman-init.sh"
sdk install java 22.0.2-graalce
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ GraalVM Native Image 的背景信息可参考 https://www.graalvm.org 。

## 注意事项

本节涉及的所有 Docker Image 均不通过 https://downloads.apache.orghttps://repository.apache.org/ 等 ASF 官方渠道进行分发。
Docker Image 仅在 `ghcr.io` 等下游渠道提供以方便使用。
本节涉及的所有 Docker Image 均不通过 https://downloads.apache.orghttps://repository.apache.org 等 ASF 官方渠道进行分发。
Docker Image 仅在 `GitHub Packages``Docker Hub` 等下游渠道提供以方便使用。

Proxy 的 Native Image 产物在 https://github.com/apache/shardingsphere/pkgs/container/shardingsphere-proxy-native 存在每夜构建。
假设存在包含 `global.yaml``conf` 文件夹为 `./custom/conf`,你可通过如下的 `docker-compose.yml` 文件进行测试。
Expand Down Expand Up @@ -53,6 +53,9 @@ ShardingSphere Proxy Native 可执行 DistSQL,这意味着实际上不需要
若处于 Linux(riscv64)等 Graal compiler 不支持的系统环境,
请根据 https://medium.com/graalvm/graalvm-native-image-meets-risc-v-899be38eddd9 的内容启用 LLVM backend 来使用 LLVM compiler。

用户必须对需要运行 GraalVM Native Image 的每个目标操作系统和目标体系结构,来独立构建不同的 GraalVM Native Image。
用户可以考虑通过 Docker Image 来部分绕开此限制。

本节依然受到 ShardingSphere JDBC 一侧的 [GraalVM Native Image](/cn/user-manual/shardingsphere-jdbc/graalvm-native-image) 的已记录内容的限制。

## 前提条件
Expand Down Expand Up @@ -114,17 +117,23 @@ cd ./shardingsphere/
第 3 个参数为要侦听的主机,如果为 `0.0.0.0` 则允许任意数据库客户端均可访问 ShardingSphere Proxy Native
第 4 个参数为 Force Start,如果为 `true` 则保证 ShardingSphere Proxy Native 无论能否连接都能正常启动。

假设已存在文件夹`./custom/conf`,示例为,
已完成构建的 GraalVM Native Image 的二进制文件仅可设置命令行参数。这意味着,

(1) 用户仅可在构建 GraalVM Native Image 的过程中设置 JVM 参数
(2) 用户无法针对已完成构建的 GraalVM Native Image 的二进制文件设置 JVM 参数

假设已存在文件夹`/customAbsolutePath/conf`,示例为,

```bash
cd ./shardingsphere/
cd ./distribution/proxy-native/target/apache-shardingsphere-5.5.1-SNAPSHOT-shardingsphere-proxy-native-bin/
./proxy-native "3307" "./custom/conf" "0.0.0.0" "false"
./proxy-native "3307" "/customAbsolutePath/conf" "0.0.0.0" "false"
```

4. 如果需要构建 Docker Image, 在添加存在 SPI 实现的依赖或第三方依赖后, 在命令行执行如下命令,

```shell
cd ./shardingsphere/
./mvnw -am -pl distribution/proxy-native -T1C -Prelease.native,docker.native -DskipTests clean package
```

Expand All @@ -140,23 +149,31 @@ services:
- "3307:3307"
```

如果你不对 Git Source 做任何更改, 上文提及的命令将使用 `oraclelinux:9-slim` 作为 Base Docker Image。
但如果你希望使用 `busybox:glic`,`gcr.io/distroless/base` 或 `scratch` 等更小体积的 Docker Image 作为 Base Docker
Image,你需要根据 https://www.graalvm.org/jdk23/reference-manual/native-image/guides/build-static-executables/ 的要求,
做为 `pom.xml`的 `native profile` 添加 `-H:+StaticExecutableWithDynamicLibC` 的 `jvmArgs` 等操作。
另请注意,某些第三方依赖将需要在 `Dockerfile` 安装更多系统库,例如 `libdl`。
因此请确保根据你的使用情况调整 `distribution/proxy-native` 下的 `pom.xml` 和 `Dockerfile` 的内容。
如果用户不对 Git Source 做任何更改,上文提及的命令将使用 `oraclelinux:9-slim` 作为 Base Docker Image。
但如果用户希望使用 `scratch`,`alpine:3`,`gcr.io/distroless/base-debian12`,
`gcr.io/distroless/java-base-debian12` 或 `gcr.io/distroless/static-debian12` 等更小体积的 Docker Image 作为 Base Docker Image,
用户可能需要根据 https://www.graalvm.org/jdk23/reference-manual/native-image/guides/build-static-executables/ 的要求,
做为 `pom.xml`的 `Maven Profile` 添加 `--static`,`--libc=musl` 或 `--static-nolibc` 的 `buildArgs` 等操作。

构建静态链接的 GraalVM Native Image 需要更多系统依赖,且目前不支持为 Linux(aarch64)等环境构建静态链接的 GraalVM Native Image。
完全静态链接的 GraalVM Native Image 采用的是 musl libc。
大多数 Linux 系统内置的 musl 已过时,比如 Ubuntu 22.04.5 LTS 使用的 [musl (1.2.2-4) unstable](https://packages.ubuntu.com/jammy/musl)。
用户总是需要从源代码构建和安装新版本的 musl。

另请注意,某些第三方 Maven 依赖将需要在 `Dockerfile` 安装更多系统库,
因此请确保根据使用情况调整 `distribution/proxy-native` 下的 `pom.xml` 和 `Dockerfile` 的内容。

## 可观察性

针对 GraalVM Native Image 形态的 ShardingSphere Proxy,其提供的可观察性的能力与[可观察性](/cn/user-manual/shardingsphere-proxy/observability)并不一致。

用户可以使用 https://www.graalvm.org/jdk23/tools/ 提供的一系列命令行工具或可视化工具观察 GraalVM Native Image 的内部行为,
并根据其要求使用 VSCode 完成调试工作。如果用户正在使用 IntelliJ IDEA 并且希望调试生成的 GraalVM Native Image,
并根据其要求在 Linux 下使用 VSCode 完成 Debug 工作。如果用户正在使用 IntelliJ IDEA 并且希望调试生成的 GraalVM Native Image,
用户可以关注 https://blog.jetbrains.com/idea/2022/06/intellij-idea-2022-2-eap-5/#Experimental_GraalVM_Native_Debugger_for_Java 及其后继。

如果用户使用的不是 Linux,则无法对 GraalVM Native Image 进行 Debug,请关注尚未关闭的 https://github.com/oracle/graal/issues/5648 。

对于使用 `ShardingSphere Agent` 等 Java Agent 的情形, GraalVM 的 `native-image` 组件尚未完全支持在构建 Native Image 时使用 javaagent,
用户需要关注尚未关闭的 https://github.com/oracle/graal/issues/1065
用户需要关注尚未关闭的 https://github.com/oracle/graal/issues/8177

若用户期望在 ShardingSphere Proxy Native 下使用这类 Java Agent,则需要关注 https://github.com/oracle/graal/pull/8077 涉及的变动。
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ For background information about GraalVM Native Image, please refer to https://w

## Notice

All Docker Images mentioned in this section are not distributed through ASF official channels such as https://downloads.apache.org and https://repository.apache.org/ .
Docker Images are only provided on downstream channels such as `ghcr.io` for convenience.
All Docker images mentioned in this section are not distributed through ASF official channels such as https://downloads.apache.org and https://repository.apache.org .
Docker images are only provided in downstream channels such as `GitHub Packages` and `Docker Hub` for easy use.

Native Image products of Proxy exist in nightly builds at https://github.com/apache/shardingsphere/pkgs/container/shardingsphere-proxy-native .
Assuming that there is a `conf` folder containing `global.yaml` as `./custom/conf`, you can test it with the following `docker-compose.yml` file.
Expand Down Expand Up @@ -54,6 +54,9 @@ This section assumes that you are in one of the following system environments:
If you are in a system environment that Graal compiler does not support, such as Linux (riscv64),
please enable LLVM backend according to the content of https://medium.com/graalvm/graalvm-native-image-meets-risc-v-899be38eddd9 to use the LLVM compiler.

Users must build the different GraalVM Native Image for each target operating system and target architecture that they need to run the GraalVM Native Image on.
Users can consider partially circumventing this limitation by using Docker Image.

This section is still limited by the documented content of the [GraalVM Native Image](/us/user-manual/shardingsphere-jdbc/graalvm-native-image) on the ShardingSphere JDBC side.

## Premise
Expand Down Expand Up @@ -116,18 +119,24 @@ cd ./shardingsphere/
The third parameter is the host to listen to. If it is `0.0.0.0`, any database client is allowed to access ShardingSphere Proxy Native.
The fourth parameter is Force Start. If it is `true`, it ensures that ShardingSphere Proxy Native can start normally regardless of whether it can be connected.

Assuming the folder `./custom/conf` already exists, the example is.
Only command line parameters can be set for binaries of built GraalVM Native Images. This means that:

(1) Users can only set JVM parameters during the process of building a GraalVM Native Image
(2) Users cannot set JVM parameters for binaries of built GraalVM Native Images

Assuming the folder `/customAbsolutePath/conf` already exists, the example is.

```bash
cd ./shardingsphere/
cd ./distribution/proxy-native/target/apache-shardingsphere-5.5.1-SNAPSHOT-shardingsphere-proxy-native-bin/
./proxy-native "3307" "./custom/conf" "0.0.0.0" "false"
./proxy-native "3307" "/customAbsolutePath/conf" "0.0.0.0" "false"
```

4. If you need to build a Docker Image, after adding the dependencies that have SPI implementation or third-party dependencies,
execute the following command in the command line:

```shell
cd ./shardingsphere/
./mvnw -am -pl distribution/proxy-native -T1C -Prelease.native,docker.native -DskipTests clean package
```

Expand All @@ -144,25 +153,34 @@ services:
- "3307:3307"
```

If you do not make any changes to the Git Source,
the commands mentioned above will use `oraclelinux:9-slim` as the Base Docker Image.
But if you want to use a smaller Docker Image such as `busybox:glic`,
`gcr.io/distroless/base` or `scratch` as the Base Docker Image,
you need to follow the requirements of https://www.graalvm.org/jdk23/reference-manual/native-image/guides/build-static-executables/ and add `-H:+StaticExecutableWithDynamicLibC` to the `jvmArgs` of the `native profile` in `pom.xml`.
Also note that some third-party dependencies will require more system libraries to be installed in the `Dockerfile`, such as `libdl`.
So make sure to adjust the contents of `pom.xml` and `Dockerfile` under `distribution/proxy-native` according to your usage.
If the user does not make any changes to the Git Source, the command mentioned above will use `oraclelinux:9-slim` as the Base Docker Image.
But if the user wants to use a smaller Docker Image such as `scratch`, `alpine:3`, `gcr.io/distroless/base-debian12`,
`gcr.io/distroless/java-base-debian12` or `gcr.io/distroless/static-debian12` as the Base Docker Image,
the user may need to add `--static`,
`--libc=musl` or `--static-nolibc` to the `Maven Profile` of `pom.xml` as required by https://www.graalvm.org/jdk23/reference-manual/native-image/guides/build-static-executables/ and other operations such as `buildArgs`.

Building a statically linked GraalVM Native Image requires more system dependencies,
and currently does not support building statically linked GraalVM Native Images for environments such as Linux (aarch64).
Fully statically linked GraalVM Native Images use musl libc.
Most Linux systems come with outdated musl, such as Ubuntu 22.04.5 LTS which uses [musl (1.2.2-4) unstable](https://packages.ubuntu.com/jammy/musl).
Users always need to build and install a new version of musl from source.

Also note that some third-party Maven dependencies will require more system libraries to be installed in the `Dockerfile`,
so make sure to adjust the contents of `pom.xml` and `Dockerfile` under `distribution/proxy-native` according to your usage.

## Observability

The observability provided by ShardingSphere Proxy in the form of GraalVM Native Image is not consistent with [observability](/cn/user-manual/shardingsphere-proxy/observability).

Users can use a series of command-line tools or visualization tools provided by https://www.graalvm.org/jdk23/tools/ to observe the internal behavior of GraalVM Native Image,
and use VSCode to complete debugging work according to their requirements.
If users are using IntelliJ IDEA and want to debug the generated GraalVM Native Image,
users can follow https://blog.jetbrains.com/idea/2022/06/intellij-idea-2022-2-eap-5/#Experimental_GraalVM_Native_Debugger_for_Java and its successors.
If users are not using Linux, they cannot debug GraalVM Native Image. Please follow https://github.com/oracle/graal/issues/5648 which has not been closed.
and use VSCode under Linux to complete debugging work according to their requirements.
If the user is using IntelliJ IDEA and wants to debug the generated GraalVM Native Image,
the user can follow https://blog.jetbrains.com/idea/2022/06/intellij-idea-2022-2-eap-5/#Experimental_GraalVM_Native_Debugger_for_Java and its successors.

If the user is not using Linux, the GraalVM Native Image cannot be debugged.
Please follow https://github.com/oracle/graal/issues/5648 which has not been closed.

For Java Agents such as `ShardingSphere Agent`, the `native-image` component of GraalVM does not fully support the use of javaagent when building Native Image.
Users need to pay attention to https://github.com/oracle/graal/issues/1065 which has not been closed.
Users need to pay attention to https://github.com/oracle/graal/issues/8177 which has not been closed.

If users expect to use such Java Agents under ShardingSphere Proxy Native, they need to pay attention to the changes involved in https://github.com/oracle/graal/pull/8077 .
Loading

0 comments on commit a4342ca

Please sign in to comment.