Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RAM management bug fix #30

Merged
merged 18 commits into from
Jun 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
ca4e9ea
Bump maven-compiler-plugin from 3.8.0 to 3.10.1
dependabot[bot] Sep 19, 2022
6620be7
Bump actions/checkout from 2 to 3
dependabot[bot] Sep 19, 2022
308c59f
Bump maven-shade-plugin from 3.2.1 to 3.4.0
dependabot[bot] Sep 19, 2022
c60597b
Bump actions/download-artifact from 2 to 3
dependabot[bot] Sep 19, 2022
8c60017
Merge pull request #25 from HSLdevcom/dependabot/github_actions/devel…
mjaakko Sep 19, 2022
be20b1d
Merge pull request #24 from HSLdevcom/dependabot/github_actions/devel…
mjaakko Sep 19, 2022
c2195f6
Merge pull request #23 from HSLdevcom/dependabot/maven/develop/org.ap…
mjaakko Sep 19, 2022
ae827f0
Merge pull request #22 from HSLdevcom/dependabot/maven/develop/org.ap…
mjaakko Sep 19, 2022
6f905e1
Bump actions/upload-artifact from 2 to 3
dependabot[bot] Oct 1, 2022
e86d8dd
Merge pull request #26 from HSLdevcom/dependabot/github_actions/devel…
mjaakko Oct 4, 2022
d95b948
Update common version to 1.5.2
mjaakko Mar 2, 2023
2f8c864
Bump maven-shade-plugin from 3.4.0 to 3.4.1
dependabot[bot] Apr 27, 2023
12a3e26
Bump maven-compiler-plugin from 3.10.1 to 3.11.0
dependabot[bot] Apr 27, 2023
a57cbbf
Merge pull request #27 from HSLdevcom/dependabot/maven/develop/org.ap…
mjaakko Apr 27, 2023
25bf68b
Merge pull request #28 from HSLdevcom/dependabot/maven/develop/org.ap…
mjaakko Apr 27, 2023
b79cc81
Update README.md
mjaakko Apr 27, 2023
1f2df27
Complete migration to Java 11 and Temurin based Docker image
thjarvin Jun 3, 2024
2b7ceb1
Common dependency version updated to 1.5.4
thjarvin Jun 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .github/workflows/test-and-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Set up JDK 11
uses: actions/setup-java@v3
with:
Expand All @@ -18,7 +18,7 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Upload .jar file
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: pulsar-mqtt-gateway.jar
path: target/pulsar-mqtt-gateway.jar
Expand All @@ -28,9 +28,9 @@ jobs:
# Run only on develop branch
if: github.ref == 'refs/heads/develop'
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Download .jar file
uses: actions/download-artifact@v2
uses: actions/download-artifact@v3
with:
name: pulsar-mqtt-gateway.jar
path: target
Expand All @@ -47,9 +47,9 @@ jobs:
# Run only for tagged commits
if: github.event_name == 'push' && contains(github.ref, 'refs/tags/')
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Download .jar file
uses: actions/download-artifact@v2
uses: actions/download-artifact@v3
with:
name: pulsar-mqtt-gateway.jar
path: target
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ FROM eclipse-temurin:11-alpine
RUN apk add --no-cache curl

ADD target/pulsar-mqtt-gateway.jar /usr/app/pulsar-mqtt-gateway.jar
ENTRYPOINT ["java", "-jar", "/usr/app/pulsar-mqtt-gateway.jar"]
ENTRYPOINT ["java", "-XX:InitialRAMPercentage=10.0", "-XX:MaxRAMPercentage=95.0", "-jar", "/usr/app/pulsar-mqtt-gateway.jar"]
39 changes: 19 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# pulsar-mqtt-gateway [![Test and create Docker image](https://github.com/HSLdevcom/pulsar-mqtt-gateway/actions/workflows/test-and-build.yml/badge.svg)](https://github.com/HSLdevcom/pulsar-mqtt-gateway/actions/workflows/test-and-build.yml)

## Description

Application for publishing Pulsar messages to MQTT. The application doesn't care about
Expand All @@ -9,13 +11,13 @@ the data content, it only publishes the binary message as-is.

This project depends on [transitdata-common](https://github.com/HSLdevcom/transitdata-common) project.

Either use released versions from public maven repository or build your own and install to local maven repository:
- ```cd transitdata-common && mvn install```
Either use released versions from the public GitHub Packages repository (Maven) or build your own and install to local Maven repository:
- `cd transitdata-common && mvn install`

### Locally

- ```mvn compile```
- ```mvn package```
- `mvn compile`
- `mvn package`

### Docker image

Expand All @@ -24,22 +26,19 @@ Either use released versions from public maven repository or build your own and

## Running

Requirements:
- Pulsar Cluster
- By default uses localhost, override host in PULSAR_HOST if needed.
- Tip: f.ex if running inside Docker in OSX set `PULSAR_HOST=host.docker.internal` to connect to the parent machine
- You can use [this script](https://github.com/HSLdevcom/transitdata/blob/master/bin/pulsar/pulsar-up.sh) to launch it as Docker container
- Connection to an external MQTT server.
- Configure username and password via files
- Set filepath for username via env variable FILEPATH_USERNAME_SECRET, default is `/run/secrets/mqtt_broker_username`
- Set filepath for password via env variable FILEPATH_PASSWORD_SECRET, default is `/run/secrets/mqtt_broker_password`
- Mandatory: Set mqtt-topic via env variable MQTT_TOPIC


All other configuration options are configured in the [config file](src/main/resources/environment.conf)
which can also be configured externally via env variable CONFIG_PATH
### Dependencies

Launch Docker container with
* Pulsar
* Connection to a MQTT broker

```docker-compose -f compose-config-file.yml up <service-name>```
### Environment variables

* `MQTT_HAS_AUTHENTICATION`: whether the MQTT broker uses authentication
* `FILEPATH_USERNAME_SECRET`: path to the file containing the username, default is `/run/secrets/mqtt_broker_username`
* `FILEPATH_PASSWORD_SECRET`: path to the file containing the password, default is `/run/secrets/mqtt_broker_password`
* `MQTT_BROKER_HOST`: URL of the MQTT broker
* `MQTT_TOPIC`: MQTT topic where to publish messages
* `MQTT_MAX_INFLIGHT`: maximum amount of MQTT messages in-flight
* `MQTT_CLIENT_ID`: MQTT client ID
* `MQTT_RETAIN_MESSAGE`: whether to send MQTT messages with retained flag
* `MQTT_KEEP_ALIVE_INTERVAL`: interval for MQTT keep-alive, in seconds
6 changes: 3 additions & 3 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<common.version>1.3.29</common.version>
<common.version>1.5.4</common.version>
<paho.version>1.2.5</paho.version>
</properties>

Expand Down Expand Up @@ -52,7 +52,7 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.1</version>
<version>3.4.1</version>
<executions>
<!-- Run shade goal on package phase -->
<execution>
Expand Down Expand Up @@ -96,7 +96,7 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<version>3.11.0</version>
<configuration>
<source>11</source>
<target>11</target>
Expand Down
Loading