Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aggies Sprint 9 #117

Open
wants to merge 39 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
7e0462e
Update version to 1.1.9 SNAPSHOT
jmicah Apr 2, 2024
6fc61f7
Update service module pom.xml to version 1.1.9-SNAPSHOT
wwelling Apr 11, 2024
71c21fe
Update components module pom.xml to version 1.1.9-SNAPSHOT
wwelling Apr 11, 2024
1d2f431
Merge pull request #129 from TAMULib/1.1.9-SNAPSHOT
kaladay May 9, 2024
36a67ca
Merge branch 'main' into sprint17-staging
kaladay May 9, 2024
b60c66c
Merge pull request #131 from TAMULib/sprint17-staging
kaladay May 10, 2024
bb145af
Merge pull request #132 from folio-org/master
kaladay May 30, 2024
d675385
Merge pull request #133 from folio-org/master
kaladay Jun 18, 2024
a73340d
Merge pull request #134 from folio-org/master
kaladay Sep 12, 2024
5a39ca7
Update README.md
wwelling Oct 1, 2024
757eb6d
Null check headers
wwelling Oct 1, 2024
fa34781
Merge pull request #138 from TAMULib/srptin20-null-check-headers
wwelling Oct 1, 2024
b44c9bd
Merge pull request #139 from TAMULib/sprint20-staging
wwelling Oct 3, 2024
b3efed2
Merge pull request #137 from TAMULib/wwelling-patch-1
wwelling Oct 3, 2024
4725fcc
Work in progress
wwelling Oct 7, 2024
ea8d7ee
Add Kafka producer config
wwelling Oct 7, 2024
37fa6f3
Merge pull request #115 from TAMULib/main
wwelling Oct 17, 2024
b0f32c4
Remove ActiveMQ properties
wwelling Oct 17, 2024
be274d0
Update module descriptor template
wwelling Oct 17, 2024
d2fb42a
Undo a bit
wwelling Oct 17, 2024
862c78c
Add kafka bootstrap-servers property
wwelling Oct 17, 2024
b4af7d3
Test send to topic workflow.event
wwelling Oct 17, 2024
8e0a7c8
Update EventProducer.java
wwelling Oct 17, 2024
41cc02b
Remove ActiveMQ port binding in launch descriptor
wwelling Oct 18, 2024
fe96067
Add nodes to module descriptor
wwelling Oct 18, 2024
9ae8c6c
Add some logging and only path according to okapi
wwelling Oct 19, 2024
7be5cb6
Log exchange
wwelling Oct 19, 2024
c74bb6b
Add workflow engine path default /camunda
wwelling Oct 19, 2024
c2b6244
Update WorkflowEngineService.java
wwelling Oct 19, 2024
01f4ee7
Update WorkflowEngineService.java
wwelling Oct 19, 2024
e903edc
Update WorkflowEngineService.java
wwelling Oct 19, 2024
a2785de
Add event producer topic property
wwelling Oct 19, 2024
c7d52bb
Enable spring kafka auto create
wwelling Oct 19, 2024
ee458cb
Switch some info logs to debugging
wwelling Oct 21, 2024
c56e830
Update README.md
wwelling Oct 21, 2024
591d7db
Add descriptions to env and add env to readme
wwelling Oct 21, 2024
6e9a60f
Update README.md
wwelling Oct 21, 2024
82dfe7f
Update WorkflowEngineService.java
wwelling Oct 21, 2024
d75fd0d
Merge pull request #116 from wwelling/MODWRKFLOW-34
wwelling Oct 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 42 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ The [Workflow Documentation](docs/README.md) describes additional information on

```
docker build -t folio/mod-workflow .
docker run -d -p 9001:9001 folio/mod-workflow
docker run -d -p 9001:8081 folio/mod-workflow
```

### Publish docker image
Expand All @@ -35,6 +35,47 @@ docker build -t [docker repo]/folio/mod-workflow:[version] .
docker push [docker repo]/folio/mod-workflow:[version]
```

### Environment variables:

| Name | Default value | Description |
|:-------------------------------|:-------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------|
| JAVA_OPTIONS | -XX:MaxRAMPercentage=75.0 | Java options |
| DB_HOST | postgres | Postgres hostname |
| DB_PORT | 5432 | Postgres port |
| DB_USERNAME | folio_admin | Postgres username |
| DB_PASSWORD | - | Postgres username password |
| DB_DATABASE | okapi_modules | Postgres database name |
| DB_QUERYTIMEOUT | 60000 | Database query timeout. |
| DB_CHARSET | UTF-8 | Database charset. |
| DB_MAXPOOLSIZE | 5 | Database max pool size. |
| KAFKA_HOST | kafka | Kafka broker hostname |
| KAFKA_PORT | 9092 | Kafka broker port |
| KAFKA_SECURITY_PROTOCOL | PLAINTEXT | Kafka security protocol used to communicate with brokers (SSL or PLAINTEXT) |
| KAFKA_SSL_KEYSTORE_LOCATION | - | The location of the Kafka key store file. This is optional for client and can be used for two-way authentication for client. |
| KAFKA_SSL_KEYSTORE_PASSWORD | - | The store password for the Kafka key store file. This is optional for client and only needed if 'ssl.keystore.location' is configured. |
| KAFKA_SSL_TRUSTSTORE_LOCATION | - | The location of the Kafka trust store file. |
| KAFKA_SSL_TRUSTSTORE_PASSWORD | - | The password for the Kafka trust store file. If a password is not set, trust store file configured will still be used, but integrity checking is disabled. |
| OKAPI_URL | http://okapi:9130 | OKAPI URL used to login system user, required |
| SERVER_PORT | 8081 | The port to listen on that must match the PortBindings. |
| SERVER_SERVLET_CONTEXTPATH | / | The context path, or base path, to host at. |
| SPRING_FLYWAY_ENABLED | false | Database migration support via Spring Flyway. |
| SPRING_JPA_HIBERNATE_DDLAUTO | update | Auto-configure database on startup. |
| TENANT_DEFAULTTENANT | diku | The name of the default tenant to use. |
| TENANT_FORCETENANT | false | Forcibly add or overwrite the tenant name using the default tenant. |
| TENANT_INITIALIZEDEFAULTTENANT | true | Perform initial auto-creation of tenant in the DB (schema, tables, etc..). |
| TENANT_RECREATEDEFAULTTENANT | false | When TENANT_INITIALIZEDEFAULTTENANT is true and the DB already exists, then drop and re-create. |


### Required Permissions
Institutional users should be granted the following permissions in order to use this workflow API:
- `workflow.actions.all`
- `workflow.events.all`
- `workflow.nodes.all`
- `workflow.nodes.item.post`
- `workflow.triggers.all`
- `workflow.tasks.all`
- `workflow.workflows.all`

### Issue tracker

See project [FOLIO](https://issues.folio.org/browse/FOLIO)
Expand Down
105 changes: 86 additions & 19 deletions service/descriptors/ModuleDescriptor-template.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,37 @@
}
]
},
{
"id": "workflow-nodes",
"version": "1.0",
"handlers": [
{
"methods": ["GET"],
"pathPattern": "/nodes",
"permissionsRequired": ["workflow.nodes.collection.get"]
},
{
"methods": ["POST"],
"pathPattern": "/nodes",
"permissionsRequired": ["workflow.nodes.item.post"]
},
{
"methods": ["DELETE"],
"pathPattern": "/nodes/{id}",
"permissionsRequired": ["workflow.nodes.item.delete"]
},
{
"methods": ["GET"],
"pathPattern": "/nodes/{id}",
"permissionsRequired": ["workflow.nodes.item.get"]
},
{
"methods": ["PUT"],
"pathPattern": "/nodes/{id}",
"permissionsRequired": ["workflow.nodes.item.put"]
}
]
},
{
"id": "workflow-triggers",
"version": "1.0",
Expand Down Expand Up @@ -251,6 +282,44 @@
],
"visible": false
},
{
"permissionName": "workflow.nodes.collection.get",
"displayName": "Workflow - node - get node collection",
"description": "Get node collection"
},
{
"permissionName": "workflow.nodes.item.delete",
"displayName": "Workflow - node - delete node item",
"description": "Delete node item"
},
{
"permissionName": "workflow.nodes.item.get",
"displayName": "Workflow - node - get node item",
"description": "Get node item"
},
{
"permissionName": "workflow.nodes.item.post",
"displayName": "Workflow - node - post node item",
"description": "Create node item"
},
{
"permissionName": "workflow.nodes.item.put",
"displayName": "Workflow - node - put node item",
"description": "Update node item"
},
{
"permissionName": "workflow.nodes.all",
"displayName": "Workflow - node - all permissions",
"description": "Entire set of permissions needed to use the workflow triggers",
"subPermissions": [
"workflow.nodes.collection.get",
"workflow.nodes.item.delete",
"workflow.nodes.item.get",
"workflow.nodes.item.post",
"workflow.nodes.item.put"
],
"visible": false
},
{
"permissionName": "workflow.triggers.collection.get",
"displayName": "Workflow - trigger - get trigger collection",
Expand Down Expand Up @@ -404,32 +473,30 @@
"HostConfig": {
"Memory": 402653184,
"PortBindings": {
"8081/tcp": [ { "HostPort": "%p" } ],
"61616/tcp": [ { "HostPort": "61616" } ]
"8081/tcp": [ { "HostPort": "%p" } ]
}
}
},
"env": [
{
"name": "JAVA_OPTIONS",
"value": "-XX:MaxRAMPercentage=66.0"
},
{ "name": "DB_HOST", "value": "postgres" },
{ "name": "DB_PORT", "value": "5432" },
{ "name": "DB_USERNAME", "value": "folio_admin" },
{ "name": "DB_PASSWORD", "value": "folio_admin" },
{ "name": "DB_DATABASE", "value": "okapi_modules" },
{ "name": "DB_QUERYTIMEOUT", "value": "60000" },
{ "name": "DB_CHARSET", "value": "UTF-8" },
{ "name": "DB_MAXPOOLSIZE", "value": "16" },
{ "name": "EVENT_UPLOADS_PATH", "value": "events", "description": "The ActiveMQ event uploads path." },
{ "name": "EVENT_QUEUE_NAME", "value": "event.queue", "description": "The ActiveMQ event queue name." },
{ "name": "OKAPI_CAMUNDA_BASEPATH", "value": "/", "description": "The context path, or base path of Mod-Camunda." },
{ "name": "OKAPI_CAMUNDA_RESTPATH", "value": "/rest", "description": "The path Mod-Camunda uses for REST requests." },
{ "name": "JAVA_OPTIONS", "value": "-XX:MaxRAMPercentage=66.0", "description": "Java options" },
{ "name": "DB_HOST", "value": "postgres", "description": "Postgres hostname" },
{ "name": "DB_PORT", "value": "5432", "description": "Postgres port" },
{ "name": "DB_USERNAME", "value": "folio_admin", "description": "Postgres username" },
{ "name": "DB_PASSWORD", "value": "folio_admin", "description": "Postgres username password" },
{ "name": "DB_DATABASE", "value": "okapi_modules", "description": "Postgres database name" },
{ "name": "DB_QUERYTIMEOUT", "value": "60000", "description": "Database query timeout." },
{ "name": "DB_CHARSET", "value": "UTF-8", "description": "Database charset." },
{ "name": "DB_MAXPOOLSIZE", "value": "16", "description": "Database max pool size." },
{ "name": "KAFKA_HOST", "value": "kafka", "description": "Kafka broker hostname" },
{ "name": "KAFKA_PORT", "value": "9092", "description": "Kafka broker port" },
{ "name": "KAFKA_SECURITY_PROTOCOL", "value": "PLAINTEXT", "description": "Kafka security protocol used to communicate with brokers (SSL or PLAINTEXT)" },
{ "name": "KAFKA_SSL_KEYSTORE_LOCATION", "value": "", "description": "The location of the Kafka key store file. This is optional for client and can be used for two-way authentication for client." },
{ "name": "KAFKA_SSL_KEYSTORE_PASSWORD", "value": "", "description": "The store password for the Kafka key store file. This is optional for client and only needed if 'ssl.keystore.location' is configured." },
{ "name": "KAFKA_SSL_TRUSTSTORE_LOCATION", "value": "", "description": "The location of the Kafka trust store file." },
{ "name": "KAFKA_SSL_TRUSTSTORE_PASSWORD", "value": "", "description": "The password for the Kafka trust store file. If a password is not set, trust store file configured will still be used, but integrity checking is disabled." },
{ "name": "OKAPI_URL", "value": "http://10.0.2.15:9130", "description": "The URL to the OKAPI service." },
{ "name": "SERVER_PORT", "value": "8081", "description": "The port to listen on that must match the PortBindings." },
{ "name": "SERVER_SERVLET_CONTEXTPATH", "value": "/", "description": "The context path, or base path, to host at." },
{ "name": "SPRING_ACTIVEMQ_BROKERURL", "value": "tcp://localhost:61616", "description": "The URL to the ActiveMQ server." },
{ "name": "SPRING_FLYWAY_ENABLED", "value": "false", "description": "Database migration support via Spring Flyway." },
{ "name": "SPRING_JPA_HIBERNATE_DDLAUTO", "value": "update", "description": "Auto-configure database on startup." },
{ "name": "TENANT_DEFAULTTENANT", "value": "diku", "description": "The name of the default tenant to use." },
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
package org.folio.rest.workflow.config;

import java.util.HashMap;
import java.util.Map;

import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.common.serialization.StringSerializer;
import org.folio.spring.messaging.model.Event;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.kafka.core.DefaultKafkaProducerFactory;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.core.ProducerFactory;
import org.springframework.kafka.support.serializer.JsonSerializer;

@Configuration
@Profile({ "messaging", "!test" })
public class KafkaProducerConfig {

@Value(value = "${spring.kafka.bootstrap-servers}")
private String bootstrapAddress;

@Bean
public ProducerFactory<String, Event> eventProducerFactory() {
Map<String, Object> configProps = new HashMap<>();
configProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapAddress);
configProps.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
configProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);

return new DefaultKafkaProducerFactory<>(configProps);
}

@Bean
public KafkaTemplate<String, Event> eventKafkaTemplate() {
return new KafkaTemplate<>(eventProducerFactory());
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;
import jakarta.jms.JMSException;
import jakarta.servlet.http.HttpServletRequest;
import java.io.File;
import java.io.IOException;
Expand All @@ -20,7 +19,7 @@
import org.folio.rest.workflow.dto.TriggerDto;
import org.folio.rest.workflow.enums.HttpMethod;
import org.folio.rest.workflow.exception.EventPublishException;
import org.folio.rest.workflow.jms.EventProducer;
import org.folio.rest.workflow.kafka.EventProducer;
import org.folio.rest.workflow.model.repo.TriggerRepo;
import org.folio.spring.messaging.model.Event;
import org.folio.spring.tenant.annotation.TenantHeader;
Expand Down Expand Up @@ -173,7 +172,7 @@ private void processEvent(TriggerDto trigger, Event event) throws EventPublishEx
logger.debug("Publishing event: {}: {}", trigger.getName(), trigger.getDescription());
try {
eventProducer.send(event);
} catch (JMSException | IOException e) {
} catch (Exception e) {
throw new EventPublishException("Unable to publish event!", e);
}
}
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
package org.folio.rest.workflow.kafka;

import org.folio.spring.messaging.model.Event;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Service;

@Service
public class EventProducer {

private static final Logger logger = LoggerFactory.getLogger(EventProducer.class);

@Value("${application.kafka.producer.events.topic}")
private String topic;

private KafkaTemplate<String, Event> eventTemplate;

@Autowired
public EventProducer(KafkaTemplate<String, Event> eventTemplate) {
this.eventTemplate = eventTemplate;
}

public void send(Event event) {
logger.debug("Send []: {}, {}", event.getMethod(), event.getPath());
this.eventTemplate.send(topic, event);
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,6 @@ public class OkapiDiscoveryService {
@Value("${okapi.url}")
private String okapiUrl;

@Value("${okapi.camunda.base-path}")
private String basePath;

private HttpService httpService;

private ObjectMapper objectMapper;
Expand Down Expand Up @@ -94,7 +91,7 @@ public Map<String, List<Handler>> getHandlers(String tenant, String id) throws I
}

public JsonNode getModules(String tenant) {
String url = okapiUrl + basePath + PROXY_TENANT_BASE_PATH + tenant + MODULES_SUB_PATH;
String url = okapiUrl + PROXY_TENANT_BASE_PATH + tenant + MODULES_SUB_PATH;
ResponseEntity<JsonNode> response = request(url, tenant);
if (logger.isDebugEnabled()) {
logger.debug("Response status code {}", response.getStatusCode());
Expand All @@ -104,7 +101,7 @@ public JsonNode getModules(String tenant) {
}

public JsonNode getModuleDescriptor(String tenant, String id) {
String url = okapiUrl + basePath + PROXY_MODULE_BASE_PATH + id;
String url = okapiUrl + PROXY_MODULE_BASE_PATH + id;
ResponseEntity<JsonNode> response = request(url, tenant);
if (logger.isDebugEnabled()) {
logger.debug("Response status code {}", response.getStatusCode());
Expand Down
Loading