A Knowledge Integration project

Introduction

The direct consortial borrowing project aims to address the need of consortia that have already agreed to lend within the network.

Running

There are currently no other requirements other than having the application.

Linux and Mac

After pulling the repository the application can be started by issuing the command

$ ./gradlew run

Windows

After pulling the repository the application can be started by issuing the command

> gradle.bat run

Deploying

Environment variables

The following environment variables are common to all the methods of deployment.

Note: Today flyway does not support r2dbc datasources, so we need to configure both JDBC and R2DBC datasources - same DB connection effectively, 2 different connections - with JDBC only being used for database migrations.

Note: Both ELASTICSEARCH_HTTP_HOSTS and OPENSEARCH_HTTP_HOSTS are optional but you must configure one of them to enable push to OS/ES functionality. Not configuring one of these options will result in a 404 error when POSTing to /admin/reindex.
ENV Description Required Example

R2DBC_DATASOURCES_DEFAULT_URL

R2DBC Connection URL

yes

r2dbc:postgresql://localhost:5432/dcb

R2DBC_DATASOURCES_DEFAULT_USERNAME

R2DBC Username

yes

dcb

R2DBC_DATASOURCES_DEFAULT_PASSWORD

R2DBC Password

yes

dcb

DATASOURCES_DEFAULT_URL

JDBC Connection URL

yes

jdbc:postgresql://localhost:5432/dcb

DATASOURCES_DEFAULT_USERNAME

JDBC Username

yes

dcb

DATASOURCES_DEFAULT_PASSWORD

JDBC Password

yes

dcb

DCB_INDEX_NAME

Name of the index representing the shared index

no

si-index

DCB_INDEX_USERNAME

Username to communicate with the index service

no

si-user

DCB_INDEX_PASSWORD

Password to communicate with the index service

no

si-pass

Choose 1 of the below

ELASTICSEARCH_HTTP_HOSTS

1 or more Elasticsearch hosts (Comma separated)

no (See note)

http://localhost:9200

OPENSEARCH_HTTP_HOSTS

1 or more OpenSearch hosts (Comma separated)

no (See note)

http://localhost:9200

Docker

The module is built and published as a docker container on the K-Int repository

The images are tagged using the major/minor version numbers, so you are free to decide at which level you wish to track the software.

For example to deploy the latest snapshot pre-release of version 1 you would do the following inside a console:

1. Command prompt
$ docker pull docker.libsdev.k-int.com/knowledgeintegration/dcb:1-SNAPSHOT
$ docker run -it <your env vars> docker.libsdev.k-int.com/knowledgeintegration/dcb:1-SNAPSHOT
Click to view available environment variables

Note: Today flyway does not support r2dbc datasources, so we need to configure both JDBC and R2DBC datasources - same DB connection effectively, 2 different connections - with JDBC only being used for database migrations.

Note: Both ELASTICSEARCH_HTTP_HOSTS and OPENSEARCH_HTTP_HOSTS are optional but you must configure one of them to enable push to OS/ES functionality. Not configuring one of these options will result in a 404 error when POSTing to /admin/reindex.

ENV Description Required Example

R2DBC_DATASOURCES_DEFAULT_URL

R2DBC Connection URL

yes

r2dbc:postgresql://localhost:5432/dcb

R2DBC_DATASOURCES_DEFAULT_USERNAME

R2DBC Username

yes

dcb

R2DBC_DATASOURCES_DEFAULT_PASSWORD

R2DBC Password

yes

dcb

DATASOURCES_DEFAULT_URL

JDBC Connection URL

yes

jdbc:postgresql://localhost:5432/dcb

DATASOURCES_DEFAULT_USERNAME

JDBC Username

yes

dcb

DATASOURCES_DEFAULT_PASSWORD

JDBC Password

yes

dcb

DCB_INDEX_NAME

Name of the index representing the shared index

no

si-index

DCB_INDEX_USERNAME

Username to communicate with the index service

no

si-user

DCB_INDEX_PASSWORD

Password to communicate with the index service

no

si-pass

Choose 1 of the below

ELASTICSEARCH_HTTP_HOSTS

1 or more Elasticsearch hosts (Comma separated)

no (See note)

http://localhost:9200

OPENSEARCH_HTTP_HOSTS

1 or more OpenSearch hosts (Comma separated)

no (See note)

http://localhost:9200

Building from source

Last updated 2023-01-30

DCB is a micronaut application at time of writing (Jan 2023) compiled as a Java 17 source. At that time, k-int are building using the Temurin 7.0.6 Java distro. The project itself has gradle wrapper installed, so no external gradle installation is necessary.

This guidance is intended for organisations wishing to build the raw service executable (I.E. not have the build emit a container).

There is a policy decision which hosting providers need to make: Use the graalvm executable or build a plain old java fat-jar (Or build the skinny jar and do something else). Both options are cleanly described in the gradle build target - run "gradlew tasks" to see the targets possible.

Native executable build

You will need the graalvm JDK to build a native executable - instructions tested Jan 2023 with Graalvm 22.3.r17-grl

2. Command prompt
./gradlew nativeCompile

Will create a native executable including ONLY the parts of the JDK needed to run the code. This will output artefacts to dcb/build/native/nativeCompile named dcb

This executable is a native executable and can be executed on any linux like environment, without the need to install the Java Virtual Machine.

Jar build

3. Command prompt
./gradlew jar

Is the command to execute to build jar options. After execution the dcb/build/libs/

  • dcb-all.jar

  • dcb-jar

  • dcb-runner.jar

Where the -all.jar jar is the fatjar. .jar is the standard library, and -runner is executable

Running

Set any environment variables:

4. Command prompt
export r2dbc_datasources_default_url="r2dbc:postgresql://localhost:5432/dcb"
export r2dbc_datasources_default_username="dcb"
export r2dbc_datasources_default_password="dcb"
export datasources_default_url="jdbc:postgresql://localhost:5432/dcb"
export datasources_default_username="dcb"
export datasources_default_password="dcb"

you can then run the jar or the native executable from the relevant directory:

5. Command prompt (Run Jar)
java -jar ./dcb-all.jar
6. Command prompt (Run native)
./dcb
Click to view available environment variables

Note: Today flyway does not support r2dbc datasources, so we need to configure both JDBC and R2DBC datasources - same DB connection effectively, 2 different connections - with JDBC only being used for database migrations.

Note: Both ELASTICSEARCH_HTTP_HOSTS and OPENSEARCH_HTTP_HOSTS are optional but you must configure one of them to enable push to OS/ES functionality. Not configuring one of these options will result in a 404 error when POSTing to /admin/reindex.

ENV Description Required Example

R2DBC_DATASOURCES_DEFAULT_URL

R2DBC Connection URL

yes

r2dbc:postgresql://localhost:5432/dcb

R2DBC_DATASOURCES_DEFAULT_USERNAME

R2DBC Username

yes

dcb

R2DBC_DATASOURCES_DEFAULT_PASSWORD

R2DBC Password

yes

dcb

DATASOURCES_DEFAULT_URL

JDBC Connection URL

yes

jdbc:postgresql://localhost:5432/dcb

DATASOURCES_DEFAULT_USERNAME

JDBC Username

yes

dcb

DATASOURCES_DEFAULT_PASSWORD

JDBC Password

yes

dcb

DCB_INDEX_NAME

Name of the index representing the shared index

no

si-index

DCB_INDEX_USERNAME

Username to communicate with the index service

no

si-user

DCB_INDEX_PASSWORD

Password to communicate with the index service

no

si-pass

Choose 1 of the below

ELASTICSEARCH_HTTP_HOSTS

1 or more Elasticsearch hosts (Comma separated)

no (See note)

http://localhost:9200

OPENSEARCH_HTTP_HOSTS

1 or more OpenSearch hosts (Comma separated)

no (See note)

http://localhost:9200
Validate

A docker-compose.yml and init.sql file are provided in this directory which will work with the above config for starting the application.

Configuring the shared index synchronization

DCB supports either Opensearch or Elasticsearch when maintaining a shared index. Start by configuring the necessary shared values below and then add the connection properties for your chosen search provider.

ENV

Description

Required

Example

DCB_INDEX_NAME

Name of the index representing the shared index

no

si-index

DCB_INDEX_USERNAME

Username to communicate with the index service

no

si-user

DCB_INDEX_PASSWORD

Password to communicate with the index service

no

si-pass

Choose 1 of the below

ELASTICSEARCH_HTTP_HOSTS

1 or more Elasticsearch hosts (Comma separated)

no (See note)

http://localhost:9200

OPENSEARCH_HTTP_HOSTS

1 or more OpenSearch hosts (Comma separated)

no (See note)

http://localhost:9200

Metrics

Metrics are available from running instances of the application at /metrics.

Prometheus

DCB supports providing metrics in the format understood by Prometheus. By default Prometheus looks for metrics to be published under /metrics, but seeing as that is the home of our internal metrics, the prometheus format is instead published under /prometheus

API Docs

View the HTTP API documentation (Opens in new window).

There is also an interactive version of the API documentation available from a running instance of the application. Assuming the application is running at http://localhost:8080 visit http://localhost:8080/openapi/ui/index.html