refactor: adjust configuration management and smoke test setup to support production config by default

[ata-nas]

Description:

• make the default values in config records to be the production ones;
• the app.properties file should not be setting the production values, by default it should be empty, we should only set props there if we want to override the production defaults;
• when executing smoke tests, the config records should be with test values;
• explore options to achieve the result: (1) use env variables - does not seem to work as expected, the config dependency gets the vars via System.getenv() and it only shows case-sensitive keys, so adding a value like mediator.ringBufferSize=1024 will result into not being visible in runtime, it seems to me to be inconsistent. (2) build the jar for the smoke tests with the app.properties from src/test/resources - this works, the file gets packaged into the jar, only hurdle currently is to find a good and reliable way to do it with a gradle task.
• a single gradle task execution should be sufficient to build and run the needed environment for the test to pass

UPDATE:

• after some discussions with @mattp-swirldslabs & @AlfredoG87, we have discussed some issues that we have discovered:

  1. environment variables that are set to the docker container are read only if they are all upper case and separated only via _.;
  2. the initial idea of making a single gradle task (see some previous commits in this PR) that will build the project with correct test props, create the docker image and run it (so we can have an easy single entry point) did not seem to work when I tried it, since when in gh actions I execute either a shell script or the gradle task that contains a shell invocation of the docker executable, gh actions failed with exit code 14 (insufficient privileges). After some research, I see that I probably need to use the docker executable directly in the run part of the step. That worked and the test passed after commiting. So this part is still debatable, can I or can I not use the docker executable from a shell script.;

• since this discussions, there are several discoveries:

  1. using the IntelliJ IDE, if we modify the run config to include an environment variable with lowercase and . separator, it gets discovered. I am still to understand how that works and why, it does not seem to get passed as an CLI arg when starting the jar, so the IDE is doing something else. But that only means that it is indeed possible to discover non-conventional keys for environment variables.;
  2. as an approach, we could use PropertyFileConfigSource.class and read vars directly from a file (this is different from ClasspathFileConfigSource.java where we technically still load a file app.properties but this is packaged into the jar, here we are talking about an arbitrary file outside the classpath), this worked and is a potential approach. However, it has the lowest precedence, so if the same key comes from basically any other source, this will get overridden, we need to keep that in mind. Ordinals could be set in the constructor, so we can decide precedence. (see ConfigSourceOrdinalConstants.java);

• final approach (or the usage of multiple approaches so more could be tested if decided so) is still under debate;

UPDATE:

• after a discussion with @jsync-swirlds, he noted that we will be able to support by default unconventional casing for environment variables (meaning an environment variable like mediator.ringBufferSize=1024 will be picked up without having to do anything special). Until then, we can use the MappedConfigSource in order to support the mapping between uppercase variables and the formate we need them (for example the env variable MEDIATOR_RING_BUFFER_SIZE=1024 will be available with that key as well as with this one mediator.ringBufferSize=1024). For now, we can only pick up uppercase variables with an _ separator, but with the mapping, we will be able to read the values with the mapped key.

Related issue(s):

Fixes #185

Notes for reviewer:

Tested locally:

• Start the jar directly and use app.properties
• Start the jar directly and use env vars
• Start the docker container and use app.properties
• Start the docker container and use env vars
• Start the smoke-test action locally and use app.properties
• Start the smoke-test action locally and use env vars
• Start k8s cluster with the container and use app.properties
• Start k8s cluster with the container and use env vars
• Started all of the above with both app.properties and env vars, expected that the env vars take precedence (as is by default and we have not changed the default precedence)

Checklist

• Documented (Code comments, README, etc.)
• Tested (unit, integration, etc.)

[georgi-l95]

I know it's still on draft, but so far it looks good and I don't have any questions or nits 👍

[georgi-l95]

LGTM

[mattp-swirldslabs]

I added a few comments

[jsync-swirlds]

Looks good, just one nitpick (which can be addressed later), and one block of text that should be a github discussion rather than a comment in code.

[mattp-swirldslabs]

Looks good. Since you changed something in .github/workflows you might need approval from someone in RE or DevOps.

[AlfredoG87]

LGTM

[rbarkerSL]

Review applies to

• .github/workflows/smoke-test.yaml
• README.md

LGTM

[jsync-swirlds]

@ata-nas, it appears that most of your commits on this PR are unsigned.
You can fix this with the following git commands:

git rebase --gpg-sign -f HEAD~20
git push --force

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (main@611e91b). Learn more about missing BASE report.
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #232   +/-   ##
=======================================
  Coverage        ?   99.78%           
  Complexity      ?      259           
=======================================
  Files           ?       52           
  Lines           ?      938           
  Branches        ?       64           
=======================================
  Hits            ?      936           
  Misses          ?        2           
  Partials        ?        0           

Files with missing lines	Coverage Δ
...er/config/ServerMappedConfigSourceInitializer.java	100.00% <100.00%> (ø)
...m/hedera/block/server/mediator/MediatorConfig.java	100.00% <ø> (ø)
...m/hedera/block/server/notifier/NotifierConfig.java	100.00% <ø> (ø)