PIO: sketch build flags and compilation time

[mcspr]

Is your feature request related to a problem? Please describe.

• pio run -e nodemcu-lolin compiles framework + libraries + sketch the very first time
• every consecutive run rebuilds only the sketch. any changes to the sketch keep framework + libraries .o cached in .pio/build/nodemcu-lolin/*
• if .pio/libdeps/nodemcu-lolin/* source changes, build notices this too, rebuilding library objects
• if build_flags arguments change, framework + libraries are rebuilt again
• framework + libraries .o are not shared between envs

Describe the solution you'd like

2 additional configuration options to add:

https://docs.platformio.org/en/latest/projectconf/section_env_build.html#src-build-flags

An option src_build_flags has the same behavior like build_flags but will be applied only for the >project source code from src_dir directory.

PLATFORMIO_SRC_BUILD_FLAGS is an alternative to the PLATFORMIO_BUILD_FLAGS
If feels like board flags and every other espurna option should be provided via those instead of global build_flags. Unless, if these are for external configuration (like no EEPROM global for framework, pubsubclient library max packet size, etc..)

https://docs.platformio.org/en/latest/projectconf/section_platformio.html#build-cache-dir

PlatformIO Core (CLI) uses this folder to store derived files from a build system (objects, firmwares, ELFs). These files are shared between all build environments. To speed up a build process, you can use the same cache folder between different projects if they depend on the same development platform and framework.

Therefore, check if similar boards can be combined under those conditions.
Also check if this can be used in travis release process

Describe alternatives you've considered

This should still apply under v2 / #1876

Additional context

[mcspr]

@Niek ee6ee89#r35457680
Turns out that was only the case with platformio 4.0.0, I forgot that this change was in 4.0.1+
platformio/platformio-core@8059e04 changed it so that src_build_flags change (PLATFORMIO_SRC_BUILD_FLAGS, src_build_flags = ${sysenv.SOMETHING} etc.) will be treated as platformio.ini change:
https://github.com/platformio/platformio-core/blob/82ec0164b0384c59dc434582cb65ca89b42ac6d4/platformio/project/helpers.py#L111
For example, .to_json() output is:

{"env:d1_mini": {"platform": "https://github.com/platformio/platform-espressif8266#feature/stage", "build_flags": [""], "board":
"d1_mini", "framework": ["arduino"], "src_build_flags": ["-DMY_FIRST_BUILD_FLAG"]}}

Feels like a bug (...will link the upstream issue...).
And this can be worked around by using custom.h and modifying it's contents each build.

[mcspr]

Rough draft:
https://travis-ci.org/mcspr/espurna/builds/597221420
dev...mcspr:travis/all
Now every job is testing all of existing TRAVISN hardware profiles + some extras, pre-defined and enabled via cmd args.
If src_build_flags could be excluded from build dir purge condition, cfg_... become arrays of -DKEY=VALUE (or maybe not, bc I am not sure why ${!cfg[@]} syntax does not work)

Secure client is failing as expected.

[Niek]

Nice approach! It's failing but with a different error ;)

espurna/config/custom.h:2:22: error: token "=" is not valid in preprocessor expressions

It also shouldn't fail because you added the ; fix in the branch, right?

[mcspr]

At that commit there was no mqtt.ino fix present, it is there now though -> #1943
(and let's pretend that was intentional, and not failure to remember that those are no longer -D flags)
BUILD_CACHE is next one to test, if it does matter at all.

[mcspr]

BUILD_CACHE does indeed work as intended. In case global BUILD_FLAGS stay the same, instead of "Compiling" it will "Retrieve" each Core's or libraries object files from cache. Speed-up is on par with using custom.h substitution, 10...15seconds instead of ~1 minute per env
But now it begs the question wheter it is useful with build tests like these, unless there is any need to give some other flags to gcc (-Wextra,-fsomething, -include ... etc.)

Releases on the other hand only need a single build flag in most cases and build cache option could be provided via environment.