Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup build automation script and CI workflow #2

Closed
wants to merge 67 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
b7ed3fb
Setup script to check example name and directory
sudotensor Jul 30, 2021
8c455b2
Add requirements.txt with dependencies for venv
sudotensor Jul 30, 2021
31d5239
Create .gitignore
sudotensor Jul 30, 2021
09a0fae
Update script to setup virtual environment
sudotensor Jul 30, 2021
4fdd7bd
Update .gitignore with example dependencies
sudotensor Jul 30, 2021
b05f409
Update script to install example-specific requirements
sudotensor Jul 30, 2021
da73c96
Add cmake build directory to .gitignore
sudotensor Jul 30, 2021
ea5af60
Update script with build instructions for mock example
sudotensor Jul 30, 2021
90d245a
Update script with build instructions for MCUboot example
sudotensor Jul 30, 2021
4287874
Add application build and .mbed files to .gitignore
sudotensor Jul 30, 2021
2c4fac3
Rename MCUboot to mcuboot
sudotensor Jul 30, 2021
c454386
Rename Mock to mock
sudotensor Jul 30, 2021
f4379f6
Fix typos in build instructions script
sudotensor Aug 2, 2021
789a886
Update check to see if script is sourced
sudotensor Aug 2, 2021
da539aa
Create fota.sh script with usage display function
sudotensor Aug 2, 2021
9dfde9f
Add function to parse and store options
sudotensor Aug 2, 2021
dff4da1
Add variable to store path of repository root
sudotensor Aug 2, 2021
5bc6c1b
Setup utils.sh file for constants and helper functions
sudotensor Aug 2, 2021
4d95793
Add "say" function to utils to print error messages
sudotensor Aug 2, 2021
a58f3b9
Add "fail" function to utils
sudotensor Aug 2, 2021
5395451
Update parse options to parse space-separated options
sudotensor Aug 3, 2021
dd6d7d4
Fix indefinite looping bug in options parsing function
sudotensor Aug 3, 2021
53e4c73
Revert to equals-separated parsing of options
sudotensor Aug 3, 2021
680fe35
Update usage message with equals-separated options
sudotensor Aug 3, 2021
a099196
Update fota script to use utils functions
sudotensor Aug 3, 2021
4618965
Indent sections in usage display by two spaces
sudotensor Aug 3, 2021
9858c1a
Refactor variables and function calls into main function
sudotensor Aug 3, 2021
a316787
Add comments describing usage, clean and parse functions
sudotensor Aug 3, 2021
839fecd
Implement success and message formatting options for say
sudotensor Aug 3, 2021
bf1df85
Add check to verify that mount point is valid
sudotensor Aug 3, 2021
b790dbd
Add check to verify that example is valid
sudotensor Aug 3, 2021
63d6606
Add check to verify that the target board is valid
sudotensor Aug 3, 2021
f820c28
Add check to verify that the toolchain is GCC_ARM
sudotensor Aug 3, 2021
ad47cf3
Fix root directory path
sudotensor Aug 3, 2021
ba15ff4
Add function to setup virtual environment folder
sudotensor Aug 3, 2021
67aee91
Add function to install general requirements
sudotensor Aug 3, 2021
bf83d77
Fix mount point directory check and fail
sudotensor Aug 3, 2021
75b3f09
Add dummy example build functions
sudotensor Aug 3, 2021
1316fff
Update build_example to pass repository root as argument
sudotensor Aug 3, 2021
28bf5c4
Fix the argument list generation in build_example
sudotensor Aug 3, 2021
270a9b6
Add note formatting to say function
sudotensor Aug 3, 2021
ae47e62
Update missing mount message in valid_mount
sudotensor Aug 3, 2021
2b23956
Update mock_build with build instructions
sudotensor Aug 3, 2021
554c6f2
Update clean routine for both examples
sudotensor Aug 3, 2021
3958932
Update comments and messages in mock.sh
sudotensor Aug 3, 2021
01098f1
Update mcuboot_build routine to install example requirements
sudotensor Aug 3, 2021
fde4e6d
Update mcuboot_build with more build instructions
sudotensor Aug 3, 2021
c2bab19
Update mcuboot_build to handle changes to mbed_app.json
sudotensor Aug 3, 2021
d205e61
Update mcuboot_build with steps to create update binary
sudotensor Aug 3, 2021
3742dac
Update mcuboot_clean to clean build files and dependencies
sudotensor Aug 3, 2021
34287c0
Add trap to cleanup upon termination
sudotensor Aug 3, 2021
7172635
Add say message to clean function
sudotensor Aug 3, 2021
83d22ec
Remove old build.sh script
sudotensor Aug 3, 2021
741b9fe
Setup CI workflow folder and file
sudotensor Aug 4, 2021
5252ed9
Add jobs in workflow to build both examples
sudotensor Aug 4, 2021
67d89c5
Refactor fail function to remove mode parameter
sudotensor Aug 4, 2021
8b6e749
Merge pull request #5 from lambda-shuttle/build-process-automation
sudotensor Aug 4, 2021
0106481
Merge branch 'main' of github.com:lambda-shuttle/mbed-os-example-ble-…
sudotensor Aug 4, 2021
9779527
Merge pull request #6 from lambda-shuttle/ci-setup
sudotensor Aug 4, 2021
6c4d669
Revert "CI Setup"
sudotensor Aug 4, 2021
7680ac5
Merge pull request #7 from lambda-shuttle/revert-6-ci-setup
sudotensor Aug 4, 2021
95f0a00
Update workflow to use mbed-os-env container
sudotensor Aug 4, 2021
431c6df
Merge branch 'main' of github.com:lambda-shuttle/mbed-os-example-ble-…
sudotensor Aug 4, 2021
5f0c889
Add python3-venv dependency to workflow
sudotensor Aug 4, 2021
40139a7
Update apt-get before installing dependencies
sudotensor Aug 4, 2021
60e666a
Merge pull request #8 from lambda-shuttle/ci-setup
sudotensor Aug 4, 2021
82b0f0f
Remove on push from build-examples.yml
sudotensor Aug 4, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .github/workflows/build-examples.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Build Examples
on:
workflow_dispatch:
pull_request:
jobs:
build-mock:
runs-on: ubuntu-latest
container: mbedos/mbed-os-env:latest
steps:
# Checkout the repo and download it to the runner
- name: Checkout
uses: actions/checkout@v2
# Install the venv module
- name: Install dependencies
run: |
apt-get update -y
apt-get install -y python3-venv
# Run the fota build tool with mock as the option
- name: Build example
run: ./scripts/fota.sh -e=mock

build-mcuboot:
runs-on: ubuntu-latest
container: mbedos/mbed-os-env:latest
steps:
# Checkout the repo and download it to the runner
- name: Checkout
uses: actions/checkout@v2
# jq is a command-line JSON processor
- name: Install dependencies
run: |
apt-get update -y
apt-get install -y jq python3-venv
# Pipe "yes" into the script to select the appropriate option
# More information about this in the documentation
# Run the fota build tool with mcuboot as the option
- name: Build example
run: yes | ./scripts/fota.sh -e=mcuboot
38 changes: 38 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# macOS Desktop Services Store
.DS_Store

# Virtual environment directory
venv/

# IDE-specific directories
.vscode/
.idea/

# Example application dependencies
mcuboot/target/application/mbed-os-experimental-ble-services/
mcuboot/target/application/mbed-os/
mcuboot/target/application/mcuboot/
mcuboot/target/bootloader/dist/
mcuboot/target/bootloader/imgtool.egg-info/
mcuboot/target/bootloader/mbed-os/
mcuboot/target/bootloader/mcuboot/
mock/target/mbed-os-experimental-ble-services/
mock/target/mbed-os/

# CMake build directory
**/cmake_build

# Example application build files
mcuboot/target/application/BUILD/
mcuboot/target/bootloader/application.hex
mcuboot/target/bootloader/build/
mcuboot/target/bootloader/merged.hex
mcuboot/target/bootloader/signed_application.hex
mcuboot/target/bootloader/signed_update.bin
mcuboot/target/bootloader/signed_update.hex
mcuboot/target/bootloader/signing-keys.pem
mcuboot/target/bootloader/signing_keys.c

# .mbed files
mcuboot/target/application/.mbed
mcuboot/target/bootloader/.mbed
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
6 changes: 6 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
bleak==0.12.1
cmake==3.20.5
mbed-cli==1.10.5
mbed-tools==7.28.1
mercurial==5.8
ninja==1.10.0.post2
201 changes: 201 additions & 0 deletions scripts/fota.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
#!/bin/bash
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Author's Note:
# The fota cli tool is used to setup and build the ble fota examples in this repository. The tool takes in as
# arguments the example, target board, mount point of the target board, and toolchain. Note that if the mount point
# is not provided (it could be the case that an end-user is trying to build without the board connected), then the
# target binary is not flashed. In the case of the MCUboot example, the "factory firmware" is saved and would require
# manual transfer when the board is indeed connected. In either case, the demonstration step would be the only part
# that requires manual intervention.
#
# Important: The tool assumes the target board and toolchain unless otherwise specified by the end-user. However,
# currently, the only board and toolchain supported are NRF52840_DK and GCC_ARM respectively.

set -e
trap 'cleanup $?' SIGINT SIGTERM ERR EXIT

source scripts/utils.sh
source scripts/mock.sh
source scripts/mcuboot.sh

# Display a neatly formatted message on how to use this tool
usage () {
cat <<EOF
Usage: $(basename "${BASH_SOURCE[0]}") [options]

A simple cli tool to automate the setup and build process for Bluetooth-LE
Firmware Over the Air (FOTA) examples

Options:
-e=, --example=TEXT The example you are trying to setup and build.
[default: mock]
-t=, --toolchain=[GCC_ARM|ARM] The toolchain you are using to build your app.
[default: GCC_ARM]
-b=, --board=TEXT A build target for an Mbed-enabled device.
[default: NRF52840_DK]
-m=, --mount=TEXT Path to the mount point of the target board.
-c, --clean Clean the example builds and environment
-h, --help Print this message and exit

Note:
For now, only the NRF52840_DK board and GCC_ARM toolchain are supported. Also,
if a mount point isn't provided, then the target binary is not flashed.
EOF
exit
}

# This function would clean up the example builds and generated files
clean () {
say message "Cleaning builds and generated files..."
Copy link
Member

@paul-szczepanek-arm paul-szczepanek-arm Aug 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this? nevermind, figured it out

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tool accepts a -c or --clean option to remove any generated build folders or files. It removes the venv folder and any example-specific generated folders and files by invoking the clean functions of each example - mock_clean and mcuboot_clean; the definitions of these functions are located in mock.sh and mcuboot.sh respectively. Also, it's worth mentioning that if the build were to fail at any point with an exit code of 1, it would trap into the cleanup routine, which would call this function.


rm -rf "$root/venv"

# Clean example-specific files and folders
mock_clean "$root"
mcuboot_clean "$root"

say success "All neat and tidy now" && exit
}

# Parses the options specified by the end-user and either assigns the corresponding variable or calls a function and
# exits (i.e. in the case of help and clean). If the option is unrecognised, the function returns 1, which triggers
# a fail that exits the tools and displays an error on stderr.
parse_options () {
for i in "$@"; do
case $i in
-e=*|--example=*) example="${i#*=}" ; shift ;;
-t=*|--toolchain=*) toolchain="${i#*=}" ; shift ;;
-b=*|--board=*) board="${i#*=}" ; shift ;;
-m=*|--mount=*) mount="${i#*=}" ; shift ;;
-c|--clean) clean ;;
-h|--help) usage ;;
*) return 1 ;;
esac
done
}

# Check that the provided target board mount point is valid. If the mount point is not provided, display a short message
# indicating that binaries will have to be flashed manually.
valid_mount () {
if [[ -z $mount ]]; then
say note "Mount point not provided - binaries will not be flashed."
skip=1
else
[ -d "$mount" ] || fail "Mount point invalid" \
"Please check the path and if the board is connected" \
"Tip: Use mbed-ls to identify the mount point path"
fi
}

# Checks if the example is either mock or mcuboot. If more examples, are added in the future, this function would have
# to be modified accordingly.
valid_example () {
case $example in
mock|mcuboot) ;;
*) fail "Invalid example" "Supported examples - [mock|mcuboot]" ;;
esac
}

# Checks if the board is NRF52840_DK as it's currently the only supported board. This function will have to be modified
# when DISCO_L475VG_IOT01A (and maybe other boards) are supported at a later point.
valid_board () {
case $board in
NRF52840_DK) ;;
*) fail "Unsupported board" "The only supported board is NRF52840_DK"
esac
}

# Checks if the toolchain is GCC_ARM as it's the only one supported. The ARM toolchain may be supported in the future.
valid_toolchain () {
case $toolchain in
GCC_ARM) ;;
*) fail "Unsupported toolchain" "The only supported toolchain is GCC_ARM"
esac
}

# A series of checks to make sure that the program options are valid
check_usage () {
valid_mount
valid_example
valid_board
valid_toolchain
}

# Setups up the main virtual environment and activates it
setup_virtualenv () {
if [[ -d venv ]]; then
say message "Using existing virtual environment venv..."
else
say message "Creating virtual environment..."

# Create the venv directory and setup the virtual environment
# shellcheck disable=SC2015
mkdir venv && python3 -m venv venv \
|| fail "Virtual environment creation failed!" "Tip: Check your python installation!"
fi
source venv/bin/activate
say success "Virtual environment activated"
}

# Installs the required python dependencies silently and notify if there's any issue in the installation.
install_requirements () {
say message "Installing/updating requirements silently..."
# shellcheck disable=SC2015
pip install -q --upgrade pip && pip install -q -r "$root/requirements.txt" \
|| fail "Unable to install requirements!" "Please take a look at requirements.txt"
say success "General requirements installed/updated"
}

# Call the build functions corresponding to the selected example
# Pre: The example is valid and so are all other arguments.
build_example () {
args=("$toolchain" "$board" "$mount" "$skip" "$root")
case $example in
mock) mock_build "${args[@]}" ;;
mcuboot) mcuboot_build "${args[@]}" ;;
esac
}

# Cleanup routine that runs when the program exits (either successfully or abruptly)
cleanup () {
# If unsuccessful termination, then clean the build
if [[ "$1" -eq 1 ]]; then clean; fi
}

#-----------------------------------------------------------------------------------------------------------------------
# Main Program
#-----------------------------------------------------------------------------------------------------------------------
main () {
# Default Options:
example="mock"
toolchain="GCC_ARM"
board="NRF52840_DK"
skip=0 # Skip the binary flashing if mount point is missing - Default: false

# Root directory of repository
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." &>/dev/null && pwd -P)

setup_formatting
parse_options "$@" || fail "Unrecognised option" "Please use -h or --help for usage"

check_usage
setup_virtualenv
install_requirements
build_example
}

main "$@"
exit
Loading