Skip to content

Commit f9193f9

Browse files
h2zeroh2zero
h2zero
authored and
h2zero
committed
Update build script and add workflows for PR/release.
1 parent 4c4adc0 commit f9193f9

File tree

6 files changed

+143
-89
lines changed

6 files changed

+143
-89
lines changed

‎.github/workflows/build.yml

+23
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
name: Build
2+
3+
on: [push, pull_request]
4+
5+
jobs:
6+
build-upload:
7+
runs-on: ubuntu-latest
8+
name: Build
9+
steps:
10+
- uses: actions/checkout@v2
11+
12+
- name: Set up Python
13+
uses: actions/setup-python@v2
14+
with:
15+
python-version: "3.8"
16+
17+
- name: Install build tools
18+
run: |
19+
sudo apt-get update
20+
sudo apt-get -y install libusb-1.0-0 cmake srecord
21+
pip install -U pyserial
22+
- name: Build
23+
run: ./scripts/build-firmware

‎.github/workflows/release.yml

+43
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
name: Release
2+
3+
on:
4+
release:
5+
types: [created]
6+
7+
jobs:
8+
build-upload:
9+
runs-on: ubuntu-latest
10+
name: Build
11+
steps:
12+
- uses: actions/checkout@v2
13+
14+
- name: Set up Python
15+
uses: actions/setup-python@v2
16+
with:
17+
python-version: "3.8"
18+
19+
- name: Install build tools
20+
run: |
21+
sudo apt-get update
22+
sudo apt-get -y install libusb-1.0-0 cmake srecord
23+
pip install -U pyserial
24+
25+
- name: Get release
26+
id: get_release
27+
uses: bruceadams/[email protected]
28+
env:
29+
GITHUB_TOKEN: ${{ github.token }}
30+
31+
- name: Prepare Release Assets
32+
run: |
33+
./scripts/build-firmware
34+
mkdir assets
35+
cp build/konnected-esp8266-latest.bin assets/konnected-esp8266-${{ steps.get_release.outputs.tag_name }}.bin
36+
37+
- name: Upload asset to release
38+
uses: bgpat/release-asset-action@03b0c30db1c4031ce3474740b0e4275cd7e126a3
39+
with:
40+
pattern: "assets/*"
41+
release-url: ${{ steps.get_release.outputs.upload_url }}
42+
allow-overwrite: true
43+
github-token: ${{ secrets.GITHUB_TOKEN }}

‎.gitignore

+3
Original file line numberDiff line numberDiff line change
@@ -48,3 +48,6 @@ firmware/builds
4848

4949
# Private files
5050
private
51+
52+
# NodeMCU
53+
nodemcu-firmware

‎README.md

+10-13
Original file line numberDiff line numberDiff line change
@@ -51,34 +51,31 @@ Go to the [releases section](https://github.com/konnected-io/konnected-security/
5151
that you can flash on your Konnected Alarm Panel or ESP8266 device.
5252

5353
### Building the Firmware Yourself
54-
Konnected leverages the [NodeMCU](https://github.com/nodemcu/nodemcu-firmware) codebase and [Docker builder](https://hub.docker.com/r/marcelstoer/nodemcu-build/) to create a base nodeMCU firmware image and a filesystem containing the Konnected application. Building only requires a few steps.
54+
Konnected leverages the [NodeMCU](https://github.com/nodemcu/nodemcu-firmware) codebase to create a base nodeMCU firmware image and a filesystem containing the Konnected application. Building only requires a few steps.
5555

56-
1. Download and install [Docker](https://www.docker.com/products/docker-desktop)
5756
1. Clone the Konnected repo
5857

5958
git clone https://github.com/konnected-io/konnected-security.git
6059

61-
1. Use the build-firmware script to kick off the build - providing a semantic version command line argument as shown below. The build script will automatically pull down the correct nodeMCU image, and use this nodeMCU docker builder to create base firmware, an LFS image, and a SPIFFS file system containing the entire Konnected application.
60+
1. Use the build-firmware script to kick off the build. The build script will automatically pull down the correct nodeMCU image, and use this to create base firmware, an LFS image, and a SPIFFS file system containing the entire Konnected application.
6261

6362
cd konnected-security
64-
./scripts/build-firmware 2-2-99
63+
./scripts/build-firmware
6564

66-
1. Once the build completes a folder will be created in `firmware/builds` named after the version specified in the previous step. This folder will contain three files also reflecting the version.
67-
1. konnected-filesystem-0x100000-2-2-99.img
68-
1. konnected-firmware-2-2-99.bin
69-
1. konnected-esp8266-2-2-99.bin
65+
1. Once the build completes a folder will be created in `build`. This folder will contain three files reflecting the version or branch.
66+
1. konnected-filesystem-0x100000-{BRANCH/VERSION}.img
67+
1. konnected-firmware-{BRANCH/VERSION}.bin
68+
1. konnected-esp8266-*.bin
7069

7170
The `konnected-firmware-*` contains the firmware partition and should be flashed at location `0x0`.
7271
The `konnected-filesystem-*` image contains the Konnected application and should be flashed at the memory location in
7372
the filename.
74-
For convenience, the `konnected-esp8266-*` image is an all-in-one image to be flashed at memory location `0x0` containing the two images above.
73+
For convenience, the `konnected-esp8266-latest` image is an all-in-one image to be flashed at memory location `0x0` containing the two images above.
7574

76-
*Note: Each time you build it will remove any prior build outputs corresponding to the same version.*
77-
*Note: Versions in this project should always be formatted `<major>-<minor>-<patch>`.*
7875

7976
### Flashing a Build
8077
Flashing a build is simple with the [NodeMCU PyFlasher](https://github.com/marcelstoer/nodemcu-pyflasher/releases). Simply flash
81-
the `konnected-esp8266-*.bin` file to your device. Typically use baud rate 115200 and flash mode `dio`.
78+
the `konnected-esp8266-latest.bin` file to your device. Typically use baud rate 115200 and flash mode `dio`.
8279

8380
Mac and Linux users can also easily flash from the command line using `esptool`
8481

@@ -91,7 +88,7 @@ Mac and Linux users can also easily flash from the command line using `esptool`
9188

9289
1. Flash the downloaded image using `esptool`:
9390

94-
esptool.py --port=/dev/cu.SLAB_USBtoUART write_flash --flash_mode dio --flash_size detect 0x0 konnected-esp8266-3-0-0.bin
91+
esptool.py --port=/dev/cu.SLAB_USBtoUART write_flash --flash_mode dio --flash_size detect 0x0 konnected-esp8266-latest.bin
9592

9693
*Note: The USB port may vary depending on your computer platform and board.*
9794

‎scripts/build-firmware

+63-70
Original file line numberDiff line numberDiff line change
@@ -2,118 +2,111 @@
22

33
function usage() {
44
echo
5-
echo "usage: $0 <version>"
5+
echo "usage: $0 [-h][-c]"
66
echo
7-
echo ' -v version should be formatted <major>-<minor>-<patch>'
8-
echo ' -c <nodemcu-repo> optional abs path to custom nodemcu-firmware repo'
7+
echo ' -h displays usage information'
8+
echo ' -c perform a clean build'
99
echo
10-
echo 'Execute from the root dir of the repo. Upon successful completion the release will be created in firmware/releases/<version>'
11-
echo " ex: '$0 2-3-0'"
10+
echo 'Execute from the root dir of the repo. Upon successful completion the firmware image will be build/konnected-pro-fw_*'
11+
echo " ex: '$0'"
1212
exit 1
1313
}
14-
if [ "$#" == 0 ]; then
15-
usage
16-
fi
1714

1815
set -e
1916

17+
if ! git describe --exact-match --tags &> /dev/null; then
18+
BRANCH="$(git branch | sed -n -e 's/^\* \(.*\)/\1/p')"
19+
else
20+
BRANCH="$(git describe --exact-match --tags $(git log -n1 --pretty='%h'))"
21+
fi
22+
2023
FW_TAG="3.0.0-release_20210201"
21-
VERSION="$1"
2224
FIRMWARE_OVERLAY_PATH="firmware/nodemcu-firmware-overlay"
23-
2425
LFS_BASE="${PWD}/src/lfs"
2526
OUTPUT_BASE="${PWD}/build"
2627
OUTPUT_APP="${OUTPUT_BASE}/app"
27-
FW_BASE="${OUTPUT_BASE}/nodemcu-firmware"
28+
FW_BASE="${PWD}/nodemcu-firmware"
29+
IMAGE_NAME="konnected-firmware-${BRANCH}"
2830

2931
# process cmd args
30-
while getopts "v:c:h" opt; do
31-
case ${opt} in
32-
h )
33-
usage
34-
;;
35-
c )
36-
FW_BASE="$OPTARG"
37-
FW_TAG=''
38-
echo "using custom nodemcu-repo ${FW_BASE}"
39-
;;
40-
v )
41-
VERSION="$OPTARG"
42-
if ! [[ "${VERSION}" =~ ^[0-9]+-[0-9]+-[0-9]+ ]] ; then
43-
echo "Version not formatted correctly..."
44-
usage
45-
fi
46-
echo "tagging as version ${VERSION}"
47-
;;
48-
\? )
49-
echo "Invalid Option: -$OPTARG" 1>&2
50-
exit 1
51-
;;
52-
esac
32+
while getopts "ch" opt; do
33+
case ${opt} in
34+
h )
35+
usage
36+
;;
37+
c )
38+
cd "${FW_BASE}"
39+
make clean
40+
cd -
41+
# copy in our config overlay
42+
cp -rf "${FIRMWARE_OVERLAY_PATH}"/* "${FW_BASE}/"
43+
rm -f "${FW_BASE}/local/fs/*"
44+
echo "performing clean build"
45+
;;
46+
\? )
47+
echo "Invalid Option: -$OPTARG" 1>&2
48+
exit 1
49+
;;
50+
esac
5351
done
5452

55-
BUILD_PATH="firmware/builds/${VERSION}"
56-
IMAGE_NAME="konnected-firmware-${VERSION}"
57-
58-
rm -rf ${OUTPUT_BASE}
53+
rm -rf "${OUTPUT_BASE}"
5954
mkdir -p "${OUTPUT_BASE}";
6055
chown -R $USER "${OUTPUT_BASE}"
6156

62-
# pull and update the FW repo if none is supplied
63-
if [ ! -z "$FW_TAG" ] ; then
57+
# pull and update the FW repo if not already existing
58+
if [ ! -d "$FW_BASE" ] ; then
6459
echo "Checking out nodemcu-firmware:${FW_TAG}"
6560
git clone --recurse-submodules https://github.com/nodemcu/nodemcu-firmware.git \
6661
"${FW_BASE}"
62+
# Copy firmware configuration from this repository to the nodemcu-firmware repo
63+
cp -rf "${FIRMWARE_OVERLAY_PATH}"/* "${FW_BASE}/"
64+
rm -f "${FW_BASE}/local/fs/*"
6765
cd "${FW_BASE}"
6866
git checkout "${FW_TAG}"
6967
git submodule update --recursive
7068
cd -
7169
fi
7270

73-
# Copy firmware configuration from this repository to the nodemcu-firmware repo
74-
cp -rf "${FIRMWARE_OVERLAY_PATH}"/* "${FW_BASE}/"
75-
76-
rm -f "${FW_BASE}/local/fs/*"
71+
# Build it
72+
cd "${FW_BASE}"
73+
make
74+
srec_cat -output "${OUTPUT_BASE}/${IMAGE_NAME}.bin" -binary bin/0x00000.bin -binary -fill 0xff 0x00000 0x10000 bin/0x10000.bin -binary -offset 0x10000
75+
cd -
7776

78-
# Build NodeMCU firmware image
79-
docker run -e "IMAGE_NAME=${IMAGE_NAME}" \
80-
--rm -ti -v "${FW_BASE}:/opt/nodemcu-firmware" marcelstoer/nodemcu-build build
77+
# Build spiffsimg tool (manual for now because of https://github.com/nodemcu/nodemcu-firmware/issues/2943)
78+
make -C "${FW_BASE}"/tools/spiffsimg
8179

8280
# Build LFS image with application files
83-
LUA_FILES=$(find ${LFS_BASE} -iname "*.lua" | sed "s|${LFS_BASE}|/lfs|g")
81+
mkdir -p "${OUTPUT_APP}"
82+
LUA_FILES=$(find ${LFS_BASE} -iname "*.lua")
8483

8584
echo Adding files to LFS:
8685
echo ${LUA_FILES}
87-
docker run --rm -ti \
88-
-v ${FW_BASE}:/nodemcu-firmware \
89-
-v ${OUTPUT_APP}:/build \
90-
-v ${LFS_BASE}:/lfs \
91-
marcelstoer/nodemcu-build /nodemcu-firmware/luac.cross.int -m 0x40000 -f -o "/build/lfs.img" ${LUA_FILES}
9286
echo
87+
"${FW_BASE}/luac.cross.int" -m 0x40000 -f -o "${OUTPUT_APP}/lfs.img" ${LUA_FILES}
9388

9489
# transfer source files not in the lfs to the build output (ignore the status)
9590
cp -f ${LFS_BASE}/../* "${OUTPUT_APP}" 2>/dev/null || true; echo "Building SPIFFS..."
9691

97-
# Build spiffsimg tool (manual for now because of https://github.com/nodemcu/nodemcu-firmware/issues/2943)
98-
docker run --rm -ti \
99-
-v ${FW_BASE}:/nodemcu-firmware \
100-
marcelstoer/nodemcu-build make -C /nodemcu-firmware/tools/spiffsimg
92+
# create the fs list from /build/app
93+
rm -rf "${OUTPUT_BASE}/spiffs.lst"
94+
find "${OUTPUT_APP}" -type f | sed "s|${OUTPUT_APP}/||" | awk '{printf "import build/app/%s %s\n", $1, $1}' > "${OUTPUT_BASE}/spiffs.lst"
10195

102-
# Create SPIFFS image
103-
docker run \
104-
--rm -ti -v "${FW_BASE}:/opt/nodemcu-firmware" \
105-
-v "${PWD}/scripts:/scripts" \
106-
-v "${OUTPUT_APP}:/opt/lua" marcelstoer/nodemcu-build bash /scripts/build-spiffs
96+
echo Adding files to spiffs:
97+
cat "${OUTPUT_BASE}/spiffs.lst"
98+
echo
10799

108-
# Create or clear the output directory and transfer files
109-
mkdir -p "${BUILD_PATH}"
110-
rm -rf "${BUILD_PATH}/*"
111-
cp "${FW_BASE}/bin/nodemcu_integer_${IMAGE_NAME}.bin" "${BUILD_PATH}/${IMAGE_NAME}.bin"
112-
cp "${FW_BASE}/bin/konnected-filesystem-0x100000.img" "${BUILD_PATH}/konnected-filesystem-0x100000-${VERSION}.img"
100+
"${FW_BASE}/tools/spiffsimg/spiffsimg" -f "${OUTPUT_BASE}/konnected-filesystem-0x100000-${BRANCH}.img" -S 10m -U 0x100000 -r "${OUTPUT_BASE}/spiffs.lst" -d
113101

114-
srec_cat -output "${BUILD_PATH}/konnected-esp8266-${VERSION}.bin" -binary "${BUILD_PATH}/${IMAGE_NAME}.bin" -binary -fill 0xff 0x0000 0x100000 "${BUILD_PATH}/konnected-filesystem-0x100000-${VERSION}.img" -binary -offset 0x100000
102+
# Transfer firmware file
103+
#cp "${FW_BASE}/bin/nodemcu_integer_${IMAGE_NAME}.bin" "${OUTPUT_BASE}/${IMAGE_NAME}.bin"
104+
105+
# Build full image
106+
srec_cat -output "${OUTPUT_BASE}/konnected-esp8266-${BRANCH}.bin" -binary "${OUTPUT_BASE}/${IMAGE_NAME}.bin" -binary -fill 0xff 0x0000 0x100000 "${OUTPUT_BASE}/konnected-filesystem-0x100000-${BRANCH}.img" -binary -offset 0x100000
115107

116108
# Copy to common name for ease of flashing
117-
cp "${BUILD_PATH}/konnected-esp8266-${VERSION}.bin" "${BUILD_PATH}/../konnected-esp8266-latest.bin"
109+
cp "${OUTPUT_BASE}/konnected-esp8266-${BRANCH}.bin" "${OUTPUT_BASE}/konnected-esp8266-latest.bin"
110+
111+
echo "Build Complete: Flash this build with './scripts/flash ${BRANCH} <port>'"
118112

119-
echo "Build Complete: Flash this build with './scripts/flash ${VERSION} <port>'"

‎scripts/flash

+1-6
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,8 @@ if [ "$#" == 0 ]; then
1515
fi
1616

1717
VERSION="$1"
18-
if ! [[ "${VERSION}" =~ ^[0-9]+-[0-9]+-[0-9]+ ]] ; then
19-
echo "Version not formatted correctly..."
20-
usage
21-
fi
22-
2318
PORT=${2:-/dev/cu.wchusbserial1410}
24-
BUILD_PATH="firmware/builds/${VERSION}"
19+
BUILD_PATH=build
2520

2621
if [ ! -d "${BUILD_PATH}" ] ; then
2722
echo "No matching build version - trying releases"

0 commit comments

Comments
 (0)