Skip to content

Commit

Permalink
Merge branch 'master' into tag_str
Browse files Browse the repository at this point in the history
  • Loading branch information
lhstrh authored Nov 5, 2023
2 parents c20e090 + 63a3d9c commit 05ef99e
Show file tree
Hide file tree
Showing 1,371 changed files with 30,251 additions and 18,583 deletions.
38 changes: 38 additions & 0 deletions .github/actions/latest-release/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Latest release
description: Report the latest release of the current repo
outputs:
ref:
value: ${{ steps.find.outputs.ref }}
description: The latest semver tag
ver:
value: ${{ steps.find.outputs.ver }}
description: The semver corresponding to the latest semver tag

runs:
using: "composite"
steps:
- name: Install semver-tool
run: |
wget -O /usr/local/bin/semver https://raw.githubusercontent.com/fsaintjacques/semver-tool/master/src/semver
chmod +x /usr/local/bin/semver
semver --version
shell: bash
- name: Fetch all tags
run: git fetch --all --tags
shell: bash
- name: Fetch latest-release script
run: |
wget https://raw.githubusercontent.com/lf-lang/lingua-franca/master/.github/scripts/latest-release.sh
chmod +x latest-release.sh
shell: bash
- name: Find the latest release
id: find
run: |
export tag=$(./latest-release.sh)
echo "ref=${tag}" >> $GITHUB_OUTPUT
shopt -s extglob
export ver="${tag##v}"
echo "ver=${ver}" >> $GITHUB_OUTPUT
echo "Latest release tag: ${tag}"
echo "Without a leading 'v': ${ver}"
shell: bash
60 changes: 23 additions & 37 deletions .github/actions/prepare-build-env/action.yml
Original file line number Diff line number Diff line change
@@ -1,42 +1,28 @@
name: Prepare build environment (Linux only)
description: Set up Java, Maven, Gradle, etc.
name: Set up build environment
description: Set up Java and Gradle (including caching).

inputs:
lingua-franca-dir:
description: 'Path to the lingua-franca directory'
required: false

runs:
using: "composite"
steps:
- name: Set up Java 17
run: |
echo "$JAVA_HOME_17_X64/bin" >> $GITHUB_PATH
echo "org.gradle.java.home=${JAVA_HOME_17_X64//\\/\/}" >> gradle.properties
echo "JAVA_HOME=$JAVA_HOME_17_X64" >> $GITHUB_ENV
shell: bash
- name: Check settings
run: |
echo $(which java)
cat gradle.properties
echo $JAVA_HOME
shell: bash
- name: Create hash of Gradle configuration (macOS only)
run: |
echo "gradle-hash"="$(find . -type f \( -name "gradle.properties" -o -name "gradle-wrapper.properties" \) -exec cat {} + | shasum -a 256 | cut -d ' ' -f 1)" >> $GITHUB_ENV
if: ${{ runner.os == 'macOS' }}
shell: bash
- name: Create hash of Gradle configuration (Linux and Windows only)
run: |
echo "gradle-hash"="$(find . -type f \( -name "gradle.properties" -o -name "gradle-wrapper.properties" \) -exec cat {} + | sha256sum | cut -d ' ' -f 1)" >> $GITHUB_ENV
if: ${{ runner.os == 'Windows' || runner.os == 'Linux' }}
shell: bash
- name: Cache
uses: actions/cache@v3
- uses: actions/setup-java@v3
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: gradle-${{ runner.os }}-${{ env.gradle-hash }}
# restore-keys: |
# ${{ runner.os }}-gradle-
- name: Bring down Gradle daemon (Windows)
uses: webiny/[email protected]
id: post-run-command
distribution: temurin
java-version: 17
- name: Gradle Build Action
uses: gradle/[email protected]
with:
run: ./gradlew --stop
if: ${{ runner.os == 'Windows' }}
cache-read-only: false
- name: Download Gradle and print version
working-directory: ${{ inputs.lingua-franca-dir }}
run: |
# Retry 3 times before the steps actually fails
(echo "==== Gradle Download Attempt: 1 ====" && ./gradlew --version) || \
(sleep 30 && echo "==== Gradle Download Attempt: 2 ====" && ./gradlew --version) || \
(sleep 30 && echo "==== Gradle Download Attempt: 3 ====" && ./gradlew --version) || \
(echo "==== Gradle Download Failed ====" && exit 1)
shell: bash
44 changes: 44 additions & 0 deletions .github/actions/report-code-coverage/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Report code coverage to CodeCov
inputs:
files:
description: 'list of coverage files to send to CodeCov'
required: true
runs:
using: "composite"
steps:
- name: Report code coverage to CodeCov (1st try)
id: codecov1
uses: codecov/[email protected]
with:
files: ${{ inputs.files }}
fail_ci_if_error: true
verbose: true
token: 18fd5ab8-d6ba-4f5d-b0f4-7c26340ab98c
continue-on-error: true
- name: Wait 20 seconds
run: sleep 20
shell: bash
if: steps.codecov1.outcome == 'failure'
- name: Report code coverage to CodeCov (2nd try)
id: codecov2
uses: codecov/[email protected]
with:
files: ${{ inputs.files }}
fail_ci_if_error: true
verbose: true
token: 18fd5ab8-d6ba-4f5d-b0f4-7c26340ab98c
continue-on-error: true
if: steps.codecov1.outcome == 'failure'
- name: Wait 20 seconds
run: sleep 20
shell: bash
if: steps.codecov2.outcome == 'failure'
- name: Report code coverage to CodeCov (3rd try)
id: codecov3
uses: codecov/[email protected]
with:
files: ${{ inputs.files }}
fail_ci_if_error: true
verbose: true
token: 18fd5ab8-d6ba-4f5d-b0f4-7c26340ab98c
if: steps.codecov2.outcome == 'failure'
34 changes: 34 additions & 0 deletions .github/actions/setup-francabot/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: Load global configuration settings for francabot
description: Set up author information and GPG signature
author: Marten Lohstroh <[email protected]>

inputs:
gpg-key:
description: 'francabot GPG key'
required: true
gpg-passphrase:
description: 'francabot GPG passphrase'
required: true

runs:
using: composite
steps:
- name: Set environment variables
run: |
echo "username=lingua-franca[bot]" >> "$GITHUB_ENV"
echo "[email protected]" >> "$GITHUB_ENV"
echo "user-and-email=lingua-franca[bot] <[email protected]>" >> "$GITHUB_ENV"
shell: bash
- name: Configure git username and email
run: |
git config --global user.name '${{ env.username }}'
git config --global user.email '${{ env.email }}'
shell: bash
- name: Import GPG key
uses: crazy-max/ghaction-import-gpg@v5
with:
gpg_private_key: ${{ inputs.gpg-key }}
passphrase: ${{ inputs.gpg-passphrase }}
git_config_global: true
git_user_signingkey: true
git_commit_gpgsign: true
13 changes: 6 additions & 7 deletions .github/actions/setup-zephyr/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,7 @@ runs:
steps:
- name: Setup environment variables
run: |
echo "SDK_VERSION=0.16.1" >> $GITHUB_ENV
echo "ZEPHYR_VERSION=3.3.0" >> $GITHUB_ENV
echo "SDK_VERSION=0.16.3" >> $GITHUB_ENV
shell: bash
- name: Dependencies
run: |
Expand All @@ -27,14 +26,14 @@ runs:
cd "/opt/zephyr-sdk-${{env.SDK_VERSION}}"
sudo ./setup.sh -t all -h -c
shell: bash
- name: Download and install Zephyr RTOS
- name: Download and install Zephyr RTOS with the lf-west-template
run: |
cd $HOME
west init zephyrproject --mr "v${{env.ZEPHYR_VERSION}}"
cd zephyrproject
git clone https://github.com/lf-lang/lf-west-template && cd lf-west-template
west update
west zephyr-export
pip install -r zephyr/scripts/requirements.txt
echo "ZEPHYR_BASE=$HOME/zephyrproject/zephyr" >> $GITHUB_ENV
pip install -r deps/zephyr/scripts/requirements.txt
echo "LF_WEST_TEMPLATE_BASE=$HOME/lf-west-template" >> $GITHUB_ENV
echo "ZEPHYR_BASE=$HOME/lf-west-template/deps/zephyr" >> $GITHUB_ENV
echo "ZEPHYR_SDK_INSTALL_DIR=/opt/zephyr-sdk-${{env.SDK_VERSION}}/" >> $GITHUB_ENV
shell: bash
8 changes: 5 additions & 3 deletions .github/scripts/check-diff.sh
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
#!/bin/bash

changes() {
git diff --name-only HEAD $(git merge-base HEAD origin/master)
}

if changes | grep -q $1; then
echo "CHANGED_$2=1" >> $GITHUB_OUTPUT
if changes | grep "$1" | grep -q -v "^.*md\|txt$"; then
echo "changed_$2=1" >> $GITHUB_OUTPUT
else
echo "CHANGED_$2=0" >> $GITHUB_OUTPUT
echo "changed_$2=0" >> $GITHUB_OUTPUT
fi
37 changes: 33 additions & 4 deletions util/RunZephyrTests.sh → .github/scripts/run-zephyr-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ num_successes=0
num_failures=0
failed_tests=""

# Skip
skip=("FileReader" "FilePkgReader" "Tracing" "ThreadedThreaded")
# Skip tests doing file IO and tracing
skip=("FileReader" "FilePkgReader" "Tracing" "ThreadedThreaded" "CountTest" "AsyncCallback")

find_kconfig_folders() {
if [ -f "$folder/CMakeLists.txt" ]; then
Expand All @@ -20,7 +20,7 @@ find_kconfig_folders() {
echo "Skipping: $test_name"
else
echo "Running: $test_name"
if run_zephyr_test "$folder"; then
if run_qemu_zephyr_test "$folder"; then
echo "Test $test_name successful"
let "num_successes+=1"
else
Expand All @@ -38,11 +38,40 @@ find_kconfig_folders() {
find_kconfig_folders "$folder"
fi
done
}

run_native_zephyr_test() {
return_val=0
pushd $1/build

rm -f res.text

timeout 60s make run | tee res.txt
result=$?

if [ $result -eq 0 ]; then
echo "Command completed within the timeout."
return_val=0
else
echo "Command terminated or timed out."
echo "Test output:"
echo "----------------------------------------------------------------"
cat res.txt
echo "----------------------------------------------------------------"
return_val=1
fi

popd
return "$return_val"




}

# Run Zephyr test until either: Timeout or finds match in output
# https://www.unix.com/shell-programming-and-scripting/171401-kill-process-if-grep-match-found.html
run_zephyr_test() {
run_qemu_zephyr_test() {
success=false
pushd $1/build

Expand Down
11 changes: 8 additions & 3 deletions .github/scripts/test-lfc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,12 @@ function test_with_links() {
foo/bar/baz/link-${1} --help
}

bin/lfc test/C/src/Minimal.lf
bin/lfc-dev test/C/src/Minimal.lf

# Ensure that lfc is robust to symbolic links.
test_with_links "lfc"
# Ensure that lfc can be invoked via symbolic links.
test_with_links "lfc-dev"

# Ensure that lfc can be invoked from outside the root directory.
cd bin
./lfc-dev --help
cd ..
25 changes: 25 additions & 0 deletions .github/scripts/test-lfd.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/usr/bin/env bash

# Exit 1 if any command returns with a non-zero exit code.
set -euo pipefail

cd $GITHUB_WORKSPACE

function test_with_links() {
rm -rf foo
mkdir -p foo/bar/baz
ln -s ../bin/${1} foo/link-foo
ln -s ../link-foo foo/bar/link-bar
ln -s ../link-bar foo/bar/baz/link-${1}
foo/bar/baz/link-${1} --help
}

bin/lfd-dev test/C/src/Minimal.lf

# Ensure that lfd can be invoked via symbolic links.
test_with_links "lfd-dev"

# Ensure that lfd can be invoked from outside the root directory.
cd bin
./lfd-dev --help
cd ..
21 changes: 13 additions & 8 deletions .github/scripts/test-lff.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,19 @@ function test_with_links() {
}

# just a couple of smoke tests
bin/lff --help
bin/lff --version
bin/lff-dev --help
bin/lff-dev --version

bin/lff -d test/C/src/Minimal.lf
bin/lff --dry-run test/Cpp/src/Minimal.lf
bin/lff-dev -d test/C/src/Minimal.lf
bin/lff-dev --dry-run test/Cpp/src/Minimal.lf

bin/lff -d test/C/src/Minimal.lf
bin/lff --dry-run test/Cpp/src/Minimal.lf
bin/lff-dev -d test/C/src/Minimal.lf
bin/lff-dev --dry-run test/Cpp/src/Minimal.lf

# Ensure that lff is robust to symbolic links.
test_with_links "lff"
# Ensure that lff can be invoked via symbolic links.
test_with_links "lff-dev"

# Ensure that lfc can be invoked from outside the root directory.
cd bin
./lff-dev --help
cd ..
3 changes: 1 addition & 2 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
# Lingua Franca workflows

## Continuous Integration
The main CI configuration can be found in [ci.yml](https://github.com/lf-lang/lingua-franca/blob/master/.github/workflows/ci.yml) and gets triggered by pushes to `master` and by pushes to branches involved in an open pull request.
The main two CI configurations can be found in [all-misc.yml](https://github.com/lf-lang/lingua-franca/blob/master/.github/workflows/all-misc.yml) and [all-targets.yml](https://github.com/lf-lang/lingua-franca/blob/master/.github/workflows/all-targets.yml). The [all-misc.yml](https://github.com/lf-lang/lingua-franca/blob/master/.github/workflows/all-misc.yml) workflow runs the non-target specific tests, and [all-targets.yml](https://github.com/lf-lang/lingua-franca/blob/master/.github/workflows/all-targets.yml) workflow runs the target specific tests. They get triggered by pushes to `master` and by pushes to branches involved in an open pull request.

**NOTE: [ci.yml](https://github.com/lf-lang/lingua-franca/blob/master/.github/workflows/ci.yml) references these workflows with respect to master (signified by the "@master" suffix). If you edit a workflow and want your changes reflected in the CI run for your pull request, then make sure that the workflow of your feature branch gets invoked instead of the one on master (and change back to "@master" before merging so that the feature branch can be safely deleted).**

### Benchmark tests
The [benchmark-tests.yml](https://github.com/lf-lang/lingua-franca/blob/master/.github/workflows/lfc-tests.yml) workflow compiles and runs benchmarks for a given target. The purpose of this workflow is not to gather performance results but to ensure that the benchmark programs remain functional. This workflow has one (optional) argument:
Expand Down
Loading

0 comments on commit 05ef99e

Please sign in to comment.