TSC Meeting (10 Oct 2023) - Fixing builds, Upstream dependencies, stackstorm.com, Python difficulties, new OS support

[arm4b]

October 2023 @StackStorm/tsc 1 hour meeting:

• Tuesday, 10 October 2023, 09:30 AM US Pacific / 06:30 PM EU CET
• See StackStorm TSC Meetings #33 for the schedule and how to join
• TLDR; Jitsi link: https://meet.jit.si/StackStormTSC

Meeting Agenda

Fixing the 🔴 broken st2 builds (help wanted, needs volunteers) (10mins)

WIP: Let's fix the builds (Updated 06 Nov):

• st2
  • Use codecov-cli over python-codecov st2#6036 - thanks to @FileMagic
  • Fix Orquesta Integration Test Codecov Upload st2#6046 - thanks to @FileMagic
  • CI: Update ruby 2.4 -> 2.6 to fix CircleCI Packagecloud deploy step st2#6045
  • Remove unused codecov for Orquesta integration tests st2#6048
• orquesta
  • Fix CI Python 3.6 not available on ubuntu-latest github workflow orquesta#257 - thanks @amanda11 @nzlosh
• st2-packages
  • CI: Update ruby 2.4 -> 2.6 ro fix PackageCloud deploy step st2-packages#729
• ansible-st2
  • Fix e2e tests, drop Ubuntu18 ansible-st2#332 - thanks to @setswei
• puppet-st2
  • TBD by @bishopbm1
• docker
  • Change BATS image st2-docker#259 by @ZoeLeah
• k8s
  • Update K8s to latest version stackstorm-k8s#379 by @zoeLeah
• e2e-tests
  • Fix win e2e test - update the location for WinRM powershell setup script st2cd#473
  • https://github.com/StackStorm/ops-infra/pull/424
  • Switch to MongoDB 4.4 fo U18 st2-packages#730
• st2web
  • Fix build, pin lerna version st2web#1008 - thanks @guzzijones

Updating StackStorm dependencies and upstream CVEs (10mins)

The StackStorm's upstream dependencies need updating. Multiple projects : st2, st2chatops, st2web, orquesta, OS-level dependencies (docker). Identify dependencies to bump, update. Do we need to release a quick patch v3.8.1 afterwards before committing to a more involved v3.9.0?

WIP: Let's fix security (Updated 06 Nov):

• orquesta
  • Allow networkx<3 for Python 3.7 or newer orquesta#255
  • Reduce jsonschema requirements to allow 3.2 orquesta#251
  • Prepare for orquesta v1.6.0 release orquesta#258
• st2 (WIP)
  • Update Orquesta to v1.6.0 st2#6050
  • Update cryptography and pyOpenSSL (security) st2#6055
  • Fix EL7 build for Orquesta v1.6.0 update with 'python_version' markers st2-packages#728
  • Bump eventlet to fix setting SSLContext minimum_version property that results in RecursionErrors st2#6061
  • Update requests to fix CVEs (security) st2#6062
• st2web
  • Update dependencies st2web#1009 thanks @enykeev
  • Update Node version to current st2web#1010 thanks @enykeev
  • Update additional dependencies st2web#1020 @enykeev
• st2chatops
  • !HELP WANTED! - Regenerate npm-shrinkwrap to up-to-date dependencies (security) st2chatops#133

Related: StackStorm project security initiatives/ideas/problems are tracked in a separated Github project: https://github.com/orgs/StackStorm/projects/25. Security-interested folks, driving the coordination on these topics would be welcome. Some of the security hardening topics were raised by Scott and Haven in the last meeting #124.

OS Support and Proposal to use Amazon Linux 2 as supported OS (10mins)

See Add ability to use AL2 as base OS for Stackstorm (ST2) #6016. What's involved to add a new OS? What could be the blockers? @khushboobhatia01 able to assist on this? (the original issue was raised by her colleague?)

Other OS-level support topics:

• Ubuntu 22.04 LTS support
  • Python 3.10
• CentOS Stream 9 support
  • Python 3.9?

Using Eleventy vs Hugo for stackstorm.com website templating (10mins)

We've migrated from Wordpress to GH static pages (Migrate StackStorm blog and CMS to GH Static Pages #76) for better security, allow community to add new content (blogs, website updates) via PR and TSC reviews/approval. It works as https://github.com/stackStorm/stackstorm.com -> stackstorm.com, but uses raw HTML which is harder to manage. Having a template engine for static HTML generation would be better (Hugo template for stackstorm.com #4, Enable Netlify integration (preview) #3) and these tasks were not finished.
Dale Smith proposed to Restart Static Site Migration Using Eleventy and CloudCannon #123.

TLDR; Need to decide on the framework used for generating the stackstorn.com website from templates. Hugo - more popularity, experience by the TSC members (bus factor, maintenance burden), heavily used in DevOps/SRE repos for website generation (ex: Docker, Kubernetes). Eleventy, - Javascript/npm tool (we're moving away from Javascript where possible #8), but Dale has experience with it and can help with the migration.

Who is familiar with Evelenty or Hugo in the TSC and can assist/pair w Dale?

Future Python version (3.10) support issues, Pants builds (10mins)

See #103 as we're blocked to add new releases, new OSes having difficulties adding newer Python versions.
@carlos @amanda11 @cognifloyd to update on where we are, the direction and what kind of help is needed.
Community assistance wanted with adding the support for python v3.10.

• WIP:
  • update builtin collections imports to be forward-compatible with Python3.10+ orquesta#253 by @AndroxxTraxxon
  • Add Jammy and CentOS Stream 9 st2packaging-dockerfiles#111 docker builds experiments by @mamercad

[mickmcgrath13]

I'm excited to announce that we're doing a StackStorm training at the end of the month:
https://www.bitovi.com/events/stackstorm-training

[nzlosh]

Minutes for the meeting are now available.

[FileMagic]

I saw that you were looking for the build of the st2 project to be fixed. while reading through @guzzijones's PR from StackStorm/st2 #5995, that they fixed the tests in that PR. We could look at those updates for those changes.

[dzimine]

as agreed I'll shadow @dalesmith as he uses Evelenty to update StackStorm.com per #123.

[arm4b]

@FileMagic Thanks for looking at that! Replied in your Slack thread.
Also, if there are CI fixes that are part of that PR, - makes sense to extract them first. Eg. fix the working master branch first thing.

[setswei]

Hey, I logged a PR to fix the builds on ansible-st2. Let me know if you need anything else. I added some details to the PR as to what was wrong.

[arm4b]

Very helpful, thanks a lot @setswei