Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: AWS SQS Example #297

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions aws-sqs-node/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
AWS_ACCOUNT_ID=yourAccountId
AWS_REGION=us-east-1
QUEUE_NAME=somequeue.fifo
AWS_ACCESS_KEY_ID=somekeyId
AWS_SECRET_ACCESS_KEY=somekey
3 changes: 3 additions & 0 deletions aws-sqs-node/.eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
node_modules
lib
.eslintrc.js
48 changes: 48 additions & 0 deletions aws-sqs-node/.eslintrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
const { builtinModules } = require('module');

const ALLOWED_NODE_BUILTINS = new Set(['assert']);

module.exports = {
root: true,
parser: '@typescript-eslint/parser',
parserOptions: {
project: './tsconfig.json',
tsconfigRootDir: __dirname,
},
plugins: ['@typescript-eslint', 'deprecation'],
extends: [
'eslint:recommended',
'plugin:@typescript-eslint/eslint-recommended',
'plugin:@typescript-eslint/recommended',
'prettier',
],
rules: {
// recommended for safety
'@typescript-eslint/no-floating-promises': 'error', // forgetting to await Activities and Workflow APIs is bad
'deprecation/deprecation': 'warn',

// code style preference
'object-shorthand': ['error', 'always'],

// relaxed rules, for convenience
'@typescript-eslint/no-unused-vars': [
'warn',
{
argsIgnorePattern: '^_',
varsIgnorePattern: '^_',
},
],
'@typescript-eslint/no-explicit-any': 'off',
},
overrides: [
{
files: ['src/workflows.ts', 'src/workflows-*.ts', 'src/workflows/*.ts'],
rules: {
'no-restricted-imports': [
'error',
...builtinModules.filter((m) => !ALLOWED_NODE_BUILTINS.has(m)).flatMap((m) => [m, `node:${m}`]),
],
},
},
],
};
5 changes: 5 additions & 0 deletions aws-sqs-node/.prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"printWidth": 120,
"singleQuote": true,
"semi": false
}
27 changes: 27 additions & 0 deletions aws-sqs-node/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# AWS SQS Node.js Sample

This sample demonstrates how to use Temporal to process messages from an AWS SQS queue, all within a single container.
This example uses a FIFO based queue, but it can be modified to use a standard queue.

In essence, this code sets up a system where multiple processes are spawned. Some of these processes are responsible for polling an SQS queue and starting Temporal workflows based on the messages they receive. Another process is responsible for running a Temporal worker that processes these workflows. If any of these processes die, the primary process respawns them.

### Primary-Worker Architecture

The `run`` function sets up a primary-worker architecture using Node's cluster module.

If in the primary process:
- It determines the number of available CPUs.
- Spawns child processes based on the available CPUs. One of these processes is dedicated to running the Temporal worker, while the rest are for polling the SQS queue.
- Listens for child process exits and respawns them if needed.
If in a child process:
- Depending on the worker type, it either polls the SQS queue or runs the Temporal worker.

### Running this sample

1. `temporal server start-dev` to start [Temporal Server](https://github.com/temporalio/cli/#installation).
2. `pnpm install` to install dependencies (`npm install`).
3. rename `.env.example` to `.env` and fill in the values for your AWS account and queue (FIFO).
4. `npm run start:worker` to start the Worker.
5. In another shell, `npm run start:client` to send a sample SQS message.
6. use `:watch` for development

33 changes: 33 additions & 0 deletions aws-sqs-node/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
{
"name": "aws-sqs-example",
"version": "1.0.0",
"scripts": {
"start:worker": "tsx src/worker.ts | pino-pretty",
"start:client": "tsx src/client.ts | pino-pretty",
"start:worker:watch": "tsx watch src/worker.ts | pino-pretty",
"start:client:watch": "tsx watch src/client.ts | pino-pretty"
},
"dependencies": {
"@aws-sdk/client-sqs": "^3.427.0",
"@temporalio/activity": "^1.8.6",
"@temporalio/client": "^1.8.6",
"@temporalio/common": "^1.8.6",
"@temporalio/worker": "^1.8.6",
"@temporalio/workflow": "^1.8.6",
"dotenv": "^16.3.1",
"nanoid": "^5.0.1",
"pino": "^8.15.6",
"zod": "^3.22.4"
},
"devDependencies": {
"@typescript-eslint/eslint-plugin": "^6.7.4",
"@typescript-eslint/parser": "^6.7.4",
"eslint": "^8.51.0",
"eslint-config-prettier": "^9.0.0",
"eslint-plugin-deprecation": "^2.0.0",
"pino-pretty": "^10.2.3",
"prettier": "^3.0.3",
"tsx": "^3.13.0",
"typescript": "^5.2.2"
}
}
Loading
Loading