Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename WorkContext to ExeUnit #982

Merged
merged 20 commits into from
Jun 20, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
70c8ed0
refactor(work-context): rename WorkContext to ExeUnit
mgordel Jun 18, 2024
12a488f
docs: fixed typedoc link
mgordel Jun 18, 2024
31c4dcc
refactor: changed ctx to exe unit in logs
mgordel Jun 18, 2024
1fd6eac
feat: added setup and teardown functions as options to exe-unit
mgordel Jun 19, 2024
5fa2fb8
chore: sync beta
mgordel Jun 19, 2024
5c120ba
docs: added example of the setup and terdown functions
mgordel Jun 19, 2024
2c73ba0
test: fixed oneOf params in unit test
mgordel Jun 19, 2024
387aee4
docs: fixed oneOf params in example
mgordel Jun 19, 2024
1cd8f00
refactor(exe-unit): renamed `before` method to `setup` and added `tea…
mgordel Jun 19, 2024
9dda67d
docs: added typedoc comments for oneOf and manyOf
mgordel Jun 19, 2024
dee024e
docs: added example (rendering) of using the setup and teardown funct…
mgordel Jun 19, 2024
8b27838
test: fixed after removing fixtures
mgordel Jun 19, 2024
55fcaa5
chore: sync beta
mgordel Jun 20, 2024
80f0736
docs: removed rendering example
mgordel Jun 20, 2024
f65e0e0
docs: example for setup and teardown
SewerynKras Jun 20, 2024
413cde3
docs: document setup and teardown in README
SewerynKras Jun 20, 2024
d04e8f4
Merge pull request #986 from golemfactory/decryption-example
mgordel Jun 20, 2024
a9336a7
docs: use golem image in setup and teardown example
SewerynKras Jun 20, 2024
0ba718a
refactor: changed remaining `lease` names to `rental`
mgordel Jun 20, 2024
042c76b
Merge remote-tracking branch 'origin/mgordel/JST-984/exe-unit-rename'…
mgordel Jun 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/advanced/local-image/serveLocalGvmi.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ const getImagePath = (path: string) => new URL(path, import.meta.url).toString()
},
};

const lease = await glm.oneOf(order);
const lease = await glm.oneOf({ order });
// in our Dockerfile we have created a file called hello.txt, let's read it
const result = await lease
.getExeUnit()
Expand Down
2 changes: 1 addition & 1 deletion examples/advanced/override-module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ const order: MarketOrderSpec = {

try {
await glm.connect();
const lease = await glm.oneOf(order);
const lease = await glm.oneOf({ order });
await lease
.getExeUnit()
.then((exe) => exe.run("echo Hello, Golem! 👋"))
Expand Down
2 changes: 1 addition & 1 deletion examples/advanced/payment-filters.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ const order: MarketOrderSpec = {

try {
await glm.connect();
const lease = await glm.oneOf(order);
const lease = await glm.oneOf({ order });
await lease
.getExeUnit()
.then((exe) => exe.run("echo Hello, Golem! 👋"))
Expand Down
2 changes: 1 addition & 1 deletion examples/advanced/proposal-filter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ const order: MarketOrderSpec = {

try {
await glm.connect();
const lease = await glm.oneOf(order);
const lease = await glm.oneOf({ order });
await lease
.getExeUnit()
.then((exe) => exe.run(`echo [provider:${exe.provider.name}] Hello, Golem! 👋`))
Expand Down
2 changes: 1 addition & 1 deletion examples/advanced/proposal-predefined-filter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ const order: MarketOrderSpec = {

try {
await glm.connect();
const lease = await glm.oneOf(order);
const lease = await glm.oneOf({ order });
await lease
.getExeUnit()
.then((exe) => exe.run(`echo [provider:${exe.provider.name}] Hello, Golem! 👋`))
Expand Down
2 changes: 1 addition & 1 deletion examples/advanced/proposal-selector.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ const order: MarketOrderSpec = {

try {
await glm.connect();
const lease = await glm.oneOf(order);
const lease = await glm.oneOf({ order });
await lease
.getExeUnit()
.then((exe) => exe.run(`echo [provider:${exe.provider.name}] Hello, Golem! 👋`))
Expand Down
4 changes: 2 additions & 2 deletions examples/advanced/reuse-allocation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,8 @@ import { pinoPrettyLogger } from "@golem-sdk/pino-logger";
},
};

const lease1 = await glm.oneOf(firstOrder);
const lease2 = await glm.oneOf(secondOrder);
const lease1 = await glm.oneOf({ order: firstOrder });
const lease2 = await glm.oneOf({ order: secondOrder });

await lease1
.getExeUnit()
Expand Down
64 changes: 64 additions & 0 deletions examples/advanced/setup-and-teardown.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
import { MarketOrderSpec, GolemNetwork, LifecycleFunction } from "@golem-sdk/golem-js";
import { pinoPrettyLogger } from "@golem-sdk/pino-logger";

const order: MarketOrderSpec = {
demand: {
workload: { imageTag: "golem/alpine:latest" },
},
market: {
rentHours: 0.5,
pricing: {
model: "linear",
maxStartPrice: 0.5,
maxCpuPerHourPrice: 1.0,
maxEnvPerHourPrice: 0.5,
},
},
};

(async () => {
const glm = new GolemNetwork({
logger: pinoPrettyLogger({
level: "info",
}),
});

try {
await glm.connect();

const setup: LifecycleFunction = async (exe) =>
exe
.run(`echo This code is run on the start of the exe-unit ${exe.provider.name}`)
.then((res) => console.log(res.stdout));

const teardown: LifecycleFunction = async (exe) =>
exe
.run(`echo This code is run before the exe-unit ${exe.provider.name} is destroyed`)
.then((res) => console.log(res.stdout));

const pool = await glm.manyOf({
concurrency: 2,
order,
setup,
teardown,
});
await Promise.allSettled([
pool.withLease(async (lease) =>
lease
.getExeUnit()
.then((exe) => exe.run(`echo Hello, Golem from the first machine! 👋 ${exe.provider.name}`))
.then((res) => console.log(res.stdout)),
),
pool.withLease(async (lease) =>
lease
.getExeUnit()
.then((exe) => exe.run(`echo Hello, Golem from the second machine! 👋 ${exe.provider.name}`))
.then((res) => console.log(res.stdout)),
),
]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a better example would be to show that if you re-use a lease the setup and teardown are only executed once (for example make w pool with concurrency 2 and run 10 tasks on it)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or we could borrow the example from TE where we upload a file, add lines to it in every task function and then display the contents at the end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or maybe... an example with rendering... 🫣 intentionally change the name to rendering.ts, this is the original example where it is necessary to use the setup .. (?)
I added it for comparison - rendering.ts

} catch (err) {
console.error("Failed to run the example", err);
} finally {
await glm.disconnect();
}
})().catch(console.error);
6 changes: 3 additions & 3 deletions examples/advanced/step-by-step.ts
Original file line number Diff line number Diff line change
Expand Up @@ -136,11 +136,11 @@ import { filter, map, switchMap, take } from "rxjs";
// Start the computation
// First lets start the activity - this will deploy our image on the provider's machine
const activity = await glm.activity.createActivity(agreement);
// Then let's create a WorkContext, which is a set of utilities to interact with the
// Then let's create a ExeUnit, which is a set of utilities to interact with the
// providers machine, like running commands, uploading files, etc.
const ctx = await glm.activity.createWorkContext(activity);
const exe = await glm.activity.createExeUnit(activity);
// Now we can run a simple command on the provider's machine
const result = await ctx.run("echo Hello, Golem 👋!");
const result = await exe.run("echo Hello, Golem 👋!");
console.log("Result of the command ran on the provider's machine:", result.stdout);

// We're done, let's clean up
Expand Down
22 changes: 12 additions & 10 deletions examples/basic/events.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,16 +36,18 @@ import { pinoPrettyLogger } from "@golem-sdk/pino-logger";
});

const lease = await glm.oneOf({
demand: {
workload: { imageTag: "golem/alpine:latest" },
},
market: {
rentHours: 0.5,
pricing: {
model: "linear",
maxStartPrice: 0.5,
maxCpuPerHourPrice: 1.0,
maxEnvPerHourPrice: 0.5,
order: {
demand: {
workload: { imageTag: "golem/alpine:latest" },
},
market: {
rentHours: 0.5,
pricing: {
model: "linear",
maxStartPrice: 0.5,
maxCpuPerHourPrice: 1.0,
maxEnvPerHourPrice: 0.5,
},
},
},
});
Expand Down
2 changes: 1 addition & 1 deletion examples/basic/one-of.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ const order: MarketOrderSpec = {

try {
await glm.connect();
const lease = await glm.oneOf(order);
const lease = await glm.oneOf({ order });
await lease
.getExeUnit()
.then((exe) => exe.run("echo Hello, Golem! 👋"))
Expand Down
2 changes: 1 addition & 1 deletion examples/basic/run-and-stream.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ const order: MarketOrderSpec = {

try {
await glm.connect();
const lease = await glm.oneOf(order);
const lease = await glm.oneOf({ order });
const exe = await lease.getExeUnit();

const remoteProcess = await exe.runAndStream(
Expand Down
4 changes: 2 additions & 2 deletions examples/docs-examples/quickstarts/retrievable-task/task.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ const job = golem.createJob({
imageTag: "golem/alpine:latest",
},
});
job.startWork(async (ctx) => {
const response = await ctx.run("echo 'Hello, Golem!'");
job.startWork(async (exe) => {
const response = await exe.run("echo 'Hello, Golem!'");
return response.stdout;
});

Expand Down
4 changes: 2 additions & 2 deletions examples/experimental/express/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,9 @@ app.post("/tts", async (req, res) => {
console.log("Job succeeded", job.results);
});

job.startWork(async (ctx) => {
job.startWork(async (exe) => {
const fileName = `${Math.random().toString(36).slice(2)}.wav`;
await ctx
await exe
.beginBatch()
.run(`espeak "${req.body}" -w /golem/output/output.wav`)
.downloadFile("/golem/output/output.wav", `public/${fileName}`)
Expand Down
4 changes: 2 additions & 2 deletions examples/experimental/job/cancel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ async function main() {
console.log("Job ended event emitted");
});

job.startWork(async (ctx) => {
return String((await ctx.run("echo -n 'Hello Golem!'")).stdout);
job.startWork(async (exe) => {
return String((await exe.run("echo -n 'Hello Golem!'")).stdout);
});

console.log("Canceling job...");
Expand Down
4 changes: 2 additions & 2 deletions examples/experimental/job/getJobById.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ function startJob() {
console.log("Job ended event emitted");
});

job.startWork(async (ctx) => {
return String((await ctx.run("echo -n 'Hello Golem!'")).stdout);
job.startWork(async (exe) => {
return String((await exe.run("echo -n 'Hello Golem!'")).stdout);
});
return job.id;
}
Expand Down
4 changes: 2 additions & 2 deletions examples/experimental/job/waitForResults.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ async function main() {
console.log("Job ended event emitted");
});

job.startWork(async (ctx) => {
return String((await ctx.run("echo -n 'Hello Golem!'")).stdout);
job.startWork(async (exe) => {
return String((await exe.run("echo -n 'Hello Golem!'")).stdout);
});

const result = await job.waitForResult();
Expand Down
2 changes: 1 addition & 1 deletion examples/web/hello.html
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ <h3>Results</h3>
appendResults("Establishing a connection to the Golem Network");
await glm.connect();
appendResults("Request for leasing a provider machine");
const lease = await glm.oneOf(order);
const lease = await glm.oneOf({ order });
await lease
.getExeUnit()
.then(async (exe) =>
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
"docs": "typedoc src/ --plugin typedoc-plugin-merge-modules --plugin typedoc-theme-hierarchy",
"docs:md": "typedoc src/ --plugin typedoc-plugin-markdown --plugin .docs/typedoc-frontmatter-theme.cjs --hideBreadcrumbs true && node .docs/summary-generator.cjs",
"test": "npm run test:unit && npm run test:e2e",
"test:unit": "jest --config tests/unit/jest.config.json",
"test:unit": "jest --config tests/jest.config.json",
"test:e2e": "cross-env NODE_OPTIONS=--experimental-vm-modules jest --config tests/e2e/jest.config.json tests/e2e/**.spec.ts --runInBand --forceExit",
"test:cypress": "cypress run",
"test:examples": "tsx tests/examples/examples.test.ts",
Expand Down
26 changes: 13 additions & 13 deletions src/activity/activity.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { Agreement } from "../market/agreement";
import { Activity, IActivityApi, ActivityEvents, Result } from "./index";
import { defaultLogger } from "../shared/utils";
import { GolemServices } from "../golem-network/golem-network";
import { WorkContext, WorkOptions } from "./work";
import { ExeUnit, ExeUnitOptions } from "./exe-unit";
import { ExeScriptExecutor, ExeScriptRequest, ExecutionOptions } from "./exe-script-executor";
import { Observable, catchError, tap } from "rxjs";
import { StreamingBatchEvent } from "./results";
Expand Down Expand Up @@ -38,11 +38,11 @@ export interface ActivityModule {
findActivityById(activityId: string): Promise<Activity>;

/**
* Create a work context "within" the activity so that you can perform commands on the rented resources
* Create a exe-unit "within" the activity so that you can perform commands on the rented resources
*
* @return An WorkContext that's fully commissioned and the user can execute their commands
* @return An ExeUnit that's fully commissioned and the user can execute their commands
*/
createWorkContext(activity: Activity, options?: WorkOptions): Promise<WorkContext>;
createExeUnit(activity: Activity, options?: ExeUnitOptions): Promise<ExeUnit>;

/**
* Factory method for creating a script executor for the activity
Expand Down Expand Up @@ -274,30 +274,30 @@ export class ActivityModuleImpl implements ActivityModule {
return await this.activityApi.getActivity(activityId);
}

async createWorkContext(activity: Activity, options?: WorkOptions): Promise<WorkContext> {
this.logger.info("Creating work context for activity", { activityId: activity.id });
const ctx = new WorkContext(activity, this, {
async createExeUnit(activity: Activity, options?: ExeUnitOptions): Promise<ExeUnit> {
this.logger.info("Creating exe-unit for activity", { activityId: activity.id });
const exe = new ExeUnit(activity, this, {
yagnaOptions: this.services.yagna.yagnaOptions,
logger: this.logger.child("work-context"),
logger: this.logger.child("exe-unit"),
...options,
});

this.logger.debug("Initializing the exe-unit for activity", { activityId: activity.id });
try {
await ctx.before();
await exe.setup();
this.events.emit(
"workContextInitialized",
"exeUnitInitialized",
await this.refreshActivity(activity).catch(() => {
this.logger.warn("Failed to refresh activity after work context initialization", { activityId: activity.id });
return activity;
}),
);
return ctx;
return exe;
} catch (error) {
this.events.emit(
"errorInitializingWorkContext",
"errorInitializingExeUnit",
await this.refreshActivity(activity).catch(() => {
this.logger.warn("Failed to refresh activity after work context initialization error", {
this.logger.warn("Failed to refresh activity after exe-unit initialization error", {
activityId: activity.id,
});
return activity;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Activity, ActivityStateEnum, Agreement } from "../../src";
import { Activity, ActivityStateEnum, Agreement } from "../index";
import { instance, mock } from "@johanblumenberg/ts-mockito";

const mockAgreement = mock(Agreement);
Expand Down
4 changes: 2 additions & 2 deletions src/activity/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ export type ActivityEvents = {
activityDestroyed: (activity: Activity) => void;
errorDestroyingActivity: (activity: Activity, error: Error) => void;

workContextInitialized: (activity: Activity) => void;
errorInitializingWorkContext: (activity: Activity, error: Error) => void;
exeUnitInitialized: (activity: Activity) => void;
errorInitializingExeUnit: (activity: Activity, error: Error) => void;

activityStateChanged: (activity: Activity, previousState: ActivityStateEnum) => void;
errorRefreshingActivity: (activity: Activity, error: Error) => void;
Expand Down
4 changes: 2 additions & 2 deletions src/activity/exe-script-executor.test.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import { Activity } from "./activity";
import { _, anything, imock, instance, mock, reset, verify, when } from "@johanblumenberg/ts-mockito";
import { Capture, Deploy, DownloadFile, Run, Script, Start, Terminate, UploadFile } from "./script";
import { buildExeScriptSuccessResult } from "../../tests/unit/helpers";
import { GolemWorkError, WorkErrorCode } from "./work";
import { buildExeScriptSuccessResult } from "../../tests/utils/helpers";
import { GolemWorkError, WorkErrorCode } from "./exe-unit";
import { Logger, sleep } from "../shared/utils";
import { GolemAbortError, GolemError } from "../shared/error/golem-error";
import { ExeScriptExecutor } from "./exe-script-executor";
Expand Down
Loading
Loading