feat: support package nodes in the enclave builder ui #2250
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
adschwartz
approved these changes
Mar 4, 2024
Dartoxian
force-pushed
the
bgazzard/package-node
branch
from
e8878d8 to
43a2af7
Compare
github-merge-queue bot
pushed a commit
that referenced
this pull request
Mar 13, 2024
## Description:
This change implements the generation of a yaml that represents the
effect of a sequence of instructions on an enclave. The major changes
are as follows:
- Adds gRPC endpoints `GetStarlarkPackge/ScriptPlanYaml` to APIC and
Enclave Manager for returning this yaml
- Implements `PlanYaml` object and yaml generation logic in
`startosis_engine`
- Adds `UpdatePlan(plan *PlanYaml)` method to `KurtosisInstruction`
interface so each instruction implements logic for updating the plan
yaml
- Most of the knowledge needed to generate the yaml comes from the
interpretation phase and is simply passed into yaml generation logic
Tests are in `startosis_interpreter_plan_yaml_tests.go` and demonstrate
how the `InstructionsPlan` generates the yaml via the `PlanYaml` object.
eg. starlark script turned plan yaml:
```
def run(plan, hi_files_artifact):
service = plan.add_service(
name="db",
config=ServiceConfig(
image="postgres:latest",
env_vars={
"POSTGRES_DB": "kurtosis",
"POSTGRES_USER": "kurtosis",
"POSTGRES_PASSWORD": "kurtosis",
},
files = {
"/root": hi_files_artifact,
}
)
)
execResult = plan.exec(
service_name="db",
recipe=ExecRecipe(
command=["echo", service.ip_address + " " + service.hostname]
),
acceptable_codes=[0],
)
runShResult = plan.run_sh(
run="echo " + execResult["code"] + " " + execResult["output"],
)
plan.run_sh(
run="echo " + runShResult.code + " " + runShResult.output,
)
```
plan yaml:
```
packageId: DEFAULT_PACKAGE_ID_FOR_SCRIPT
services:
- uuid: "1"
name: db
image:
name: postgres:latest
envVars:
- key: POSTGRES_DB
value: kurtosis
- key: POSTGRES_PASSWORD
value: kurtosis
- key: POSTGRES_USER
value: kurtosis
files:
- mountPath: /root
filesArtifacts:
- uuid: "2"
name: hi-file
filesArtifacts:
- uuid: "2"
name: hi-file
tasks:
- uuid: "3"
taskType: exec
command:
- echo
- '{{ kurtosis.1.ip_address }} {{ kurtosis.1.hostname }}'
serviceName: db
acceptableCodes:
- 0
- uuid: "4"
taskType: sh
command:
- echo {{ kurtosis.3.code }} {{ kurtosis.3.output }}
image: badouralix/curl-jq
- uuid: "5"
taskType: sh
command:
- echo {{ kurtosis.4.code }} {{ kurtosis.4.output }}
image: badouralix/curl-jq
```
## Is this change user facing?
NO
## References:
The Enclave Manager uses this plan yaml to render packages in the Enclave Builder:
#2250
---------
Co-authored-by: Ben Gazzard <[email protected]>
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 9583343 | Triggered | Generic Password | 0d154ac | core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go | View secret |
| 9583343 | Triggered | Generic Password | 0d154ac | core/server/api_container/server/startosis_engine/plan.yml | View secret |
| 9583343 | Triggered | Generic Password | 0d154ac | core/server/api_container/server/startosis_engine/plan.yml | View secret |
| 9583343 | Triggered | Generic Password | 0d154ac | core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go | View secret |
| 9583343 | Triggered | Generic Password | 0d154ac | core/server/api_container/server/startosis_engine/startosis_interpreter_plan_yaml_test.go | View secret |
| 9583343 | Triggered | Generic Password | 0d154ac | core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go | View secret |
| 9966505 | Triggered | Generic Password | 0d154ac | core/server/api_container/server/startosis_engine/startosis_interpreter_plan_yaml_test.go | View secret |
| 9966505 | Triggered | Generic Password | 0d154ac | core/server/api_container/server/startosis_engine/startosis_interpreter_plan_yaml_test.go | View secret |
| 9966505 | Triggered | Generic Password | 0d154ac | core/server/api_container/server/startosis_engine/startosis_interpreter_plan_yaml_test.go | View secret |
| 9966505 | Triggered | Generic Password | 0d154ac | core/server/api_container/server/startosis_engine/startosis_interpreter_plan_yaml_test.go | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secrets safely. Learn here the best practices.
- Revoke and rotate these secrets.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
Our GitHub checks need improvements? Share your feedbacks!
tedim52
force-pushed
the
bgazzard/package-node
branch
from
3879f14 to
77cd134
Compare
github-merge-queue bot
pushed a commit
that referenced
this pull request
Mar 13, 2024
## Description THIS PR IS A COPY OF THIS PR: #2250 The frontend PR was initially made off the backend PR branch. I merged the backend PR branch to main. Then merged the frontend PR into the backend PR branch. Now merging into main again. --------- Co-authored-by: Ben Gazzard <[email protected]>
tedim52
added a commit
that referenced
this pull request
Mar 21, 2024
## Description:
This change implements the generation of a yaml that represents the
effect of a sequence of instructions on an enclave. The major changes
are as follows:
- Adds gRPC endpoints `GetStarlarkPackge/ScriptPlanYaml` to APIC and
Enclave Manager for returning this yaml
- Implements `PlanYaml` object and yaml generation logic in
`startosis_engine`
- Adds `UpdatePlan(plan *PlanYaml)` method to `KurtosisInstruction`
interface so each instruction implements logic for updating the plan
yaml
- Most of the knowledge needed to generate the yaml comes from the
interpretation phase and is simply passed into yaml generation logic
Tests are in `startosis_interpreter_plan_yaml_tests.go` and demonstrate
how the `InstructionsPlan` generates the yaml via the `PlanYaml` object.
eg. starlark script turned plan yaml:
```
def run(plan, hi_files_artifact):
service = plan.add_service(
name="db",
config=ServiceConfig(
image="postgres:latest",
env_vars={
"POSTGRES_DB": "kurtosis",
"POSTGRES_USER": "kurtosis",
"POSTGRES_PASSWORD": "kurtosis",
},
files = {
"/root": hi_files_artifact,
}
)
)
execResult = plan.exec(
service_name="db",
recipe=ExecRecipe(
command=["echo", service.ip_address + " " + service.hostname]
),
acceptable_codes=[0],
)
runShResult = plan.run_sh(
run="echo " + execResult["code"] + " " + execResult["output"],
)
plan.run_sh(
run="echo " + runShResult.code + " " + runShResult.output,
)
```
plan yaml:
```
packageId: DEFAULT_PACKAGE_ID_FOR_SCRIPT
services:
- uuid: "1"
name: db
image:
name: postgres:latest
envVars:
- key: POSTGRES_DB
value: kurtosis
- key: POSTGRES_PASSWORD
value: kurtosis
- key: POSTGRES_USER
value: kurtosis
files:
- mountPath: /root
filesArtifacts:
- uuid: "2"
name: hi-file
filesArtifacts:
- uuid: "2"
name: hi-file
tasks:
- uuid: "3"
taskType: exec
command:
- echo
- '{{ kurtosis.1.ip_address }} {{ kurtosis.1.hostname }}'
serviceName: db
acceptableCodes:
- 0
- uuid: "4"
taskType: sh
command:
- echo {{ kurtosis.3.code }} {{ kurtosis.3.output }}
image: badouralix/curl-jq
- uuid: "5"
taskType: sh
command:
- echo {{ kurtosis.4.code }} {{ kurtosis.4.output }}
image: badouralix/curl-jq
```
## Is this change user facing?
NO
## References:
The Enclave Manager uses this plan yaml to render packages in the Enclave Builder:
#2250
---------
Co-authored-by: Ben Gazzard <[email protected]>
tedim52
added a commit
that referenced
this pull request
Mar 21, 2024
## Description THIS PR IS A COPY OF THIS PR: #2250 The frontend PR was initially made off the backend PR branch. I merged the backend PR branch to main. Then merged the frontend PR into the backend PR branch. Now merging into main again. --------- Co-authored-by: Ben Gazzard <[email protected]>
tedim52
added a commit
that referenced
this pull request
Mar 21, 2024
## Description:
This change implements the generation of a yaml that represents the
effect of a sequence of instructions on an enclave. The major changes
are as follows:
- Adds gRPC endpoints `GetStarlarkPackge/ScriptPlanYaml` to APIC and
Enclave Manager for returning this yaml
- Implements `PlanYaml` object and yaml generation logic in
`startosis_engine`
- Adds `UpdatePlan(plan *PlanYaml)` method to `KurtosisInstruction`
interface so each instruction implements logic for updating the plan
yaml
- Most of the knowledge needed to generate the yaml comes from the
interpretation phase and is simply passed into yaml generation logic
Tests are in `startosis_interpreter_plan_yaml_tests.go` and demonstrate
how the `InstructionsPlan` generates the yaml via the `PlanYaml` object.
eg. starlark script turned plan yaml:
```
def run(plan, hi_files_artifact):
service = plan.add_service(
name="db",
config=ServiceConfig(
image="postgres:latest",
env_vars={
"POSTGRES_DB": "kurtosis",
"POSTGRES_USER": "kurtosis",
"POSTGRES_PASSWORD": "kurtosis",
},
files = {
"/root": hi_files_artifact,
}
)
)
execResult = plan.exec(
service_name="db",
recipe=ExecRecipe(
command=["echo", service.ip_address + " " + service.hostname]
),
acceptable_codes=[0],
)
runShResult = plan.run_sh(
run="echo " + execResult["code"] + " " + execResult["output"],
)
plan.run_sh(
run="echo " + runShResult.code + " " + runShResult.output,
)
```
plan yaml:
```
packageId: DEFAULT_PACKAGE_ID_FOR_SCRIPT
services:
- uuid: "1"
name: db
image:
name: postgres:latest
envVars:
- key: POSTGRES_DB
value: kurtosis
- key: POSTGRES_PASSWORD
value: kurtosis
- key: POSTGRES_USER
value: kurtosis
files:
- mountPath: /root
filesArtifacts:
- uuid: "2"
name: hi-file
filesArtifacts:
- uuid: "2"
name: hi-file
tasks:
- uuid: "3"
taskType: exec
command:
- echo
- '{{ kurtosis.1.ip_address }} {{ kurtosis.1.hostname }}'
serviceName: db
acceptableCodes:
- 0
- uuid: "4"
taskType: sh
command:
- echo {{ kurtosis.3.code }} {{ kurtosis.3.output }}
image: badouralix/curl-jq
- uuid: "5"
taskType: sh
command:
- echo {{ kurtosis.4.code }} {{ kurtosis.4.output }}
image: badouralix/curl-jq
```
## Is this change user facing?
NO
## References:
The Enclave Manager uses this plan yaml to render packages in the Enclave Builder:
#2250
---------
Co-authored-by: Ben Gazzard <[email protected]>
tedim52
added a commit
that referenced
this pull request
Mar 21, 2024
## Description THIS PR IS A COPY OF THIS PR: #2250 The frontend PR was initially made off the backend PR branch. I merged the backend PR branch to main. Then merged the frontend PR into the backend PR branch. Now merging into main again. --------- Co-authored-by: Ben Gazzard <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description:
This PR adds package nodes to the enclave builder UI. It's based on the backend work added by @tedim52 in #2177.
Demo
This demo has two paths - the first one shows how a package can be added to the enclave builder, and used by a service defined in the builder as a dependency. The second one shows how two packages can be combined in the same enclave - allowing connectivity between them at runtime.
packagenodes-2.mp4
Is this change user facing?
YES
References (if applicable):