plugins for kurtosis

[Mr-Nobody21]

How can I add plugins to kurtosis

[leeederek]

Hi @Mr-Nobody21! Thanks for asking a question here on Github Discussions! To help us help you, can you share with me:

1. What kind of plugins are you referring to and can you give us an example of one?
2. What is the workflow or operation you're trying to accomplish that you believe would require a plugin?

Thank you!
Derek

[ericobi]

I also would like to use native starlark plugins in Kurtosis.

[mieubrisse]

Hey @Mr-Nobody21 @ericobi , thanks for the question! I could see three concepts of "plugins" for Kurtosis, so I'll address all of them here (and if they don't answer your question, let me know!):

The ability to add extra functionality to Starlark itself

I.e., adding extra functionality to the language, sort of like C or Rust macros (for example, redefining the + operator).

We don't want to do this because part of the reason we chose Starlark was for its simplicity, approachability, and ease-of-reading (it's pretty much Just Python). In our experience, macros - while powerful - also add a lot of mental complexity to the reader.

Further, we don't want to build a new general-purpose, Turing-complete programming language, because there are already dozens of great options out there. If we're trying to morph Starlark (which is intentionally basic and simple) into a general-purpose language, we're fighting a losing battle. Instead, we want to use Starlark for what it's good at: an orchestration language. In essence, we want users to have enough power to write DRY orchestration logic to define their environments, but not attempt to replace the general-purpose languages.

So how do we do this? Basically, by keeping Starlark functionality intentionally limited, and making it very easy to call down to general-purpose programming languages. Our philosophy is basically, "use Starlark for the top-level orchestration, to stitch together a bunch of different pieces, but the pieces are written in actual programming languages". That way users can use whatever general-purpose programming language they please, or mix-and-match as desired.

In practice, the ways to call down to different programming languages in Kurtosis are plan.add_service to start a service in an arbitrary language (because containers) that runs indefinitely, and plan.run_sh to run a job in an arbitrary language (because containers) that starts, runs, and finishes.

We'll likely continue to add conveniences to make it easier to call down to general-purpose programming languages (e.g. maybe we add plan.run_python or plan.run_javascript or plan.run_golang), but we intentionally don't want to contort Starlark into a general-purpose language.

Starlark libraries

With that being said, part of writing DRY orchestration logic is code reuse and we do want to promote reusing the Starlark code that you write.

This is available in Kurtosis right now through the packaging system for publishing code, and import_module for including it.

For example, you can start a Postgres service using the Postgres package simply by writing & running this Starlark on your machine:

postgres_package = import_module("github.com/kurtosis-tech/postgres-package/main.star")

def run(plan, args):
    postgres_package.run(plan)

You aren't limited to just the packages we publish though; as long as your package is on Github and has a kurtosis.yml defining its name, you can include your own code into your Starlark scripts.

Adding extra plan functions

If you're thinking about adding extra plan functions (e.g. plan.do_something_custom), I'd be curious to hear more about the usecase. So far, the package/import system has solved all the cases where folks want to reuse code since you can import pretty much anything you want from another package (it doesn't just have to be the run function).

For example, if you had this package published to Github:

# ------------- github.com/my-org/my-package/main.star --------------
def run_hello(plan):
    plan.run_sh("echo 'Hello, world!'")

You could use the run_hello functionality elsewhere:

# main.star on my local machine
hello = import_module("github.com/my-org/my-package/main.star")

def run(plan, args):
    hello.run_hello(plan)

Fin

Does all of the above answer your questions @ericobi and @Mr-Nobody21 ?

[mieubrisse]

Hey @Mr-Nobody21 , wanted to ping again and check if this answers your question?

[Mr-Nobody21]

Hi @mieubrisse
Apologies for the delay

Yes it does answer my questions and thanks for the elaborate reply