diff --git a/docs/get-details/.pages b/docs/get-details/.pages index c52893385..5926cf847 100644 --- a/docs/get-details/.pages +++ b/docs/get-details/.pages @@ -2,13 +2,13 @@ title: Get details arrange: - index.md + - algokit - accounts - transactions - - asa.md - atomic_transfers.md - atc.md + - asa.md - dapps - - algokit.md - indexer.md - conduit.md - stateproofs diff --git a/docs/get-details/algokit.md b/docs/get-details/algokit.md deleted file mode 100644 index 7aaa20b64..000000000 --- a/docs/get-details/algokit.md +++ /dev/null @@ -1,43 +0,0 @@ -title: AlgoKit - -AlgoKit is a collection of tools and libraries that make it easy to build applications on Algorand. - -# CLI Tool - -The [AlgoKit CLI Tool](https://github.com/algorandfoundation/algokit-cli) provides a command line interface developers building applications on Algorand. - -Using the CLI tool a developer can, among other things, spin up or manage a [local network](#localnet) or initialize a new project from a [Template](#templates). - -## LocalNet - -The AlgoKit `localnet` sub-command allows a developer to quickly spin up a private network in a local docker container. This is useful for testing and development. This command replaces the need to install a local node manually or with the Sandbox. - -Simply running -```sh -$ algokit localnet start -``` - -Will download the required docker containers and spin up a network listening on the same ports as Sandbox and create and fund a set of accounts for the developer to use. - -[More information](https://github.com/algorandfoundation/algokit-cli/blob/main/docs/features/localnet.md) - -## Templates - -The AlgoKit `init` sub-command allows a developer to instantly initialize a new project from a template. A default set of templates is provided by the AlgoKit CLI tool, but any template can be used by passing the `--template-url` flag. - -Simply running -```sh -$ algokit init -``` - -Will take the developer through a set of prompts to initialize a new project. The developer can choose from a set of templates and the CLI tool will download the template and initialize the project in a new directory. All parameters can also be provided as command line arguments. - -[More information](https://github.com/algorandfoundation/algokit-cli/blob/main/docs/features/init.md) - -# Utilities - -There are a set of libraries that `AlgoKit` provides to make it easier to build solutions on Algorand. These libraries are written with developer experience in mind as a wrapper over the core algosdk libraries and abstract the most common tasks away from the developer as well as providing some advanced features such as idempotent smart contract deployment. - - - [Python](https://github.com/algorandfoundation/algokit-utils-py) - - [TypeScript/JS](https://github.com/algorandfoundation/algokit-utils-ts) - \ No newline at end of file diff --git a/docs/get-details/algokit/.pages b/docs/get-details/algokit/.pages new file mode 100644 index 000000000..e4ee6efda --- /dev/null +++ b/docs/get-details/algokit/.pages @@ -0,0 +1,9 @@ +title: AlgoKit + +arrange: +- index.md +- features +- cli-reference.md +- tutorials +- architecture-decisions +- articles diff --git a/docs/get-details/algokit/architecture-decisions/2022-11-14_sandbox-approach.md b/docs/get-details/algokit/architecture-decisions/2022-11-14_sandbox-approach.md new file mode 100644 index 000000000..d30a8f3ca --- /dev/null +++ b/docs/get-details/algokit/architecture-decisions/2022-11-14_sandbox-approach.md @@ -0,0 +1,94 @@ +# AlgoKit sandbox approach + +- **Status**: Approved +- **Owner:** Rob Moore +- **Deciders**: Anne Kenyon (Algorand Inc.), Alessandro Cappellato (Algorand Foundation), Will Winder (Algorand Inc.) +- **Date created**: 2022-11-14 +- **Date decided:** 2022-11-14 +- **Date updated**: 2022-11-16 + +## Context + +In order for AlgoKit to facilitate a productive development experience it needs to provide a managed Algorand sandbox experience. This allows developers to run an offline (local-only) private instance of Algorand that they can privately experiment with, run automated tests against and reset at will. + +## Requirements + +- The sandbox works cross-platform (i.e. runs natively on Windows, Mac and Linux) +- You can spin up algod and indexer since both have useful use cases when developing +- The sandbox is kept up to date with the latest version of algod / indexer +- There is access to KMD so that you can programmatically fund accounts to improve the developer experience and reduce manual effort +- There is access to the tealdbg port outside of algod so you can attach a debugger to it +- The sandbox is isolated and (once running) works offline so the workload is private, allows development when there is no internet (e.g. when on a plane) and allows for multiple instances to be run in parallel (e.g. when developing multiple independent projects simultaneously) +- Works in continuous integration and local development environments so you can facilitate automated testing + +## Principles + +- **[AlgoKit Guiding Principles](../index.md#Guiding-Principles)** - specifically Seamless onramp, Leverage existing ecosystem, Meet devs where they are +- **Lightweight** - the solution should have as low an impact as possible on resources on the developers machine +- **Fast** - the solution should start quickly, which makes for a nicer experience locally and also allows it to be used for continuous integration automation testing + +## Options + +### Option 1 - Pre-built DockerHub images + +Pre-built application developer-optimised DockerHub images that work cross-platform; aka an evolved AlgoKit version of . + +**Pros** + +- It's quick to download the images and quick to start the container since you don't need to compile Algod / indexer and the images are optimised for small size +- The only dependency needed is Docker, which is a fairly common dependency for most developers to use these days +- The images are reasonably lightweight +- The images provide an optimised application developer experience with: (devmode) algo, KMD, tealdbg, indexer +- It natively works cross-platform + +**Cons** + +- Some people have reported problems running WSL 2 on a small proportion of Windows environments (to get the latest Docker experience) +- Docker within Docker can be a problem in some CI environments that run agents on Docker in the first place +- Work needs to be done to create an automated CI/CD that automatically releases new versions to keep it up to date with latest algod/indexer versions + +### Option 2 - Lightweight algod client implementation + +Work with the Algorand Inc. team to get a lightweight algod client that can run outside of a Docker container cross-platform. + +**Pros** + +- Likely to be the most lightweight and fastest option - opening up better/easier isolated/parallelised automated testing options +- Wouldn't need Docker as a dependency + +**Cons** + +- Indexer wouldn't be supported (Postgres would require Docker anyway) +- Algorand Inc. does not distribute Windows binaries. + +### Option 3 - Sandbox + +Use the existing [Algorand Sandbox](https://github.com/algorand/sandbox). + +**Pros** + +- Implicitly kept up to date with Algorand - no extra thing to maintain +- Battle-tested by the core Algorand team day-in-day-out +- Supports all environments including unreleased feature branches (because it can target a git repo / commit hash) + +**Cons** + +- Sandbox is designed for network testing, not application development - it's much more complex than the needs of application developers +- Slow to start because it has to download and built algod and indexer (this is particularly problematic for ephemeral CI/CD build agents) +- It's not cross-platform (it requires bash to run sandbox.sh, although a sandbox.ps1 version could be created) + +## Preferred option + +Option 1 and Option 2. + +Option 1 provides a fully-featured experience that will work great in most scenarios, having option 2 as a second option would open up more advanced parallel automated testing scenarios in addition to that. + +## Selected option + +Option 1 + +We're aiming to release the first version of AlgoKit within a short timeframe, which won't give time for Option 2 to be developed. Sandbox itself has been ruled out since it's not cross-platform and is too slow for both development and continuous integration. + +Option 1 also results in a similar result to running Sandbox, so existing Algorand documentation, libraries and approaches should work well with this option making it a good slot-in replacement for Sandbox for application developers. + +AlgoKit is designed to be modular: we can add in other approaches over time such as Option 2 when/if it becomes available. diff --git a/docs/get-details/algokit/architecture-decisions/2022-11-22_beaker-testing-strategy.md b/docs/get-details/algokit/architecture-decisions/2022-11-22_beaker-testing-strategy.md new file mode 100644 index 000000000..57597c919 --- /dev/null +++ b/docs/get-details/algokit/architecture-decisions/2022-11-22_beaker-testing-strategy.md @@ -0,0 +1,95 @@ +# Beaker testing strategy + +- **Status**: Draft +- **Owner:** Rob Moore +- **Deciders**: Anne Kenyon (Algorand Inc.), Alessandro Cappellato (Algorand Foundation), Michael Diamant (Algorand Inc.), Benjamin Guidarelli (Algorand Foundation) +- **Date created**: 2022-11-22 +- **Date decided:** TBD +- **Date updated**: 2022-11-28 + +## Context + +AlgoKit will be providing a smart contract development experience built on top of [PyTEAL](https://pyteal.readthedocs.io/en/stable/) and [Beaker](https://developer.algorand.org/articles/hello-beaker/). Beaker is currently in a pre-production state and needs to be productionised to provide confidence for use in generating production-ready smart contracts by AlgoKit users. One of the key things to resolve to productionisation of Beaker is to improve the automated test coverage. + +Beaker itself is currently split into the PyTEAL generation related code and the deployment and invocation related code (including interacting with Sandbox). This decision is solely focussed on the PyTEAL generation components of Beaker. The current automated test coverage of this part of the codebase is ~50% and is largely based on compiling and/or executing smart contracts against Algorand Sandbox. While it's generally not best practice to try and case a specific code coverage percentage, a coverage of ~80%+ is likely indicative of good coverage in a dynamic language such as Python. + +The Sandbox tests provide a great deal of confidence, but are also slow to execute, which can potentially impair Beaker development and maintenance experience, especially as the coverage % is grown and/or features are added over time. + +Beaker, like PyTEAL, can be considered to be a transpiler on top of TEAL. When generating smart contracts, the individual TEAL opcodes are significant, since security audits will often consider the impact at that level, and it can have impacts on (limited!) resource usage of the smart contract. As such, "output stability" is potentially an important characteristic to test for. + +## Requirements + +- We have a high degree of confidence that writing smart contracts in Beaker leads to expected results for production smart contracts +- We have reasonable regression coverage so features are unlikely to break as new features and refactorings are added over time +- We have a level of confidence in the "output stability" of the TEAL code generated from a Beaker smart contract + +## Principles + +- **Fast development feedback loops** - The feedback loop during normal development should be as fast as possible to improve the development experience of developing Beaker itself +- **Low overhead** - The overhead of writing and maintaining tests is as low as possible; tests should be quick to read and write +- **Implementation decoupled** - Tests aren't testing the implementation details of Beaker, but rather the user-facing experience and output of it; this reduces the likelihood of needing to rewrite tests when performing refactoring of the codebase + +## Options + +### Option 1: TEAL Approval tests + +Writing [approval tests](https://approvaltests.com/) of the TEAL output generated from a given Beaker smart contract. + +**Pros** + +- Ensures TEAL output stability and focussing on asserting the output of Beaker rather than testing whether Algorand Protocol is working +- Runs in-memory/in-process so will execute in low 10s of milliseconds making it easy to provide high coverage with low developer feedback loop overhead +- Tests are easy to write - the assertion is a single line of code (no tedious assertions) +- The tests go from Beaker contract -> TEAL approval so don't bake implementation detail and thus allow full Beaker refactoring with regression confidence without needing to modify the tests +- Excellent regression coverage characteristics - fast test run and quick to write allows for high coverage and anchoring assertions to TEAL output is a very clear regression marker + +**Cons** + +- The tests rely on the approver to understand the TEAL opcodes that are emitted and verify they match the intent of the Beaker contract - anecdotally this can be difficult at times even for experienced (Py)TEAL developers +- Doesn't assert the correctness of the TEAL output, just that it matches the previously manually approved output + +### Option 2: Sandbox compile tests + +Writing Beaker smart contracts and checking that the TEAL output successfully compiles against algod. + +**Pros** + +- Ensures that the TEAL output compiles, giving some surety about the intactness of it and focussing on asserting the output of Beaker rather than testing whether Algorand Protocol is working +- Faster than executing the contract +- Tests are easy to write - the assertion is a single line of code (no tedious assertions) + +**Cons** + +- Order of magnitude slower than asserting TEAL output (out of process communication) +- Doesn't assert the correctness of the TEAL output, just that it compiles + +### Option 3: Sandbox execution tests + +Execute the smart contracts and assert the output is as expected. This can be done using dry run and/or actual transactions. + +**Pros** + +- Asserts that the TEAL output _executes_ correctly giving the highest confidence +- Doesn't require the test writer to understand the TEAL output +- Tests don't bake implementation detail and do assert on output so give a reasonable degree of refactoring confidence without modifying tests + +**Cons** + +- Tests are more complex to write +- Tests take an order of magnitude longer to run than just compilation (two orders of magnitude to run than checking TEAL output) +- Harder to get high regression coverage since it's slower to write and run the tests making it impractical to get full coverage +- Doesn't ensure output stability +- Is testing that the Algorand Protocol itself works (TEAL `x` when executed does `y`) so the testing scope is broader than just Beaker itself + +## Preferred option + +Option 1 (combined with Option 2 to ensure the approved TEAL actually compiles, potentially only run on CI by default to ensure fast local dev loop) for the bulk of testing to provide a rapid feedback loop for developers as well as ensuring output stability and great regression coverage. + +## Selected option + +Combination of option 1, 2 and 3: + +- While Option 1 + 2 provides high confidence with fast feedback loop, it relies on the approver being able to determine the TEAL output does what they think it does, which isn't always the case +- Option 3 will be used judiciously to provide that extra level of confidence that the fundamentals of the Beaker output are correct for each main feature; this will involve key scenarios being tested with execution based tests, the goal isn't to get combinatorial coverage, which would be slow and time-consuming, but to give a higher degree of confidence +- The decision of when to use Option 3 as well as Option 1+2 will be made on a per-feature basis and reviewed via pull request, over time a set of principles may be able to be revised that outline a clear delineation +- Use of PyTest markers to separate execution so by default the dev feedback loop is still fast, but the full suite is always run against pull requests and merges to main diff --git a/docs/get-details/algokit/architecture-decisions/2023-01-11_beaker_productionisation_review.md b/docs/get-details/algokit/architecture-decisions/2023-01-11_beaker_productionisation_review.md new file mode 100644 index 000000000..7039d9554 --- /dev/null +++ b/docs/get-details/algokit/architecture-decisions/2023-01-11_beaker_productionisation_review.md @@ -0,0 +1,412 @@ +# Beaker productionisation review + +- **Status**: Approved +- **Owners:** Rob Moore, Adam Chidlow +- **Deciders**: Anne Kenyon (Algorand Inc.), Alessandro Cappellato (Algorand Foundation), Jason Weathersby (Algorand Foundation), Benjamin Guidarelli (Algorand Foundation), Bob Broderick (Algorand Inc.) +- **Date created**: 2023-01-11 +- **Date decided:** 2023-02-04 +- **Date updated**: 2023-02-04 + +## Context + +Beaker is a smart contract development framework for [Algorand](https://www.algorand.com/) that provides a wrapper over [PyTeal](https://pyteal.readthedocs.io/en/stable/) that focusses on providing a great developer experience through terse, expressive language constructs and making common tasks easier. Beaker is useful because it creates a higher level programming construct from PyTEAL that is easier to get started when learning and results in code that is terser and easier to read and write. + +Beaker is an important part of the [AlgoKit strategy](https://github.com/algorandfoundation/algokit-cli/#algokit-cli). It helps create a more seamless onramp to Algorand development by providing an easier starting point for developers. As part of the lead up to releasing AlgoKit, it was desired to perform a v1.0 release of Beaker and explicitly mark it as being production ready. In order to provide confidence a productionisation review was conducted by [MakerX](https://www.makerx.com.au/); this document summarises the recommendations from that review. + +An architecture decision was made in the lead up to this review on a [testing strategy for Beaker](https://github.com/algorandfoundation/algokit-cli/blob/main/docs/architecture-decisions/2022-11-22_beaker-testing-strategy.md). + +## Goal + +The goals of this productionisation review are to: + +- Get Beaker ready for production use +- Gain confidence in Beaker's software architecture and maintainability +- Reduce the likelihood of need for breaking changes soon after release by getting key recommended breaking changes identified now + +## Findings summary + +The Beaker codebase is well factored and had a decent initial test coverage (albeit some of that test coverage is via a series of examples that while they provide high code coverage, don't actually validate all of the functionality). + +A series of changes have been landed to improve some of the fundamentals of Beaker in preparation for production launch: + +- [Various improvements](https://github.com/algorand-devrel/beaker/pull/142) - Improved test coverage, improved dev experience (setup + ongoing) via Poetry, improvements to the code quality setup (linting, automatic formatting, typing), allowed Windows development on Beaker itself, significantly improved CI/CD pipeline speed, removing the examples directory and tests from being distributed wit hthe PyPi package +- [Typing improvements](https://github.com/algorand-devrel/beaker/pull/147) +- [Removed inline imports](https://github.com/algorand-devrel/beaker/pull/148) +- [Removed dead code](https://github.com/algorand-devrel/beaker/pull/149/files) +- [Added automated release management and versioning](https://github.com/algorand-devrel/beaker/pull/161) + +In addition, there are a remaining set of more major (breaking) changes that are recommended. The recommendations are split into 2 categories, recommendations for immediate improvement (i.e. included in v1.0) and future suggestions that can be addressed post v1.0 launch. + +The recommended additional areas for immediate improvement are: + +- **Replace the class-based structure with an instance based one** - remove some areas of potential surprise for developers and simplify the Beaker codebase by moving to a composable instance-based structure rather than a static class-based structure +- **Defer PyTEAL compilation** - improve flexibility and future contract output stability by deferring PyTEAL compilation (i.e. Beaker -> TEAL transpilation) to not happen when the Beaker contract is initialised +- **Renamings** - There are some clear parameters that make sense to rename for various reasons +- **Key decorator improvements** - Refactor some of the Beaker decorators to fix some bugs and improve user experience +- **Beaker state refactor** - Refactor of the Beaker state interfaces to improve user extensibility and significantly simplify the Beaker codebase to improve maintainability + +The recommended areas for future improvement are: + +- Typed client generation from app spec to improve deploy-time and run-time dev experience +- `Tmpl` values in app spec so you can have type-safe deployment clients that substitute any template values reliably at contract deploy time +- Refactor storage types (blob, reserved, etc.) to allow use of in-built Python types and operators (terser, more intuitive) +- Box storage implementation improved to match local/global behaviour and also automatically delete itself on contract deletion +- Composable and stackable authorization and `@authorize` as a standalone decorator +- PyTEAL typings to be improved to support types beyond `Expr` where a more explicit type can be specified (improves typing and extensibility) +- Support referencing an app/lsig via ID/address (deployed separately, potentially automatically as part reading a Directed Acyclic Graph (DAG) in application.json of application dependencies) or bytes (deployed inline, what was previously called precompile, noting this would be deploy-time substitution, not smart contract run-time substitute like `TemplateVariable`), this also may allow precompile to be deprecated (it's a very complex implementation for what we believe to be an advanced edge case) + +## Immediate recommendations + +### (1) Replace the class-based structure with an instance based one + +#### What? + +Beaker is currently structured around users sub-classing the `beaker.Application` class. They then hold state variables (from `beaker.state.*`) as [class variables](https://pynative.com/python-class-variables/) and also contain methods which are forwarded to the `pyteal.abi.Router` instance created during `Application.compile(...)` based on decorators from `beaker.decorators.*`. We propose replacing this with an "instance based structure", drawing inspiration from highly popular Python web frameworks such as `flask` ([example](https://flask.palletsprojects.com/en/2.2.x/quickstart/#a-minimal-application)). + +This change will simplify Beaker's code (improving maintainability) and, more importantly, reduce the potential for end-user error and confusion. + +#### Why? + +**User-facing benefits** + +1. The current structure, by encouraging and supporting [bound instance methods](https://www.geeksforgeeks.org/bound-methods-python/) alongside [class variables](https://pynative.com/python-class-variables/), is a potential source of confusion for users new to writing smart contracts or PyTEAL. The distinction between what runs on `beaker.Application` instantiation, evaluation by PyTeal during compile, and finally what runs on-chain, can be difficult to grasp at first. One might assume (wrongly) that Beaker is somehow maintaining the state of `self.*` between methods, but this is not the case. Contrast this with Solidity, for example, where state can be directly manipulated because it's help within the class instance. +2. Currently, actually using `self.*` can easily lead to problems, since if they are not defined before calling `super().__init__(...)` they won't be defined when compiling. This can be fixed by not automatically compiling in `Application.__init__()` (which is also proposed in (2) below) for simple constants, however another issue is that using `self.foo = `, would not currently work with the introspection beaker is performing. This could potentially be fixed by itself, but developers will still need to define these values _before_ calling `super().__init__()` which is a source of confusion. Usually, idiomatic Python will call super init sooner rather than later so this is something that can trip up experienced Python developers. +3. In order to compose applications together, say if there were two ARC standard implementations that we wanted to combine into the same contract, the user doesn't need to understand Python's multiple-inheritance idiosyncrasies like [Method Resolution Order](https://www.geeksforgeeks.org/method-resolution-order-in-python-inheritance/). Additionally, by taking a functional composition approach, we can have easy to understand entry points where you can check any pre-conditions. +4. Since state variables are currently defined as class variables, this makes them "globals", which can lead to errors/bugs that are non-obvious. + + For instance, consider: + + ```python + class MyBaseApp(beaker.Application): + counter = beaker.ApplicationStateValue(stack_type=pyteal.TealType.uint64) + + @beaker.create + def create(self) -> pyteal.Expr: + return self.initialize_application_state() + + class MyApp(MyBaseApp): + pass + MyApp.counter.default = pyteal.Int(10) + + class MyOtherApp(MyBaseApp): + pass + + app1 = MyApp() + app2 = MyOtherApp() + assert app1.approval_program != app2.approval_program # fails + ``` + +5. Setting parameters that control the program creation is awkward with the current approach of extending `beaker.Application`, currently this impacts just the `version` parameter (which specifies the TEAL version), but there are clear examples we can see for other variables that are useful to define at this point in the future (e.g. a state allocation override if you know that the state a contract will need grows in the future). +6. There are bugs in beaker which are directly caused by the class-based structure. For example, bare methods are currently evaluated as a subroutine only once: + + ```python + class MyApp(beaker.Application): + price = beaker.ApplicationStateValue(stack_type=pyteal.TealType.uint64) + + def __init__(self, default_price: int, version: int = pyteal.MAX_TEAL_VERSION): + self.price.default = pyteal.Int(default_price) + super().__init__(version=version) + + + class CorrectApp(MyApp): + @beaker.create + def create(self, *, output: pyteal.abi.Uint64) -> pyteal.Expr: + return pyteal.Seq(self.initialize_application_state(), output.set(self.price)) + + + class IncorrectApp(MyApp): + @beaker.create + def create(self) -> pyteal.Expr: + return self.initialize_application_state() + + + correct_app1 = CorrectApp(default_price=123) + correct_app2 = CorrectApp(default_price=456) + + incorrect_app1 = IncorrectApp(default_price=123) + incorrect_app2 = IncorrectApp(default_price=456) + + assert correct_app1.approval_program != correct_app2.approval_program # success + assert incorrect_app1.approval_program != incorrect_app2.approval_program # failure + + ``` + +**Beaker maintainability benefit** + +The main benefit to Beaker is the removal the complex code that modifies function signatures to remove `self` before passing to PyTEAL. Removing the instance method implementation will significantly reduce the complexity of the code and likelihood of unknown bugs surfacing from that part of the codebase. + +#### Before & After (user's perspective) + +While the proposed changes are fairly substantial internally, and propose a radically different architecture conceptually for beaker Applications, the migration should actually be relatively straight forward for users with existing Beaker code: + +The following examples assume the import of relevant names from `beaker` and/or `pyteal` are present to simplify the code. + +**Before:** + +```python + +class CounterApp(Application): + counter = ApplicationStateValue( + stack_type=TealType.uint64, + descr="A counter for showing how to use application state", + ) + + @create + def create(self): + return self.initialize_application_state() + + @external(authorize=Authorize.only(Global.creator_address())) + def increment(self, *, output: abi.Uint64): + """increment the counter""" + return Seq( + self.counter.set(self.counter + Int(1)), + output.set(self.counter), + ) + + @external(authorize=Authorize.only(Global.creator_address())) + def decrement(self, *, output: abi.Uint64): + """decrement the counter""" + return Seq( + self.counter.set(self.counter - Int(1)), + output.set(self.counter), + ) + +``` + +**After:** + +The changes are: + +- State is moved into a dedicated class `CounterState` +- `beaker.Application` is directly instantiated (along with the state, and optionally the teal `version`) +- Class methods are de-indented, `self` is removed and the decorator is prefixed with `app.` (which in turn reduces the number of imports needed from the `beaker` namespace and provides better exploratory intellisense for users) + +```python + +class CounterState(beaker.State): + counter = ApplicationStateValue( + stack_type=TealType.uint64, + descr="A counter for showing how to use application state", + ) + + +app = beaker.Application(state=CounterState()) + +@app.create +def create(): + return app.state.initialize_application_state() + +@app.external(authorize=Authorize.only(Global.creator_address())) +def increment(*, output: abi.Uint64): + """increment the counter""" + return Seq( + app.state.counter.set(app.state.counter + Int(1)), + output.set(app.state.counter), + ) + +@app.external(authorize=Authorize.only(Global.creator_address())) +def decrement(*, output: abi.Uint64): + """decrement the counter""" + return Seq( + app.state.counter.set(app.state.counter - Int(1)), + output.set(app.state.counter), + ) + +``` + +### (2) Defer PyTEAL compilation + +#### What? + +Currently, `beaker.Application.compile()` is called as part of `__init__()`, assuming there are no `precompiles` defined. We recommend that `compile()` always be deferred to a later point, and further that `compile()` does not mutate `Application` in any way, but instead returns a new object. + +#### Why? + +The deferment of the `compile()` call is actually a necessary part of recommendation #1 that we have skipped over thus far, but would be recommended anyway. + +The immediate `compile()` has issues such as requiring implementors (i.e. subclasses) to call `super().__init__()` as a **final** step in their own `__init__` method - any code that runs after the super init call will have no effect on the application produced! + +Immediate compilation also reduces the control the user has over the output. Although currently the only parameter that `compile` takes is a `client`, it might be useful to add (optional) parameters here to control the compilation. For example, if you can pass in the list of optimisations that should be applied, that allows you to have [output stability](../articles/output_stability.md) of your smart contract code if new optimisations are added in the future. + +The separation of compiled state outside of `Application` simplifies the design, and can be done mostly transparently to end-users. + +The separation of compiled state will also benefit future interoperability. It allows for more explicit decoupling of PyTEAL compilation (Beaker / PyTEAL transpilation -> TEAL) and deploment (TEAL -> compiled byte code -> Algorand network). Once `beaker.client` is split into a separate package, if the compiled state can be both generated from a beaker Application object _or_ loaded from disk (or similar), this means Beaker's ApplicationClient could be used in more situations, such as for a (say) tealish smart contract, or a C# smart contract, or a raw PyTEAL or TEAL smart contract, etc. This conforms better to the modularity principle in AlgoKit and also vice versa allows for a Beaker smart contract to be deployed by a TypeScript deployer, or C# deployer, etc. + +#### Before & After - user's perspective + +For most use cases, this should be a relatively small and probably imperceptible change. + +We believe there are two common usage scenarios that use the output of PyTEAL compilation currently: + +1. Output the `Application` via `Application.dump(...)` +2. Interact with the `Application` by passing it to `ApplicationClient(app=..., ...)`. + +We propose maintaining those two scenarios without any immediate external changes, but internally: + +1. `Application.dump(...)` will call `Application.compile().dump()`, and potentially trigger a `DeprecationWarning` if we decide that we want users to always explicitly call compile. +2. `ApplicationClient(app=..., ...)` will call `Application.compile()` and not retain any reference to `app`. + +To make use of scenarios 1 _and_ 2, or to control compilation parameters, a user should also be able to (for instance): + +```python +app = Application(...) +compiled_app: CompiledApplication = app.compile(...) +compiled_app.dump(...) +client = ApplicationClient(app=compiled_app, ...) +``` + +We suggest also potentially renaming `CompiledApplication.dump()`, perhaps to something along the lines of `serialize()`. + +The `compile()` call is actually a transpilation call (Beaker / PyTEAL transpilation -> TEAL), although it's called compile in PyTEAL so consideration should be made to either keep consistency with PyTEAL or use the more accurate `transpile()` (which also reduces confusion around the fact that you then have to call `compile` on algod to compile the TEAL to byte code before deployment). + +The exact details of what `CompiledApplication` will look like are TBD, but should be driven by the principles outlined in the "Why?" section above. Broadly, it stands to reason it would contain the approval and clear TEAL, the ABI spec and the app spec though at least. + +Finally, there is likely need to use metadata from transpilation such as the mapping of source code to line numbers, but we are confident these use cases will be able to be implemented on top of the proposed change. + +### (3) Renamings + +Renaming `version` parameter in `Application.__init__(version: int = pyteal.MAX_VERSION)` to (e.g.) `avm_version`, to be more explicit. Otherwise developers may be confused that it's the version of the specific smart contract. It may be desirable to allow `version` to continue to be specified for some time, but to raise a `DeprecationWarning`. + +Rename methods in `beaker.lib.*` to start with an uppercase. Although going against PEP-8, this prevents collisions with `builtins` such as `min` and `max`, and also follows the useful convention from PyTeal where methods that produce TEAL code (vs just running Python code at transpilation time) start with uppercase such as `Add`, `Or`, `Concat`, etc. + +### (4) Key decorator improvements + +Refactor some of the Beaker decorators to fix some bugs and improve user experience. + +End state: + +```python +# for user convenience, rather than having to import + use MethodConfig +OnCompleteActionName: TypeAlias = Literal[ + "no_op", + "opt_in", + "close_out", + "clear_state", + "update_application", + "delete_application", +] + +HandlerFunc: TypeAlias = Callable[..., Expr] +DecoratorFunc: TypeAlias = Callable[[HandlerFunc], HandlerFunc] + +class Application: + # the main decorator, capable of handling both ABI and Bare method registration + def external( + self, + fn: HandlerFunc | None = None, + /, + *, + # note: retain existing behaviour of if method_config is None, default to no_op with CallConfig.CALL + method_config: MethodConfig | dict[OnCompleteActionName, CallConfig] | None = None, + name: str | None = None, + authorize: SubroutineFnWrapper | None = None, + bare: bool = False, + read_only: bool = False, + override: bool | None = False, + ) -> HandlerFunc | DecoratorFunc: + ... + + # the below are just "shortcuts" to @external for simple/common use cases + def create( + self, + fn: HandlerFunc | None = None, + /, + *, + allow_call: bool = False, + name: str | None = None, + authorize: SubroutineFnWrapper | None = None, + bare: bool = False, + read_only: bool = False, + override: bool | None = False, + ) -> HandlerFunc | DecoratorFunc: + ... + + + def ( + self, + fn: HandlerFunc | None = None, + /, + *, + allow_call: bool = True, + allow_create: bool = False, + name: str | None = None, + authorize: SubroutineFnWrapper | None = None, + bare: bool = False, + read_only: bool = False, + override: bool | None = False, + ) -> HandlerFunc | DecoratorFunc: + ... +``` + +For reference, the current state: + +```python +def internal( + return_type_or_handler: TealType | HandlerFunc, +) -> HandlerFunc | DecoratorFunc: + ... + +def external( + func: HandlerFunc | None = None, + /, + *, + name: str | None = None, + authorize: SubroutineFnWrapper | None = None, + method_config: MethodConfig | None = None, + read_only: bool = False, +) -> HandlerFunc | DecoratorFunc: + ... + +def bare_external( + no_op: CallConfig | None = None, + opt_in: CallConfig | None = None, + clear_state: CallConfig | None = None, + delete_application: CallConfig | None = None, + update_application: CallConfig | None = None, + close_out: CallConfig | None = None, +) -> Callable[..., HandlerFunc]: + ... + + +def create( + fn: HandlerFunc | None = None, + /, + *, + authorize: SubroutineFnWrapper | None = None, + method_config: Optional[MethodConfig] | None = None, +) -> HandlerFunc | DecoratorFunc: + ... + + +def ( + fn: HandlerFunc | None = None, /, *, authorize: SubroutineFnWrapper | None = None +) -> HandlerFunc | DecoratorFunc: + ... +``` + +Changes: + +- Remove `@internal`: + - if you don't pass a TealType parameter to it, i.e. intend to create an ABI internal routine, it actually just inlines the code currently due to a bug + - when passing in a TealType parameter to it, i.e. intent to create a normal subroutine, then in combination with (1) it will be unneeded since you can use `Subroutine` from PyTEAL (since the methods don't need to be artificially modified to remove `self` anymore) +- Add `bare: bool` option: + - Currently, this is not able to be controlled by the user - for `` decorators, they will create a bare method if the function takes no parameters other than maybe a `self` parameter. This has some down-sides: + 1. The user might want an ABI method rather than a bare method. In this case, currently they could use `@external(method_config=...)`, but for simple cases this is not as easy to read/type and is not intuitive to discover in the first place. + 2. The user might have more than one method that takes no parameters that is able to be called with a given `OnCompletionAction`, currently this would produce a `BareOverwriteError` in Beaker. Again, the work-around exists of calling `@external` instead, but it would be nicer and more intuitive to add a `bare` option to control this explicitly. + - The above Python methods have `bare: bool = False`. An alternative option would be to make this `bare: bool | None = None`, where `None` would retain the current behaviour of inspecting the method signature to see if it takes parameters or not. +- Remove `@bare_external`: + - Mostly unused, and doesn't provide the same options as the other decorators (e.g. `authorize`) + - Instead, we can replace the case of a single option being passed to it, with the equivalent named method: for example `@bare_external(opt_in=CallConfig.CALL)` becomes `@opt_in(bare=True)` + - For the multi-argument case: `@bare_external(no_op=CallConfig.CREATE, opt_in=CallConfig.CALL)` becomes `@external(method_config={"no_op": CallConfig.CREATE, "opt_in": CallConfig.CALL}, bare=True)` + +* Add optional `name` option to all decorators, not just `@external`. +* Add `allow_call` and `allow_create` options to shortcut methods (except `@create` shortcut which should always allow `CallConfig.CREATE`). +* Remove `method_config` from `@create` shortcut - the default behaviour will remain unchanged, but any usages with `method_config` specified would be equivalent to just using `@external` directly. +* Add `override: bool | None = False` parameter. + - If `False` (the suggested default), an error will be raised if an ABI or Bare method would replace one already registered in the Application. For bare methods, this would be keyed on the `OnCompleteAction`, and for ABI methods should be based on the method signature (ie `ABIReturnSubroutine.method_signature()`). This is suggested as the default to prevent unexpected cases of overriding, especially when using blueprints/templates from the future Smart Contracts Library. + - If `True`, then an error will be raised if it _does not_ replace an already registered ABI or Bare method. This is similar to Java's `@Override` annotation, and can allow the user to be explicit and thus prevent unexpectedly _not_ replacing an existing method. + - If `None`, then methods will be overwritten if present, and no error will be raised if not already present. This option is here for maximum flexibility, but should perhaps be discouraged. + +### (5) Beaker state refactor + +Refactor of the `beaker.state` internal interfaces to simplify Beaker code base, make it easier to add new state wrappers, and to pave the way for future enhancements. This will have a side effect of allowing users to create their own state wrappers without having to modify `beaker` itself, although we recommend marking these interfaces as internal and subject to change - at least initially. diff --git a/docs/get-details/algokit/architecture-decisions/2023-01-11_brew_install.md b/docs/get-details/algokit/architecture-decisions/2023-01-11_brew_install.md new file mode 100644 index 000000000..552d4a077 --- /dev/null +++ b/docs/get-details/algokit/architecture-decisions/2023-01-11_brew_install.md @@ -0,0 +1,86 @@ +# HomeBrew install strategy + +- **Status**: Approved +- **Owner:** Daniel McGregor +- **Deciders**: Daniel McGregor, Rob Moore, John Woods, Alessandro Cappellato +- **Date created**: 2023-01-11 +- **Date decided:** 2023-01-11 + +## Context + +HomeBrew (brew) is a MacOS package manager, and is commonly used on MacOS systems to install applications and tools like AlgoKit. Brew offers two main installation methods. + + * Formula - A source based install, this is typically recommended for open source and command line based applications. Formula can also be "bottled" to provide pre built packages for quick installs on supported MacOS platforms. + * Cask - A binary based install, this is typically recommended for closed source or GUI based applications. + +Additionally there are also two options for how the brew install scripts could be distributed. + +* Homebrew repository - This is the official repository for homebrew installs, and provides bottle support for all moderns MacOS platforms (MacOS 11+, Intel and ARM) +* Algorand hosted repository - A homebrew repository managed by Algorand Foundation. + +Creating a HomeBrew Formula initially seemed like the best option for AlgoKit as it meets the [criteria](https://docs.brew.sh/Acceptable-Formulae) for a Formula. However there is a much higher maintenance cost with this approach as everything is built from source. We encountered an issue where one of our newly added python dependencies (pyclip) did not build from source [correctly](https://github.com/algorandfoundation/homebrew-tap/actions/runs/3884956190/jobs/6628201057#step:8:2871). + +The alternative install method of a cask was then considered, and while AlgoKit does not meet the [criteria](https://docs.brew.sh/Acceptable-Casks) for a cask, it does remove the need for a source build on each MacOS platform and the additional maintenance overhead of the Formula approach. + +## Requirements + +- **Low maintenance**: The ongoing maintenance for supporting brew installs of AlgoKit should be low and not require manual effort for each release. +- **Fast and easy install experience**: The install experience for end users of AlgoKit should be easy and not require multiple complicated steps, additionally it should install in seconds, not minutes. + +## Options + +### Option 1: Formula on Official Homebrew Repo + +This would be the preferred option except for the two notable issues. Firstly there is a high risk of ongoing maintenance overhead due to the need to support source building all the dependencies. Ideally this would not be an issue, but we have already hit a problem with a dependency (pyclip) early on in AlgoKit's development. Secondly inclusion into the official repo is subject to Homebrew's criteria, which AlgoKit won't reach until it is more mature. + +**Pros** +* Most discoverable option for end users `brew install algokit`. +* Homebrew supports automatically bottling on all modern MacOS platforms (MacOS 11+ both Intel and ARM variants) meaning fast installs for users. + +**Cons** +* Inclusion is subject to Homebrew's approval process, which algokit won't meet for now at least. +* Higher maintenance cost given the source build is more fragile and is likely to break and require investigation, plus build and install approach differs significantly from Chocolatey and pipx +* Longer build time on release +* Not possible to fully automate release, it relies on a Brew maintainer approving the pull request, so there's extra operational overhead to keep track of the release pull requests + +### Option 2: Formula on Algorand Repo + +This option is similar to Option 1, but allows Algorand to self publish the installer without meeting Homebrews formula criteria. However one issue is that Platform support is more limited, GitHub provides action runners for intel variants for MacOS 11 + 12, but [MacOS 13](https://github.com/github/roadmap/issues/620) and [ARM](https://github.com/github/roadmap/issues/528) support are not yet available. Additional platforms could be supported by using a combination of self-hosted runners and/or third party solutions. This means pre-built bottles aren't easy to build for ARM or MacOS 13 and installation on those environments will take 5+ minutes. + +**Pros** +* Algorand Foundation has control over this process and it can be fully automated +* It's what we have already implemented and working today +* Easier to move to the official Brew core repository once AlgoKit is stable and demonstrably popular (thus meeting the constraints Brew place) + +**Cons** +* Supporting all modern MacOS platforms may require use of a 3rd party service and more effort, in the meantime the installation experience on ARM and MacOS 13 is slow (5+ min install) +* Less discoverable install for end users `brew install algorandfoundation/algokit` (relies on them following documentation) +* Higher maintenance cost given the source build is more fragile and is likely to break and require investigation, plus build and install approach differs significantly from Chocolatey and pipx +* Longer build time on release + +### Option 3: Cask on Algorand Repo + +This option uses a cask which does not have the maintenance overhead of a formula, and can be hosted in an Algorand Foundation repo to get around the fact AlgoKit does not meet the normal cask criteria. + +**Pros** +* Algorand Foundation has control over this process and it can be fully automated +* Lower maintenance cost as we do not need to support source builds of dependencies and it's consistent with how algokit cli is installed via Chocolatey and pipx +* Fast install for all MacOS platforms +* Fast build time on release + +**Cons** +* Less discoverable install for end users `brew install algorandfoundation/algokit` +* AlgoKit does not meet the stated criteria for a cask and as such it would unlikely to be approved as a cask in the official Homebrew Repo if that was a desired future state +* More effort to implement a new way of installing via brew + +### Option 4: Cask on Official Homebrew Repo + +This is not a viable option as AlgoKit does not meet the criteria for an official cask. + +## Preferred option + +Option 1 because it would be the best end user experience. + +## Selected option + +Option 3 because Option 1 isn't possible right now and it's also a higher overhead to maintain. The install experience for end users is similar with option 3 (just with a bit more typing). diff --git a/docs/get-details/algokit/architecture-decisions/2023-01-12_smart-contract-deployment.md b/docs/get-details/algokit/architecture-decisions/2023-01-12_smart-contract-deployment.md new file mode 100644 index 000000000..e835298a5 --- /dev/null +++ b/docs/get-details/algokit/architecture-decisions/2023-01-12_smart-contract-deployment.md @@ -0,0 +1,375 @@ +# Smart Contract Deployment + +- **Status**: Approved +- **Owner:** Rob Moore +- **Deciders**: Anne Kenyon (Algorand Inc.), Alessandro Cappellato (Algorand Foundation), Fabrice Benhamouda (Algorand Foundation) +- **Date created**: 2023-01-12 +- **Date decided:** 2023-02-04 +- **Date updated**: 2023-02-04 + +## Context + +AlgoKit will provide an end-to-end development and deployment experience that includes support for the end-to-end smart contract development lifecycle: + +1. Development + 1. **Write** smart contracts + 2. **Transpile** smart contracts with development-time parameters to TEAL Templates + 3. **Verify** the TEAL Templates maintain [output stability](../articles/output_stability.md) and any other static code quality checks +2. Deployment + 1. **Substitute** deploy-time parameters into TEAL Templates to create final TEAL code + 2. **Compile** the TEAL to create byte code using algod + 3. **Deploy** the byte code to one or more Algorand networks (e.g. LocalNet, TestNet, MainNet) to create Deployed Application(s) +3. Runtime + 1. **Validate** the deployed app via automated testing of the smart contracts to provide confidence in their correctness + 2. **Call** deployed smart contract with runtime parameters to utilise it + +![Smart Contract Development Lifecycle](./lifecycle.jpg) + +The default Development experience that AlgoKit exposes will be via Beaker, however AlgoKit is modular and extensible so other tooling can also be used. + +This decision record covers the different options and high level design for how AlgoKit aims to cover Deployment and Runtime. + +## Requirements + +- We support the different activities defined above under Deployment and Runtime: Substitute, Compile, Deploy, Validate and Call +- We support the ability to provide dev-time (e.g. static values that are passed into instances of a contract that get output), deploy-time (e.g. network specific addresses or IDs, etc.) and run-time (e.g. call arguments) values to smart contracts +- We support deploying smart contracts that have been output by any means (Beaker or otherwise) that creates TEAL templates (logic signature or approval & clear) and (for an app) an [ABI](https://github.com/algorandfoundation/ARCs/blob/main/ARCs/arc-0004.md) and an [app spec](https://github.com/algorandfoundation/ARCs/pull/150) +- We support calling smart contracts with multiple languages / programming ecosystems (with AlgoKit providing Python and TypeScript implementations) +- We support generating type-safe smart contract clients based on the smart contract definition +- We support deploying smart contracts to AlgoKit LocalNet, TestNet and Mainnet +- We support deploying manually and via continuous deployment pipeline + +## Principles + +- [AlgoKit Guiding Principles](../index.md#Guiding-Principles) - specifically: + - **Cohesive developer tool suite** + - **Seamless onramp** + - **Secure by default** + - **Modular components** +- **Continuous Delivery** - support the ability for software developers to adopt a [Continuous Delivery](https://continuousdelivery.com/) approach to reduce risk, namely by supporting: + - [Deployment pipelines](https://continuousdelivery.com/implementing/patterns/#the-deployment-pipeline) that build once and deploy to similar environments (that bit is nicely facilitated by the blockchain!) consistently + - [Automated testing](https://continuousdelivery.com/implementing/architecture/) +- **Facilitate correctness** - smart contract development is higher risk than many other types of development, standard practice involves deploying an immutable contract that must be right from the beginning; AlgoKit should help developers fall into the pit of success and produce higher quality output that is more likely to be correct while having flexibility to opt-in to other behaviours as needed + +## Decisions and design + +The following design decisions need to be considered, and are discussed below: + +- TEAL Templates and deploy-time parameter substitution +- Generated / Type-safe clients +- Deployment and development decoupling +- Upgradeable and deletable contracts +- Mnemonic storage and retrieval +- Contract identification +- Automated vs manual deployments +- Output stability testing +- Validation testing + +### TEAL Templates and deploy-time parameter substitution + +The above diagram includes a TEAL Templates step separate from the final TEAL that gets deployed. A fair question may be to ask if this extra step is really needed? + +There are two key considerations to help answer this question: + +1. Should development and deployment be decoupled from each other (i.e. happen at a separate time)? + - If we couple development and deployment together then it necessitates that at deploy time you have the same programming environment running that's needed for the smart contract development. So, if you (for instance) were building a smart contract in Python using PyTEAL or Beaker, but deploying the smart contract using TypeScript that means you need a deployment environment that supports both Node.js _and_ Python. This makes it harder to follow the Modular components principle. + - If development and deployment are coupled together it rules out using Continuous Delivery since it forces you to build the deployment artifact at the same time as you are deploying it. This means you miss out on the confidence and risk benefit of knowing that when you are deploying to (say) MainNet you are deploying the same artifact that was successfully deployed and tested on (say) TestNet and AlgoKit LocalNet (let alone passes any other checks you decide to run as part of Continuous Integration like automated tests, static code analysis, etc.). + - If development and deployment are coupled together it means we aren't perform an [output stability](../articles/output_stability.md) test so we don't get notified if we make a change that results in a different smart contract (which may then affect things like hashes for smart contract auditing review comparison, unintended introduction of security vulnerabilities, etc.). + - Based on all of this, decoupling development and deployment is a very helpful thing for a smart contract and aligns with all of the above-stated principles more closely. +2. Do we need to provide deploy-time parameters? + - When deploying a smart contract to a network (say MainNet), there are likely to be certain parameters that will be different from deploying to a different network (say TestNet), e.g.: + - If you are calling another smart contract, say an Oracle, then the application ID will change between networks. + - If you have standard prices you may decide to make them smaller on the TestNet contract given it's much harder to get access to a reasonable number of ALGOs on TestNet (without knowing the right people, or painfully clicking repeatedly on a CAPTCHA on one of the dispensers repeatedly to get 10 ALGOs at a time). + - If you are providing admin permissions for a statically defined account (hardcoded for security reasons) then it's likely you would use a different account address for MainNet vs TestNet so you don't expose a production mnemonic in test infrastructure. + - etc. + - Based on all of this, being able to provide deploy-time parameters is an important feature. + +Because it makes sense to decouple development and deployment, but also important to be able to provide deploy-time parameters, that means it's necessary to support deploy-time parameter substitution and thus: TEAL that is output from the development stage should be considered a template that may have deploy-time substitutions performed on it. + +Thankfully, this is supported as a first-class concept in PyTEAL via the [`Tmpl` feature](https://pyteal.readthedocs.io/en/stable/api.html?highlight=TMPL#pyteal.Tmpl) and could be similarly mimicked in any other TEAL transpilation language. + +### Generated / Type-safe clients + +Smart contract development results in an on-chain program that can be invoked from a "client". The development of the client itself has two broad options: + +1. Hand-code the client for a given smart contract using basic primitives (such as being able to issue a smart contract call of a certain type with an array of arguments) +2. Generate the client based on the smart contract definition and then call methods on the client that correspond to methods in the smart contract + +The second option, while more complex, results in an easier, faster, and safer developer experience: + +- You don't need to understand as much about the underlying blockchain calls since they will be constructed for you so the frontends (e.g. dApps) don't have to be constructed by smart contract / web3 experts +- You can have type-safe / intellisensed client SDKs, in multiple programming languages with no extra effort beyond writing the smart contract - making the developer experience much easier and meeting devs where they are +- Using a typed client means that smart contract calls (against the same version of the smart contract the client was generated from) will always be correct and should succeed so the client code is more likely to be correct and can be statically checked for correctness + +Because of this, the desired experience for AlgoKit is to encourage and directly support a generated / type-safe client experience. The intention is to drive this from a combination of [ARC-0004](https://github.com/algorandfoundation/ARCs/blob/main/ARCs/arc-0004.md) and [ARC-0032](https://github.com/algorandfoundation/ARCs/pull/150). + +To illustrate what the end result looks like consider the following Beaker smart contract: + +```python +from beaker.application import Application +from beaker.decorators import Authorize, delete, external +from pyteal import Approve, Bytes, Concat, Expr, Global +from pyteal.ast import abi + + +class HelloWorld(Application): + @external(read_only=True) + def hello(self, name: abi.String, *, output: abi.String) -> Expr: + return output.set(Concat(Bytes("Hello, "), name.get())) + + @delete(authorize=Authorize.only(Global.creator_address())) + def delete(self) -> Expr: + return Approve() +``` + +Let's say you wanted to deploy and interact with that smart contract using TypeScript; if you didn't have a client generated from that code then you would need to construct the method call: + +```typescript +// Assume `appId`, `algod` and `senderAccount` are already in scope +const composer = new AtomicTransactionComposer(); +composer.addMethodCall({ + appID: appId, + method: new ABIMethod({ + name: "hello", + args: [{ name: "name", type: "string" }], + returns: { type: "string" }, + }), // Not type-safe, no intellisense + sender: senderAccount.addr, + signer: makeBasicAccountTransactionSigner(senderAccount), + suggestedParams: await algod.getTransactionParams().do(), + methodArgs: ["World!"], // Not type-safe, no intellisense +}); +const result = await composer.execute(algod, 5); +console.log(result.methodResults[0].returnValue); // Hello, World! +``` + +If instead you generated a client you could having something like this, which gives you intellisense and is type-safe: + +```typescript +// Assume `appId`, `algod` and `senderAccount` are already in scope +// HelloWorldAppClient is generated from the smart contract definition (ABI json and app spec json) +const app = new HelloWorldAppClient(appId, algod, senderAccount); +const result = app.hello({ name: "World!" }); // Type-safe and intellisense +console.log(result); // Hello, World! +``` + +To be fair, you could have a middle-ground and load the ABI json to populate the `method` parameter of the `addMethodCall` call, but the `methodArgs` are still problematic and there is still no intellisense. + +The suggested implementation for AlgoKit v1 is to provide a basic type-safe TypeScript client (leveraging either the MakerX TypeScript generator or [beaker-ts](https://github.com/algorand-devrel/beaker-ts)) and leave Python with the semi-typed implementation that Beaker currently exposes (with implementing a fully typed Python client as a future implementation effort). + +### Deployment and development decoupling + +As discussed above, decoupling deployment and development of smart contracts is a useful technique. + +One of the advantages is it allows you to use separate programming languages for the writing of a smart contract and the deployment and automated testing of it. Separate, but related, it also makes it easier to generate type-safe clients (per the previous point) because there is an intermediate output from the development stage that can then be used to generate a client, in a separate (or the same) programming language, before using that client to then deploy and interact with the smart contract (for an end user experience, a programmatic interaction or an automated test). + +This helps us follow the "Meet devs where they are" principle and provide optionality for developers to select the programming environment they are most comfortable with for writing clients and automated tests. + +### Upgradeable and deletable contracts + +Smart contracts are a powerful capability that can be used for anything from locking billions of dollars of value to implementing the mechanics of a game's state storage. This means there are different risk profiles, rigour and functionality evolution characteristics in different circumstances. + +These different risk profiles have an impact on the functionality that is exposed within a smart contract. Two key examples of this are: + +- **Upgradeable smart contracts** - Whether or not a smart contract can be updated inline (and keep existing state and app ID / address characteristics) or they are immutable +- **Deletable smart contracts** - Whether or not a smart contract can be deleted or is permanent + +Immutability and permanence are useful architectural properties in certain circumstances (similarly mutability and impermanence in others). For example: + +- If you have a smart contract that locks billions of dollars of value, then allowing that smart contract to be upgradeable allows for the manager of the smart contract to make a change that lets them steal the value. +- If you have a smart contract that provides an Oracle service (say for betting odds), then allowing that smart contract to be deletable could break many other applications that are hard-coded to call that Oracle contract's app ID. +- If you have a smart contract that runs the mechanics of a game engine the players of said game may have a reasonable expectation that the game engine is evolved and enhanced over time, but they don't want to lose their state (so upgrading the smart contract is useful). +- If you have a smart contract that handles a one-off trade of value across a cross-chain bridge then it makes sense to delete it (only once trade has been concluded) to remove noise, potential confusion and operational overhead for the operators of said bridge. + +All 4 scenarios above provide different situations where immutability and permanence are desired or undesired. If you choose incorrectly for your circumstance, particularly for high-risk scenarios, the consequences could be major. + +If a contract is immutable, it does limit the ability to evolve the functionality over time based on user feedback and also discourages use of best practice software delivery techniques that encourage evolution and rapid deployment like Continuous Delivery. + +Another consideration in favour of immutability is smart contract auditing. If a smart contract is audited for security, but then the smart contract is upgradeable or is changed between when the audit occurred and when the contract was deployed to MainNet then the smart contract audit is somewhat invalidated and certainly any hashes that are provided of the smart contract code in an audit report will no longer match. + +There are techniques that can be used to allow for immutable smart contracts, but let them be evolved somewhat: + +- You could release a new version of a smart contract and include an ability for that smart contract to communicate to/from the existing smart contract to migrate state. +- You could ensure that clients can dynamically find the latest smart contract so the application ID / address doesn't need to be hardcoded and the smart contract remains addressable. + - This could be done via some sort of on-chain or off-chain lookup, and/or by encoding information into the creation transaction note of the smart contract app. +- You could limit MainNet releases to major upgrades that happen infrequently and let users opt-in to whether or not they use the new version or not, and having a set of calls the user can sign to migrate their state/value from one contract to the next. + +Lastly, it's worth noting that having fast development feedback loops during development and testing of smart contracts is likely a very useful feature to improve the development experience and speed. For this reason, allowing contracts to be upgraded or at the very least deleted (and then recreated) is likely very useful, but potentially when deployed to MainNet a switch could be made to disallow upgrading / deleting (as relevant). + +The goal of AlgoKit is to create a development experience that is productive and easy, but also one that is secure by default and helps developers fall into the pit of success. For that reason, and given the consequences of getting this wrong the suggested approach AlgoKit takes is: + +- All provided smart contract templates are by default immutable +- An immutability automated test is included by default to ensure that smart contracts can't be upgraded by the contract creator (this would have to be deleted by a developer, who is then opting in to the consequences of that) +- All provided smart contract templates are by default permanent when deployed to MainNet, but deletable elsewhere to facilitate an iterative development experience +- Client code will include mechanisms to dynamically find deployed applications in LocalNet and TestNet environments to support delete/recreate flows and improve the developer experience +- MainNet deployments will immediately (i.e. before any usage occurs) check that smart contracts are not upgradeable by the creator account by default (with an explicit opt-out option available for smart contracts that are meant to be upgradeable, which in turn will issue a warning to the developer to explain the implications) +- MainNet deployments will immediately (i.e. before any usage occurs) check that smart contracts are not deletable by the creator account by default (with an explicit opt-out option available for smart contracts that are meant to be deletable, which in turn will issue a warning to the developer to explain the implications) + +### Mnemonic storage and retrieval + +When deploying and interacting with a smart contract, you need to have access to the private key of an account. This is a secret and must be handled with care, as exposing a private key can be disastrous, and while [rekeying](https://developer.algorand.org/docs/get-details/accounts/rekey/) is possible if it's not done fast enough you can still lose assets, be victim to malicious calls and experience a painful user experience going forward (wallet support for rekeyed accounts is limited). + +Another consideration is the network being deployed to / called. If you are interacting with the LocalNet network then mnemonics are all, but meaningless since you can simply reset the LocalNet and regenerate new accounts on the fly (and fund them with essentially unlimited ALGOs). If you are interacting with TestNet then mnemonics may hold TestNet ALGOs, which while difficult to get in large numbers, are more an inconvenience than a serious commercial problem to lose. + +Finally, when interacting with LocalNet to create a smooth developer experience it's ideal to automatically generate and fund any accounts that are being used so the developer doesn't have to manually do this every time the LocalNet is reset. Even better, it's ideal if this can be done in a way that idempotently gets a consistently private key for a given "named account" so that subsequent calls use the same account (mimicking what happens in TestNet or MainNet when using a particular private key for a given "named account"). + +Given all of this, the suggested approach that AlgoKit takes is: + +- LocalNet accounts are by default automatically and idempotently generated against a named account by using a named wallet via [Kmd](https://developer.algorand.org/docs/clis/kmd/) and are automatically funded using the LocalNet faucet account (the private key for which is automatically retrieved using Kmd). +- Where they are needed mnemonics will be provided using environment variables to follow [twelve factor app conventions](https://12factor.net/config), this is an industry standard approach to handling secrets and is easy to support cross-platform and cross-programming language as well as using encrypted secrets on CI/CD pipelines. +- An option will be provided for deployments that allows for deployments using ephemeral accounts that then get rekeyed to a separate, known [break-glass](https://www.beyondtrust.com/blog/entry/provide-security-privileged-accounts-with-break-glass-process) account (the private key of which is not available to the deploying process) will be provided to allow for developers to deploy using a break-glass setup. +- A `DISPENSER_MNEMONIC` environment variable will be expected when deploying to non-LocalNet environments, and will be encouraged to be a separate account just used for that purpose to limit [blast radius](https://www.lepide.com/blog/what-is-a-blast-radius-in-data-security-terms/), so that funds needed for deployments or calls can be automatically provided by convention, including for ephemeral accounts. +- The Algokit CLI will allow for mnemonics to be provided to it for a given project, which will get stored in a encrypted form within the .gitignore'd `.env` file with a project-specific random encryption key stored on that machine. This prevents cursory exploitation and accidental exposure through screen-sharing, but won't protect users with an exploited machine from having them exposed. For this reason, developers will be discouraged from storing MainNet mnemonics in that way and will need to acknowledge that risk. +- AlgoKit will provide example CI/CD templates that illustrate how to construct a deployment pipeline that includes MainNet deployments using secret storage for mnemonics so developers won't need to handle MainNet mnemonics on their local machine. + +### Contract identification + +Being able to identify an existing deployed instance of a given smart contract is very useful: + +- It avoids the need to hardcode application IDs +- It makes things easier to automate, including automated deployments and testing of smart contracts and the apps that call them +- It allows the deployer of a smart contract to detect if that smart contract is already deployed and if so handle it appropriately (e.g. do nothing vs upgrade vs delete and create vs leave alone and create) depending on whether that smart contract is immutable and/or permanent and the network being deployed to (e.g. LocalNet vs TestNet vs MainNet) + +As soon as a contract is not immutable, or is immutable and not permanent then the application ID of the smart contract for a given network will change over time. And, if a smart contract is immutable and permanent then net new versions may still be deployed, or at the very least the contract will change across networks (e.g. LocalNet vs TestNet vs MainNet). Because of this it's important to support dynamic resolution of application IDs. + +It's important to consider whether the smart contract needs to be resolved on-chain or off-chain. + +Resolving on-chain is harder to achieve dynamically, but there are some patterns that can be used, e.g.: + +- Storing the application ID in (e.g. Global) state and providing a creator-only ABI method that can be called as part of deployment of the dependant contract to update the stored application ID. +- [Lookup/registry contract](https://research.csiro.au/blockchainpatterns/general-patterns/contract-structural-patterns/contract-registry/) that returns the ID of a named contract and that lookup contract allows said ID to be updated. +- [Proxy contract](https://blog.openzeppelin.com/proxy-patterns/) that mirrors the interface of the parent contract but delegates calls to an underlying contract whose ID can be updated. +- [Other patterns](https://ethereum.org/en/developers/docs/smart-contracts/upgrading/#what-is-a-smart-contract-upgrade). + +Resolving off-chain can be done through a variety of ways, e.g.: + +- Create a contract discovery service API that allows new contracts to be registered by API call after being deployed. +- If the contract creator account is always known (i.e. ephemeral creator accounts aren't being used) then it's possible to identify a contract by name by encoding a payload into the application creation transaction note and then using the indexer to find the application creation transactions for a given account and working backwards to find the relevant application ID. +- If the contract creator account is always known (i.e. ephemeral creator accounts aren't being used) then it's possible to identify a contract by name by encoding the name in to one of the [application params](https://developer.algorand.org/docs/rest-apis/algod/v2/#applicationparams) (e.g. approval program or using a global state variable) and looking up the `created-apps` property when [retrieving an account via algod](https://developer.algorand.org/docs/rest-apis/algod/v2/#account). +- Create a contract discovery service API that scans the blockchain transactions ([example](https://developer.algorand.org/articles/developer-preview-of-conduit-a-new-way-to-access-algorand-chain-data/)) to automatically maintain a record of smart contract IDs. +- Use CI/CD variables ([example](https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-output-parameter)) to propagate the deployed app ID to the configuration on the deployment of the client (assuming the same pipeline is used for deployment of smart contract and client(s)). +- Manually set the application ID via the CI/CD pipeline of the clients with environment-specific values (that are passed in via environment variables) and change them to reflect any changes to the smart contracts (note: this isn't sustainable for frequently changing contracts, but is a simple method for infrequently changing contracts). + +In order to provide the simplest implementation possible to handle this complexity, AlgoKit v1 will: + +- Add payloads to application creation transactions to allow them to be resolved by name for a known creator account using the indexer. +- Allow application IDs to be resolved by name using environment variables for situations where a developer is using ephemeral creator accounts. +- On-chain resolution will be left for developers to implement (likely by hardcoding or Global-state-based configurable application IDs). + +More sophisticated options could potentially by implemented in the future. + +### Automated vs manual deployments + +In order to support Continuous Delivery for smart contracts there is a need to support automated deployments of smart contracts via deployment pipeline. + +This presents a number of challenges to solve for though: + +- The deployment pipeline needs to determine if the smart contract already exists or not in the network being deployed to +- Depending on whether the contract is immutable and/or permanent and if it already exists or not in the network being deployed to, the pipeline needs to handle upgrade, deletion and/or creation of the contract (and this behaviour may need to be switched depending on the network being deployed to) +- In order to protect integrity of MainNet contracts the deployment process should probably have some safeguards for MainNet that by default prevent destructive operations and require some kind of human opt-in to perform those operations +- Exposing private key mnemonics for MainNet necessitates they are "hot wallets" which may be undesirable for privileged accounts responsible for high value smart contracts (e.g. smart contracts locking $m's or $b's), alternatively low privilege scenarios may benefit from (encrypted secret) mnemonic storage to improve developer experience and operational overhead +- Supporting cold storage wallets for high privilege accounts, without resorting the manual deployments (which present their own risks) is tricky +- There are many deployment pipeline technologies available on the market and providing support for many of them is an impractical amount of effort. +- Should manual deployments be supported as well as automated deployments to make it easier for developers to perform ad hoc deployments (e.g. when quickly testing concepts, or doing a one-off deployment) and improve developer flexibility + +The proposed AlgoKit implementation for v1 that provides a balance of flexibility vs implementation effort, while aligning to the principles is as follows: + +- A GitHub Actions implementation is provided in the default template, since GitHub is a highly capable and prevalent option and is free for Open Source and basic private usage (the community can contribute other CI/CD implementations if desired) +- The "Contract identification" v1 implementation suggested above is followed to determine if a contract already exists for the network being deployed to +- The "Mnemonic storage and retrieval" v1 implementation suggested above is followed to facilitate a secure, but flexible implementation for how to handle private keys for ad hoc manual deployments and CI/CD pipelines +- Cold wallets won't be supported for now, but is recommended as a future exploration, in the meantime that scenario can be implemented by using the rekey technique mentioned in "Mnemonic storage and retrieval" + +### Output stability testing + +Ensuring [output stability](../articles/output_stability.md) is useful for smart contract development. It helps ensure that you can refactor code without making an inadvertent change to the smart contract, ensure that a smart contract output isn't changed from any output that is audited, and ensure that there is a clear mechanism to manually review the smart contract output before it's (re-)deployed. + +There are three broad approaches that could be taken by AlgoKit to help facilitate output stability testing: + +1. Include documentation that recommends this kind of testing and provides examples for how to implement it +2. Include a Git-based approach in default AlgoKit template(s), that requires you to stage changes to the existing smart contract output (per decoupling development and deployment) using normal Git workflows otherwise the test will fail (meaning it will fail locally and on continuous integration pipeline) +3. Include an [approval-testing](https://approvaltests.com/) based approach in default AlgoKit template(s), that results in an approved TEAL file that is committed + +A documentation-only approach strays away from the "Facilitate correctness" and "Secure by default" principles where we want to help developers fall into the pit of success by default. + +Approval testing is a handy technique that is established in the industry, but in this case results in the TEAL output being committed twice, which is a confusing duplication. Furthermore, by ensuring the automated test is against the output that will get deployed it ensures there is a coherence between the TEAL being tested and the TEAL being deployed. + +With this in mind, the proposal is for AlgoKit to include a Git-based output stability test by default, but per the "Modular components" principle there is a template option to exclude those tests. + +### Validation testing + +In order to provide confidence in the correctness of a smart contract it's important to exercise testing to validate the smart contract operates as expected. + +There are a few possible approaches that could be taken by AlgoKit to help facilitate this: + +1. **Documentation** - Include documentation that recommends this kind of testing and provides examples for how to implement it +2. **Manual testing** - Encourage a manual testing approach using (for example) the ABI user interface in dAppFlow, by providing an AlgoKit CLI command that sends the user there along with the ABI definition and contract ID resulting in a manual testing experience for the deployed contract with low friction +3. **Automated integration tests** - Facilitate automated testing by issuing real transactions against a LocalNet and/or TestNet network +4. **Automated dry run tests** - Facilitate automated testing using the [Dry Run endpoint](https://developer.algorand.org/docs/rest-apis/algod/v2/#post-v2tealdryrun) to simulate what would happen when executing the contract under certain scenarios (e.g. [Graviton](https://github.com/algorand/graviton/blob/main/graviton/README.md)) +5. **TEAL emulator** - Facilitate automated testing against a TEAL emulator (e.g. [Algo Builder Runtime](https://algobuilder.dev/api/runtime/index.html)) + +#### Documentation + +**Pros** + +- Least effort to implement + +**Cons** + +- Doesn't follow the principles of "Seamless onramp", "Continuous Delivery" or "Facilitate correctness" +- Easy for users to miss + +#### Manual testing + +**Pros** + +- Low effort to implement +- Facilitates a great manual testing experience for exploratory testing or situations where you want/need to manual test a contract in addition to automated testing + +**Cons** + +- Doesn't follow the principle of "Continuous Delivery" or "Facilitate correctness" +- Doesn't provide regression coverage as the smart contract evolves during development + +#### Automated integration tests + +**Pros** + +- Prior art can be leveraged from MakerX (TypeScript) and [algopytest](https://github.com/DamianB-BitFlipper/algopytest) (Python), both of which provide abstractions to make it easier to produce tests that avoid intermittent failures +- High degree of confidence - exercising the smart contract in a similar way to real users means we have a high degree of confidence in the validation +- Good regression coverage + +**Cons** + +- This type of testing is naturally slower making it impractical to provide combinatorial coverage (relevant: [first 5 minutes of the Microtesting presentation Rob and Matt delivered in 2016](https://www.youtube.com/watch?v=pls1Vk_bw_Y)) +- This type of testing can be hard to make reliable (it's easy to get intermittent timing errors if you aren't careful) + +#### Automated dry run tests + +**Pros** + +- Faster test runs allows for testing a larger proportion of combinatorial coverage and adopting approaches like property-based testing to provide higher degree of confidence +- Allows for validation of additional properties like opcode usage etc. + +**Cons** + +- Verbose / unfamiliar test setup to specify the desired state of the blockchain before the dry run (which won't be shareable code with any clients like dApps since it will be test specific) +- Highly likely there will be a coherence gap between test setup and real-life call since +- High effort to implement, particularly cross-platform since there isn't existing TypeScript-based prior art +- Dry run endpoint is being replaced with a new simulate endpoint, but there isn't much available about what that endpoint will look like yet so need to decide between implementing something that is + +#### TEAL emulator + +**Pros** + +- Likely to be the best speed properties allowing for full combinatorial coverage + +**Cons** + +- Requires implementation and/or maintenance of a TEAL emulator, which would duplicate effort being put in by Algorand Inc. on the simulate endpoint +- Algo Builder implementation, while being an existing solution and OpenSource requires TypeScript (so is harder to use for Python testing) and also requires a highly bespoke syntax to interact with it that won't allow for easy interoperability with AlgoKit or other things, thus not confirming with "Modular components" principle) + +#### Selected option + +Based on all of this the suggested option for AlgoKit v1 is **Automated integration tests** since it conforms to the principles well, has prior art across TypeScript and Python that can be utilised and provides developers with a lot of confidence. + +Post v1, it's recommended that dAppFlow integration for exploratory testing and Graviton (or similar) support should be explored to provide a range of options to empower developers with a full suite of techniques they can use. diff --git a/docs/get-details/algokit/architecture-decisions/2023-06-06_frontend-templates.md b/docs/get-details/algokit/architecture-decisions/2023-06-06_frontend-templates.md new file mode 100644 index 000000000..0df58389a --- /dev/null +++ b/docs/get-details/algokit/architecture-decisions/2023-06-06_frontend-templates.md @@ -0,0 +1,164 @@ +# Frontend Templates + +- **Status**: Approved +- **Owner:** Altynbek Orumbayev +- **Deciders**: Rob Moore, Daniel McGregor, Adam Chidlow +- **Date created**: 2023-06-06 +- **Date decided:** 2023-06-09 +- **Date updated**: 2023-06-08 + +## Context + +AlgoKit v2 aims provide an end-to-end development and deployment experience that includes support for the end-to-end smart contract and dApp development lifecycle. With the release of the typed clients feature - developers are now able to reduce the time it takes to fully integrate the interactions between the contract and the frontend components powering the end-to-end dapp user experience. Hence, as a logical continuation, the following Architecture Decision Record aims to expand on the current capabilities of AlgoKit to support the `frontend` templates, an [AlgoKit Principles](https://github.com/algorandfoundation/algokit-cli/blob/main/docs/algokit.md#guiding-principles) compliant approach towards simplifying integrations of smart contracts with the dApp frontend components. + +## Requirements + +### Independent Frontend Templates + +1. Create official frontend template that complies with AlgoKit's principles of modularity, maintainability, and flexibility while also serving as a reference for template builders. + +2. Expand algokit functionality to provide a clear process for developers to link frontend templates with the typed clients generated by backend templates. + +3. Design frontend templates such that backend and frontend have no dependencies on each other, ensuring high modularity. + +### End-to-End Starter Repositories + +1. Establish a starter repository that allows developers to bootstrap end-to-end dApp projects within a single command, while also serving as an example for template builders on how to efficiently couple their backend and frontend templates. +2. Implement a bundling process for backend and frontend templates in the starter repositories that reduce code duplication and maintenance overhead. +3. Develop deployment pipelines for frontend components inside end-to-end starter repository that supports TestNet, and MainNet deployments to hosting providers of choice (Netlify, Vercel and etc to be decided). +4. Ensure that the end-to-end starter repository supports both manual deployment and deployment via a continuous deployment pipeline. + +## Principles + +- **Modularity**: The dApp templating feature should follow guiding AlgoKit principles and expand on approaches already utilized in existing smart contract templates feature. This implies that giving developers flexibility to mix and match different `smart contract` templates with `frontend` templates should serve as key consideration. +- **Maintainability**: The dApp templating feature should be easy to maintain and extend. This implies that the feature should be implemented in a way that allows for easy addition of new templates and/or modification of existing ones as the complexity and variety of templates scale. +- **Seamless onramp**: The dApp templating feature should provide a seamless onramp for developers to get started with dApp development. This implies that the feature should provide a simple and intuitive way to get started with dApp development and deployment. Providing developers a choice on whether they want more flexibility or rely on default recommended practices. + +All of the aforementioned requirements should be met in a way that is consistent with the guiding principles of AlgoKit or attempt to find a balanced trade of between the principles that satisfies the requirements. Refer to [AlgoKit Guiding Principles](../index.md#Guiding-Principles) for detailed reference on the principles. + +## Explored options + +Enhancing AlgoKit's templating capabilities involves a notable shift from managing individual templates to co-locating multiple templates or projects together. This is a marked increase in complexity that requires careful consideration of the varying trade-offs between different approaches. The subsequent sections explore these options and their associated trade-offs, providing a concise overview of this new challenge and our proposed solutions. + +### Option 1: Monolithic template + +This option suggests that the frontend and backend templates should be bundled together in a single repository. This approach aims to simplify the maintenance of the templates by reducing the number of repositories that need to be maintained. Both official and community templates should be build on top of this approach where frontend is always tailored for specific needs of the smart contracts that are being used. + +Drawbacks: + +- **Lack of flexibility**: The monorepo approach does not allow developers to choose their preferred frontend technology stack. This is because the frontend is tightly coupled with the backend and the two cannot be separated. This is a major drawback as it limits the flexibility of the templates and does not allow developers to choose their preferred frontend technology stack. +- **Increased maintenance overhead**: Despite monorepos generally being easier to maintain and collaborate on, this approach has risks of increasing the maintenance overhead as it forces for code duplication when creating new template variations. + +### Option 2: Separate individual frontend templates and official full-stack starter templates as an end-to-end reference + +This option suggests that the frontend and backend templates should be maintained in separate repositories. This approach aims to provide developers with the flexibility to choose their preferred frontend technology stack. It also allows the templates to be maintained independently, thereby reducing the maintenance overhead. + +Drawbacks: + +- **Compatibility overhead**: The separate repositories approach increases the maintenance overhead as it requires the templates to be updated independently. This can lead to issues with compatibility between the templates and the typed clients. +- **Lack of guidance**: The separate repositories approach does not provide developers with a clear process for linking the frontend templates with the typed clients generated by the backend templates. This can lead to confusion and issues with the integration of the templates. + +The outlined drawbacks can be mitigated by providing a clear process or reference for template builders on how to link various combinations of frontend and backend templates. The compatibility overhead can be mitigated by ensuring that backend and frontend modules are self sufficient and completely decoupled from each other. This will ensure that the templates can be updated independently. The implementation proposal below is a detailed exploration of this approach and ways to mitigate the drawbacks. + +### Implementation proposal for Option 2 + +The proposal consists of 2 main parts. On a high level, the ideas revolve around giving developers a choice on how they want to couple AlgoKit backend and frontend templates if they like to use custom templates. The other is focused on providing an official full-stack template that bundles both backend and frontend templates together, while also serving as an example for template builders on how to efficiently couple any combination of templates. + +The addendums to the proposal also explores orthogonal ideas that can further improve the CLI tooling itself by providing a way for any existing non web-3 frontend project be converted into a full-stack dApp with minimal efforts. Lastly, it expands on improving incentives for developers to build and maintain their own templates. + +--- + +### Part 1. Independent frontend templates + +> TLDR: Independent frontend templates will be created to provide developers with a highly customizable dApp starter project, built on AlgoKit's principles of modularity, maintainability, and flexibility. The templates will also serve as a reference for template builders. + +The following aims to provide a seamless onramp for developers to get started with highly customizable dApp starter projects. The idea is to create a set of separate official frontend template repositories to serve as: +a) A reference for template builders on how to create standalone frontend templates that can be then further coupled with any backend template. +b) Expand on AlgoKit principles of modularity, maintainability and flexibility by giving developers a choice of preferred technological stack. + +The official standalone frontend templates can be build by reusing already established best practices and templates from official backend repositories and by continuing reliance on `copier` for template automation. Another important consideration to keep in mind is that with the introduction of frontend templates we need to establish a clear separation of concerns between the backend and frontend templates to ensure modularity. + +![Diagram 1](assets/2023-06-06_frontend-templates/modular_templates.jpg) + +As demonstrated on the diagram above, the only glue connecting the backend and frontend is the generated typed client. Neither backend or frontend templates should be concerned with the other but instead provide modular interfaces that clearly indicate to developers on how to integrate the two. From a perspective of a backend template, the typed client shall be seen as a static asset that can be reused by any frontend template. Frontend templates on the other hand are mostly standard web projects with an additional layer of utilities that optionally allow them to be integrated with typed clients produced by backend templates. + +#### Higher level overview + +The main scenario to support for this part is to allow developers to use official starter templates to bootstrap end to end dApp projects. + +![Diagram 2](assets/2023-06-06_frontend-templates/scenario_1.jpg) + +As demonstrated above the dev experience will consist of executing an `algokit init` command for the preferred backends and frontends. + +It gives user a choice and responsibility to then decide how to integrate the two components depending on their project needs. To improve this however, we should **additionally introduce a new utility** that will serve as a tool to automate linking with the typed client that backend templates will be generating. Implementation specific details can be discussed separately is it goes out of scope of this Architecture Decision Record. + +--- + +### Part 2. End-to-end starter repositories + +> TLDR: End-to-end starter repositories are designed to offer developers an official starter template for bootstrapping dApp projects. This is achieved by efficiently bundling backend and frontend templates to facilitate easy maintenance and smooth onboarding. + +#### Higher level overview + +The main scenario to support for this part is to allow developers to use official starter templates to bootstrap end to end dApp projects. + +![Diagram 2](assets/2023-06-06_frontend-templates/scenario_2.jpg) + +As demonstrated above the user experience will consist of a single execution of `algokit init` command pointed at official full-stack template repository. The full stack templates are responsible for bundling both backend and frontend templates together and providing a seamless onramp for developers to get started with dApp development. The way repositories are bundled should be easy to maintain and should not duplicate individual backend and frontend repositories to avoid redundant maintenance, instead it should expand on metatemplating capabilities of `copier` to allow for efficient reuse of existing standalone backend/frontend templates. + +### Addendum 1. Converting _ANY_ frontend projects into dApps. + +> TLDR: The approach aims to enhance the algokit-cli codebase's adaptability, enabling easy transformation of existing frontend projects into web3 dApps. + +This orthogonal approach proposes to improve capabilities of the algokit-cli codebase by making it adaptable to various frontend stacks to allow anyone to easily convert their existing frontend projects into web3 dApps. + +The implementation specific details will consist of deriving a set of bare minimum requirements for such feature to scan and understand the structure of frontend where algokit is being embedded and performing necessary modifications to project's files. A detailed discussion can be held in a scope separate from this Architecture Decision Record. + +> This approach can be explored and maintained without overlapping with main proposal on frontend templates. + +### Addendum 2. Website for choosing preferred frontend and backend repositories. + +> TLDR: The approach proposes a website to enhance the discoverability of official and community-based algokit templates, thereby incentivizing template builders to create and maintain their own templates. + +This orthogonal approach proposes to improve discoverability of official and community based algokit templates by providing a simple static website. The website can consist of minimalistic UI components for picking preferred backend, frontend and then a `Generate` button that will output copyable algokit CLI commands to spin up a project with the selected templates. + +As a specific example, the website can be hosted on [AwesomeAlgo](https://awesomealgo.com) website, thus ensuring that this is an open-source community maintained entry-point for discovering and using algokit templates. Removing the need and maintenance overhead on our teams to maintain it as official resource. + +Lastly, community template builders will get a platform to increase discoverability of their templates and further incentivize them to build and maintain them. While developers using the templates can support creators of templates by donating to their projects (a simple tipping mechanism for Algo and ASAs can be embedded into the website) or by contributing to the templates themselves. + +> This approach can be explored and maintained without overlapping with main proposal on frontend templates. + +--- + +## Final decision + +After several review rounds team reached a conclusion to expand on [Option 2](#option-2-separate-individual-frontend-templates-and-official-full-stack-starter-templates-as-an-end-to-end-reference) and respective [implementation proposal](#implementation-proposal-for-option-2) given better alignment with AlgoKit design principles such as flexibility and modularity. Hence, the following proposal is further exploration of ideas based on this approach and propose solution to mitigate the potential drawbacks and risks outlined. + +## Open questions + +- Python typed clients are not going to be too relevant for scenarios where they need to be integrated with frontends given that in majority of cases a developer would prefer a TS typed client. Hence, how do we ensure that user gets a clear information and indication on when to use Python typed clients vs TS typed clients? + - `Answer: This is a fair comment, but to my mind it's a clear dilineation in that it's dependant on what language you are using. While there are Python website libraries, we aren't using them we will use JavaScript (TypeScript).` +- Would we want to introduce a notion of non smart contract based backend templates that can be used to plug in the python typed clients and spin up servers that can be used to build APIs that interact with the smart contracts? + - `Answer: This is definitely out of scope of dApp stuff.` + +## Next steps + +After the final decision is made, the action items necessary to implement the described proposal can be outlined as follows: + +1. **Design and Development of Independent Frontend Template**: This involves selecting appropriate technology stacks and building out the templates. The template should be able to work with the generated typed client from the backend and will be used as a dependency for end-to-end starter repository. + +2. **Development of End-to-End Starter Repository**: This involves building comprehensive template that bundles both frontend and backend components. The template should be designed to be easy to maintain and should leverage the metatemplating capabilities of `copier` for efficient reuse of existing standalone templates. + +3. **Integration and Testing**: Ensure proper integration between frontend and backend templates. Also, extensive testing should be done to ensure smooth functioning and a seamless onramp experience for the developers. + +4. **Documentation**: Write comprehensive documentation covering the use of the new templates, how to integrate them, and the utility for linking them with the typed client. This documentation should also include how-to guides and sample applications to help developers get started. + +5. **Addendum 1 - Converting ANY Frontend Projects into dApps**: Start a separate discussion and potentially a project to research the feasibility of this idea. If feasible, design and implement a method that allows developers to convert their existing non-web3 frontend projects into dApps using AlgoKit. + +6. **Addendum 2 - Development of Template Selection Website**: Plan and execute the development of a minimalist UI that allows developers to easily discover and select their preferred templates. This should also allow them to easily generate the necessary AlgoKit CLI commands for their project setup. + +7. **Community Engagement**: Engage the developer community to drive contributions to the template repositories. This includes encouraging template builders to contribute and supporting developers using the templates with issues and suggestions. + +8. **Continuous Review and Maintenance**: Regularly review the templates to ensure they are up to date with changes in technology and AlgoKit principles. Continuous maintenance should also be carried out to ensure the templates remain functional and relevant. + +Regarding the open questions, these should be discussed in detail to clarify how to handle Python typed clients and the potential introduction of non-smart contract based backend templates. These discussions may lead to additional actions as required. diff --git a/docs/get-details/algokit/architecture-decisions/2023-07-19_advanced_generate_command.md b/docs/get-details/algokit/architecture-decisions/2023-07-19_advanced_generate_command.md new file mode 100644 index 000000000..59a897694 --- /dev/null +++ b/docs/get-details/algokit/architecture-decisions/2023-07-19_advanced_generate_command.md @@ -0,0 +1,203 @@ +# Advanced `algokit generate` command + +- **Status**: Approved +- **Owner:** Altynbek Orumbayev, Inaie Ignacio +- **Deciders**: Rob Moore, Daniel McGregor, Alessandro Ferrari +- **Date created**: 2023-07-19 +- **Date decided:** 2023-07-24 +- **Date updated**: 2023-07-24 + +## Context + +The [Frontend Templates ADR](./2023-06-06_frontend-templates.md) introduced and expanded on AlgoKit's principles of Modularity and Maintainability by introducing a new set of official templates for quickly bootstrapping standalone `react` and `fullstack` projects showcasing best practices and patterns for building frontend and fullstack applications with Algorand. As a logical next step, we want to enable developers to extend existing projects instantiated from official templates with new files and features. + +## Requirements + +### 1. AlgoKit user should be able to use generate command to extend existing algokit compliant projects with new `files` of any kind + +This implies scenarios like: + +- Adding new contracts into existing algokit compliant projects. + > Algokit compliant projects are projects that were instantiated from official or community templates and follow the same structure and conventions. +- Overriding existing files with new ones. +- Adding new files into existing projects. + +Overall, we want to introduce a notion of `generators` which can be viewed as a modular self-sufficient template units that are hosted within template repositories and describe how to create or update files within projects instantiated from AlgoKit templates. + +Ruby on Rails has a similar concept of [generators](https://guides.rubyonrails.org/generators.html) which are used to create or update files within Rails projects. This can be used as a reference for inspiration. + +### 2. Template builder should be able to access a clear guideline and refer to official templates for examples on how to create `generators` + +This implies extension of existing starter guidelines available for template builders on [AlgoKit Docs](https://github.com/algorandfoundation/algokit-cli/blob/main/docs/tutorials/algokit-template.md) and using one or several official templates as a reference point. + +## Principles + +- **Modularity**: Artifacts dependant on `advanced algokit generate` command capabilities embedded into templates should follow guiding AlgoKit principles and expand on approaches already utilized in `react`, `fullstack` and `beaker` templates. This implies that giving developers flexibility to define any extra templating logic, allowing to create or update any files within projects instantiated from algokit templates. +- **Maintainability**: The `advanced algokit generate` capabilities on `algokit-cli` and related artifacts on respective official templates should be easy to maintain and extend. +- **Seamless onramp**: Great developer experience for template builders to create their own `generators` and user experience to use them via `advanced algokit generate` command should be a priority. + +All of the aforementioned requirements should be met in a way that is consistent with the guiding principles of AlgoKit or attempt to find a balanced trade of between the principles that satisfies the requirements. Refer to [AlgoKit Guiding Principles](../index.md#Guiding-Principles) for detailed reference on the principles. + +## Considered Options + +Based on preliminary research, all of the options below assume that: +A `generator` is a self contained copier/jinja template that is hosted within a template repository and describes how to create or update files within projects instantiated from algokit templates. Hosting it along with the template is a necessity given that community based templates can follow different conventions, patterns and structure making it hard to attempt to generalize the logic of `generators` and make them work for all templates. + +### Option 1: Wrapping generators into self contained copier templates hidden within algokit templates + +This option implies that `generators` are self contained copier templates that are hidden within algokit templates and are not exposed to the end user. This option is inspired by [Ruby on Rails generators](https://guides.rubyonrails.org/generators.html) and [Yeoman generators](https://yeoman.io/authoring/). + +The main idea is to rely on `_templates_suffix` in copier.yamls to define 2 separate types of suffixes for `templates` and for `generators`: + +- Existing templates under all official algokit templates are already prefixed with `.jinja` hence we just need to explicitly prefix it with `.jinja` on root copier +- The new generators jinja templates can be prefixed (for example) with alternative file extension for jinja files such as `.j2`. Which is also a common convention for jinja templates. +- - This only works for files though for regular folders and cases like `{% if %}folder_name{% endif %}.j2` we need to wrap them into {% raw %} to that first pass when template initially initialized unwraps the content allowing second pass via generator to then use them as jinja templates. The only downside here is slightly longer file names for folders, but I think it's a reasonable tradeoff considering simplicity of the solution. + +Overview of the proposal can be summarized via the following diagram: + +```mermaid +graph TB + T["Template Folder"] + T --> C["copier.yaml"] + T --> C1["..."] + T --> G[".algokit/generators"] + + G --> G1["Generator 1"] + G1 --> E1["copier.yaml"] + G1 --> E2["..."] + + G --> G2["..."] + G2 --> E3["..."] + + G --> G3["Generator N"] + G3 --> E["copier.yaml"] + G3 --> E4["..."] +``` + +#### Pros + +- Generators are hidden within algokit templates and are not exposed to the end user. When user runs `algokit generate` command, cli presents a list of available generators to choose from. This makes it easier for user to understand what generators are available and what they do. +- Generators are self contained copier templates giving template builders flexibility to do any kind of templating logic similar to what they can already do with regular templates. +- Majority of implementation complexity is reduced by relying on copier as a backbone for generators feature. + +#### Cons + +- Generators are somewhat tightly coupled with individual algokit templates, which implies its not necessarily a matter of copy pasting generators from one template to another. This can be a problem for community template builders who want to reuse generators from official templates. However, this can be mitigated by providing clear guidelines on how to create generators and referring to official templates as a reference point. Additionally, it seems like a reasonable tradeoff given that templates can vastly differ in type, structure, conventions and patterns and it's significantly harder to generalize the logic of generators to make them work for all templates. + +#### Implementation details + +**1. Adjusting templates structure** + +For instance, if we assume existing `beaker` template, the new file/folder structure can look as follows: + +``` +template_content/.algokit # alternatively could be just `.algokit-generators` +└── generators + └── {generator_name} # generator name can be anything + ├── copier.yaml # copier config for generator + └── smart_contracts # logic for adding new contracts to beaker template + └── {% raw %}{{ contract_name }}{% endraw %} + ├── contract.py.j2 + ├── {% raw %}{% if language == 'python' %}deploy_config.py{% endif %}{% endraw %}.j2 + └── {% raw %}{% if language == 'typescript' %}deploy-config.ts{% endif %}{% endraw %}.j2 +...rest of the template is left as is +copier.yml +``` + +The `index.ts` and `config.py` files on beaker template need to be updated to auto import all contracts from sub folders at `smart_contracts` to eliminate the need for developers to manually import them after running the smart contract generator. + +> Please note, above is just an example that assumes the generator for adding new contracts, but the proposal is generic enough to support any kind of jinja-based templating logic. + +**2. Adjusting `.algokit.toml`** + +The proposal for new structure for defining generators in root algokit toml is as follows: + +```toml +[generators.create_contract] # [generators.] +description = "Adds new smart contract to existing project" # description of the generator, can appear in cli for extra info +path = ".algokit/generators/create_contract" # path that cli should grab to forward to copier copy +``` + +**3. Adjusting `algokit generate` command on cli** + +Next step in implementation part of the proposal is adjusting the `generator` command on `algokit-cli` to make sure it knows how to look for generators. The available generators can be provided to user via list picker (a.k.a ruby on rails style) by letting algokit scan contents of `algokit.toml` and look for `.generators` folder. + +a) Has no generators +If algokit-cli can't find any generator configured then nothing change from current implementation. + +b) Has generators and user runs `algokit generate` command to see the list of available generators +A new `click` command per each generator in `.algokit.toml` is added to the list of generate commands, that will list all available generators for the user to choose from. + +```bash +algokit-cli generate + +--- + +Usage: algokit generate [OPTIONS] COMMAND [ARGS]... + + Generate code for an Algorand project. + +Options: + -h, --help Show this message and exit. + +Commands: + client Create a typed ApplicationClient from an ARC-32 application.json + smart-contract Adds new smart contract to existing project +``` + +Then, to invoke the interactive flow via copier user runs: + +```bash +algokit-cli generate smart-contract `name of the contract` +``` + +c) Has generators, user knows what to pick and wants to run it non interactively +Using the generator configuration the `generator` will automatically add new `click` commands to the existing list of generate commands allowing the user to run them in the command line. + +```bash +algokit-cli generate smart-contract -a contract_name=`name of the contract` -a language=python # passing extra arguments to generator similar to algokit init +``` + +**4. Testing and documentation** + +Lastly we need to make sure that the new feature is properly tested and documented. This includes: + +- Testing the new feature works as expected on all official templates that will host generators. This will imply adding a separate test suite that picks some default template state and run generators on top of them confirming that created artifacts are placed in expected locations and have expected content. Tests should be easy to follow and expand on existing tests suite on templates without introducing extra complexity. +- Introduce new tutorial sections on [AlgoKit Docs](https://github.com/algorandfoundation/algokit-cli/blob/main/docs/tutorials/algokit-template.md) on how to create generators and refer to official templates as a reference point. + +### Option 2: Wrapping generators into self contained copier templates hosted on separate repositories + +This option proposes to host `generators` on separate set of repositories and use them as a dependency for algokit templates. +`algokit.toml` can be extended on template repositories to list generators they depend on. + +The only distinction between this option and option 1 is that generators are hosted on separate repositories and are not hidden within algokit templates. Implying that they are not tightly coupled with algokit templates and can be reused/forked by community template builders to build their own generators. + +#### Pros + +- Generators are not tightly coupled with algokit templates and can be reused/forked by community template builders to build their own generators. +- Generators can be versioned and updated independently from algokit templates. + +#### Cons + +- Developing and maintaining generators is significantly more complex due to the need to maintain separate repositories and versioning. +- Official maintainers and community template builders need to be put extra effort at keeping generators generic enough to be reused by other templates. Given that templates can vastly differ in type, structure, conventions and patterns, this can be a challenge. +- Given that copier is being considered as a backbone for generators feature, drawbacks outlined for hosting templates on monorepos in the [previous adr](2023-06-06_frontend-templates.md#option-1-monolithic-template) apply here as well. + +## Open questions for reviewers + +1. What kinds of generators other than `add new contract` do we want to support on initial release (if any)? +2. Are there any other template repositories that we want to integrate with generators other than `beaker` (`fullstack` will contain those as well as it uses `beaker` as a dependency)? + +> Please note an MVP PoC is already implemented and available to play around on algokit-cli and beaker template repo under `advanced-generate-command` and `generators` branches respectively. +> To test it out checkout the branche on cli do `pipx install . --force`, navigate to beaker template repo and checkout the branch as well, then navigate to any of the sub folders in `tests_generated`. Lastly do `algokit bootstrap all`, build the contract and execute `algokit generate` from root of that folder to play around with the feature based on the implementation proposal from Option 1. + +## Final Decision + +The team approved the proposal for Option 1: Wrapping generators into self contained copier templates hidden within algokit templates. + +## Next steps + +1. Polishing the PoC on algokit-cli and adding tests +2. Polishing the PoC on beaker template and adding tests +3. Adding documentation for new capabilities of the generate command +4. Adding documentation for template builders on how to create generators diff --git a/docs/get-details/algokit/architecture-decisions/assets/2023-06-06_frontend-templates/modular_templates.jpg b/docs/get-details/algokit/architecture-decisions/assets/2023-06-06_frontend-templates/modular_templates.jpg new file mode 100644 index 000000000..b8250a68b Binary files /dev/null and b/docs/get-details/algokit/architecture-decisions/assets/2023-06-06_frontend-templates/modular_templates.jpg differ diff --git a/docs/get-details/algokit/architecture-decisions/assets/2023-06-06_frontend-templates/scenario_1.jpg b/docs/get-details/algokit/architecture-decisions/assets/2023-06-06_frontend-templates/scenario_1.jpg new file mode 100644 index 000000000..e6fa2e36f Binary files /dev/null and b/docs/get-details/algokit/architecture-decisions/assets/2023-06-06_frontend-templates/scenario_1.jpg differ diff --git a/docs/get-details/algokit/architecture-decisions/assets/2023-06-06_frontend-templates/scenario_2.jpg b/docs/get-details/algokit/architecture-decisions/assets/2023-06-06_frontend-templates/scenario_2.jpg new file mode 100644 index 000000000..1b48bdb15 Binary files /dev/null and b/docs/get-details/algokit/architecture-decisions/assets/2023-06-06_frontend-templates/scenario_2.jpg differ diff --git a/docs/get-details/algokit/architecture-decisions/lifecycle.jpg b/docs/get-details/algokit/architecture-decisions/lifecycle.jpg new file mode 100644 index 000000000..4c4c2af60 Binary files /dev/null and b/docs/get-details/algokit/architecture-decisions/lifecycle.jpg differ diff --git a/docs/get-details/algokit/articles/output_stability.md b/docs/get-details/algokit/articles/output_stability.md new file mode 100644 index 000000000..31bbe99e2 --- /dev/null +++ b/docs/get-details/algokit/articles/output_stability.md @@ -0,0 +1,8 @@ +# Smart Contract Output Stability + +Smart contracts development is analogous to low level firmware software development; it's a highly constrained environment in terms of both compute power and memory storage, with a high risk of vulnerabilities due to lower level access to memory and less developer-oriented security tooling. +Because of this, the assembly language code that is output for a smart contract is important - a seemingly innocuous minor change could inadvertently add a security vulnerability, or could significantly change the execution and memory profile. +As such it is important to ensure that, even if higher level code is refactored, there are no unintended changes to the generated smart contract assembly language output. +We refer to this property as **output stability**. + +We recommend having "output stability tests" that require a developer to explicitly opt-in to accepting a change in the output of a smart contract's assembly code. This can be implemented as part of an automated build process which fails if the output changes aren't committed to source control, thus preventing deployment of the smart contract without a human review taking place (assuming automated deployment). diff --git a/docs/get-details/algokit/cli-reference.md b/docs/get-details/algokit/cli-reference.md new file mode 100644 index 000000000..273553e7d --- /dev/null +++ b/docs/get-details/algokit/cli-reference.md @@ -0,0 +1,439 @@ +# CLI Reference + + + +AlgoKit is your one-stop shop to develop applications on the Algorand blockchain. + +If you are getting started, please see the [quick start tutorial](tutorials/intro.md). + +```shell +algokit [OPTIONS] COMMAND [ARGS]... +``` + +### Options + + +### --version +Show the version and exit. + + +### -v, --verbose +Enable logging of DEBUG messages to the console. + + +### --color, --no-color +Force enable or disable of console output styling. + + +### --skip-version-check +Skip version checking and prompting. + +## bootstrap + +Expedited initial setup for any developer by installing and configuring dependencies and other +key development environment setup activities. + +```shell +algokit bootstrap [OPTIONS] COMMAND [ARGS]... +``` + +### Options + + +### --force +Continue even if minimum AlgoKit version is not met + +### all + +Runs all bootstrap sub-commands in the current directory and immediate sub directories. + +```shell +algokit bootstrap all [OPTIONS] +``` + +### Options + + +### --interactive, --non-interactive, --ci +Enable/disable interactive prompts. If the CI environment variable is set, defaults to non-interactive + +### env + +Copies .env.template file to .env in the current working directory and prompts for any unspecified values. + +```shell +algokit bootstrap env [OPTIONS] +``` + +### Options + + +### --interactive, --non-interactive, --ci +Enable/disable interactive prompts. If the CI environment variable is set, defaults to non-interactive + +### npm + +Runs npm install in the current working directory to install Node.js dependencies. + +```shell +algokit bootstrap npm [OPTIONS] +``` + +### poetry + +Installs Python Poetry (if not present) and runs poetry install in the current working directory to install Python dependencies. + +```shell +algokit bootstrap poetry [OPTIONS] +``` + +## completions + +Install and Uninstall AlgoKit shell integrations. + +```shell +algokit completions [OPTIONS] COMMAND [ARGS]... +``` + +### install + +Install shell completions, this command will attempt to update the interactive profile script +for the current shell to support algokit completions. To specify a specific shell use –shell. + +```shell +algokit completions install [OPTIONS] +``` + +### Options + + +### --shell +Specify shell to install algokit completions for. + + +* **Options** + + bash | zsh + + +### uninstall + +Uninstall shell completions, this command will attempt to update the interactive profile script +for the current shell to remove any algokit completions that have been added. +To specify a specific shell use –shell. + +```shell +algokit completions uninstall [OPTIONS] +``` + +### Options + + +### --shell +Specify shell to install algokit completions for. + + +* **Options** + + bash | zsh + + +## config + +Configure settings used by AlgoKit + +```shell +algokit config [OPTIONS] COMMAND [ARGS]... +``` + +### version-prompt + +Controls whether AlgoKit checks and prompts for new versions. +Set to [disable] to prevent AlgoKit performing this check permanently, or [enable] to resume checking. +If no argument is provided then outputs current setting. + +Also see –skip-version-check which can be used to disable check for a single command. + +```shell +algokit config version-prompt [OPTIONS] [[enable|disable]] +``` + +### Arguments + + +### ENABLE +Optional argument + +## deploy + +Deploy smart contracts from AlgoKit compliant repository. + +```shell +algokit deploy [OPTIONS] [ENVIRONMENT_NAME] +``` + +### Options + + +### -C, --command +Custom deploy command. If not provided, will load the deploy command from .algokit.toml file. + + +### --interactive, --non-interactive, --ci +Enable/disable interactive prompts. If the CI environment variable is set, defaults to non-interactive + + +### -P, --path +Specify the project directory. If not provided, current working directory will be used. + +### Arguments + + +### ENVIRONMENT_NAME +Optional argument + +## doctor + +Diagnose potential environment issues that may affect AlgoKit + +Will search the system for AlgoKit dependencies and show their versions, as well as identifying any +potential issues. + +```shell +algokit doctor [OPTIONS] +``` + +### Options + + +### -c, --copy-to-clipboard +Copy the contents of the doctor message (in Markdown format) in your clipboard. + +## explore + +Explore the specified network in the browser using Dappflow. + +```shell +algokit explore [OPTIONS] [[localnet|testnet|mainnet]] +``` + +### Arguments + + +### NETWORK +Optional argument + +## generate + +Generate code for an Algorand project. + +```shell +algokit generate [OPTIONS] COMMAND [ARGS]... +``` + +### client + +Create a typed ApplicationClient from an ARC-32 application.json + +Supply the path to an application specification file or a directory to recursively search +for “application.json” files + +```shell +algokit generate client [OPTIONS] APP_SPEC_PATH_OR_DIR +``` + +### Options + + +### -o, --output +Path to the output file. The following tokens can be used to substitute into the output path: {contract_name}, {app_spec_dir} + + +### -l, --language +Programming language of the generated client code + + +* **Options** + + python | typescript + + +### Arguments + + +### APP_SPEC_PATH_OR_DIR +Required argument + +## goal + +Run the Algorand goal CLI against the AlgoKit LocalNet. + +Look at [https://developer.algorand.org/docs/clis/goal/goal/](https://developer.algorand.org/docs/clis/goal/goal/) for more information. + +```shell +algokit goal [OPTIONS] [GOAL_ARGS]... +``` + +### Options + + +### --console +Open a Bash console so you can execute multiple goal commands and/or interact with a filesystem. + +### Arguments + + +### GOAL_ARGS +Optional argument(s) + +## init + +Initializes a new project from a template, including prompting +for template specific questions to be used in template rendering. + +Templates can be default templates shipped with AlgoKit, or custom +templates in public Git repositories. + +Includes ability to initialise Git repository, run algokit bootstrap and +automatically open Visual Studio Code. + +This should be run in the parent directory that you want the project folder +created in. + +```shell +algokit init [OPTIONS] +``` + +### Options + + +### -n, --name +Name of the project / directory / repository to create. + + +### -t, --template +Name of an official template to use. To see a list of descriptions, run this command with no arguments. + + +* **Options** + + beaker | react | fullstack | playground + + + +### --template-url +URL to a git repo with a custom project template. + + +### --template-url-ref +Specific tag, branch or commit to use on git repo specified with –template-url. Defaults to latest. + + +### --UNSAFE-SECURITY-accept-template-url +Accept the specified template URL, acknowledging the security implications of arbitrary code execution trusting an unofficial template. + + +### --git, --no-git +Initialise git repository in directory after creation. + + +### --defaults +Automatically choose default answers without asking when creating this template. + + +### --bootstrap, --no-bootstrap +Whether to run algokit bootstrap to install and configure the new project’s dependencies locally. + + +### --ide, --no-ide +Whether to open an IDE for you if the IDE and IDE config are detected. Supported IDEs: VS Code. + + +### -a, --answer +Answers key/value pairs to pass to the template. + +## localnet + +Manage the AlgoKit LocalNet. + +```shell +algokit localnet [OPTIONS] COMMAND [ARGS]... +``` + +### console + +Run the Algorand goal CLI against the AlgoKit LocalNet via a Bash console so you can execute multiple goal commands and/or interact with a filesystem. + +```shell +algokit localnet console [OPTIONS] +``` + +### explore + +Explore the AlgoKit LocalNet using Dappflow + +```shell +algokit localnet explore [OPTIONS] +``` + +### logs + +See the output of the Docker containers + +```shell +algokit localnet logs [OPTIONS] +``` + +### Options + + +### --follow, -f +Follow log output. + + +### --tail +Number of lines to show from the end of the logs for each container. + + +* **Default** + + `all` + + +### reset + +Reset the AlgoKit LocalNet. + +```shell +algokit localnet reset [OPTIONS] +``` + +### Options + + +### --update, --no-update +Enable or disable updating to the latest available LocalNet version, default: don’t update + +### start + +Start the AlgoKit LocalNet. + +```shell +algokit localnet start [OPTIONS] +``` + +### status + +Check the status of the AlgoKit LocalNet. + +```shell +algokit localnet status [OPTIONS] +``` + +### stop + +Stop the AlgoKit LocalNet. + +```shell +algokit localnet stop [OPTIONS] +``` diff --git a/docs/get-details/algokit/features/bootstrap.md b/docs/get-details/algokit/features/bootstrap.md new file mode 100644 index 000000000..7833d204b --- /dev/null +++ b/docs/get-details/algokit/features/bootstrap.md @@ -0,0 +1,114 @@ +# AlgoKit Bootstrap + +The AlgoKit Bootstrap feature allows you to bootstrap different project dependencies by looking up specific files in your current directory and immediate sub directories by convention. + +This is useful to allow for expedited initial setup for each developer e.g. when they clone a repository for the first time. It's also useful to provide a quick getting started experience when initialising a new project via [AlgoKit Init](./init.md) and meeting our goal of "nothing to debugging code in 5 minutes". + +It can bootstrap one or all of the following (with other options potentially being added in the future): + +- Python Poetry projects - Installs Poetry via pipx if its not present and then runs `poetry install` +- Node.js project - Checks if npm is installed and runs `npm install` +- dotenv (.env) file - Checks for `.env.template` files, copies them to `.env` (which should be in `.gitignore` so developers can safely make local specific changes) and prompts for any blank values (so the developer has an easy chance to fill in their initial values where there isn't a clear default). + +## Usage + +Available commands and possible usage as follows: + +``` +$ ~ algokit bootstrap +Usage: algokit bootstrap [OPTIONS] COMMAND [ARGS]... + +Options: + -h, --help Show this message and exit. + +Commands: + all Bootstrap all aspects of the current directory and immediate sub directories by convention. + env Bootstrap .env file in the current working directory. + npm Bootstrap Node.js project in the current working directory. + poetry Bootstrap Python Poetry and install in the current working directory. +``` + +## Functionality + +### Bootstrap .env file + +The command `algokit bootstrap env` runs two main tasks in the current directory: + +- Searching for `.env.template` file in the current directory and use it as template to create a new `.env` file in the same directory. +- Prompting the user to enter a value for any empty token values in the `env.` including printing the comments above that empty token + +For instance, a sample `.env.template` file as follows: + +``` +SERVER_URL=https://myserver.com +# This is a mandatory field to run the server, please enter a value +# For example: 5000 +SERVER_PORT= +``` + +Running the `algokit bootstrap env` command while the above `.env.template` file in the current directory will result in the following: + +``` +$ ~ algokit bootstrap env +Copying /Users/me/my-project/.env.template to /Users/me/my-project/.env and prompting for empty values +# This is a mandatory field to run the server, please enter a value value +# For example: 5000 + +? Please provide a value for SERVER_PORT: +``` + +And when the user enters a value for `SERVER_PORT`, a new `.env` file will be created as follows (e.g. if they entered `4000` as the value): + +``` +SERVER_URL=https://myserver.com +# This is a mandatory field to run the server, please enter a value +# For example: 5000 +SERVER_PORT=4000 +``` + +### Bootstrap Node.js project + +The command `algokit bootstrap npm` installs Node.js project dependencies if there is a `package.json` file in the current directory by running `npm install` command to install all node modules specified in that file. If you don't have npm available it will show a clear error message and resolution instructions. + +Here is an example outcome of running `algokit bootstrap npm` command: + +``` +$ ~ algokit bootstrap npm +Installing npm dependencies +npm: +npm: added 17 packages, and audited 18 packages in 3s +npm: +npm: 2 packages are looking for funding +npm: run `npm fund` for details +npm: +npm: found 0 vulnerabilities +``` + +### Bootstrap Python poetry project + +The command `algokit bootstrap poetry` does two main actions: + +- Checking for Poetry version by running `poetry --version` and upgrades it if required +- Installing Python dependencies and setting up Python virtual environment via Poetry in the current directory by running `poetry install`. + +Here is an example of running `algokit bootstrap poetry` command: + +``` +$ ~ algokit bootstrap poetry +Installing Python dependencies and setting up Python virtual environment via Poetry +poetry: +poetry: Installing dependencies from lock file +poetry: +poetry: Package operations: 1 installs, 1 update, 0 removals +poetry: +poetry: • Installing pytz (2022.7) +poetry: • Updating copier (7.0.1 -> 7.1.0a0) +poetry: +poetry: Installing the current project: algokit (0.1.0) +``` + +### Bootstrap all + +You can run `algokit bootstrap all` which will run all three commands `algokit bootstrap env`, `algokit bootstrap npm` and `algokit bootstrap poetry` inside the current directory and all immediate sub-directories. This command is executed by default after initialising a new project via the [AlgoKit Init](./init.md) command. + +To learn more about the `algokit bootstrap` command, please refer to [bootstrap](../cli-reference.md#bootstrap) in the AlgoKit CLI reference documentation. diff --git a/docs/get-details/algokit/features/completions.md b/docs/get-details/algokit/features/completions.md new file mode 100644 index 000000000..e548c8df2 --- /dev/null +++ b/docs/get-details/algokit/features/completions.md @@ -0,0 +1,56 @@ +# AlgoKit Completions + +AlgoKit supports shell completions for zsh and bash shells, e.g. + +**bash** + +``` +$ algokit +bootstrap completions config doctor explore goal init sandbox +``` + +**zsh** + +``` +$ ~ algokit +bootstrap -- Bootstrap AlgoKit project dependencies. +completions -- Install and Uninstall AlgoKit shell integration. +config -- Configure AlgoKit options. +doctor -- Run the Algorand doctor CLI. +explore -- Explore the specified network in the... +goal -- Run the Algorand goal CLI against the AlgoKit Sandbox. +init -- Initializes a new project. +sandbox -- Manage the AlgoKit sandbox. +``` + +## Installing + +To setup the completions, AlgoKit provides commands that will modify the current users interactive shell script (`.bashrc`/`.zshrc`). + +> **Note** +> If you would prefer AlgoKit to not modify your interactive shell scripts you can install the completions yourself by following the instructions [here](https://click.palletsprojects.com/en/8.1.x/shell-completion/). + +To [install](../cli-reference.md#install) completions for the current shell execute `algokit completions install`. You should see output similar to below: + +``` +$ ~ algokit completions install +AlgoKit completions installed for zsh 🎉 +Restart shell or run `. ~/.zshrc` to enable completions +``` + +After installing the completions don't forget to restart the shell to begin using them! + +## Uninstalling + +To [uninstall](../cli-reference.md#uninstall) completions for the current shell run `algokit completions uninstall`: + +``` +$ ~ algokit completions uninstall +AlgoKit completions uninstalled for zsh 🎉 +``` + +## Shell Option + +To install/uninstall the completions for a specific [shell](../cli-reference.md#shell) the `--shell` option can be used e.g. `algokit completions install --shell bash`. + +To learn more about the `algokit completions` command, please refer to [completions](../cli-reference.md#completions) in the AlgoKit CLI reference documentation. diff --git a/docs/get-details/algokit/features/deploy.md b/docs/get-details/algokit/features/deploy.md new file mode 100644 index 000000000..24e737034 --- /dev/null +++ b/docs/get-details/algokit/features/deploy.md @@ -0,0 +1,127 @@ +# AlgoKit Deploy + +Deploy your smart contracts effortlessly to various networks with the AlgoKit Deploy feature. This feature is essential for automation in CI/CD pipelines and for seamless deployment to various Algorand network environments. + +## Usage + +```sh +$ algokit deploy [OPTIONS] [ENVIRONMENT_NAME] +``` + +This command deploys smart contracts from an AlgoKit compliant repository to the specified network. + +### Options + +- `--command, -C TEXT`: Specifies a custom deploy command. If this option is not provided, the deploy command will be loaded from the `.algokit.toml` file. +- `--interactive / --non-interactive, --ci`: Enables or disables the interactive prompt for mnemonics. When the CI environment variable is set, it defaults to non-interactive. +- `--path, -P DIRECTORY`: Specifies the project directory. If not provided, the current working directory will be used. +- `-h, --help`: Show this message and exit. + +## Environment files + +AlgoKit `deploy` employs both a general and network-specific environment file strategy. This allows you to set environment variables that are applicable across all networks and others that are specific to a given network. + +The general environment file (`.env`) should be placed at the root of your project. This file will be used to load environment variables that are common across deployments to all networks. + +For each network you're deploying to, you can optionally have a corresponding `.env.[network_name]` file. This file should contain environment variables specific to that network. Network-specific environment variables take precedence over general environment variables. + +The directory layout would look like this: + +```md +. +├── ... (your project files and directories) +├── .algokit.toml # Configuration file for AlgoKit +├── .env # (OPTIONAL) General environment variables common across all deployments +└── .env.[{mainnet|testnet|localnet|betanet|custom}] # (OPTIONAL) Environment variables specific to deployments to a network +``` + +> ⚠️ Please note that creating `.env` and `.env.[network_name]` files is only necessary if you're deploying to a custom network or if you want to override the default network configurations provided by AlgoKit. AlgoKit comes with predefined configurations for popular networks like `TestNet`, `MainNet`, `BetaNet`, or AlgoKit's `LocalNet`. + +The logic for loading environment variables is as follows: + +- If a `.env` file exists, the environment variables contained in it are loaded first. +- If a `.env.[network_name]` file exists, the environment variables in it are loaded, overriding any previously loaded values from the `.env` file for the same variables. + +## AlgoKit Configuration File + +AlgoKit uses a configuration file called `.algokit.toml` in the root of your project. The configuration file can be created using the `algokit init` command. This file will define the deployment commands for the various network environments that you want to target. + +Here's an example of what the `.algokit.toml` file might look like. When deploying it will prompt for the `DEPLOYER_MNEMONIC` secret unless it is already defined as an environment variable or is deploying to localnet. + +```toml +[algokit] +min_version = "v{lastest_version}" + +[deploy] +command = "poetry run python -m smart_contracts deploy" +environment_secrets = [ + "DEPLOYER_MNEMONIC", +] + +[deploy.localnet] +environment_secrets = [] +``` + +The `command` key under each `[deploy.{network_name}]` section should contain a string that represents the deployment command for that particular network. If a `command` key is not provided in a network-specific section, the command from the general `[deploy]` section will be used. + +The `environment_secrets` key should contain a list of names of environment variables that should be treated as secrets. This can be defined in the general `[deploy]` section, as well as in the network-specific sections. The environment-specific secrets will be added to the general secrets during deployment. + +The `[algokit]` section with the `min_version` key allows you to specify the minimum version of AlgoKit that the project requires. + +This way, you can define common deployment logic and environment secrets in the `[deploy]` section, and provide overrides or additions for specific environments in the `[deploy.{environment_name}]` sections. + +## Deploying to a Specific Network + +The command requires a `ENVIRONMENT` argument, which specifies the network environment to which the smart contracts will be deployed. Please note, the `environment` argument is case-sensitive. + +Example: + +```sh +$ algokit deploy testnet +``` + +This command deploys the smart contracts to the testnet. + +## Custom Project Directory + +By default, the deploy command looks for the `.algokit.toml` file in the current working directory. You can specify a custom project directory using the `--project-dir` option. + +Example: + +```sh +$ algokit deploy testnet --project-dir="path/to/project" +``` + +## Custom Deploy Command + +You can provide a custom deploy command using the `--custom-deploy-command` option. If this option is not provided, the deploy command will be loaded from the `.algokit.toml` file. + +Example: + +```sh +$ algokit deploy testnet --custom-deploy-command="your-custom-command" +``` + +## CI Mode + +By using the `--ci` or `--non-interactive` flag, you can skip the interactive prompt for mnemonics. + +This is useful in CI/CD environments where user interaction is not possible. When using this flag, you need to make sure that the mnemonics are set as environment variables. + +Example: + +```sh +$ algokit deploy testnet --ci +``` + +## Example of a Full Deployment + +```sh +$ algokit deploy testnet --custom-deploy-command="your-custom-command" +``` + +This example shows how to deploy smart contracts to the testnet using a custom deploy command. This also assumes that .algokit.toml file is present in the current working directory, and .env.testnet file is present in the current working directory and contains the required environment variables for deploying to TestNet environment. + +## Further Reading + +For in-depth details, visit the [deploy](../cli-reference.md#deploy) section in the AlgoKit CLI reference documentation. diff --git a/docs/get-details/algokit/features/doctor.md b/docs/get-details/algokit/features/doctor.md new file mode 100644 index 000000000..b7f418274 --- /dev/null +++ b/docs/get-details/algokit/features/doctor.md @@ -0,0 +1,51 @@ +# AlgoKit Doctor + +The AlgoKit Doctor feature allows you to check your AlgoKit installation along with its dependencies. This is useful for diagnosing potential issues with using AlgoKit. + +## Functionality + +The AlgoKit Doctor allows you to make sure that your system has the correct dependencies installed and that they satisfy the minimum required versions. All passed checks will appear in your command line natural color while warnings will be in yellow (warning) and errors or missing critical services will be in red (error). The critical services that AlgoKit will check for (since they are directly used by certain commands): Docker, docker compose and git. + +Please run this command to if you are facing an issue running AlgoKit. It is recommended to run it before [submitting an issue to AlgoKit](https://github.com/algorandfoundation/algokit-cli/issues/new). You can copy the contents of the Doctor command message (in Markdown format) to your clipboard by providing the `-c` flag to the command as follows `algokit doctor -c`. + +# Examples + +For example, running `algokit doctor` with all prerequisites installed will result in output similar to the following: + +``` +$ ~ algokit doctor +timestamp: 2023-03-29T03:58:05+00:00 +AlgoKit: 0.6.0 +AlgoKit Python: 3.11.2 (main, Mar 24 2023, 00:16:47) [Clang 14.0.0 (clang-1400.0.29.202)] (location: /Users/algokit/.local/pipx/venvs/algokit) +OS: macOS-13.2.1-arm64-arm-64bit +docker: 20.10.22 +docker compose: 2.15.1 +git: 2.39.1 +python: 3.10.9 (location: /Users/algokit/.asdf/shims/python) +python3: 3.10.9 (location: /Users/algokit/.asdf/shims/python3) +pipx: 1.2.0 +poetry: 1.3.2 +node: 18.12.1 +npm: 8.19.2 +brew: 4.0.10-34-gb753315 + +If you are experiencing a problem with AlgoKit, feel free to submit an issue via: +https://github.com/algorandfoundation/algokit-cli/issues/new +Please include this output, if you want to populate this message in your clipboard, run `algokit doctor -c` +``` + +The doctor command will indicate if there is any issues to address, for example: + +If AlgoKit detects a newer version, this will be indicated next to the AlgoKit version +``` +AlgoKit: 1.2.3 (latest: 4.5.6) +``` + +If the detected version of docker compose is unsupported, this will be shown: +``` +docker compose: 2.1.3 + Docker Compose 2.5.0 required to run `algokit localnet command`; + install via https://docs.docker.com/compose/install/ +``` + +For more details about the `AlgoKit doctor` command, please refer to the [AlgoKit CLI reference documentation](../cli-reference.md#doctor). diff --git a/docs/get-details/algokit/features/explore.md b/docs/get-details/algokit/features/explore.md new file mode 100644 index 000000000..db66ed543 --- /dev/null +++ b/docs/get-details/algokit/features/explore.md @@ -0,0 +1,21 @@ +# AlgoKit explore + +AlgoKit provides a quick shortcut to [explore](../cli-reference.md#explore) various Algorand networks using [Dappflow](https://app.dappflow.org/) including [AlogKit LocalNet](./localnet.md)! + +## LocalNet + +The following three commands are all equivalent and will open Dappflow pointing to the local [AlgoKit LocalNet](./localnet.md) instance: + +- `algokit explore` +- `algokit explore localnet` +- `algokit localnet explore` + +## Testnet + +`algokit explore testnet` will open Dappflow pointing to TestNet via the [node](https://algonode.io/api/). + +## Mainnet + +`algokit explore mainnet` will open Dappflow pointing to MainNet via the [node](https://algonode.io/api/). + +To learn more about the `algokit explore` command, please refer to [explore](../cli-reference.md#explore) in the AlgoKit CLI reference documentation. diff --git a/docs/get-details/algokit/features/generate.md b/docs/get-details/algokit/features/generate.md new file mode 100644 index 000000000..bb76deba5 --- /dev/null +++ b/docs/get-details/algokit/features/generate.md @@ -0,0 +1,167 @@ +# AlgoKit Generate + +The `algokit generate` [command](../cli-reference.md#generate) is used to generate components used in an AlgoKit project. It also allows for custom generate commands which are loaded from the .algokit.toml file in your project directory. + +## 1. Typed clients + +The `algokit generate client` [command](../cli-reference.md#client) can be used to generate a typed client from an [ARC-0032](https://arc.algorand.foundation/ARCs/arc-0032) application specification with +both Python and TypeScript available as target languages. + +### Prerequisites + +To generate Python clients AlgoKit itself is the only dependency. +To generate TypeScript clients an installation of Node.js and npx is also required. + +Each generated client will also have a dependency on `algokit-utils` libraries for the target language: + +- Python clients require: `algokit-utils@^1.2` +- TypeScript clients require: `@algorandfoundation/algokit-utils@^2.0` + +### Output tokens + +The output path is interpreted as relative to the current working directory, however an absolute path may also be specified e.g. +`algokit generate client application.json --output /absolute/path/to/client.py` + +There are two tokens available for use with the `-o`, `--output` [option](../cli-reference.md#-o---output-): + +- `{contract_name}`: This will resolve to a name based on the ARC-0032 contract name, formatted appropriately for the target language. +- `{app_spec_dir}`: This will resolve to the parent directory of an `application.json` which can be useful to output a client relative to its source application.json. + +### Usage + +Usage examples of using a generated client are below, typed clients allow your favourite IDE to provide better intellisense to provide better discoverability +of available operations and parameters. + +#### Python + +```python +# A similar working example can be seen in the beaker_production template, when using Python deployment +from smart_contracts.artifacts.HelloWorldApp.client import ( + HelloWorldAppClient, +) + +app_client = HelloWorldAppClient( + algod_client, + creator=deployer, + indexer_client=indexer_client, +) +deploy_response = app_client.deploy( + on_schema_break=OnSchemaBreak.ReplaceApp, + on_update=OnUpdate.UpdateApp, + allow_delete=True, + allow_update=True, +) + +response = app_client.hello(name="World") +``` + +#### TypeScript + +```typescript +// A similar working example can be seen in the beaker_production template, when using TypeScript deployment +import { HelloWorldAppClient } from './artifacts/HelloWorldApp/client' + +const appClient = new HelloWorldAppClient( + { + resolveBy: 'creatorAndName', + findExistingUsing: indexer, + sender: deployer, + creatorAddress: deployer.addr, + }, + algod, +) +const app = await appClient.deploy({ + allowDelete: isLocal, + allowUpdate: isLocal, + onSchemaBreak: isLocal ? 'replace' : 'fail', + onUpdate: isLocal ? 'update' : 'fail', +}) +const response = await appClient.hello({ name: 'world' }) +``` + +### Examples + +To output a single application.json to a python typed client: +`algokit generate client path/to/application.json --output client.py` + +To process multiple application.json in a directory structure and output to a typescript client for each in the current directory: +`algokit generate client smart_contracts/artifacts --output {contract_name}.ts` + +To process multiple application.json in a directory structure and output to a python client alongside each application.json: +`algokit generate client smart_contracts/artifacts --output {app_spec_path}/client.py` + +## 2. Using Custom Generate Commands + +Custom generate commands are defined in the `.algokit.toml` file within the project directory, typically supplied by community template builders or official AlgoKit templates. These commands are specified under the `generate` key and serve to execute a generator at a designated path with provided answer key/value pairs. + +### Understanding `Generators` + +A `generator` is essentially a compact, self-sufficient `copier` template. This template can optionally be defined within the primary `algokit templates` to offer supplementary functionality after a project is initialized from the template. For instance, the official [`algokit-beaker-default-template`](https://github.com/algorandfoundation/algokit-beaker-default-template/tree/main/template_content) provides a generator within the `.algokit/generators` directory. This generator can be employed for executing extra tasks on AlgoKit projects that have been initiated from this template, such as adding new smart contracts to an existing project. For a comprehensive explanation, please refer to the [`architecture decision record`](../architecture-decisions/2023-07-19_advanced_generate_command.md). + +### Requirements + +To utilize custom generate commands, you must have `copier` installed. This installation is included by default in the AlgoKit CLI. Therefore, no additional installation is necessary if you have already installed the `algokit cli`. + +### How to Use + +A custom command can be defined in the `.algokit.toml` as shown: + +```toml +[generate.my_generator] +path = "path/to/my_generator" +description = "A brief description of the function of my_generator" +``` + +Following this, you can execute the command as follows: + +`algokit generate my_generator --answer key value --path path/to/my_generator` + +If no `path` is given, the command will use the path specified in the `.algokit.toml`. If no `answer` is provided, the command will initiate an interactive `copier` prompt to request answers (similar to `algokit init`). + +The custom command employs the `copier` library to duplicate the files from the generator's path to the current working directory, substituting any values from the `answers` dictionary. + +### Examples + +As an example, let's use the `smart-contract` generator from the `algokit-beaker-default-template` to add new contract to an existing project based on that template. The `smart-contract` generator is defined as follows: + +```toml +[algokit] +min_version = "v1.3.1" + +... # other keys + +[generate.smart_contract] +description = "Adds a new smart contract to the existing project" +path = ".algokit/generators/create_contract" +``` + +To execute this generator, ensure that you are operating from the same directory as the `.algokit.toml` file, and then run: + +```bash +$ algokit generate + +# The output will be as follows: +# Note how algokit dynamically injects a new `smart-contract` command based +# on the `.algokit.toml` file + +Usage: algokit generate [OPTIONS] COMMAND [ARGS]... + + Generate code for an Algorand project. + +Options: + -h, --help Show this message and exit. + +Commands: + client Create a typed ApplicationClient from an ARC-32 application.json + smart-contract Adds a new smart contract to the existing project +``` + +To execute the `smart-contract` generator, run: + +```bash +$ algokit generate smart-contract + +# or + +$ algokit generate smart-contract -a contract_name "MyCoolContract" +``` diff --git a/docs/get-details/algokit/features/goal.md b/docs/get-details/algokit/features/goal.md new file mode 100644 index 000000000..ed8e01ab9 --- /dev/null +++ b/docs/get-details/algokit/features/goal.md @@ -0,0 +1,118 @@ +# AlgoKit goal + +AlgoKit goal command provides the user with a mechanism to run [goal cli](https://developer.algorand.org/docs/clis/goal/goal/) commands against the current [AlgoKit LocalNet](./localnet.md). + +You can explore all possible goal commands by running `algokit goal` e.g.: + +``` +$ ~ algokit goal + GOAL is the CLI for interacting Algorand software instance. The binary 'goal' is installed alongside the algod binary and is considered an integral part of the complete installation. The binaries should be used in tandem - you should not try to use a version of goal with a different version of algod. + + Usage: + goal [flags] + goal [command] + + Available Commands: + account Control and manage Algorand accounts + app Manage applications + asset Manage assets + clerk Provides the tools to control transactions + completion Shell completion helper + help Help about any command + kmd Interact with kmd, the key management daemon + ledger Access ledger-related details + license Display license information + logging Control and manage Algorand logging + network Create and manage private, multi-node, locally-hosted networks + node Manage a specified algorand node + protocols + report + version The current version of the Algorand daemon (algod) + wallet Manage wallets: encrypted collections of Algorand account keys + + Flags: + -d, --datadir stringArray Data directory for the node + -h, --help help for goal + -k, --kmddir string Data directory for kmd + -v, --version Display and write current build version and exit + + Use "goal [command] --help" for more information about a command. +``` + +For instance, running `algokit goal report` would result in output like: + +``` +$ ~ algokit goal report + 12885688322 + 3.12.2.dev [rel/stable] (commit #181490e3) + go-algorand is licensed with AGPLv3.0 + source code available at https://github.com/algorand/go-algorand + + Linux ff7828f2da17 5.15.49-linuxkit #1 SMP PREEMPT Tue Sep 13 07:51:32 UTC 2022 aarch64 GNU/Linux + + Genesis ID from genesis.json: sandnet-v1 + + Last committed block: 0 + Time since last block: 0.0s + Sync Time: 0.0s + Last consensus protocol: future + Next consensus protocol: future + Round for next consensus protocol: 1 + Next consensus protocol supported: true + Last Catchpoint: + Genesis ID: sandnet-v1 + Genesis hash: vEg1NCh6SSXwS6O5HAfjYCCNAs4ug328s3RYMr9syBg= +``` + +If the AlgoKit Sandbox `algod` docker container is not present or not running, the command will fail with a clear error, e.g.: + +``` +$ ~ algokit goal + Error: No such container: algokit_algod + Error: Error executing goal; ensure the Sandbox is started by executing `algokit sandbox status` +``` + +``` +$ ~ algokit goal + Error response from daemon: Container 5a73961536e2c98e371465739053d174066c40d00647c8742f2bb39eb793ed7e is not running + Error: Error executing goal; ensure the Sandbox is started by executing `algokit sandbox status` +``` + +## Working with Files in the Container +When interacting with the container, especially if you're using tools like goal, you might need to reference files or directories. Here's how to efficiently deal with files and directories: + +### Automatic File Mounting +When you specify a file or directory path in your `goal` command, the system will automatically mount that path from your local filesystem into the container. This way, you don't need to copy files manually each time. + +For instance, if you want to compile a `teal` file: + +``` +algokit goal clerk compile /Path/to/inputfile/approval.teal -o /Path/to/outputfile/approval.compiled +``` +Here, `/Path/to/inputfile/approval.teal` and `/Path/to/outputfile/approval.compiled` are paths on your local file system, and they will be automatically accessible to the `goal` command inside the container. + +### Manual Copying of Files + +In case you want to manually copy files into the container, you can do so using `docker cp`: + +``` +docker cp foo.txt algokit_algod:/root +``` +This command copies the `foo.txt` from your local system into the root directory of the `algokit_algod` container. + +Note: Manual copying is optional and generally only necessary if you have specific reasons for doing so since the system will auto-mount paths specified in commands. + +## Running multiple commands + +If you want to run multiple commands or interact with the filesystem you can execute `algokit goal --console`. This will open a [Bash](https://www.gnu.org/software/bash/) shell session on the `algod` Docker container and from there you can execute goal directly, e.g.: + +``` +$ algokit goal --console +Opening Bash console on the algod node; execute `exit` to return to original console +root@82d41336608a:~# goal account list +[online] C62QEFC7MJBPHAUDMGVXGZ7WRWFAF3XYPBU3KZKOFHYVUYDGU5GNWS4NWU C62QEFC7MJBPHAUDMGVXGZ7WRWFAF3XYPBU3KZKOFHYVUYDGU5GNWS4NWU 4000000000000000 microAlgos +[online] DVPJVKODAVEKWQHB4G7N6QA3EP7HKAHTLTZNWMV4IVERJQPNGKADGURU7Y DVPJVKODAVEKWQHB4G7N6QA3EP7HKAHTLTZNWMV4IVERJQPNGKADGURU7Y 4000000000000000 microAlgos +[online] 4BH5IKMDDHEJEOZ7T5LLT4I7EVIH5XCOTX3TPVQB3HY5TUBVT4MYXJOZVA 4BH5IKMDDHEJEOZ7T5LLT4I7EVIH5XCOTX3TPVQB3HY5TUBVT4MYXJOZVA 2000000000000000 microAlgos +``` + +For more details about the `AlgoKit goal` command, please refer to the [AlgoKit CLI reference documentation](../cli-reference.md#goal). diff --git a/docs/get-details/algokit/features/init.md b/docs/get-details/algokit/features/init.md new file mode 100644 index 000000000..98a1c4a12 --- /dev/null +++ b/docs/get-details/algokit/features/init.md @@ -0,0 +1,116 @@ +# AlgoKit Init + +The `algokit init` [command](../cli-reference.md#init) is used to quickly initialize new projects using official Algorand Templates or community provided templates. It supports a fully guided command line wizard experience, as well as fully scriptable / non-interactive functionality via command options. + +## Quick start + +For a quick start template with all of the defaults you can run: `algokit init --name my_project --template beaker --git --defaults` and then you can run `cd my_project && code .` and you should immediately be able to hit F5 to compile the hello world smart contract to the `smart_contracts/artifacts` folder (with breakpoint debugging - try setting a breakpoint in `smart_contracts/helloworld.py`) and open the `smart_contracts/helloworld.py` file and get linting, automatic formatting and syntax highlighting. + +## Prerequisites + +Git is a prerequisite for the init command as it is used to clone templates and initialize git repos. Please consult the [install guide](../tutorials/intro.md#prerequisites) for installation instructions. + +## Functionality + +The simplest use of the command is to just run `algokit init` and you will then be guided through selecting a template and configuring options for that template. e.g. + +``` +$ ~ algokit init +? Select a project template: beaker +? Name of project / directory to create the project in: my-smart-contract +🎤 Package author name + Algorand Foundation +🎤 Package author email + info@algorand.foundation +🎤 Do you want to add VSCode configuration? + Yes +🎤 Do you want to use a Python linter? + Ruff +🎤 Do you want to use a Python formatter (via Black)? + Yes +🎤 Do you want to use a Python type checker (via mypy)? + Yes +🎤 Do you want to include unit tests (via pytest)? + Yes +🎤 Do you want to include Python dependency vulnerability scanning (via pip-audit)? + Yes +``` + +If you want to accept the default for each option simply hit [enter] or alternatively to speed things up you can run `algokit init --defaults` and they will be auto-accepted. + +## Bootstrapping + +You will also be prompted if you wish to run the [bootstrap](../cli-reference.md#bootstrap) command, this is useful if you plan to immediately begin developing in the new project. If you passed in `--defaults` or `--bootstrap` then it will automatically run bootstrapping unless you passed in `--no-bootstrap`. + +``` +? Do you want to run `algokit bootstrap` to bootstrap dependencies for this new project so it can be run immediately? Yes +Installing Python dependencies and setting up Python virtual environment via Poetry +poetry: Creating virtualenv my-smart-contract in /Users/algokit/algokit-init/my-smart-contract/.venv +poetry: Updating dependencies +poetry: Resolving dependencies... +poetry: +poetry: Writing lock file +poetry: +poetry: Package operations: 53 installs, 0 updates, 0 removals +poetry: +poetry: • Installing pycparser (2.21) + +---- other output omitted for brevity ---- + +poetry: • Installing ruff (0.0.171) +Copying /Users/algokit/algokit-init/my-smart-contract/smart_contracts/.env.template to /Users/algokit/algokit-init/my-smart-contract/smart_contracts/.env and prompting for empty values +? Would you like to initialise a git repository and perform an initial commit? Yes +🎉 Performed initial git commit successfully! 🎉 +🙌 Project initialized at `my-smart-contract`! For template specific next steps, consult the documentation of your selected template 🧐 +Your selected template comes from: +➡️ https://github.com/algorandfoundation/algokit-beaker-default-template +As a suggestion, if you wanted to open the project in VS Code you could execute: +> cd my-smart-contract && code . +``` + +After bootstrapping you are also given the opportunity to initialize a git repo, upon successful completion of the init command the project is ready to be used. If you pass in `--git` it will automatically initialise the git repository and if you pass in `--no-git` it won't. + +## Options + +There are a number of options that can be used to provide answers to the template prompts. Some of the options requiring further explanation are detailed below, but consult the CLI reference for all available [options](../cli-reference.md#init). + +## Community Templates + +As well as the official Algorand templates shown when running the init command, community templates can also be provided by providing a URL via the prompt or the `--template-url` option. + +e.g. `algokit init --template-url https://github.com/algorandfoundation/algokit-beaker-default-template` (that being the url of the official beaker template, the same as `algokit init -t beaker`). + +The `--template-url` option can be combined with `--template-url-ref` to specify a specific commit, branch or tag + +e.g. `algokit init --template-url https://github.com/algorandfoundation/algokit-beaker-default-template --template-url-ref 9985005b7389c90c6afed685d75bb8e7608b2a96` + +If the URL is not an official template there is a potential security risk and so to continue you must either acknowledge this prompt, or if you are in a non-interactive environment you can pass the `--UNSAFE-SECURITY-accept-template-url` option (but we generally don't recommend this option so users can review the warning message first) e.g. + +``` +Community templates have not been reviewed, and can execute arbitrary code. +Please inspect the template repository, and pay particular attention to the values of _tasks, _migrations and _jinja_extensions in copier.yml +? Continue anyway? Yes +``` + +If you want to create a community template, you can use the [AlgoKit guidelines on template building](../tutorials/algokit-template.md#creating-algokit-templates) and [Copier documentation](https://copier.readthedocs.io/en/stable/) as a starting point. + +## Template Answers + +Answers to specific template prompts can be provided with the `--answer {key} {value}` option, which can be used multiple times for each prompt. Quotes can be used for values with spaces e.g. `--answer author_name "Algorand Foundation"`. + +To find out the key for a specific answer you can either look at `.copier-answers.yml` in the root folder of a project created via `algokit init` or in the `copier.yaml` file of a template repo e.g. for the [beaker template](https://github.com/algorandfoundation/algokit-beaker-default-template/blob/main/copier.yaml). + +## Non-interactive project initialization + +By combining a number of options, it is possible to initialize a new project without any interaction. For example, to create a project named `my-smart-contract` using the `beaker` template with no git, no bootstrapping, the author name of `Algorand Foundation`, and defaults for all other values, you could execute the following: + +``` +$ ~ algokit init -n my-smart-contract -t beaker --no-git --no-bootstrap --answer author_name "Algorand Foundation" --defaults +🙌 Project initialized at `my-smart-contract`! For template specific next steps, consult the documentation of your selected template 🧐 +Your selected template comes from: +➡️ https://github.com/algorandfoundation/algokit-beaker-default-template +As a suggestion, if you wanted to open the project in VS Code you could execute: +> cd my-smart-contract && code . +``` + +For more details about the `AlgoKit init` command, please refer to the [AlgoKit CLI reference documentation](../cli-reference.md#init). diff --git a/docs/get-details/algokit/features/localnet.md b/docs/get-details/algokit/features/localnet.md new file mode 100644 index 000000000..9583ca615 --- /dev/null +++ b/docs/get-details/algokit/features/localnet.md @@ -0,0 +1,101 @@ +# AlgoKit LocalNet + +The AlgoKit LocalNet feature allows you to manage (start, stop, reset, manage) a locally sandboxed private Algorand network. This allows you to interact and deploy changes against your own Algorand network without needing to worry about funding TestNet accounts, information you submit being publicly visible or being connected to an active Internet connection (once the network has been started). + +AlgoKit LocalNet uses Docker images that are optimised for a great dev experience. This means the Docker images are small and start fast. It also means that features suited to developers are enabled such as KMD (so you can programmatically get faucet private keys). + +The philosophy we take with AlgoKit LocalNet is that you should treat it as an ephemeral network. This means assume it could be reset at any time - don't store data on there that you can't recover / recreate. We have optimised the AlgoKit LocalNet experience to minimise situations where the network will get reset to improve the experience, but it can and will still happen in a number of situations. + +## Prerequisites + +AlgoKit LocalNet relies on Docker and Docker Compose being present and running on your system. + +You can install Docker by following the [official installation instructions](https://docs.docker.com/get-docker/). Most of the time this will also install Docker Compose, but if not you can [follow the instructions](https://docs.docker.com/compose/install/) for that too. + +If you are on Windows then you will need WSL 2 installed first, for which you can find the [official installation instructions](https://learn.microsoft.com/en-us/windows/wsl/install). If you are using Windows 10 then ensure you are on the latest version to reduce likelihood of installation problems. + +Alternatively, the Windows 10/11 Pro+ supported [Hyper-V backend](https://docs.docker.com/desktop/install/windows-install/) for Docker can be used instead of the WSL 2 backend. + +## Known issues + +The AlgoKit LocalNet is built with 30,000 participation keys generated and after 30,000 rounds is reached it will no longer be able to add rounds. At this point you can simply reset the LocalNet to continue development. Participation keys are slow to generate hence why they are pre-generated to improve experience. + +## Supported operating environments + +We publish DockerHub images for `arm64` and `amd64`, which means that AlgoKit LocalNet is supported on Windows, Linux and Mac on Intel and AMD chipsets (including Mac M1). + +## Functionality + +### Creating / starting the LocalNet + +To create / start your AlgoKit LocalNet instance you can run `algokit localnet start`. This will: + +- Detect if you have Docker and Docker Compose installed +- Detect if you have the Docker engine running +- Create a new Docker Compose deployment for AlgoKit LocalNet if it doesn't already exist +- (Re-)Start the containers + +If it's the first time running it on your machine then it will download the following images from DockerHub: + +- [`makerxau/algorand-sandbox-dev`](https://hub.docker.com/r/makerxau/algorand-sandbox-dev) (~150-200 MB) +- [`makerxau/algorand-indexer-dev`](https://hub.docker.com/r/makerxau/algorand-indexer-dev) (~25 MB) +- [`postgres:13-alpine`](https://hub.docker.com/_/postgres) (~80 MB) + +Once they have downloaded, it won't try and re-download images unless you perform a `algokit localnet reset`. + +Once the LocalNet has started, the following endpoints will be available: + +- [algod](https://developer.algorand.org/docs/rest-apis/algod/v2/): + - address: http://localhost:4001 + - token: `aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa` +- [kmd](https://developer.algorand.org/docs/rest-apis/kmd/): + - address: http://localhost:4002 + - token: `aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa` +- [indexer](https://developer.algorand.org/docs/rest-apis/indexer/): + - address: http://localhost:8980 +- tealdbg port: + - address: http://localhost:9392 + +### Stopping and Resetting the LocalNet + +To stop the LocalNet you can execute `algokit localnet stop`. This will turn off the containers, but keep them ready to be started again in the same state by executing `algokit localnet start`. + +To reset the LocalNet you can execute `algokit localnet reset`, which will tear down the existing containers, refresh the container definition from the latest stored within AlgoKit and update to the latest Docker images. If you want to keep the same container spec and versions as you currently have, but quickly tear down and start a new instance then run `algokit localnet reset --no-update`. + +### Viewing transactions in the LocalNet + +You can see a web-based user interface of the current state of your LocalNet including all transactions by using the [AlgoKit Explore](./explore.md) feature, e.g. by executing `algokit localnet explore`. + +### Executing goal commands against AlgoKit LocalNet + +See the [AlgoKit Goal](./goal.md) feature. You can also execute `algokit localnet console` to open a [Bash shell which allows you to run the goal commandline](./goal.md#running-multiple-commands). + +Note: if you want to copy files into the container so you can access them via goal then you can use the following: + +``` +docker cp foo.txt algokit_algod:/root +``` + +### Getting access to the private key of the faucet account + +If you want to use the LocalNet then you need to get the private key of the initial wallet so you can transfer ALGOs out of it to other accounts you create. + +There are two ways to do this: + +**Option 1: Manually via goal** + +``` +algokit goal account list +algokit goal account export -a {address_from_an_online_account_from_above_command_output} +``` + +**Option 2: Automatically via kmd API** + +Needing to do this manual step every time you spin up a new development environment or reset your LocalNet is frustrating. Instead, it's useful to have code that uses the Sandbox APIs to automatically retrieve the private key of the default account. + +AlgoKit Utils provides methods to help you do this: + +* TypeScript - [`ensureFunded`](https://github.com/algorandfoundation/algokit-utils-ts/blob/main/docs/capabilities/transfer.md#ensurefunded) and [`getDispenserAccount`](https://github.com/algorandfoundation/algokit-utils-ts/blob/main/docs/capabilities/transfer.md#dispenser) +* Python - [`ensure_funded`](https://algorandfoundation.github.io/algokit-utils-py/html/apidocs/algokit_utils/algokit_utils.html#algokit_utils.ensure_funded) and [`get_dispenser_account`](https://algorandfoundation.github.io/algokit-utils-py/html/apidocs/algokit_utils/algokit_utils.html#algokit_utils.get_dispenser_account) + +For more details about the `AlgoKit localnet` command, please refer to the [AlgoKit CLI reference documentation](../cli-reference.md#localnet). diff --git a/docs/get-details/algokit/imgs/algokit-intro-video-thumbnail.jpg b/docs/get-details/algokit/imgs/algokit-intro-video-thumbnail.jpg new file mode 100644 index 000000000..aaeb7e967 Binary files /dev/null and b/docs/get-details/algokit/imgs/algokit-intro-video-thumbnail.jpg differ diff --git a/docs/get-details/algokit/imgs/algokit-map.png b/docs/get-details/algokit/imgs/algokit-map.png new file mode 100644 index 000000000..b491631b2 Binary files /dev/null and b/docs/get-details/algokit/imgs/algokit-map.png differ diff --git a/docs/get-details/algokit/imgs/algokitartifacts.png b/docs/get-details/algokit/imgs/algokitartifacts.png new file mode 100644 index 000000000..9e9a34dbf Binary files /dev/null and b/docs/get-details/algokit/imgs/algokitartifacts.png differ diff --git a/docs/get-details/algokit/imgs/algokitdemo.png b/docs/get-details/algokit/imgs/algokitdemo.png new file mode 100644 index 000000000..dd88d909c Binary files /dev/null and b/docs/get-details/algokit/imgs/algokitdemo.png differ diff --git a/docs/get-details/algokit/imgs/algokitplayground.png b/docs/get-details/algokit/imgs/algokitplayground.png new file mode 100644 index 000000000..494de2c5b Binary files /dev/null and b/docs/get-details/algokit/imgs/algokitplayground.png differ diff --git a/docs/get-details/algokit/imgs/dappflow.png b/docs/get-details/algokit/imgs/dappflow.png new file mode 100644 index 000000000..2b8ee68d1 Binary files /dev/null and b/docs/get-details/algokit/imgs/dappflow.png differ diff --git a/docs/get-details/algokit/imgs/dappflow1.png b/docs/get-details/algokit/imgs/dappflow1.png new file mode 100644 index 000000000..2f2ccb388 Binary files /dev/null and b/docs/get-details/algokit/imgs/dappflow1.png differ diff --git a/docs/get-details/algokit/imgs/dappflow2.png b/docs/get-details/algokit/imgs/dappflow2.png new file mode 100644 index 000000000..4efe6c9dc Binary files /dev/null and b/docs/get-details/algokit/imgs/dappflow2.png differ diff --git a/docs/get-details/algokit/imgs/dappflow3.png b/docs/get-details/algokit/imgs/dappflow3.png new file mode 100644 index 000000000..4385a104b Binary files /dev/null and b/docs/get-details/algokit/imgs/dappflow3.png differ diff --git a/docs/get-details/algokit/imgs/dappflow4.png b/docs/get-details/algokit/imgs/dappflow4.png new file mode 100644 index 000000000..96fb01020 Binary files /dev/null and b/docs/get-details/algokit/imgs/dappflow4.png differ diff --git a/docs/get-details/algokit/imgs/dappflow5.png b/docs/get-details/algokit/imgs/dappflow5.png new file mode 100644 index 000000000..64d409c62 Binary files /dev/null and b/docs/get-details/algokit/imgs/dappflow5.png differ diff --git a/docs/get-details/algokit/imgs/dappflow6.png b/docs/get-details/algokit/imgs/dappflow6.png new file mode 100644 index 000000000..0030e3b55 Binary files /dev/null and b/docs/get-details/algokit/imgs/dappflow6.png differ diff --git a/docs/get-details/algokit/imgs/dappflow_transaction.png b/docs/get-details/algokit/imgs/dappflow_transaction.png new file mode 100644 index 000000000..3eda5de32 Binary files /dev/null and b/docs/get-details/algokit/imgs/dappflow_transaction.png differ diff --git a/docs/get-details/algokit/imgs/localnet.png b/docs/get-details/algokit/imgs/localnet.png new file mode 100644 index 000000000..af48b4649 Binary files /dev/null and b/docs/get-details/algokit/imgs/localnet.png differ diff --git a/docs/get-details/algokit/index.md b/docs/get-details/algokit/index.md new file mode 100644 index 000000000..e592373a6 --- /dev/null +++ b/docs/get-details/algokit/index.md @@ -0,0 +1,48 @@ +# Overview + +The Algorand AlgoKit CLI is the one-stop shop tool for developers building on the Algorand network. The goal of AlgoKit is to help developers build and launch secure, automated production-ready applications rapidly. + +## AlgoKit CLI commands + +For details on how to use individual features see the following + +- [Bootstrap](./features/bootstrap.md) - Bootstrap AlgoKit project dependencies +- [Completions](./features/completions.md) - Install shell completions for AlgoKit +- [Deploy](./features/deploy.md) - Deploy your smart contracts effortlessly to various networks +- [Doctor](./features/doctor.md) - Check AlgoKit installation and dependencies +- [Explore](./features/explore.md) - Explore Algorand Blockchains using Dappflow +- [Generate](./features/generate.md) - Generate code for an Algorand project +- [Goal](./features/goal.md) - Run the Algorand goal CLI against the AlgoKit Sandbox +- [Init](./features/init.md) - Quickly initialize new projects using official Algorand Templates or community provided templates. +- [LocalNet](./features/localnet.md) - Manage a locally sandboxed private Algorand network. + +## Common AlgoKit CLI options + +AlgoKit has a number of global options that can impact all commands. Note: these global options must be appended to `algokit` and appear before a command, e.g. `algokit -v localnet start`, but not `algokit localnet start -v`. The exception to this is `-h`, which can be appended to any command or sub-command to see contextual help information. + +- `-h, --help` The help option can be used on any command to get details on any command, its sub-commands and options. +- `-v, --verbose` Enables DEBUG logging, useful when troubleshooting or if you want to peek under the covers and learn what AlgoKit CLI is doing. +- `--color / --no-color` Enables or disables output of console styling, we also support the [NO_COLOR](https://no-color.org) environment variable. +- `--skip-version-check` Skips updated AlgoKit version checking and prompting for that execution, this can also be disabled permanently on a given machine with [`algokit config version-prompt disable`](cli/#-skip-version-check). + +See also the [AlgoKit CLI Reference](./cli-reference.md), which details every command, sub-command and option. + +## AlgoKit Tutorials + +The following tutorials guide you through various scenarios: + +- [AlgoKit quick start](./tutorials/intro.md) +- [Creating AlgoKit templates](./tutorials/algokit-template.md) + +## Guiding Principles + +Algorand AlgoKit is guided by the following solution principles which flow through to the applications created by developers. + +1. **Cohesive developer tool suite**: Using AlgoKit should feel professional and cohesive, like it was designed to work together, for the developer; not against them. Developers are guided towards delivering end-to-end, high quality outcomes on MainNet so they and Algorand are more likely to be successful. +2. **Seamless onramp**: New developers have a seamless experience to get started and they are guided into a pit of success with best practices, supported by great training collateral; you should be able to go from nothing to debugging code in 5 minutes. +3. **Leverage existing ecosystem**: AlgoKit functionality gets into the hands of Algorand developers quickly by building on top of the existing ecosystem wherever possible and aligned to these principles. +4. **Sustainable**: AlgoKit should be built in a flexible fashion with long-term maintenance in mind. Updates to latest patches in dependencies, Algorand protocol development updates, and community contributions and feedback will all feed in to the evolution of the software. +5. **Secure by default**: Include defaults, patterns and tooling that help developers write secure code and reduce the likelihood of security incidents in the Algorand ecosystem. This solution should help Algorand be the most secure Blockchain ecosystem. +6. **Extensible**: Be extensible for community contribution rather than stifling innovation, bottle-necking all changes through the Algorand Foundation and preventing the opportunity for other ecosystems being represented (e.g. Go, Rust, etc.). This helps make developers feel welcome and is part of the developer experience, plus it makes it easier to add features sustainably. +7. **Meet developers where they are**: Make Blockchain development mainstream by giving all developers an idiomatic development experience in the operating system, IDE and language they are comfortable with so they can dive in quickly and have less they need to learn before being productive. +8. **Modular components**: Solution components should be modular and loosely coupled to facilitate efficient parallel development by small, effective teams, reduced architectural complexity and allowing developers to pick and choose the specific tools and capabilities they want to use based on their needs and what they are comfortable with. diff --git a/docs/get-details/algokit/tutorials/algokit-template.md b/docs/get-details/algokit/tutorials/algokit-template.md new file mode 100644 index 000000000..b2db369c8 --- /dev/null +++ b/docs/get-details/algokit/tutorials/algokit-template.md @@ -0,0 +1,182 @@ +# Creating AlgoKit Templates + +This README serves as a guide on how to create custom templates for AlgoKit, a tool for initializing Algorand smart contract projects. +Creating templates in AlgoKit involves the use of various configuration files and a templating engine to generate project structures that are tailored to your needs. +This guide will cover the key concepts and best practices for creating templates in AlgoKit. +We will also refer to the official [`algokit-beaker-default-template`](https://github.com/algorandfoundation/algokit-beaker-default-template) as an example. + + +## Quick Start +For users who are keen on getting started with creating AlgoKit templates, you can follow these quick steps: + +1. Click on `Use this template`->`Create a new repository` on [algokit-beaker-default-template](https://github.com/algorandfoundation/algokit-beaker-default-template) Github page. This will create a new reference repository with clean git history, allowing you to start modifying and transforming the base beaker template into your own custom template. +2. Modify the cloned template according to your specific needs. You can refer to the remainder of this tutorial for an understanding of expected behaviors from the AlgoKit side, Copier - the templating framework, and key concepts related to the default files you will encounter in the reference template. + +## Overview of AlgoKit Templates + +AlgoKit templates are essentially project scaffolds that can be used to initialize new smart contract projects. These templates can include code files, configuration files, and scripts. AlgoKit uses Copier along with the Jinja templating engine to create new projects based on these templates. + +### Copier/Jinja + +AlgoKit uses Copier templates. Copier is a library that allows you to create project templates that can be easily replicated and customized. It's often used along with Jinja. Jinja is a modern and designer-friendly templating engine for Python programming language. It's used in Copier templates to substitute variables in files and file names. You can find more information in the [Copier documentation](https://copier.readthedocs.io/) and [Jinja documentation](https://jinja.palletsprojects.com/). + +### AlgoKit Functionality with Templates + +AlgoKit provides the `algokit init` command to initialize a new project using a template. You can either pass the template name using the `-t` flag or select a template from a list. + +## Key Concepts + +### .algokit.toml + +This file is the AlgoKit configuration file for this project which can be used to specify the minimum version of the AlgoKit. This is essential to ensure that projects created with your template are always compatible with the version of AlgoKit they are using. + +Example from `algokit-beaker-default-template`: + +```toml +[algokit] +min_version = "v1.1.0-beta.4" +``` + +This specifies that the template requires at least version `v1.1.0-beta.4` of AlgoKit. + +### Python Support: `pyproject.toml` + +Python projects in AlgoKit can leverage a wide range of tools for dependency management and project configuration. While Poetry and the `pyproject.toml` file are common choices, they are not the only options. +If you opt to use Poetry, you'll rely on the pyproject.toml file to define the project's metadata and dependencies. This configuration file can utilize the Jinja templating syntax for customization. + +Example snippet from `algokit-beaker-default-template`: + +```toml +[tool.poetry] +name = "{{ project_name }}" +version = "0.1.0" +description = "Algorand smart contracts" +authors = ["{{ author_name }} <{{ author_email }}>"] +readme = "README.md" + +[tool.poetry.dependencies] +python = "^3.10" +beaker-pyteal = "^1.0.0" +algokit-utils = "^1.3" +python-dotenv = "^1.0.0" +``` + +This example shows how project metadata and dependencies are defined in `pyproject.toml`, using Jinja syntax to allow placeholders for project metadata. + +### TypeScript Support: `package.json` + +For TypeScript projects, the `package.json` file plays a similar role as `pyproject.toml` can do for Python projects. It specifies metadata about the project and lists the dependencies required for smart contract development. + +Example snippet: + +```json +{ + "name": "{{ project_name }}", + "version": "1.0.0", + "description": "{{ project_description }}", + "scripts": { + "build": "tsc" + }, + "devDependencies": { + "typescript": "^4.2.4", + "tslint": "^6.1.3", + "tslint-config-prettier": "^1.18.0" + } +} +``` + +This example shows how Jinja syntax is used within `package.json` to allow placeholders for project metadata and dependencies. + +### Bootstrap Option + +When instantiating your template via AlgoKit CLI it will optionally prompt the user to automatically run [algokit bootstrap](../features/bootstrap.md) after the project is initialized and can perform various setup tasks like installing dependencies or setting up databases. + +- `env`: Searches for and copies an `.env*.template` file to an equivalent `.env*` file in the current working directory, prompting for any unspecified values. This feature is integral for securely managing environment variables, as it prevents sensitive data from inadvertently ending up in version control. + By default, Algokit will scan for network-prefixed `.env` variables (e.g., `.env.localnet`), which can be particularly useful when relying on the [Algokit deploy command](../features/deploy.md). If no such prefixed files are located, Algokit will then attempt to load default `.env` files. This functionality provides greater flexibility for different network configurations. + +- `poetry`: If your Python project uses Poetry for dependency management, the `poetry` command installs Poetry (if not present) and runs `poetry install` in the current working directory to install Python dependencies. +- `npm`: If you're developing a JavaScript or TypeScript project, the `npm` command runs npm install in the current working directory to install Node.js dependencies. +- `all`: The `all` command runs all the aforementioned bootstrap sub-commands in the current directory and its subdirectories. This command is a comprehensive way to ensure all project dependencies and environment variables are properly set up. + +### Predefined Copier Answers + +When initializing a new project, Copier can prompt the user for input, which is then passed to the template as variables. This is useful for customizing the new project based on user input. + +Example: + +```yaml +# copier.yaml +project_name: + type: str + help: What is the name of this project. + placeholder: "algorand-app" +``` + +This would prompt the user for the project name, and the input can then be used in the template using the Jinja syntax `{{ project_name }}`. + +#### Default Behaviors + +When creating an AlgoKit template, there are a few default behaviors that you can expect to be provided by algokit-cli itself without introducing any extra code to your templates: + +- **Git**: If Git is installed on the user's system and the user's working directory is a Git repository, AlgoKit CLI will commit the newly created project as a new commit in the repository. This feature helps to maintain a clean version history for the project. If you wish to add a specific commit message for this action, you can specify a `commit_message` in the `_commit` option in your `copier.yaml` file. + +- **VSCode**: If the user has Visual Studio Code (VSCode) installed and the path to VSCode is added to their system's PATH, AlgoKit CLI will automatically open the newly created VSCode window unless user provides specific flags into the init command. + +- **Bootstrap**: AlgoKit CLI is equipped to execute a bootstrap script after a project has been initialized. This script, included in AlgoKit templates, can be automatically run to perform various setup tasks, such as installing dependencies or setting up databases. This is managed by AlgoKit CLI and not within the user-created codebase. By default, if a `bootstrap` task is defined in the `copier.yaml`, AlgoKit CLI will execute it, unless the user opts out during the prompt. + +By combining predefined Copier answers with these default behaviors, you can create a smooth, efficient, and intuitive initialization experience for the users of your template. + +### Working with Generators + +After mastering the use of `copier` and building your templates based on the official AlgoKit template repositories, you can enhance your proficiency by learning to define `custom generators`. Essentially, generators are smaller-scope `copier` templates designed to provide additional functionality after a project has been initialized from the template. + +For example, the official [`algokit-beaker-default-template`](https://github.com/algorandfoundation/algokit-beaker-default-template/tree/main/template_content) incorporates a generator in the `.algokit/generators` directory. This generator can be utilized to execute auxiliary tasks on AlgoKit projects that are initiated from this template, like adding new smart contracts to an existing project. For a comprehensive understanding, please consult the [`architecture decision record`](../architecture-decisions/2023-07-19_advanced_generate_command.md) and [`algokit generate documentation`](../features/generate.md). + +#### How to Create a Generator + +Outlined below are the fundamental steps to create a generator. Although `copier` provides complete autonomy in structuring your template, you may prefer to define your generator to meet your specific needs. Nevertheless, as a starting point, we suggest: + +1. Generate a new directory hierarchy within your template directory under the `.algokit/generators` folder (this is merely a suggestion, you can define your custom path if necessary and point to it via the algokit.toml file). +2. Develop a `copier.yaml` file within the generator directory and outline the generator's behavior. This file bears similarities with the root `copier.yaml` file in your template directory, but it is exclusively for the generator. The `tasks` section of the `copier.yaml` file is where you can determine the generator's behavior. Here's an example of a generator that copies the `smart-contract` directory from the template to the current working directory: + +```yaml +_task: + - "echo '==== Successfully initialized new smart contract 🚀 ===='" + +contract_name: + type: str + help: Name of your new contract. + placeholder: "my-new-contract" + default: "my-new-contract" + +_templates_suffix: ".j2" +``` + +Note that `_templates_suffix` must be different from the `_templates_suffix` defined in the root `copier.yaml` file. This is because the generator's `copier.yaml` file is processed separately from the root `copier.yaml` file. + +3. Develop your `generator` copier content and, when ready, test it by initiating a new project for your template and executing the generator command: + +```bash +algokit generate +``` + +This should dynamically load and display your generator as an optional `cli` command that your template users can execute. + +## Recommendations + +- **Modularity**: Break your templates into modular components that can be combined in different ways. +- **Documentation**: Include README files and comments in your templates to explain how they should be used. +- **Versioning**: Use `.algokit.toml` to specify the minimum compatible version of AlgoKit. +- **Testing**: Include test configurations and scripts in your templates to encourage testing best practices. +- **Linting and Formatting**: Integrate linters and code formatters in your templates to ensure code quality. +- **Algokit Principle**: for details on generic principles on designing templates refer to [algokit design principles](../index.md#guiding-principles). + +## Conclusion + +Creating custom templates in AlgoKit is a powerful way to streamline your development workflow for Algorand smart contracts, whether you are using Python or TypeScript. Leveraging Copier and Jinja for templating, and incorporating best practices for modularity, documentation, and coding standards, can result in robust, flexible, and user-friendly templates that can be a valuable asset to both your own projects and the broader Algorand community. + +Happy coding! + +``` + +``` diff --git a/docs/get-details/algokit/tutorials/intro.md b/docs/get-details/algokit/tutorials/intro.md new file mode 100644 index 000000000..b7c916745 --- /dev/null +++ b/docs/get-details/algokit/tutorials/intro.md @@ -0,0 +1,165 @@ +# AlgoKit Quick Start Tutorial + +AlgoKit is the primary tool used by the Algorand community to develop smart contracts on the Algorand blockchain. It provides the capabilities to develop, test and deploy Algorand smart contracts within minutes! This guide is intended to help you setup AlgoKit and to start developing your application. + +## Quick start videos + +If you prefer videos, take a look at this 10 minute guide to getting started. + +[![Learn How to Build on Algorand in 10 Minutes](../imgs/algokit-intro-video-thumbnail.jpg)](https://www.youtube.com/embed/MzBRef_Res8) + +Detailed video guides for both [Windows](https://www.youtube.com/embed/22RvINnZsRo) and [Mac](https://www.youtube.com/embed/zsurtpCGmgE) are also available. + +## Prerequisites + +This guide presents installing AlgoKit using an OS agnostic procedure. For OS specific instructions take a look at the [quick start videos](#quick-start-videos) above. + +Using this procedure requires the the following components be installed already + +- [Python 3.10](https://www.python.org/downloads/) or higher +- [PipX](https://pypa.github.io/pipx/#on-linux-install-via-pip-requires-pip-190-or-later) +- [Git](https://github.com/git-guides/install-git#install-git) +- [Docker](https://docker.com/download/) +- [VSCode](https://code.visualstudio.com/download) (recommended) + +## Install AlgoKit + +To install AlgoKit, run the following command from a terminal. + +```shell +pipx install algokit +``` + +After the installation completes, **restart the terminal**. + +For more detailed installation documentation, see the [quick start videos](#quick-start-videos). + +## Verify the Installation + +To verify AlgoKit Installed correctly run the following. + +```shell +algokit --version +``` + +Output similar to the following should be displayed: + +```shell +algokit, version 0.5.0 +``` + +## Start a LocalNet + +AlgoKit supports using a [local version of the Algorand blockchain](../features/localnet.md). To start an instance of this LocalNet run the following command from the terminal: + +```shell +algokit localnet start +``` + +This should start an instance of the LocalNet within docker. If you open the Docker Desktop application you should something similar to the following: + +![Docker Desktop LocalNet Instance](../imgs/localnet.png) + +## Create an AlgoKit project + +Now that AlgoKit is installed, you can rapidly create a new project to get started quickly. This can be done by running: + +```shell +algokit init +``` + +This will launch a guided menu system to create a specific project tailored to your needs. You will first be prompted to select a specific template. The templates are basic starter applications for various Algorand development scenarios. To read more about templates checkout AlgoKit detailed documentation. For now, use the arrow keys to select the `playground` template, which is a lightweight starting point for learning and experimentation. + +Next, you will be prompted for the name of your project. Finally, select the default value for the rest of the prompts (enter). + +Once finished, (if you have it installed) VS Code should automatically be opened with the initialised project and you will be prompted to install appropriate VS Code extensions. This starter app will contain one smart contract (built using the [Beaker](https://beaker.algo.xyz/) smart contract development framework) named `helloworld.py`, in the `hello_world` folder, with one method (`hello`) that takes a `String` and returns a `String`. + +![AlgoKit Playground Contract](../imgs/algokitplayground.png) + +## Run the Demo Application + +Once the playground project is created, you will notice in the `hello_world` folder a file named `demo.py` which is a simple example of using AlgoKit to deploy and make a call to the `helloworld.py` smart contract on the LocalNet instance started earlier. + +![AlgoKit Playground Demo](../imgs/algokitdemo.png) + +Right clicking on this file and selecting `Run Python File in Terminal` will deploy the `HelloWorldApp` smart contract and then call it passing the parameter `name` with a value of `Beaker`. Alternatively, you can hit F5 (or whatever keyboard shortcut is set in VS Code for running the debugger) while you are viewing the `helloworld.py` file and it will: + +1. Start LocalNet +2. Build the smart contract +3. Deploy and call the smart contract (`demo.py`) + +This should produce something similiar to the following in the VSCode terminal. + +```shell +(playground-py3.11) User@Algo-User-MBP myalgokit project % "/Users/user/code/algokit/myalgokit pro +ject/.venv/bin/python" "/Users/user/code/algokit/myalgokit project/playground/hello_world/demo.py" +Deployed app in txid SBNIJYZUOXVPXDFJHAVDDCO5TQ4WZ7P37QOOQM6CSVSMIURETHNQ + App ID: 11 + Address: 377KDIVHB7K2LFFJIPUPQFWJGVD36MAP4EI6Y2HAGRAW6JPK4MDUBC3YVY + +Hello, Beaker +``` + +The App ID of of the deployed contract and its Algorand address is displayed, followed by the message returned from the smart contract call (`Hello, Beaker`). + +At this point you have deployed a simple contract to an Algorand network and called it successfully! + +Additionally, you can find the native TEAL smart contract code and the appropriate smart contract manifest JSON files have been output to the `artifacts` folder. + +![AlgoKit Playground Demo](../imgs/algokitartifacts.png) + +Running the `build.py` python file will also generate these artifacts without deploying. These files can be used by tools like [Dappflow](https://dappflow.org/), [goal](https://developer.algorand.org/docs/clis/goal/goal/), etc. to deploy your smart contract to the various Algorand networks. + +## Using Dappflow + +Dappflow is a web-based user interface that let's you visualise accounts, transactions, assets and applications on an Algorand network and also provides ability to deploy and call smart contracts. This works for TestNet, MainNet and also LocalNet. Furthermore, you can also create and fund accounts on LocalNet. While AlgoKit surfaces both a programming interface and a command line interface for interacting with Algorand, it also allows you to quickly open Dappflow so you can see what's happening visually. + +Dappflow can be launched from AlgoKit by running the following command from the VS Code terminal. + +```shell +algokit explore +``` + +By default it will open Dappflow to point to LocalNet (It will be displayed as `sandnet-v1` in the upper left hand corner.), but you can pass in parameters to point it to TestNet and MainNet too. + +This command will launch your default web browser and load the Dappflow web application. + +**Note:** If you are using Safari, then it won't work against LocalNet and you will need to open it in a different browser. + +![AlgoKit Dappflow](../imgs/dappflow1.png) + +### Create test account + +To issue commands against the LocalNet network you need an account with ALGOs in it. Dappflow lets you easily create one. + +Select `Dev Wallets` from the left menu and click on the `Create wallet` button. This will create an account on the LocalNet and fund it with 100 Algos that can be used to test with. + +![AlgoKit Dappflow](../imgs/dappflow2.png) + +In the bottom left hand corner of the menu, select `Connect wallet` and you will be prompted with several wallet choices. Choose the `Dev Wallet` option. This will connect the account you just created to Dappflow so you can use that account for signing transactions from the Dappflow user interface. + +### Deploy the Hello World application + +To deploy the built Beaker smart contract application, select the `Beaker studio` menu and click on the import beaker app. Select `File` and `Upload file`, browse to the artifacts created in the previous section of this guide. Select the `application.json` manifest file. This will load the specific manifest file for the Hello World sample application. + +![AlgoKit Dappflow](../imgs/dappflow3.png) + +To deploy this application again, select the `Create app` button followed by the `Create` button from the popup. You should get a `Transaction successful` message with the option to view the specific transaction in the explorer. Close out of the popup and then scroll down to the `ABI` section of the page. The `hello` method should be displayed with an execute button beside it. + +![AlgoKit Dappflow](../imgs/dappflow4.png) + +Click on the Execute button and a popup will be displayed allowing you to enter the parameter that we defined in the `HelloWorldApp` smart contract. + +![AlgoKit Dappflow](../imgs/dappflow5.png) + +Enter a string in the parameter and click on `Execute`. You should get get a confirmation that the method executed properly and what the smart contract returned. + +![AlgoKit Dappflow](../imgs/dappflow6.png) + +You have now successfully deployed and executed a smart contract method call using Dappflow! + +## Next steps + +- To learn more about AlgoKit and what you can do with it, checkout the [AlgoKit documentation](../index.md). +- To learn more about Beaker, take a look at the [Beaker documentation](https://beaker.algo.xyz/). +- More information on Algorand smart contracts is also available in the [smart contract documentation](https://developer.algorand.org/docs/get-details/dapps/smart-contracts/). diff --git a/docs/get-details/index.md b/docs/get-details/index.md index 27cb79c2f..629cf40de 100644 --- a/docs/get-details/index.md +++ b/docs/get-details/index.md @@ -2,25 +2,26 @@ title: Section Index The **Get Details** section contains explanations and how-tos for all Algorand features. Code examples are embedded throughout for each of the SDKs and command line interface tools. -- [Consensus](algorand_consensus.md) - A high-level guide on the Algorand consensus protocol. - +- [AlgoKit](algokit) - A collection of tools and libraries that make it easy to build applications on Algorand. - [Accounts](accounts) - A guide about Algorand Accounts including keys, wallets, and addresses. -- [Assets](asa) - A guide about assets and how they are implemented on Algorand at layer-1. - -- [Smart Contracts](dapps/smart-contracts/) - A detailed guide for writing and interacting with Smart Contracts. - - [Transactions](transactions) - A look at the underlying structure of the different types of transactions. - [Atomic Transfers](atomic_transfers.md) - A guide to build transactions into an atomic group on Algorand. - [Atomic Transaction Composer](atc.md) - A guide to using the Atomic Transaction Composer, a simpler interface to building atomic groups. +- [Assets](asa) - A guide about assets and how they are implemented on Algorand at layer-1. + +- [Smart Contracts](dapps/smart-contracts/) - A detailed guide for writing and interacting with Smart Contracts. + - [Indexer](indexer.md) - A guide to using the Algorand Indexer. - [Conduit](conduit.md) - A guide to using the Algorand Conduit Service to gather and filter transaction data. +- [Consensus](algorand_consensus.md) - A high-level guide on the Algorand consensus protocol. + - [Protocol Parameters](parameter_tables.md) - A table containing all Algorand blockchain parameters. - [Technical FAQ](technical_faq.md) - A collection of frequently asked questions about building on Algorand. diff --git a/scripts/algokit-docs.sh b/scripts/algokit-docs.sh new file mode 100644 index 000000000..36c81128f --- /dev/null +++ b/scripts/algokit-docs.sh @@ -0,0 +1,56 @@ +#!/usr/bin/env bash + +# SETUP +PATH_TO_SOURCE_REPOS=$1 +# ensure the following repos are checked out to the proper release/branch for doc generation and +# ensure each is a subdirectory of $PATH_TO_SOURCE_REPOS +# git clone https://github.com/algorandfoundation/algokit-cli +# git clone https://github.com/algorandfoundation/algokit-utils-ts +# git clone https://github.com/algorandfoundation/algokit-utils-py +# git clone https://github.com/algorandfoundation/algokit-client-generator-py +# git clone https://github.com/algorandfoundation/algokit-client-generator-ts + +# REMOVE EXISTING ALGOKIT CONTENT +rm -rd ../docs/get-details/algokit + +# RENDER ALGOKIT-CLI +cp -R $PATH_TO_SOURCE_REPOS/algokit-cli/docs/ ../docs/get-details/algokit +mv ../docs/get-details/algokit/algokit.md ../docs/get-details/algokit/index.md +sed -i '' "1s/.*/# Overview/" ../docs/get-details/algokit/index.md +sed -i '' "s/algokit.md/index.md/" ../docs/get-details/algokit/tutorials/intro.md +sed -i '' "s,look that the,look at the," ../docs/get-details/algokit/tutorials/intro.md +sed -i '' "s,\[AlgoKit install\](https://github.com/algorandfoundation/algokit-cli/blob/main/README.md#install) guide,\[quick start videos\](#quick-start-videos) above," ../docs/get-details/algokit/tutorials/intro.md +sed -i '' "s,\[directly used by certain commands\](../../README.md#prerequisites),directly used by certain commands," ../docs/get-details/algokit/features/doctor.md +sed -i '' "s,Prequisites,Prerequisites," ../docs/get-details/algokit/tutorials/intro.md +sed -i '' "s,\[README\](../../README.md#prerequisites),\[install guide\](../tutorials/intro.md#prerequisites)," ../docs/get-details/algokit/features/init.md +sed -i '' "s,https://github.com/algorandfoundation/algokit-cli/blob/main/docs/algokit,../index," ../docs/get-details/algokit/tutorials/algokit-template.md +sed -i '' "s,../../docs/algokit.md,../index.md," ../docs/get-details/algokit/architecture-decisions/2022-11-14_sandbox-approach.md +sed -i '' "s,../../docs/algokit.md,../index.md," ../docs/get-details/algokit/architecture-decisions/2023-01-12_smart-contract-deployment.md +sed -i '' "s,../../docs/algokit.md,../index.md," ../docs/get-details/algokit/architecture-decisions/2023-06-06_frontend-templates.md +sed -i '' "s,../../docs/algokit.md,../index.md," ../docs/get-details/algokit/architecture-decisions/2023-07-19_advanced_generate_command.md +sed -i '' "s,quick start tutorial: \[https://bit.ly/algokit-intro-tutorial\](https://bit.ly/algokit-intro-tutorial),[quick start tutorial](tutorials/intro.md)," ../docs/get-details/algokit/cli/index.md +sed -i '' "s,https://github.com/algorandfoundation/algokit-cli/blob/main/docs/,../," ../docs/get-details/algokit/features/init.md +sed -i '' "/- \[algokit\](#algokit)/,/# algokit/d" ../docs/get-details/algokit/cli/index.md +sed -i '' "s,# AlgoKit CLI Reference Documentation,# CLI Reference," ../docs/get-details/algokit/cli/index.md +sed -i '' "/## Table of Contents/,/## Quick Start/d" ../docs/get-details/algokit/tutorials/algokit-template.md +sed -i '' "s,For users who are keen,## Quick Start\nFor users who are keen," ../docs/get-details/algokit/tutorials/algokit-template.md +sed -i '' "s,https://github.com/algorandfoundation/algokit-cli/blob/main/docs/features/bootstrap,../features/bootstrap," ../docs/get-details/algokit/tutorials/algokit-template.md +sed -i '' "s,https://github.com/algorandfoundation/algokit-cli/blob/deploy-command/docs/features/deploy,../features/deploy," ../docs/get-details/algokit/tutorials/algokit-template.md +sed -i '' "s,dow6U8DxOGc,MzBRef_Res8," ../docs/get-details/algokit/tutorials/intro.md +sed -i '' "s,https://docs.docker.com/desktop/install/mac-install/,https://docker.com/download/," ../docs/get-details/algokit/tutorials/intro.md +sed -i '' "s,\[VSCode\](https://code.visualstudio.com/download),[VSCode](https://code.visualstudio.com/download) (recommended)," ../docs/get-details/algokit/tutorials/intro.md +sed -i '' "s,\[official installation guide](https://github.com/algorandfoundation/algokit-cli#install),[quick start videos](#quick-start-videos)," ../docs/get-details/algokit/tutorials/intro.md +sed -i '' "s, Feature Documentation,," ../docs/get-details/algokit/features/deploy.md +sed -i '' "s,\[permanently on a given machine\](./cli/index.md#version-prompt) with \`algokit config version-prompt disable\`,permanently on a given machine with [\`algokit config version-prompt disable\`](cli/#-skip-version-check)," ../docs/get-details/algokit/index.md + +# REORG CLI REFERENCE +mv ../docs/get-details/algokit/cli/index.md ../docs/get-details/algokit/cli-reference.md +echo "title: AlgoKit\n\narrange:\n- index.md\n- features\n- cli-reference.md\n- tutorials\n- architecture-decisions\n- articles" > ../docs/get-details/algokit/.pages +cd ../docs/get-details/algokit/ +find . -type f -name '*.md' -exec sed -i '' "s,cli/index.md,cli-reference.md," {} + +cd ../../../scripts/ + +# CLEANUP +rm ../docs/get-details/algokit/tutorials/smart-contracts.md # contains "TODO" +rm -rd ../docs/get-details/algokit/sphinx +rm -rd ../docs/get-details/algokit/cli