feat: debugger related helpers and utils (#61)

* chore: wip

* Auto stash before merge of "feat/debugger-support" and "origin/main"

* feat: initial draft implementation for debugger support

* refactor: addressing pr comments, adding extra tests

* chore: regen docs; fix spinx docs bug

* chore: renaming back to .tok.map

* chore: updating name for traces files

* chore: addressing pr comments

* chore: update src/algokit_utils/_debug_utils.py

Co-authored-by: Neil Campbell <[email protected]>

* refactor: addressing pr comments

* chore: typo fix

* chore: addressing pr comments

* chore: windows runner

* chore: testing ci

* chore: testing ci

* chore: removing windows as its unsupported by official images, adding more py versions

* chore: merge conflict fixes

* chore: minor tweaks

* refactor: minor improvements

* chore: bumping cryptography for pip-audit

* chore: fixing tests

* refactor: propagating comments from utils ts pr review

* chore: refining links to debugger

* docs: regen docs

* chore: fixing typo

* docs: updating information on how to read trace files

* chore: refining links to the marketplace

---------

Co-authored-by: Neil Campbell <[email protected]>

.github/workflows/build-python.yaml

@@ -6,21 +6,34 @@ jobs:
   build-python:
     strategy:
       matrix:
-        os: ["ubuntu-latest" ]
-        python: ["3.10"]
+        os: ["ubuntu-latest"]
+        python: ["3.10", "3.11", "3.12"]
     runs-on: ${{ matrix.os }}
     steps:
       - name: Checkout source code
         uses: actions/checkout@v3
 
-      - name: Install poetry
-        run: pipx install poetry
-
       - name: Set up Python ${{ matrix.python }}
         uses: actions/setup-python@v4
         with:
           python-version: ${{ matrix.python }}
-          cache: "poetry"
+
+      - name: Install pipx
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install pipx
+          python -m pipx ensurepath
+
+      - name: Cache Poetry
+        uses: actions/cache@v2
+        with:
+          path: ~/.local/share/pipx/venvs/poetry
+          key: ${{ runner.os }}-poetry-${{ hashFiles('**/poetry.lock') }}
+          restore-keys: |
+            ${{ runner.os }}-poetry-
+
+      - name: Install poetry
+        run: pipx install poetry
 
       - name: Install dependencies
         run: poetry install --no-interaction

.vscode/settings.json

@@ -54,9 +54,10 @@
     ]
   },
 
-  // PowerShell
-  "[powershell]": {
-    "editor.defaultFormatter": "ms-vscode.powershell"
-  },
-  "powershell.codeFormatting.preset": "Stroustrup"
+    // PowerShell
+    "[powershell]": {
+        "editor.defaultFormatter": "ms-vscode.powershell"
+    },
+    "powershell.codeFormatting.preset": "Stroustrup",
+    "python.testing.pytestArgs": ["."]
 }

docs/html/_sources/apidocs/algokit_utils/algokit_utils.md.txt

@@ -175,8 +175,8 @@ orphan: true
     :parser: myst
     :summary:
     ```
-* - {py:obj}`Program <algokit_utils.application_client.Program>`
-  - ```{autodoc2-docstring} algokit_utils.application_client.Program
+* - {py:obj}`Program <algokit_utils.common.Program>`
+  - ```{autodoc2-docstring} algokit_utils.common.Program
     :parser: myst
     :summary:
     ```
@@ -190,6 +190,11 @@ orphan: true
     :parser: myst
     :summary:
     ```
+* - {py:obj}`TransactionParametersDict <algokit_utils.models.TransactionParametersDict>`
+  - ```{autodoc2-docstring} algokit_utils.models.TransactionParametersDict
+    :parser: myst
+    :summary:
+    ```
 * - {py:obj}`TransactionResponse <algokit_utils.models.TransactionResponse>`
   - ```{autodoc2-docstring} algokit_utils.models.TransactionResponse
     :parser: myst
@@ -318,11 +323,31 @@ orphan: true
     :parser: myst
     :summary:
     ```
+* - {py:obj}`opt_in <algokit_utils.asset.opt_in>`
+  - ```{autodoc2-docstring} algokit_utils.asset.opt_in
+    :parser: myst
+    :summary:
+    ```
+* - {py:obj}`opt_out <algokit_utils.asset.opt_out>`
+  - ```{autodoc2-docstring} algokit_utils.asset.opt_out
+    :parser: myst
+    :summary:
+    ```
+* - {py:obj}`persist_sourcemaps <algokit_utils._debugging.persist_sourcemaps>`
+  - ```{autodoc2-docstring} algokit_utils._debugging.persist_sourcemaps
+    :parser: myst
+    :summary:
+    ```
 * - {py:obj}`replace_template_variables <algokit_utils.deploy.replace_template_variables>`
   - ```{autodoc2-docstring} algokit_utils.deploy.replace_template_variables
     :parser: myst
     :summary:
     ```
+* - {py:obj}`simulate_and_persist_response <algokit_utils._debugging.simulate_and_persist_response>`
+  - ```{autodoc2-docstring} algokit_utils._debugging.simulate_and_persist_response
+    :parser: myst
+    :summary:
+    ```
 * - {py:obj}`transfer <algokit_utils._transfer.transfer>`
   - ```{autodoc2-docstring} algokit_utils._transfer.transfer
     :parser: myst
@@ -1502,16 +1527,16 @@ Bases: {py:obj}`enum.Enum`
 `````
 
 ````{py:class} Program(program: str, client: algosdk.v2client.algod.AlgodClient)
-:canonical: algokit_utils.application_client.Program
+:canonical: algokit_utils.common.Program
 
-```{autodoc2-docstring} algokit_utils.application_client.Program
+```{autodoc2-docstring} algokit_utils.common.Program
 :parser: myst
 ```
 
 ```{rubric} Initialization
 ```
 
-```{autodoc2-docstring} algokit_utils.application_client.Program.__init__
+```{autodoc2-docstring} algokit_utils.common.Program.__init__
 :parser: myst
 ```
 
@@ -1713,6 +1738,144 @@ Bases: {py:obj}`enum.Enum`
 
 `````
 
+`````{py:class} TransactionParametersDict()
+:canonical: algokit_utils.models.TransactionParametersDict
+
+Bases: {py:obj}`typing.TypedDict`
+
+```{autodoc2-docstring} algokit_utils.models.TransactionParametersDict
+:parser: myst
+```
+
+```{rubric} Initialization
+```
+
+```{autodoc2-docstring} algokit_utils.models.TransactionParametersDict.__init__
+:parser: myst
+```
+
+````{py:attribute} accounts
+:canonical: algokit_utils.models.TransactionParametersDict.accounts
+:type: list[str]
+:value: >
+   None
+
+```{autodoc2-docstring} algokit_utils.models.TransactionParametersDict.accounts
+:parser: myst
+```
+
+````
+
+````{py:attribute} boxes
+:canonical: algokit_utils.models.TransactionParametersDict.boxes
+:type: collections.abc.Sequence[tuple[int, bytes | bytearray | str | int]]
+:value: >
+   None
+
+```{autodoc2-docstring} algokit_utils.models.TransactionParametersDict.boxes
+:parser: myst
+```
+
+````
+
+````{py:attribute} foreign_apps
+:canonical: algokit_utils.models.TransactionParametersDict.foreign_apps
+:type: list[int]
+:value: >
+   None
+
+```{autodoc2-docstring} algokit_utils.models.TransactionParametersDict.foreign_apps
+:parser: myst
+```
+
+````
+
+````{py:attribute} foreign_assets
+:canonical: algokit_utils.models.TransactionParametersDict.foreign_assets
+:type: list[int]
+:value: >
+   None
+
+```{autodoc2-docstring} algokit_utils.models.TransactionParametersDict.foreign_assets
+:parser: myst
+```
+
+````
+
+````{py:attribute} lease
+:canonical: algokit_utils.models.TransactionParametersDict.lease
+:type: bytes | str
+:value: >
+   None
+
+```{autodoc2-docstring} algokit_utils.models.TransactionParametersDict.lease
+:parser: myst
+```
+
+````
+
+````{py:attribute} note
+:canonical: algokit_utils.models.TransactionParametersDict.note
+:type: bytes | str
+:value: >
+   None
+
+```{autodoc2-docstring} algokit_utils.models.TransactionParametersDict.note
+:parser: myst
+```
+
+````
+
+````{py:attribute} rekey_to
+:canonical: algokit_utils.models.TransactionParametersDict.rekey_to
+:type: str
+:value: >
+   None
+
+```{autodoc2-docstring} algokit_utils.models.TransactionParametersDict.rekey_to
+:parser: myst
+```
+
+````
+
+````{py:attribute} sender
+:canonical: algokit_utils.models.TransactionParametersDict.sender
+:type: str
+:value: >
+   None
+
+```{autodoc2-docstring} algokit_utils.models.TransactionParametersDict.sender
+:parser: myst
+```
+
+````
+
+````{py:attribute} signer
+:canonical: algokit_utils.models.TransactionParametersDict.signer
+:type: algosdk.atomic_transaction_composer.TransactionSigner
+:value: >
+   None
+
+```{autodoc2-docstring} algokit_utils.models.TransactionParametersDict.signer
+:parser: myst
+```
+
+````
+
+````{py:attribute} suggested_params
+:canonical: algokit_utils.models.TransactionParametersDict.suggested_params
+:type: algosdk.transaction.SuggestedParams
+:value: >
+   None
+
+```{autodoc2-docstring} algokit_utils.models.TransactionParametersDict.suggested_params
+:parser: myst
+```
+
+````
+
+`````
+
 `````{py:class} TransactionResponse
 :canonical: algokit_utils.models.TransactionResponse
 
@@ -1957,6 +2120,30 @@ Bases: {py:obj}`algokit_utils._transfer.TransferParametersBase`
 ```
 ````
 
+````{py:function} opt_in(algod_client: algosdk.v2client.algod.AlgodClient, account: algokit_utils.models.Account, asset_ids: list[int]) -> dict[int, str]
+:canonical: algokit_utils.asset.opt_in
+
+```{autodoc2-docstring} algokit_utils.asset.opt_in
+:parser: myst
+```
+````
+
+````{py:function} opt_out(algod_client: algosdk.v2client.algod.AlgodClient, account: algokit_utils.models.Account, asset_ids: list[int]) -> dict[int, str]
+:canonical: algokit_utils.asset.opt_out
+
+```{autodoc2-docstring} algokit_utils.asset.opt_out
+:parser: myst
+```
+````
+
+````{py:function} persist_sourcemaps(*, sources: list[algokit_utils._debugging.PersistSourceMapInput], project_root: pathlib.Path, client: algosdk.v2client.algod.AlgodClient, with_sources: bool = True) -> None
+:canonical: algokit_utils._debugging.persist_sourcemaps
+
+```{autodoc2-docstring} algokit_utils._debugging.persist_sourcemaps
+:parser: myst
+```
+````
+
 ````{py:function} replace_template_variables(program: str, template_values: algokit_utils.deploy.TemplateValueMapping) -> str
 :canonical: algokit_utils.deploy.replace_template_variables
 
@@ -1965,6 +2152,14 @@ Bases: {py:obj}`algokit_utils._transfer.TransferParametersBase`
 ```
 ````
 
+````{py:function} simulate_and_persist_response(atc: algosdk.atomic_transaction_composer.AtomicTransactionComposer, project_root: pathlib.Path, algod_client: algosdk.v2client.algod.AlgodClient, buffer_size_mb: float = 256) -> algosdk.atomic_transaction_composer.SimulateAtomicTransactionResponse
+:canonical: algokit_utils._debugging.simulate_and_persist_response
+
+```{autodoc2-docstring} algokit_utils._debugging.simulate_and_persist_response
+:parser: myst
+```
+````
+
 ````{py:function} transfer(client: algosdk.v2client.algod.AlgodClient, parameters: algokit_utils._transfer.TransferParameters) -> algosdk.transaction.PaymentTxn
 :canonical: algokit_utils._transfer.transfer
 

docs/html/_sources/capabilities/debugger.md.txt

@@ -0,0 +1,30 @@
+# Debugger
+
+The AlgoKit Python Utilities package provides a set of debugging tools that can be used to simulate and trace transactions on the Algorand blockchain. These tools and methods are optimized for developers who are building applications on Algorand and need to test and debug their smart contracts via [AVM Debugger extension](link to vscode extension).
+
+## Configuration
+
+The `config.py` file contains the `UpdatableConfig` class which manages and updates configuration settings for the AlgoKit project. The class has the following attributes:
+
+- `debug`: Indicates whether debug mode is enabled.
+- `project_root`: The path to the project root directory. Can be ignored if you are using `algokit_utils` inside an `algokit` compliant project (containing `.algokit.toml` file). For non algokit compliant projects, simply provide the path to the folder where you want to store sourcemaps and traces to be used with [`AVM Debugger`](links to extension). Alternatively you can also set the value via the `ALGOKIT_PROJECT_ROOT` environment variable.
+- `trace_all`: Indicates whether to trace all operations. Defaults to false, this means that when debug mode is enabled, any (or all) application client calls performed via `algokit_utils` will store responses from `simulate` endpoint. These files are called traces, and can be used with [AVM Debugger](link to vscode extension) to debug TEAL source codes, transactions in the atomic group and etc.
+- `trace_buffer_size_mb`: The size of the trace buffer in megabytes. By default uses 256 megabytes. When output folder containing debug trace files exceedes the size, oldest files are removed to optimize for storage consumption.
+- `max_search_depth`: The maximum depth to search for a an `algokit` config file. By default it will traverse at most 10 folders searching for `.algokit.toml` file which will be used to assume algokit compliant project root path.
+
+The `configure` method can be used to set these attributes.
+
+To enable debug mode in your project you can configure it as follows:
+
+```py
+from algokit_utils.config import config
+
+config.configure(debug=True)
+```
+
+## Debugging Utilities
+
+Debugging utilities can be used to simplify gathering artifacts to be used with [AVM Debugger](link to vscode extension) in non algokit compliant projects. The following methods are provided:
+
+- `persist_sourcemaps`: This method persists the sourcemaps for the given sources as AVM Debugger compliant artifacts.
+- `simulate_and_persist_response`: This method simulates the atomic transactions using the provided `AtomicTransactionComposer` object and `AlgodClient` object, and persists the simulation response to an AVM Debugger compliant JSON file.