From de2183f1e2e43247ec5bc80c81b1a86b995c68cd Mon Sep 17 00:00:00 2001
From: rdlrt <3169068+rdlrt@users.noreply.github.com>
Date: Thu, 15 Aug 2024 01:29:28 +0000
Subject: [PATCH] Add a note regarding snapshots for dbsync

---
 Build/dbsync/index.html  |   2 +-
 search/search_index.json |   2 +-
 sitemap.xml              |  76 +++++++++++++++++++--------------------
 sitemap.xml.gz           | Bin 528 -> 528 bytes
 4 files changed, 40 insertions(+), 40 deletions(-)

diff --git a/Build/dbsync/index.html b/Build/dbsync/index.html
index 3359cf04d..c4f30bfcb 100644
--- a/Build/dbsync/index.html
+++ b/Build/dbsync/index.html
@@ -1213,7 +1213,7 @@ <h4 id="create-symlink-to-schema-folder">Create Symlink to schema folder<a class
 <div class="highlight"><pre><span></span><code>ln<span class="w"> </span>-s<span class="w"> </span>~/git/cardano-db-sync/schema<span class="w"> </span><span class="nv">$CNODE_HOME</span>/guild-db/schema
 </code></pre></div>
 <h4 id="restore-using-snapshot">Restore using Snapshot<a class="headerlink" href="#restore-using-snapshot" title="Permanent link">⚓︎</a></h4>
-<p>If you're running a mainnet/preview/preprod instance of dbsync, you might want to consider use of dbsync snapshots as documented <a href="https://github.com/intersectmbo/cardano-db-sync/blob/master/doc/state-snapshot.md">here</a>. The snapshot files as of recent epoch are available via links in <a href="https://github.com/intersectmbo/cardano-db-sync/releases">release notes</a>.</p>
+<p>If you're running a mainnet/preview/preprod instance of dbsync, you might want to consider use of dbsync snapshots as documented <a href="https://github.com/intersectmbo/cardano-db-sync/blob/master/doc/state-snapshot.md">here</a>. The snapshot files from IO for their default configs as of recent epoch are available via links in <a href="https://github.com/intersectmbo/cardano-db-sync/releases">release notes</a>. Note that the snapshots should only be used pertaining to their specific configs, if using configs from Koios - you'd want to look at snapshots <a href="https://share.koios.rest/share/xFdZDfM4/dbsync/">here</a> instead.</p>
 <p>At high-level, this would involve steps as below (read and update paths as per your environment):</p>
 <div class="highlight"><pre><span></span><code><span class="c1"># Replace the actual link below with the latest one from release notes</span>
 wget<span class="w"> </span>https://update-cardano-mainnet.iohk.io/cardano-db-sync/13/db-sync-snapshot-schema-13-block-7622755-x86_64.tgz
diff --git a/search/search_index.json b/search/search_index.json
index a6ce2618c..4aea37521 100644
--- a/search/search_index.json
+++ b/search/search_index.json
@@ -1 +1 @@
-{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":"<p>This documentation site (rather the repository itself) is created by some of the well known and experienced community members and contains instructions/information about various guild tools which simplify various stake-ops (setting up, managing and monitoring pools) for operators. Note that the guides are present to help you simplify your tasks - but as an entity responsible for creating blocks on a financial platform, we expect some basic pre-requisite skill sets - at professional level - before entering the portal:</p> <ul> <li>Learn about working with architecture, setup and essentials (SysOps);</li> <li>Understand how to secure your server (SecOps);</li> <li>Be comfortable with working with <code>cardano-cli</code>, and have worked on preview/preprod/guild networks for pool operations without use of wrapper scripts - as an education exercise;</li> <li>Read the documentation and disclaimers. The guide and tools only aim to simplify your tasks, it will not try to do everything for you - neither scope best practices;</li> <li>We recommend anyone using these instructions to use offline workflow, and NEVER have their wallet keys and pool keys to be made available to online servers, as supported by the tools.</li> </ul> <p>Everyone is welcome to contribute to the repository (via documentation, testing, code, videos, etc). Our aim is to work together and reduce confusion rather than hosting 100 versions of documentation - each marketing their pool in a way.</p>"},{"location":"#support","title":"Support","text":"<p>The Telegram Support channel is used to announce new releases and changes to the code base. This is also the place to ask general questions regarding the documentation and scripts on this site.  </p> <p>To report bugs and issues with scripts and documentation please open a GitHub Issue. Feature requests are best opened as a discussion thread.</p>"},{"location":"#getting-started","title":"Getting Started","text":"<p>Use the sidebar to navigate through the topics. Note that the instructions assume the folder structure as per here.</p> <p>Again, Feedback/Contribution and ownership of tasks is always welcome. If you're interested in collaborating regularly, make a start - and you should be part of the guild already .</p>"},{"location":"basics/","title":"Basics","text":""},{"location":"basics/#architecture","title":"Architecture","text":"<p>The architecture for various components are already described at docs.cardano.org by CF/IOHK. We will not reinvent the wheel </p>"},{"location":"basics/#manual-software-pre-requirements","title":"Manual Software Pre-Requirements","text":"<p>While we do not intend to hand out step-by-step instructions, the tools are often misused as a shortcut to avoid ensuring base skillsets mentioned on home page. Some of the common gotchas that we often find SPOs to miss out on:</p> <ul> <li>It is imperative that pools operate with highly accurate system time, in order to propogate blocks to network in a timely manner and avoid penalties to own (or at times other competing) blocks. Please refer to sample guidance here  for details - the precise steps may depend on your OS.</li> <li>Ensure your Firewall rules at Network as well as OS level are updated according to the usage of your system, you'd want to whitelist the rules that you really need to open to world (eg: You might need node and SSH ports to be open to relays and perhaps home workstation on core, while open node to internet on relays, depending on your topology and configuration that you run).</li> <li>Update your SSH Configuration to prevent password-based logon.</li> <li>Ensure that you use offline workflow, you should never require to have your offline keys on online nodes. The tools provide you backup/restore functionality to only pass online keys to online nodes.</li> </ul>"},{"location":"basics/#pre-requisites","title":"Pre-Requisites","text":"<p>Reminder !!</p> <p>You're expected to run the commands below from same session, using same working directories as indicated and using a <code>non-root user with sudo access</code>. You are expected to be familiar with this as part of pre-requisite skill sets for stake pool operators.</p>"},{"location":"basics/#os-prereqs","title":"Set up OS packages, folder structure and fetch files from repo","text":"<p>The pre-requisites for Linux systems are automated to be executed as a single script. This script uses opt-in election of what you'd like the script to do. The defaults without any arguments will only update static part of script contents for you. To download the pre-requisites scripts, execute the below:</p> <pre><code>mkdir \"$HOME/tmp\";cd \"$HOME/tmp\"\n# Install curl\n# CentOS / RedHat - sudo dnf -y install curl\n# Ubuntu / Debian - sudo apt -y install curl\ncurl -sS -o guild-deploy.sh https://raw.githubusercontent.com/cardano-community/guild-operators/master/scripts/cnode-helper-scripts/guild-deploy.sh\nchmod 755 guild-deploy.sh\n</code></pre> <p>Important !!</p> <p>Please familiarise with the syntax of <code>guild-deploy.sh</code> before proceeding (using -h as below). The exact parameters you want to run with is dependent on your, below are only sample instructions</p> <p>The usage syntax can be checked using <code>./guild-deploy.sh -h</code> , sample output below:</p> <pre><code>Usage: guild-deploy.sh [-n &lt;mainnet|guild|preprod|preview|sanchonet&gt;] [-p path] [-t &lt;name&gt;] [-b &lt;branch&gt;] [-u] [-s [p][b][l][m][d][c][o][w][x][f][s]]\nSet up dependencies for building/using common tools across cardano ecosystem.\nThe script will always update dynamic content from existing scripts retaining existing user variables\n\n-n    Connect to specified network instead of mainnet network (Default: connect to cardano mainnet network) eg: -n guild\n-p    Parent folder path underneath which the top-level folder will be created (Default: /opt/cardano)\n-t    Alternate name for top level folder - only alpha-numeric chars allowed (Default: cnode)\n-b    Use alternate branch of scripts to download - only recommended for testing/development (Default: master)\n-u    Skip update check for script itself\n-s    Selective Install, only deploy specific components as below:\n  p   Install common pre-requisite OS-level Dependencies for most tools on this repo (Default: skip)\nb   Install OS level dependencies for tools required while building cardano-node/cardano-db-sync components (Default: skip)\nl   Build and Install libsodium fork from IO repositories (Default: skip)\nm   Download latest (released) binaries for mithril-signer, mithril-client (Default: skip)\nd   Download latest (released) binaries for bech32, cardano-address, cardano-node, cardano-cli, cardano-db-sync and cardano-submit-api (Default: skip)\nc   Download latest (released) binaries for CNCLI (Default: skip)\no   Download latest (released) binaries for Ogmios (Default: skip)\nw   Download latest (released) binaries for Cardano Hardware CLI (Default: skip)\nx   Download latest (released) binaries for Cardano Signer binary (Default: skip)\nf   Force overwrite config files (backups of existing ones will be created) (Default: skip)\ns   Force overwrite entire content [including user variables] of scripts (Default: skip)\n</code></pre> <ol> <li>If you receive an error for <code>glibc</code>, it would likely be due to the build mismatch between pre-compiled binary and your OS, which is not uncommon. You may need to compile cncli manually on your OS as per instructions here - make sure to copy the output binary to <code>\"${HOME}/.local/bin\"</code> folder.</li> </ol> <p>A typical example install to install most components but not overwrite static part of existing files for preview network would be:</p> <pre><code>./guild-deploy.sh -b master -n preview -t cnode -s pdlcowx\n. \"${HOME}/.bashrc\"\n</code></pre> <p>If instead of download, you'd want to build the components yourself, you could use:</p> <pre><code>./guild-deploy.sh -b master -n preview -t cnode -s pblcowx\n. \"${HOME}/.bashrc\"\n</code></pre> <p>Lastly, if you'd want to update your scripts but not install any additional dependencies, you may simply run:</p> <pre><code>./guild-deploy.sh -b master -n preview -t cnode\n</code></pre>"},{"location":"basics/#folder-structure","title":"Folder structure","text":"<p>Running the script above will create the folder structure as per below, for your reference. You do NOT require <code>CNODE_HOME</code> to be set on shell level as scripts will derive the parent folder and assign it at runtime, the addition in <code>~/.bashrc</code> is only for your ease to switch to right folder:</p> <pre><code>/opt/cardano/cnode            # Top-Level Folder\n\u251c\u2500\u2500 ...\n\u251c\u2500\u2500 files                     # Config, genesis and topology files\n\u2502   \u251c\u2500\u2500 ...\n\u2502   \u251c\u2500\u2500 byron-genesis.json    # Byron Genesis file referenced in config.json\n\u2502   \u251c\u2500\u2500 shelley-genesis.json  # Genesis file referenced in config.json\n\u2502   \u251c\u2500\u2500 alonzo-genesis.json    # Alonzo Genesis file referenced in config.json\n\u2502   \u251c\u2500\u2500 config.json           # Config file used by cardano-node\n\u2502   \u2514\u2500\u2500 topology.json         # Map of chain for cardano-node to boot from\n\u251c\u2500\u2500 db                        # DB Store for cardano-node\n\u251c\u2500\u2500 guild-db                  # DB Store for guild-specific tools and additions (eg: cncli, cardano-db-sync's schema)\n\u251c\u2500\u2500 logs                      # Logs for cardano-node\n\u251c\u2500\u2500 priv                      # Folder to store your keys (permission: 600)\n\u251c\u2500\u2500 scripts                   # Scripts to start and interact with cardano-node\n\u2514\u2500\u2500 sockets                   # Socket files created by cardano-node\n</code></pre>"},{"location":"build/","title":"Overview","text":"<p>The documentation here uses instructions from Intersect MBO repositories as foundation, with additional info which we can contribute to where appropriate. Note that not everyone needs to build each component. You can refer to architecture to understand and qualify which of the components built by IO you want to run.</p>"},{"location":"build/#components","title":"Components","text":"<p>For most Pool Operators, simply building cardano-node should be enough. Use the below to decide whether you need other components:</p> graph TB   A([Interact with HD Walletslocally])   B([Explore blockchainlocally])   C([Easy pool-ops andfund management])   D([Create Custom Assets])   E([Monitor node using Terminal UI])   F([Sign/verify any datausing crypto keys])   N(Node)   O(Ogmios)   P(gRest/Koios)   Q(DBSync)   R(Wallet)   S(CNTools)   T(Tx Submit API)   U(GraphQL)   V(OfflineMetadataTools)   X(gLiveView)   Y(cardano-signer)   Z[(PostgreSQL)]  N --x C --x S N --x D --x S &amp; V N --x E --x X N --x B B --x U --x Q B --x P --x Q P --x O P --x T F ---x Y N --x A --x R Q --x Z  <p>Important</p> <p>We strongly prefer use of gRest over GraphQL components due to performance, security, simplicity, control and most importantly - consistency benefits. Please refer to official documentations if you're interested in <code>GraphQL</code> or <code>Cardano-Rest</code> components instead.</p> <p>Note</p> <p>The instructions are intentionally limited to stack/cabal** to avoid wait times/availability of nix/docker files on a rapidly developing codebase - this also helps us prevent managing multiple versions of instructions.</p>"},{"location":"build/#description-for-components-built-by-community","title":"Description for components built by community","text":""},{"location":"build/#cntools","title":"CNTools","text":"<p>A swiss army knife for pool operators, primarily built by Ola, to simplify typical operations regarding their wallet keys and pool management. You can read more about it here</p>"},{"location":"build/#gliveview","title":"gLiveView","text":"<p>A local node monitoring tool, primarily built by Ola, to use in addition to remote monitoring tools like Prometheus/Grafana, Zabbix or IOG's RTView. This is especially useful when moving to a systemd deployment - if you haven't done so already - as it offers an intuitive UI to monitor the node status. You can read more about it here</p>"},{"location":"build/#topology-updater","title":"Topology Updater","text":"<p>A temporary node-to-node discovery solution, run by Markus, that was started initially to bridge the gap created while awaiting completion of P2P on cardano network, but has since become an important lifeline to the network health - to allow everyone to activate their relay nodes without having to postpone and wait for manual topology completion requests. You can read more about it here</p>"},{"location":"build/#koiosgrest","title":"Koios/gRest","text":"<p>A full-featured local query layer node to explore blockchain data (via dbsync) using standardised pre-built queries served via API as per standard from Koios - for which user can opt to participate in elastic query layer. You can read more about build steps here and reference API endpoints here</p>"},{"location":"build/#ogmios","title":"Ogmios","text":"<p>A lightweight bridge interface for cardano-node. It offers a WebSockets API that enables local clients to speak Ouroboros' mini-protocols via JSON/RPC. You can read more about it here</p>"},{"location":"build/#cncli","title":"CNCLI","text":"<p>A CLI tool written in Rust by Andrew Westberg for low-level communication with cardano-node. It is commonly used by SPOs to check their leader logs (integrates with CNTools as well as gLiveView) or to send their pool's health information to https://pooltool.io. You can read more about it here</p>"},{"location":"build/#cardano-signer","title":"Cardano Signer","text":"<p>A tool written by Martin to sign/verify data (hex, text or binary) using cryptographic keys to generate data as per CIP-8 or CIP-36 standards. You can read more about it here</p>"},{"location":"catalystf11/","title":"Catalystf11","text":""},{"location":"catalystf11/#marlowehub-unifying-platform-for-marlowe-smart-contracts-phase-1-smart-contracts","title":"MarloweHub: Unifying Platform for Marlowe Smart Contracts - Phase 1 - Smart Contracts","text":"<p>Category: Concept   Applicant: mike (pooltool.io)   Requested funds: \u20b3100,000.00   Links: Ideascale, Lidonation </p>"},{"location":"catalystf11/#adastat-cardano-explorer-open-source-improved-reboot-towards-a-first-class-community-blockchain-explorer","title":"AdaStat Cardano Explorer - Open Source Improved Reboot towards a first-class community blockchain explorer","text":"<p>Category: Product   Applicant: Dmytro Stashenko (adastat.net)   Requested funds: \u20b3180,300.00   Links: Ideascale, Lidonation </p>"},{"location":"catalystf11/#adahold-decentralized-price-can-only-go-up-token-solution-for-true-ada-hodlers-smart-contract","title":"AdaHold: Decentralized Price-Can-Only-Go-Up Token, Solution For TRUE Ada Hodlers - Smart Contract","text":"<p>Category: Concept   Applicant: Dmytro Stashenko (adastat.net)   Requested funds: \u20b399,700.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#sundae-labs-next-gen-uplc-debugger-with-aiken-integration","title":"Sundae Labs Next-Gen UPLC Debugger with Aiken Integration","text":"<p>Category: Developers   Applicant: Dan Gonzalez (sundae.fi)   Requested funds: \u20b3140,000.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#margin-pool-enhanced-liquidity-for-margin-trading-on-sundaeswap","title":"Margin Pool: Enhanced Liquidity for Margin Trading on SundaeSwap","text":"<p>Category: Solution   Applicant: Dan Gonzalez (sundae.fi)   Requested funds: \u20b3300,000.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#regulated-and-permissioned-defi-with-sundae-and-kora-labs","title":"Regulated and Permissioned DeFi with Sundae and Kora Labs","text":"<p>Category: Concept   Applicant: Dan Gonzalez (sundae.fi)   Requested funds: \u20b3100,000.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#sundae-labs-comprehensive-specification-development-for-gummiworm-protocol-on-cardano","title":"Sundae Labs Comprehensive Specification Development for Gummiworm Protocol on Cardano","text":"<p>Category: Concept   Applicant: Dan Gonzalez (sundae.fi)   Requested funds: \u20b3100,000.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#nftcdnio-universal-nft-viewer-api-musicvideoweb3d","title":"[nftcdn.io] Universal NFT Viewer API (Music+Video+Web+3D+\u2026)","text":"<p>Category: Product   Applicant: Smaug (pool.pm)   Requested funds: \u20b3299,999.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#nftcdnio-nsfw-nft-detection-for-marketplaces-wallets-explorers","title":"[nftcdn.io] NSFW NFT Detection for Marketplaces, Wallets &amp; Explorers","text":"<p>Category: Product   Applicant: Smaug (pool.pm)   Requested funds: \u20b3149,999.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#cardanoscan-analytics-charts","title":"Cardanoscan Analytics Charts","text":"<p>Category: Product   Applicant: Strica (cardanoscan.io)   Requested funds: \u20b344,000.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#cardanoscan-api-javascript-sdk","title":"Cardanoscan API Javascript SDK","text":"<p>Category: Developers   Applicant: Strica (cardanoscan.io)   Requested funds: \u20b364,000.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#integrate-keystone-hardware-wallet-into-typhon","title":"Integrate Keystone Hardware Wallet into Typhon","text":"<p>Category: Product   Applicant: Strica (cardanoscan.io)   Requested funds: \u20b384,000.00   Links: Ideascale, Lidonation </p>"},{"location":"catalystf11/#add-support-for-marlowe-on-cardanoscan","title":"Add support for Marlowe on Cardanoscan","text":"<p>Category: Product   Applicant: Strica (cardanoscan.io)   Requested funds: \u20b3133,000.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#cardanoscan-data-info-bubbles","title":"Cardanoscan data info bubbles","text":"<p>Category: Ecosystem   Applicant: Strica (cardanoscan.io)   Requested funds: \u20b335,000.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#create-a-cnt-marketplace-on-norwegian-block-exchange-nbxcom","title":"Create a CNT marketplace on Norwegian Block Exchange (NBX.COM)","text":"<p>Category: Product   Applicant: Eystein Hansen (nbx.com)   Requested funds: \u20b3145,000.00   Links: Ideascale, Lidonation</p>"},{"location":"contributors/","title":"Contributors","text":"<p>Everyone is welcome to contribute to the guide, as well as the repository. Below is just a thank you to people who have been contributing consistently:</p> <p>Adam Chris Damjan Homer Markus OCG Ola Ahlman Pal Dorogi Papacarp PegasusPool Psychomb RdLrT RedOracle SmaugPool </p> <p>To start contributing, simply hit the github repository and raise Issue/Pull Request</p>"},{"location":"grest-meets/","title":"GRest Meeting summaries","text":"<p>Thank you all for joining and contributing to the project </p> <p>Below you can find a short summary of every GRest meeting held, both for logging purposes and for those who were not able to attend.</p>"},{"location":"grest-meets/#participants","title":"Participants:","text":"Participant 16Sep2021 02Sep2021 26Aug2021 19Aug2021 12Aug2021 29Jul2021 22Jul2021 15Jul2021 09Jul2021 02Jul2021 25Jun2021 Damjan Homer Markus Ola RdLrT Red Papacarp Paddy GimbaLabs 16Sep2021 02Sep2021 26Aug2021 19Aug2021 12Aug2021 29Jul2021 22Jul2021 15Jul2021 09Jul2021 02Jul2021 <p>After the initial stand-up updates from participants, we went through the entire Trello board, updating/deleting existing tickets and creating some new ones.</p> 25Jun2021"},{"location":"grest-meets/#scheduling-running-update-queries","title":"Scheduling running update queries","text":"<ul> <li>Postgres triggers are synchronous so they slow down db-sync</li> <li>Decided to explore crontab for query scheduling instead</li> </ul>"},{"location":"grest-meets/#refactor-of-queries","title":"Refactor of queries","text":"<ul> <li>Discussed how to structure RPC endpoints and what each should include</li> <li>Details have been captured in the Trello board</li> </ul>"},{"location":"grest-meets/#postgres-tuning","title":"postgres tuning","text":"<ul> <li>Discussed possible tunings to the postgres config</li> <li>Probably reducing WAL usage</li> </ul>"},{"location":"grest-meets/#updates","title":"Updates","text":"<ul> <li>making good progress on the website (koios.rest) - great job Markus!</li> <li>query tickets are now well structured and put into sections (Account, Pool, Transactions...) on the Trello board - nice work Priyank/Ola!</li> </ul>"},{"location":"grest-meets/#queries","title":"Queries","text":"<ul> <li>Transaction cache table:</li> <li>We would like to avoid 'handling' rollbacks in that table, instead simply dump if multiple entries for a transaction, as it can have much higher combination and volume to process - especially post a node/postgres/dbsync restart.</li> <li> <p>Solution being tested:</p> <ul> <li>Use an md5 hash of concatenated tx_id, tx_index and block hash to generate unique serves as primary key in that table</li> <li>On RPC built off cache tables query layer, we would add a validation for block hash being in the public.block table and exclude those who are not as part of result</li> <li>This way we don't handle rollbacks, and also keep a record if in future we require to cross check/re-run delta</li> </ul> </li> <li> <p>Pool cache table:</p> </li> <li>Need to check if transaction cache method is useful here too</li> <li>Using strace we could verify the order in which tables are touched, and avoid trigger-check to run on every block.</li> </ul>"},{"location":"grest-meets/#problems","title":"Problems","text":"<ul> <li>Priyank noticed that if including any foreign keys for tx cache, it can cause a spike in load resulting in crash of db-sync instance (with locks). There arent any visible advantages maintaining a constraint on cache table anyways, as it decreases performance. Thus, we'd keep the cache tables simple and not include any foreign keys.</li> <li>Infrastructure upgrades unlikely to help for such cases (though we may need to increase baseline specs back to initial discussion anyways, but that would be during performance testing stages).</li> </ul>"},{"location":"grest-meets/#actions","title":"Actions","text":"<ul> <li>POST/GET endpoint rules:</li> <li>Use GET for endpoints that take no input parameters (PostgREST native parameters can still be applied via URL)</li> <li>Any endpoint that we accept consumer to provide parameter for should be POST</li> <li>Switch cardanoqueries.ha to api.koios.rest on the API docs</li> <li>Load balancing:</li> <li>Until we see additional instances, the 4 trusted instances serve as Monitoring layer.</li> <li>If/when we start having community instances, we could start splitting topologies to be geographically balanced - using github as source.</li> </ul>"},{"location":"grest-meets/#queries_1","title":"Queries","text":"<ul> <li>stake distribution:</li> <li> <p>we will run the full query on regular intervals, ready for review for first iteration, will see about delta post tx cache query</p> </li> <li> <p>transaction history:</p> </li> <li>transaction history query needs to be switched to populate a cached table instead</li> <li> <p>need to think about how to approach inputs/outputs in the cached table (1 row per transaction with json objects for inputs/outputs or multiple rows for tx hash)</p> </li> <li> <p>address_txs:</p> </li> <li> <p>this endpoint should bring back list of txs, and have provision to use after and before block hash - lightweight against public schema</p> </li> <li> <p>pool cache table:</p> </li> <li>cached table to aggregate info from all the pool tables together (pool_metadata, pool_hash, pool_update...)</li> <li>the cached data should include the full history of all pools as well as the current state (latest pool update)</li> <li>will then be used for (most likely) all pool related endpoints without the need for joins</li> </ul>"},{"location":"grest-meets/#transaction-submission-feature","title":"Transaction submission feature","text":"<ul> <li>a post endpoint separate from gREST ones (different port), proxy'd over haproxy using same health check script appended for node</li> <li>will receive signed transactions in 2 formats (file and cbor) and use cardano-submit-api or CLI to submit them to the blockchain respectively</li> <li>use cases are mostly light wallets, and third-party wallets or CNTools could implement such light features with it (no need for cardano-node with CNTools)</li> </ul>"},{"location":"grest-meets/#db-replication-presentation-by-redoracle","title":"DB replication presentation by Redoracle","text":"<ul> <li>Proposition to move the gRest schema and tables required by the API to smaller instances that can be scaled more easily</li> <li>Pros and Cons to the approach discussed, worth investigating based on performance comparisons</li> </ul>"},{"location":"grest-meets/#process-for-upgrading-our-instances","title":"Process for upgrading our instances:","text":"<ul> <li>Collaboration between trusted peers will be needed, to upgrade sequentially (3 off 6 instances, for example)</li> <li>Use DNS subdomain for upgraded nodes for testing</li> <li>Ideally upgrade processes to be done between 2nd-4th day of epoch to avoid overloading smaller subset in peak hours</li> <li>Enhance grest-poll to use arguments and seperate haproxy backends, allowing for test based reduction</li> </ul>"},{"location":"grest-meets/#queries_2","title":"Queries:","text":""},{"location":"grest-meets/#stake-distribution","title":"Stake distribution","text":"<ul> <li>we need to implement triggers</li> <li>dealing with block rollbacks is tricky</li> <li>Priyank will make an example of his idea of how to deal with it that others can use/build upon</li> </ul>"},{"location":"grest-meets/#tx-history","title":"Tx History","text":"<ul> <li>Current PR to be split into two (to include value/assets and not have to return JSON that is resource intensive to generate/parse):</li> <li>Addr to Tx Hash list using start and end blocks</li> <li>Bulk Tx Hash (limit 100) query to get as much details about tx sent as possible</li> <li>Consider if cache table makes sense after above change. If yes, we also need triggers that can handle rollbacks</li> </ul>"},{"location":"grest-meets/#problems_1","title":"PROBLEMS","text":"<ul> <li>stake distribution query needs to be completed</li> <li>it's hard to use docker to replicate our current setup</li> </ul>"},{"location":"grest-meets/#actions_1","title":"ACTIONS","text":"<ul> <li>additional things to add to stake_distribution query</li> <li>Add logic to record and check tx based on block.id for last but 3rd block in existing query</li> <li>Add a control table in grest schema to record block_hash used for last update, start time, and end time. This will act as a checkpoint for polling of queries that are not live (separate backend in haproxy)</li> <li> <p>create a trigger every 2 minutes (or similar) to run stake_distribution query</p> </li> <li> <p>docker:</p> </li> <li>problems with performance due to nature of IOPs and throughput usage for resources being isolated and can only access each other through sockets.</li> <li>still useful to test whether fully dockerized (each component isolated) can keep up to chain tip</li> <li>consider dockerizing all resources in one container to give new joiners a simple one liner to get up and running - this still doesn't ensure optimal performance, Tuning will still be an additional task for any infrastructure to customise setup to best results achievable</li> </ul>"},{"location":"grest-meets/#problems_2","title":"PROBLEMS","text":"<ul> <li>Not everyone reporting to the monitoring dashboard</li> <li>We don't fully understand the execution time deviations of the stake distribution query</li> <li>catalyst rewards are hard to isolate</li> <li>branch 10.1.x has been deleted on the db-sync repo</li> <li>people have a hard time catching up with the project after being away for a while</li> </ul>"},{"location":"grest-meets/#actions_2","title":"ACTIONS","text":"<ul> <li>missing instances start reporting to monitoring</li> <li>run stake_distribution query on multiple instances, report output of <code>EXPLAIN (ANALYZE, BUFFERS)</code></li> <li>catalyst rewards can be ignored until there is a clear path to get them: Fix underway using open PR</li> <li>if someone needs help getting the right db-sync commit, message Priyank for help as the branch is now deleted</li> <li>add project metadata (requirements) to grest doc header in a checklist format that folks can use to ensure their setup is up-to-date with the current project state</li> <li>Discussed long-term plans (will be added separately in group)</li> </ul>"},{"location":"grest-meets/#problems_3","title":"PROBLEMS","text":"<ul> <li>how to sync live stake between instances (or is there need for it?)</li> </ul>"},{"location":"grest-meets/#actions_3","title":"ACTIONS","text":"<ol> <li> <p>Team</p> <ul> <li>catch live stake distributions in a separate table (in our <code>grest</code> schema)<ul> <li>these queries can run on a schedule</li> <li>response comes from the instance with the latest data</li> </ul> </li> <li>other approaches:<ul> <li>possibly distribute pools between instances (complex approach)</li> <li>run full query once and only check for new/leaving delegators (probably impossible because of existing delegator UTXO movements)</li> </ul> </li> <li>implement monitoring of execution times for all the queries</li> <li>come up with a timeline for launch (next call)</li> <li>stress test before launch</li> <li>start building queries listed on Trello board</li> </ul> </li> <li> <p>Individual</p> <ul> <li>sync db-sync instances to commit <code>84226d33eed66be8e61d50b7e1dacebdc095cee9</code> on <code>release/10.1.x</code></li> <li>update setups to reflect recent directory restructuring and updated instructions</li> </ul> </li> </ol>"},{"location":"grest-meets/#introduction-for-new-joiner-paddy","title":"Introduction for new joiner - Paddy","text":"<ul> <li>from Shamrock stake pool / poolpeek</li> <li>gRest project could be helpful for pool peek</li> <li>Paddy will probably run an individual instance</li> </ul>"},{"location":"grest-meets/#problems_4","title":"Problems","text":"<ul> <li>there is a problem with extremely high CPU usage haproxy, tuning underway.</li> <li>live stake query has multiple variations, and we need to figure out what is the correct one.</li> </ul>"},{"location":"grest-meets/#action-items","title":"Action Items","text":"<ul> <li>Everyone should add monitoring to their instances</li> <li>restructure RPC query files (separate metadata in <code>&lt;query&gt;.json</code> and sql in <code>&lt;query&gt;.sql</code>), also remove <code>get_</code> prefix</li> <li>Add new queries from the list</li> <li>fix haproxy CPU usage (use <code>nbthreads</code> in config, tune maxconn, switch to http mode)</li> <li>gather multiple variations of the live stake query and ask Erik for clarification on which one is correct</li> <li>Start working on other queries listed on trello</li> </ul>"},{"location":"grest-meets/#deployment-scripts","title":"Deployment scripts","text":"<p>Ola added automatic deployment of services to the scripts last week. We added new tasks on Trello ticket, including flags for multiple networks (guild, testnet, mainnet), haproxy service dynamically creating hosts and doc updates. Overall, the script works well with some manual interaction still required at the moment.</p>"},{"location":"grest-meets/#supported-networks","title":"Supported Networks","text":"<p>Just for the record here, a 16GB (or even 8GB) instance is enough to support both testnet and guild networks.</p>"},{"location":"grest-meets/#db-sync-versioning","title":"db-sync versioning","text":"<p>We agreed to use the <code>release/10.1.x</code> branch which is not yet released but built to include Alonzo migrations to avoid rework later. This version does require Alonzo config and hash to be in the node's <code>config.json</code>. This has to be done manually and the files are available here. Once fully released, all members should rebuild the released version to ensure each instance is running the same code.</p>"},{"location":"grest-meets/#dns-naming","title":"DNS naming","text":"<p>For the DNS setup ticket, we started to think about the instance names for the 2 DNS instances (orange in the graph). Submissions for names will be made in the Telegram group, and will probably make a poll once we have the entries finalised.</p>"},{"location":"grest-meets/#monitoring-system","title":"Monitoring System","text":"<p>Priyank started setting up the monitoring on his instance which can then easily be switched to a separate monitoring instance. We agreed to use Prometheus / Grafana combo for data source / visualisation. We'll probably need to create some custom archiving of data to keep it long term as Prometheus stores only the last 30 days of data.</p>"},{"location":"grest-meets/#next-meeting","title":"Next meeting","text":"<p>We would like to make Friday @ 07:00 UTC the standard time and keep meetings at weekly frequency. A poll will still be created for next weeks, but if there are no objections / requests for switching the time around (which we have not had so far) we can go ahead with the making Friday the standard with polls no longer required and only reminders / Google invites sent every week.</p>"},{"location":"grest-meets/#deployment-scripts_1","title":"Deployment scripts","text":"<p>During the last week, work has been done on deployment scripts for all services (db-sync, gRest and haproxy) -&gt; this is now in testing with updated instructions on trello. Everybody can put their name down on the ticket to signify when the setup is complete and note down any comments for bugs/improvements. This is the main priority at the moment as it would allow us to start transferring our setups to mainnet.</p>"},{"location":"grest-meets/#switch-to-mainnet","title":"Switch to Mainnet","text":"<p>Following on from that, we created a ticket for starting to set up mainnet instances -&gt; we can use 32GB RAM to start and increase later. While making sure everything works against the guild network is priority, people are free to start on this as well as we anticipate we are almost ready for the switch.</p>"},{"location":"grest-meets/#supported-networks_1","title":"Supported Networks","text":"<p>This brings me to another discussion point which is on which networks are to be supported. After some discussion, it was agreed to keep beefy servers for mainnet, and have small independent instances for testnet maintained by those interested, while guild instance is pretty lightweight and useful to keep.</p>"},{"location":"grest-meets/#monitoring-system_1","title":"Monitoring System","text":"<p>The ticket for creating a centralised monitoring system was discussed and updated. I would say it would be good to have at least a basic version of the system in place around the time we switch to mainnet. The system could eventually serve for:  - analysis of instance  - performances and subsequent tuning  - endpoints usage  - anticipation of system requirements increases  - etc.</p> <p>I would say that this should be an important topic of the next meeting to come up with an approach on how we will structure this system so that we can start building it in time for mainnet switch.</p>"},{"location":"grest-meets/#handling-ssl","title":"Handling SSL","text":"<p>Enabling SSL was agreed to not be required by each instance, but is optional and documentation should be created for how to automate the process of renewing SSL certificates for those wishing to add it to their instance. The end user facing endpoints \"Instance Checker\" will of course be SSL-enabled.</p>"},{"location":"grest-meets/#next-meeting_1","title":"Next meeting","text":"<p>We somewhat agreed to another meeting next week again at the same time, but some participants aren't 100% for availability. Friday at 07:00 UTC might be a good standard time we hold on to, but I will make a poll like last time so that we can get more info before confirming the meeting.</p>"},{"location":"grest-meets/#meeting-structure","title":"Meeting Structure","text":"<p>As this was the first meeting, at the start we discussed about the meeting structure. In general, we agreed to something like listed below, but this can definitely change in the future:</p> <p>1) 2-liner (60s) round the table stand-ups by everyone to sync up on what they were doing / are planning to do / mention struggles etc. This itself often sparks discussions. 2) going through the Trello board tasks with the intention of discussing and possbily assigning them to individuals / smaller groups (maybe 1-2-3 people choose to work together on a single task)</p>"},{"location":"grest-meets/#stand-ups","title":"Stand-ups","text":"<p>We then proceeded to give a status of where we are individually in terms of what's been done, a summary below:</p> <ul> <li>Homer, Ola, Markus, Priyank and Damjan have all set up their dbsync + gRest endpoints against guild network and added to topology.</li> <li>Ola laid down the groundwork for CNTools to integrate with API endpoints created so far.</li> <li>Markus has created the systemd scripts and will add them soon to repo</li> <li>Damjan is tracking live stake query that includes payment + stake address, but is awaiting fix on dbsync for pool refund (contextual reserves -&gt; account) , also need to validate reserve -&gt; MIR certs</li> <li>Priyank created initial haproxy settings for polls done, need to complete agent based on design finalisation</li> </ul>"},{"location":"grest-meets/#main-discussion-points","title":"Main discussion points","text":"<ol> <li>Directory structure on the repo -&gt; General agreement is to have anything related to db-sync/postgREST separated from the current cnode-helper-scripts directory. We can finalise the end locations of files a bit later, for now intent should be to simply add them all to /files/dbsync folder. <code>prereqs.sh</code> addendum can be done once artifacts are finalised (added a Trello ticket for tracking).</li> <li>DNS/haproxy configurations:   We have two options:   a. controlled approach for endpoints - wherein there is a layer of haproxy that will load balance and ensure tip being in sync for individual providers (individuals can provide haproxy OR gRest instances).   b. completely decentralised - each client to maintain haproxy endpoint, and fails over to other node if its not up to recent tip.   I think that in general, it was agreed to use a hybrid approach. Details are captured in diagram here. DNS endpoint can be reserved post initial testing of haproxy-agent against mainnet nodes.</li> <li>Internal monitoring system   This would be important and useful and has not been mentioned before this meeting (as far as I know). Basically, a system for monitoring all of our instances together and also handling alerts. Not only for ensuring good quality of service, but also for logging and inspection of short- and long-term trends to better understand what's happening. A ticket is added to trello board</li> </ol>"},{"location":"grest-meets/#next-meeting_2","title":"Next meeting","text":"<p>All in all, I think we saw that there is need for these meetings as there are a lot of things to discuss and new ideas come up (like the monitoring system). We went for over an hour (~1h15min) and still didn't have enough time to go through the board, we basically only touched the DNS/haproxy part of the board. This tells me that we are in a stage where more frequent meetings are required, weekly instead of biweekly, as we are in the initial stage and it's important to build things right from the start rather than having to refactor later on. With that, the participants in general agreed to another meeting next week, but this will be confirmed in the TG chat and the times can be discussed then.</p>"},{"location":"sidebar/","title":"Tree","text":"<ul> <li>Home: index.md</li> <li>Basics: basics.md</li> <li>Build and Run:<ul> <li>Overview: build.md</li> <li>Node &amp; CLI: Build/node-cli.md</li> <li>DBSync: Build/dbsync.md</li> <li>Koios gRest: Build/grest.md</li> <li>Offchain Metadata Tools: Build/offchain-metadata-tools.md</li> </ul> </li> <li>Scripts:<ul> <li>Common env: Scripts/env.md</li> <li>BlockPerf: Scripts/blockperf.md</li> <li>Koios SPO Tools:</li> <li>CNTools:<ul> <li>Overview: Scripts/cntools.md</li> <li>Common Tasks: Scripts/cntools-common.md</li> <li>Changelog: Scripts/cntools-changelog.md</li> <li>Log Monitor: Scripts/logmonitor.md</li> </ul> </li> <li>gLiveView: Scripts/gliveview.md</li> <li>Topology Updater: Scripts/topologyupdater.md</li> <li>CNCLI: Scripts/cncli.md</li> <li>Mithril:</li> <li>Overview: Mithril/mithril-overview.md</li> <li>Client: Scripts/mithril-client.md</li> <li>Signer: Scripts/mithril-signer.md</li> <li>Relay: Scripts/mithril-relay.md</li> </ul> </li> <li>Docker:<ul> <li>Overview: docker/docker.md</li> <li>Build: docker/build.md</li> <li>Run: docker/run.md</li> <li>Tips: docker/tips.md</li> </ul> </li> <li>Appendix:<ul> <li>Sample Postgres Setup: Appendix/postgres.md</li> <li>GRest meet summaries: grest-meets.md</li> </ul> </li> <li>Contributors: contributors.md</li> </ul>"},{"location":"upgrade/","title":"Upgrade","text":""},{"location":"upgrade/#steps-for-upgrading","title":"Steps for Upgrading","text":"<p>Warning</p> <p>Make sure you go through upgrade steps for your setup in a non-mainnet environment first!!</p> <ul> <li>Download the latest <code>guild-deploy.sh</code> (always double check syntax of the script with <code>guild-deploy.sh -h</code>). The scripts modified with user content (<code>env</code>, <code>gLiveView.sh</code>, <code>topologyUpdater.sh</code>, <code>cnode.sh</code>, etc) will be backed up before overwriting. The backed up files will be in the same folder as the original files, and will be named as <code>${filename}_bkp&lt;timestamp&gt;</code>. More static files (genesis files or some of the scripts themselves) will not be backed up, as they're not expected to be modified.</li> </ul> <p>Remember</p> <p>You are expected to provide appropriate environment-specific parameters (eg: custom top level folder [-p], alternate name for top level folder [-t], network flag [-n], etc) to the examples that pertain to your use case</p> <ul> <li> <p>Depending on node release, you may be able to simply perform an update-in-place of scripts and node, or for some cases, you may need to overwrite configs as well. Some Examples below:</p> <ul> <li>A typical run of the guild-deploy script that will perform update in place of current scripts on mainnet and update binaries (alongwith re-compiling libsodium dependencies). In this scenario no config files or user variables in scripts are overwritten. This is typically relevant on minor patch releases of node upgrade:</li> </ul> <pre><code>mkdir \"$HOME/tmp\";cd \"$HOME/tmp\"\ncurl -sfS -o guild-deploy.sh https://raw.githubusercontent.com/cardano-community/guild-operators/master/scripts/cnode-helper-scripts/guild-deploy.sh &amp;&amp; chmod 700 guild-deploy.sh\n./guild-deploy.sh -s dl -b master -n mainnet -t cnode -p /opt/cardano\n</code></pre> <ul> <li>Another scenario would be when you're required to overwrite configs (eg: node-8.1.2 to node-9.1.0 introduced change in genesis/config/topology file formats). In this case, you'd want to overwrite your config files as well. You should follow changelog in node release notes to verify if you'd need to overwrite configs. Note that every time you do this, you may need to re-add your customisations - if any - to the relevant config files (typically - almost always, you'd have to update the topology.json when overwriting configs). There are backups created of original file in <code>\"${CNODE_HOME}\"/files</code> folder if you'd like to compare/reuse previous version.</li> </ul> <pre><code>mkdir \"$HOME/tmp\";cd \"$HOME/tmp\"\ncurl -sfS -o guild-deploy.sh https://raw.githubusercontent.com/cardano-community/guild-operators/master/scripts/cnode-helper-scripts/guild-deploy.sh &amp;&amp; chmod 700 guild-deploy.sh\n./guild-deploy.sh -s dlf -b master -n mainnet -t cnode -p /opt/cardano\n</code></pre> </li> </ul> <p>Beware</p> <p>When upgrading node, depending on node versions (especially for major release) - you'd likely have to wait for node to revalidate/replay ledger. This can take a few hours. Please always plan ahead, do it first on a relay to ensure you've got \"${CNODE_HOME}/db\" folder ready to copy over (while source and target node have been shutdown) - prior to starting on upgrade on new machine. If mithril for target node version is ready, you can also use mithril-client to download snapshot instead of replaying, which may save you some time</p> <ul> <li>Once guild-deploy script has been run, source your bashrc file again (or restart session), and ensure <code>\"${HOME}\"/.local/bin</code> is part of your \\(PATH environment variable. If your shell does not auto-run bashrc, you may want to set it a call to \"\\)\"/.bashrc in your <code>.profile</code></li> </ul> <pre><code>source \"${HOME}\"/.bashrc\necho \"${PATH}\"\n</code></pre>"},{"location":"upgrade/#troubleshooting","title":"Troubleshooting","text":"<ul> <li> <p>We've found users often confuse between \\(PATH variable resolution between multiple shell sessions, systemd, etc. While if you only used this guide, the binaries should be in \"\\)/.local/bin\", you may have manually downloaded to another location before. To avoid this, you can edit the following files and uncomment and set the following variables to the appropriate paths as per your deployment (eg: <code>CCLI=\"${HOME}\"/.local/bin/cardano-cli</code> if following above):</p> <ul> <li>env : CCLI, CNCLI, CNODEBIN</li> <li>[If applicable] dbsync.sh: DBSYNCBIN</li> <li>[If applicable] submitapi.sh: SUBMITAPIBIN</li> <li>[If applicable] ogmios.sh: OGMIOSBIN</li> </ul> </li> <li> <p>The above should take care of tools and services. However, you might still have duplicate binaries in your $PATH (previous artifacts, re-build using old scripts, etc) - it is best that you remove any old binary files from alternate folders. You can do so by executing the below:</p> </li> </ul> <pre><code>whereis bech32 cardano-address cardano-cli cardano-db-sync cardano-hw-cli cardano-node cardano-submit-api cncli ogmios\n</code></pre> <p>For some cases - you might have no values (eg: you may not use <code>cardano-db-sync</code>, <code>cncli</code>, <code>ogmios</code> and/or <code>cardano-hw-cli</code>. You need not take any actions for the binaries you do not use.</p> <ul> <li>If you are having trouble connecting to node post upgrade from gLiveView, typically the first issue you'd want to eliminate is  whether you missed that node might need ledger replay/revalidation (which could take hours as indicated earlier on the page). You can check the node status via <code>sudo systemctl status cnode</code>. If the node shows as up, you can monitor logs via <code>tail -100f \"${CNODE_HOME}/logs/node.json</code>. If you're unable to start node using systemd itself, you can check the systemd output via <code>sudo journalctl -xeu cnode.service</code> and scroll back until you see a startup attempt with reason (typically this could be a couple of pages back). If nothing obvious (eg: showing files used as <code>/files/config.json</code> instead of <code>$CNODE_HOME/files/config.json</code> )comes up from there, you'd want to view node logs to see if there is something recorded in <code>node.json</code> instead.</li> </ul>"},{"location":"upgrade/#unintended","title":"Unintended update-in-place","text":"<p>Let's say you accidentally did an update of the files that you didnt intend to and want to revert all your scripts to previous state. While you have the backups of the scripts created while doing in-place update, you can always make use of branch flag that most scripts that perform update-in-place provide (eg: for gLiveView.sh that you want to restore to 8.1.2 state), this would be <code>-b node-8.1.2</code>. The available tags that can be used can be visited here</p>"},{"location":"upgrade/#support","title":"Support/Improvements","text":"<p>Hope the guide above helps you with the migration, but again - we could've missed some edge cases. If so, please report via chat in Koios Discussions channel or open an issue on github. Please DO NOT make edits to the script content based on forum/alternate guide/channels, while done with best intentions - there have been solutions put online that modify files unnecessarily instead of correcting configs and disabling updates, such actions will only cause trouble for future updates.</p>"},{"location":"Appendix/RecoverByronWallet/","title":"Unofficial Instructions for recovering your Byron Era funds on the new Incentivized Shelley Testnet","text":""},{"location":"Appendix/RecoverByronWallet/#1-grab-and-install-haskell","title":"1.  Grab and install Haskell","text":"<pre><code>curl -sSL https://get.haskellstack.org/ | sh\n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#2-get-the-wallet","title":"2.  Get the wallet","text":"<p>note:  you must build from source as of today as there are changes that just got into master you need <pre><code>git clone https://github.com/cardano-foundation/cardano-wallet.git\n</code></pre></p>"},{"location":"Appendix/RecoverByronWallet/#3-go-into-the-wallet-directory","title":"3.  Go into the wallet directory","text":"<pre><code>cd cardano-wallet\n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#4-build-the-wallet","title":"4. Build the wallet","text":"<p><pre><code>stack build --test --no-run-tests\n</code></pre> If it fails there are a few reasons we have found: - The cardano build instructions reference a few things that may be missing.  Check those. - or maybe one of these would help:</p>"},{"location":"Appendix/RecoverByronWallet/#libssl","title":"Libssl:","text":"<pre><code>sudo apt install libssl-dev\n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#sqlite","title":"Sqlite :","text":"<pre><code>sudo apt-get install sqlite3 libsqlite3-dev \n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#gmp","title":"gmp:","text":"<pre><code>sudo apt-get install libgmp3-dev \n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#systemd-dev","title":"systemd dev:","text":"<pre><code>sudo apt install libsystemd-dev\n</code></pre> <p>get coffee...  It takes awhile</p>"},{"location":"Appendix/RecoverByronWallet/#5-when-its-done-install-executables-to-your-path","title":"5.  When its done, install executables to your path","text":"<pre><code>stack install\n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#6-test-to-make-sure-cardano-wallet-jormungandr-works-fine","title":"6.  Test to make sure cardano-wallet-jormungandr works fine.","text":"<p>Generate your new mnemonics you will need below.  Note that this generates 15 words as opposed to your byron era mnemnomics which were only 12 words.  </p> <pre><code>cardano-wallet-jormungandr mnemonic generate\n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#7-launch-the-wallet-as-a-service","title":"7.  Launch the wallet as a service.","text":"<p>you can either open another terminal window or use screen or something.  anyway, wherever you run this next command you won't be able to use anymore for a terminal until you stop the wallet </p> <p>change --node-port 3001 to wherever you have your jormungandr rest interface running.  for me it was 5001..  so</p> <p>change --port 3002 to wherever you want to access the wallet interface at.  If you have other things running avoid those ports.  for most, 3002 should be free</p> <p>just to future proof these instructions.  genesis should be whatever genesis you are on.</p> <pre><code>cardano-wallet-jormungandr serve --node-port 3001 --port 3002 --genesis-block-hash e03547a7effaf05021b40dd762d5c4cf944b991144f1ad507ef792ae54603197\n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#8-restore-your-byron-wallet","title":"8.  Restore your byron wallet:","text":"<p>---&gt;in another window</p> <p>replace foo, foo, foo with all your mnemnomics from the byron wallet you are restoring</p> <p>Also, if you put your wallet on a different port than 3002, fix that too</p> <p><pre><code>curl -X POST -H \"Content-Type: application/json\" -d '{ \"name\": \"legacy_wallet\", \"mnemonic_sentence\": [\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\"], \"passphrase\": \"areallylongpassword\"}' http://localhost:3002/v2/byron-wallets\n</code></pre> Thats going to spit out some information about a wallet it creates, you should see the value of your wallet - hopefully its not zero.  And you need the wallet ID for the next step</p>"},{"location":"Appendix/RecoverByronWallet/#9-create-your-shelley-wallet","title":"9.  Create your shelley wallet:","text":"<p>Remember all those mnemnomics you made above.. put them here instead of all the foo's.</p> <p><pre><code>curl -X POST -H \"Content-Type: application/json\" -d '{ \"name\": \"pool_wallet\", \"mnemonic_sentence\": [\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\"], \"passphrase\": \"areallylongpasswordagain\"}' http://localhost:3002/v2/wallets\n</code></pre> Important thing to get is the wallet id from this command</p>"},{"location":"Appendix/RecoverByronWallet/#10-migrate-your-funds","title":"10.  Migrate your funds","text":"<p>Now you are ready to migrate your wallet.  replace the <code>&lt;old wallet id&gt;</code> and <code>&lt;new wallet id&gt;</code> with the values you got above</p> <pre><code>curl -X POST -H \"Content-Type: application/json\" -d '{\"passphrase\": \"areallylongpassword\"}' http://localhost:3002/v2/byron-wallets/&lt;old wallet id&gt;/migrations/&lt;new wallet id&gt;\n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#11-congratulations-your-funds-are-now-in-your-new-wallet","title":"11.  Congratulations.  your funds are now in your new wallet.","text":"<p>From here we recommend you send them to a new address entirely owned and created by jcli or whatever method you have been using for the testnet process.</p> <p>This technically may not be required.  But a lot of us did it and we know it works for setting up pools and stuff.</p> <p>send a small amount first just to make sure you are in control of the transaction and don't send your funds to la la land.</p> <p>If you want to send to another address use the command below, but replace the address that you want to send it to, the amount, and your <code>&lt;new wallet id&gt;</code> <pre><code>curl -X POST -H \"Content-Type: application/json\" -d '{\"payments\": [ { \"address\": \"&lt;address to send to&gt;\"\", \"amount\": { \"quantity\": 83333330000000, \"unit\": \"lovelace\" } } ], \"passphrase\": \"areallylongpasswordagain\"}' http://localhost:3002/v2/wallets/&lt;new wallet id&gt;/transactions\n</code></pre></p>"},{"location":"Appendix/monitoring/","title":"Monitoring","text":"<p>Ensure the Pre-Requisites are in place before you proceed.</p> <p>This is an easy-to-use script to automate setting up of monitoring tools. Tasks automates the following tasks: - Installs Prometheus, Node Exporter and Grafana Servers for your respective Linux architecture. - Configure Prometheus to connect to cardano node and node exporter jobs. - Provisions the installed prometheus server to be automatically available as data source in Grafana. - Provisions two of the common grafana dashboards used to monitor <code>cardano-node</code> by SkyLight and IOHK to be readily consumed from Grafana. - Deploy <code>prometheus</code>,<code>node_exporter</code> and <code>grafana-server</code> as systemd service on Linux. - Start and enable those services.</p> <p>Note that securing prometheus/grafana servers via TLS encryption and other security best practices are out of scope for this document, and its mainly aimed to help you get started with monitoring without much fuss.</p> <p>!&gt; Ensure that you've opened the firewall port for grafana server (default used in this script is 5000)</p>"},{"location":"Appendix/monitoring/#download-setup_monsh","title":"Download setup_mon.sh","text":"<p>If you have run <code>guild-deploy.sh</code>, you can skip this step. To download monitoring script, you can execute the commands below: <pre><code>cd $CNODE_HOME/scripts\nwget https://raw.githubusercontent.com/cardano-community/guild-operators/master/scripts/cnode-helper-scripts/setup_mon.sh\nchmod 750 setup_mon.sh\n</code></pre></p>"},{"location":"Appendix/monitoring/#customise-any-environment-variables","title":"Customise any Environment Variables","text":"<p>The default selection may not always be usable for everyone. You can customise further environment variable settings by opening in editor (eg: <code>vi setup_mon.sh</code> ), and updating variables below to your liking:</p> <pre><code>#!/usr/bin/env bash\n# shellcheck disable=SC2209,SC2164\n\n######################################################################\n#### Environment Variables\n######################################################################\nCNODE_IP=127.0.0.1\nCNODE_PORT=12798\nGRAFANA_HOST=0.0.0.0\nGRAFANA_PORT=5000\nPROJ_PATH=/opt/cardano/monitoring\nPROM_HOST=127.0.0.1\nPROM_PORT=9090\nNEXP_PORT=$(( PROM_PORT + 1 ))\n````\n\n#### Set up Monitoring\n\nExecute setup_mon.sh with full path to destination folder you want to setup monitoring in. If you're following guild folder structure, you do not need to specify `-d`. Read the usage comments below before you run the actual script.\n\nNote that to deploy services as systemd, the script expect sudo access is available to the user running the script.\n\n``` bash\ncd $CNODE_HOME/scripts\n# To check Usage parameters:\n# ./setup_mon.sh -h\n#Usage: setup_mon.sh [-d directory] [-h hostname] [-p port]\n#Setup monitoring using Prometheus and Grafana for Cardano Node\n#-d directory      Directory where you'd like to deploy the packages for prometheus , node exporter and grafana\n#-i IP/hostname    IPv4 address or a FQDN/DNS name where your cardano-node (relay) is running (check for hasPrometheus in config.json; eg: 127.0.0.1 if same machine as cardano-node)\n#-p port           Port at which your cardano-node is exporting stats (check for hasPrometheus in config.json; eg: 12798)\n./setup_mon.sh\n# \n# Downloading prometheus v2.18.1...\n# Downloading grafana v7.0.0...\n# Downloading exporter v0.18.1...\n# Downloading grafana dashboard(s)...\n#   - SKYLight Monitoring Dashboard\n#   - IOHK Monitoring Dashboard\n# \n# NOTE: Could not create directory as rdlrt, attempting sudo ..\n# NOTE: No worries, sudo worked !! Moving on ..\n# Configuring components\n# Registering Prometheus as datasource in Grafana..\n# Creating service files as root..\n# \n# =====================================================\n# Installation is completed\n# =====================================================\n# \n# - Prometheus (default): http://127.0.0.1:9090/metrics\n#     Node metrics:       http://127.0.0.1:12798\n#     Node exp metrics:   http://127.0.0.1:9091\n# - Grafana (default):    http://0.0.0.0:5000\n# \n# \n# You need to do the following to configure grafana:\n# 0. The services should already be started, verify if you can login to grafana, and prometheus. If using 127.0.0.1 as IP, you can check via curl\n# 1. Login to grafana as admin/admin (http://0.0.0.0:5000)\n# 2. Add \"prometheus\" (all lowercase) datasource (http://127.0.0.1:9090)\n# 3. Create a new dashboard by importing dashboards (left plus sign).\n#   - Sometimes, the individual panel's \"prometheus\" datasource needs to be refreshed.\n# \n# Enjoy...\n# \n# Cleaning up...\n</code></pre>"},{"location":"Appendix/monitoring/#view-dashboards","title":"View Dashboards","text":"<p>You should now be able to Login to grafana dashboard, using the public IP of your server, at port 5000. The initial credentials to login would be admin/admin, and you will be asked to update your password upon first login. Once logged on, you should be able to go to <code>Manage &gt; Dashboards</code> and select the dashboard you'd like to view. Note that if you've just started the server, you might see graphs as empty, as initial interval for dashboards is 12 hours. You can change it to 5 minutes by looking at top right section of the page.</p> <p>Thanks to Pal Dorogi for the original setup instructions used for modifying.</p>"},{"location":"Appendix/postgres/","title":"Sample Postgres Setup","text":"<p>These deployment instructions used for reference while building cardano-db-sync tool, with the scope being ease of set up, and some tuning baselines for those who are new to Postgres DB. It is recommended to customise these as per your needs for Production builds.</p> <p>Important</p> <p>You'd find it pretty useful to set up ZFS on your system prior to setting up Postgres, to help with your IOPs throughput requirements. You can find sample install instructions here. You can set up your entire root mount to be on ZFS, or you can opt to mount a file as ZFS on \"${CNODE_HOME}\"</p>"},{"location":"Appendix/postgres/#install-postgresql-server","title":"Install PostgreSQL Server","text":"<p>Execute commands below to set up Postgres Server</p> <pre><code># Determine OS platform\nOS_ID=$( (grep -i ^ID_LIKE= /etc/os-release || grep -i ^ID= /etc/os-release) | cut -d= -f 2)\nDISTRO=$(grep -i ^NAME= /etc/os-release | cut -d= -f 2)\n\nif [ -z \"${OS_ID##*debian*}\" ]; then\n#Debian/Ubuntu\nwget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -\n  RELEASE=$(lsb_release -cs)\necho \"deb [arch=amd64] http://apt.postgresql.org/pub/repos/apt/ ${RELEASE}\"-pgdg main | sudo tee  /etc/apt/sources.list.d/pgdg.list\n  sudo apt-get update\n  sudo apt-get -y install postgresql-16 postgresql-server-dev-16 postgresql-contrib libghc-hdbc-postgresql-dev\n  sudo systemctl restart postgresql\n  sudo systemctl enable postgresql\nelse\necho \"We have no automated procedures for this ${DISTRO} system\"\nfi\n</code></pre>"},{"location":"Appendix/postgres/#create-user-in-postgres","title":"Create User in Postgres","text":"<p>Login to Postgres instance as superuser:</p> <pre><code>echo $(whoami)\n# &lt;user&gt;\nsudo su postgres\npsql\n</code></pre> <p>Note the  returned as the output of <code>echo $(whoami)</code> command. Replace all instance of  in the documentation below. Execute the below in psql prompt. Replace  and PasswordYouWant with your OS user (output of <code>echo $(whoami)</code> command executed above) and a password you'd like to authenticate to Postgres with: <p><pre><code>CREATE ROLE &lt;user&gt; SUPERUSER LOGIN;\nALTER USER &lt;user&gt; PASSWORD 'PasswordYouWant';\n\\q\n</code></pre> Type <code>exit</code> at shell to return to your user from postgres</p>"},{"location":"Appendix/postgres/#verify-login-to-postgres-instance","title":"Verify Login to postgres instance","text":"<pre><code>export PGPASSFILE=$CNODE_HOME/priv/.pgpass\necho \"/var/run/postgresql:5432:cexplorer:*:*\" &gt; $PGPASSFILE\nchmod 0600 $PGPASSFILE\npsql postgres\n# psql (15.0)\n# Type \"help\" for help.\n# \n# postgres=#\n</code></pre>"},{"location":"Appendix/postgres/#tuning-your-instance","title":"Tuning your instance","text":"<p>Before you start populating your DB instance using dbsync data, now might be a good time to put some thought on to baseline configuration of your postgres instance by editing <code>/etc/postgresql/15/main/postgresql.conf</code>. Typically, you might find a lot of common standard practices parameters available in tuning guides. For our consideration, it would be nice to start with some baselines - for which we will use inputs from example here, which would need to be customised further to your environment and resources.</p> <p>In a typical Koios [gRest] setup, we use below for minimum viable specs (i.e. 64GB RAM, &gt; 8 CPUs, &gt;16K IOPs for <code>ioping -q -S512M -L -c 10 -s8k .</code> output when postgres data directory is on ZFS configured with max arc of 4GB), we find the below configuration to be the best common setup:</p> Parameter Value Comment data_directory '/opt/cardano/cnode/guild-db/pgdb/15' Move postgres data directory to ZFS mount at /opt/cardano/cnode, ensure it's writable by postgres user effective_cache_size 8GB Be conservative as Node and DBSync by themselves will need ~32-40GB of RAM if ledger-state is enabled effective_io_concurrency 4 Can go higher if you have substantially higher IOPs/IO throughputs lc_time 'en_US.UTF-8' Just to use standard server-side time formatting between instances, can adapt to your preferences log_timezone 'UTC' For consistency, to avoid timezone confusions maintenance_work_mem 512MB Helps with vacuum/index/foreign key maintainance (with 4 workers, it's set to max 2GB) max_connections 200 Allow maximum of 200 connections, the koios connections are still controlled via postgrest db-pool max_parallel_maintenance_workers 4 Max workers postgres will use for maintainance max_parallel_workers 4 Max workers postgres will use across the system max_parallel_workers_per_gather 2 Parallel threads per query, do not increase to higher values as it will multiply memory usage max_wal_size 4GB Used for WAL automatic checkpoints (disabled later) max_worker_processes 4 Maximum number of background processes system can support min_wal_size 1GB Used for WAL automatic checkpoints (disabled later) random_page_cost 1.1 Use higher value if IOPs has trouble catching up (you can use 4 instead of 1.1) shared_buffers 4GB Conservative limit to allow for node/dbsync/zfs memory usage timezone 'UTC' For consistency, to avoid timezone confusions wal_buffers 16MB WAL consumption in shared buffer (disabled later) work_mem 16MB Base memory size before writing to temporary disk files <p>In addition to above, due to the nature of usage by dbsync (synching from node and restart traversing back to last saved ledger-state snapshot), we leverage data retention on blockchain - as we're not affected by loss of volatile information upon a restart of instance. Thus, we can relax some of the data retention and protection against corruption related settings, as those are IOPs/CPU Load Average impacts that the instance does not need to spend. We'd recommend setting 3 of those below in your <code>/etc/postgresql/15/main/postgresql.conf</code>:</p> Parameter Value wal_level minimal max_wal_senders 0 synchronous_commit off <p>Once your changes are done, ensure to restart postgres service using <code>sudo systemctl restart postgresql</code>.</p>"},{"location":"Build/dbsync/","title":"DBSync","text":"<p>Important</p> <p>An average pool operator may not require cardano-db-sync at all. Please verify if it is required for your use as mentioned here.  </p> <ul> <li>Ensure the Pre-Requisites are in place before you proceed.</li> <li>The Cardano DB Sync relies on an existing PostgreSQL server. To keep the focus on building dbsync tool, and not how to setup postgres itself, you can refer to Sample Local PostgreSQL Server Deployment instructions for setting up a Postgres instance. Specifically, we expect the <code>PGPASSFILE</code> environment variable is set as per the instructions in the sample guide, for <code>db-sync</code> to be able to connect.</li> <li>One of the biggest obstacles for user experience when running dbsync is ensuring you satisfy EACH of the points mentioned in System Requirements here. Also, note that we do not advise running dbsync on mainnet if your RAM is below 48GB.</li> </ul>"},{"location":"Build/dbsync/#build-instructions","title":"Build Instructions","text":""},{"location":"Build/dbsync/#clone-the-repository","title":"Clone the repository","text":"<p>Execute the below to clone the <code>cardano-db-sync</code> repository to <code>$HOME/git</code> folder on your system:</p> <pre><code>cd ~/git\ngit clone https://github.com/intersectmbo/cardano-db-sync\ncd cardano-db-sync\n</code></pre>"},{"location":"Build/dbsync/#build-cardano-db-sync","title":"Build Cardano DB Sync","text":"<p>You can use the instructions below to build the latest release of <code>cardano-db-sync</code>.</p> <p><pre><code>git fetch --tags --all\ngit pull\n# Include the cardano-crypto-praos and libsodium components for db-sync\n# On CentOS 7 (GCC 4.8.5) we should also do\n# echo -e \"package cryptonite\\n  flags: -use_target_attributes\" &gt;&gt; cabal.project.local\n# Replace tag against checkout if you do not want to build the latest released version\ngit checkout $(curl -sLf https://api.github.com/repos/intersectmbo/cardano-db-sync/releases/latest | jq -r .tag_name)\n# Use `-l` argument if you'd like to use system libsodium instead of IOG fork of libsodium while compiling\n$CNODE_HOME/scripts/cabal-build-all.sh\n</code></pre> The above would copy the <code>cardano-db-sync</code> binary into <code>~/.local/bin</code> folder.</p>"},{"location":"Build/dbsync/#prepare-db-for-sync","title":"Prepare DB for sync","text":"<p>Now that binaries are available, let's create our database (when going through breaking changes, you may need to use <code>--recreatedb</code> instead of <code>--createdb</code> used for the first time. Again, we expect that <code>PGPASSFILE</code> environment variable is already set (refer to the top of this guide for sample instructions):</p> <pre><code>cd ~/git/cardano-db-sync\n# scripts/postgresql-setup.sh --dropdb #if exists already, will fail if it doesnt - thats OK\nscripts/postgresql-setup.sh --createdb\n# Password:\n# Password:\n# All good!\n</code></pre> <p>Verify you can see \"All good!\" as above!</p>"},{"location":"Build/dbsync/#create-symlink-to-schema-folder","title":"Create Symlink to schema folder","text":"<p>DBSync instance requires the schema files from the git repository to be present and available to the dbsync instance. You can either clone the <code>~/git/cardano-db-sync/schema</code> folder OR create a symlink to the folder and make it available to the startup command we will be using. We will use the latter in sample below:</p> <pre><code>ln -s ~/git/cardano-db-sync/schema $CNODE_HOME/guild-db/schema\n</code></pre>"},{"location":"Build/dbsync/#restore-using-snapshot","title":"Restore using Snapshot","text":"<p>If you're running a mainnet/preview/preprod instance of dbsync, you might want to consider use of dbsync snapshots as documented here. The snapshot files as of recent epoch are available via links in release notes.</p> <p>At high-level, this would involve steps as below (read and update paths as per your environment):</p> <pre><code># Replace the actual link below with the latest one from release notes\nwget https://update-cardano-mainnet.iohk.io/cardano-db-sync/13/db-sync-snapshot-schema-13-block-7622755-x86_64.tgz\nrm -rf ${CNODE_HOME}/guild-db/ledger-state ; mkdir -p ${CNODE_HOME}/guild-db/ledger-state\ncd -; cd ~/git/cardano-db-sync\nscripts/postgresql-setup.sh --restore-snapshot /tmp/dbsyncsnap.tgz ${CNODE_HOME}/guild-db/ledger-state\n# The restore may take a while, please be patient and do not interrupt the restore process. Once restore is successful, you may delete the downloaded snapshot as below:\n#   rm -f /tmp/dbsyncsnap.tgz\n</code></pre>"},{"location":"Build/dbsync/#test-running-dbsync-manually-at-terminal","title":"Test running dbsync manually at terminal","text":"<p>In order to verify that you can run dbsync, before making a start - you'd want to ensure that you can run it interactively once. To do so, try the commands below:</p> <pre><code>cd $CNODE_HOME/scripts\nexport PGPASSFILE=$CNODE_HOME/priv/.pgpass\n./dbsync.sh\n</code></pre> <p>You can monitor logs if needed via parallel session using <code>tail -10f $CNODE_HOME/logs/dbsync.json</code>. If there are no error, you would want to press Ctrl-C to stop the dbsync.sh execution and deploy it as a systemd service. To do so, use the commands below (the creation of file is done using <code>sudo</code> permissions, but you can always deploy it manually):</p> <pre><code>cd $CNODE_HOME/scripts\n./dbsync.sh -d\n# Deploying cnode-dbsync.service as systemd service..\n# cnode-dbsync.service deployed successfully!!\n</code></pre> <p>Now to start dbsync instance, you can run <code>sudo systemctl start cnode-dbsync</code></p> <p>Note</p> <p>Note that dbsync while syncs, it might defer creation of indexes/constraints to speed up initial catch up. Once relatively closer to tip, this will initiate creation of indexes - which can take a while in background. Thus, you might notice the query timings right after reaching to tip might not be as good.</p>"},{"location":"Build/dbsync/#update-dbsync","title":"Update DBSync","text":"<p>Updating dbsync can have different tasks depending on the versions involved. We attempt to briefly explain the tasks involved:</p> <ul> <li>Shutdown dbsync (eg: <code>sudo systemctl stop cnode-dbsync</code>)</li> <li>Update binaries (either download pre-compiled binaries via guild-deploy.sh or using build instructions above)</li> <li> <p>Go to your git folder, pull and checkout to latest version as in example below (if you were to switch to <code>13.1.1.3</code>):</p> <pre><code>cd ~/git/cardano-db-sync\ngit pull\ngit checkout 13.1.1.3\n</code></pre> </li> <li> <p>If going through major version update (eg: 13.x.x.x to 14.x.x.x), you might need to rebuild and resync db from scratch, you may still follow the section to restore using snapshot to save some time (as long as you use a compatible snapshot).</p> </li> <li>If the underlying <code>cardano-node</code> version has changed (specifically if it's <code>ledger-state</code> schema is different), you'd also need to clear the ledger-state directory (eg: <code>rm -rf $CNODE_HOME/guild-db/ledger-state</code>)</li> <li>Test that <code>dbsync.sh</code> starts up fine manually as described above. If it does, stop it and go ahead with startup of systemd service (i.e. <code>sudo systemctl start cnode-dbsync</code>)</li> </ul>"},{"location":"Build/dbsync/#validation","title":"Validation","text":"<p>To validate, connect to your <code>postgres</code> instance and execute commands as per below:</p> <pre><code>export PGPASSFILE=$CNODE_HOME/priv/.pgpass\npsql cexplorer\n</code></pre> <p>You should be at the <code>psql</code> prompt, you can check the tables and verify they're populated:</p> <pre><code>\\dt\nselect * from meta;\n</code></pre> <p>A sample output of the above two commands may look like below (the number of tables and names may vary between versions):</p> <pre><code>cexplorer=# \\dt\nList of relations\n Schema |           Name            | Type  | Owner\n--------+---------------------------+-------+-------\n public | ada_pots                  | table | centos\n public | admin_user                | table | centos\n public | block                     | table | centos\n public | delegation                | table | centos\n public | delisted_pool             | table | centos\n public | epoch                     | table | centos\n public | epoch_param               | table | centos\n public | epoch_stake               | table | centos\n public | ma_tx_mint                | table | centos\n public | ma_tx_out                 | table | centos\n public | meta                      | table | centos\n public | orphaned_reward           | table | centos\n public | param_proposal            | table | centos\n public | pool_hash                 | table | centos\n public | pool_meta_data            | table | centos\n public | pool_metadata             | table | centos\n public | pool_metadata_fetch_error | table | centos\n public | pool_metadata_ref         | table | centos\n public | pool_owner                | table | centos\n public | pool_relay                | table | centos\n public | pool_retire               | table | centos\n public | pool_update               | table | centos\n public | pot_transfer              | table | centos\n public | reserve                   | table | centos\n public | reserved_ticker           | table | centos\n public | reward                    | table | centos\n public | schema_version            | table | centos\n public | slot_leader               | table | centos\n public | stake_address             | table | centos\n public | stake_deregistration      | table | centos\n public | stake_registration        | table | centos\n public | treasury                  | table | centos\n public | tx                        | table | centos\n public | tx_in                     | table | centos\n public | tx_metadata               | table | centos\n public | tx_out                    | table | centos\n public | withdrawal                | table | centos\n(37 rows)\n\n\n\nselect * from meta;\n id |     start_time      | network_name\n----+---------------------+--------------\n  1 | 2017-09-23 21:44:51 | mainnet\n(1 row)\n</code></pre>"},{"location":"Build/grest-changelog/","title":"Koios gRest Changelog","text":""},{"location":"Build/grest-changelog/#112-for-all-networks","title":"[1.1.2] - For all networks.","text":"<p>This release is minor bugfix for data consistency changes behind the scenes. It has no impact to any of the API endpoints.</p>"},{"location":"Build/grest-changelog/#chores","title":"Chores:","text":"<ul> <li>Performance optimisation for <code>pool_info</code> : Add a pool_delegators_list RPC that brings across only minimal information required by pool_info, thereby improving it's performance #281</li> <li>Fix <code>pool_history</code> stats for latest epochs by restricting cache to current - 3 epoch while calculating the subsequent information using live status from the database #282</li> <li>Fix rewards/treasury/reserves calculation in <code>account_info</code> , <code>account_info_cached</code> and <code>stake_distribution_cache</code>.</li> </ul>"},{"location":"Build/grest-changelog/#111-for-all-networks","title":"[1.1.1] - For all networks.","text":"<p>This release primarily focuses on backend performance fixes and work with dbsync 13.2.0.2 - while also, we have started preparing compatibility with upcoming koios lite release, to make it a seamless swap for specific endpoints without any impact to consumers. There are no breaking (impact to existing columns or inputs) changes with this release, but we have retired 2 deprecated endpoints that were almost unused on mainnet. Due to the amount of backend changes in queries, there is a chance that we may have missed some data accuracy checks, and - hence - would like to test in non-mainnet networks first before marking final release. Accordingly, any testing/reports of data inconsistency would be welcome.</p>"},{"location":"Build/grest-changelog/#new-endpoints-added","title":"New endpoints added:","text":"<ul> <li><code>/asset_policy_mints</code> - List of mint/burn count for all assets minted under a policy #269</li> <li><code>/block_tx_info</code> - Equivalent of tx_info but uses blocks as inputs to fetch tx_info against all tx in the block[s] requested, also contains additional flags to control performance and output #255</li> <li><code>/cli_protocol_params</code> - Return protocl-parameters as returned by <code>cardano-cli</code> from Koios servers #269</li> </ul>"},{"location":"Build/grest-changelog/#data-inputoutput-changes","title":"Data Input/Output Changes:","text":"<ul> <li>Output - <code>/reserve_withdrawals</code> , <code>/treasury_withdrawals</code> - Add <code>earned_epoch</code> and <code>spendable_epoch</code> fields #269</li> <li>Output - <code>/block</code> - Add <code>parent_hash</code> field #263</li> <li>Output - <code>/account_list</code> - Add <code>stake_address_hex</code> and <code>script_hash</code> fields #263</li> <li>Output - <code>/asset_list</code> - Add <code>script_hash</code> field #263</li> <li>Output - <code>/asset_summary</code> - Add <code>addresses</code> field #263</li> <li>Output - <code>/asset_addresses</code> , <code>/asset_nft_address</code> and <code>/policy_asset_addresses</code> - Add <code>stake_address</code> field #262</li> <li>Output - Fix <code>/script_utxos</code> as it was incorrectly returning object instead of array for asset_list #269</li> <li>Output - <code>/tx_info</code> - Add <code>plutus_contract</code> -&gt; <code>spends_input</code> to <code>plutus_contracts</code> to point the input transaction being consumed by the script #269</li> </ul>"},{"location":"Build/grest-changelog/#deprecations","title":"Deprecations:","text":"<ul> <li>None</li> </ul>"},{"location":"Build/grest-changelog/#retirements","title":"Retirements:","text":"<ul> <li><code>asset_address_list</code> and <code>asset_policy_info</code> endpoints are now retired, as they were marked as deprecated in Koios 1.0.10 , and we have seen it's usage to be negligible (only a single hit in 48 hours on mainnet while marking this release). #269</li> </ul>"},{"location":"Build/grest-changelog/#chores_1","title":"Chores:","text":"<ul> <li>Retire <code>stake_distribution_new_accounts</code> and <code>stake_snapshot_cache</code> cache, as we directly perform lookup on live tables for newly registered accounts #269</li> <li>Active stake cache no longer reads the logs, but instead relies on newly added <code>epoch_sync_progress table</code> #269</li> <li>Reduce asset_info_cache rollback lookup from 1000 to 250 #269</li> <li>Replace <code>consumed_by_tx_in_id</code> references in SQL by <code>consumed_by_tx_id</code> #269</li> <li><code>pool_history_cache</code> now breaks into populating 500 epochs at a time (on guildnet, this query used to run for hours against ~20K epochs) #269</li> <li>Accomodate splitting of <code>reward</code> table into <code>instant_reward</code> #269</li> <li>Add a check in <code>stake_distribution_cache</code> to ensure that epoch info cache was run for current - 1 epoch. #269</li> <li>Change return type for internal function <code>grest.cip67_strip_label</code> from <code>text</code> to <code>bytea</code> #269</li> <li>Remove any references to <code>tx_in</code> as it is no longer required #269</li> <li>Remove references to <code>pool_offline_data</code> with <code>off_chain_pool_data</code> #269</li> <li>Disable running <code>asset-txo-cache-update</code> as the endpoints leveraging asset-txo will be moved to koios-lite</li> <li>Convert <code>block</code>, <code>account_list</code>, <code>asset_list</code> <code>asset_token_registry</code> from view to function #263</li> <li><code>asset_info_cache</code> - ensure mint tx is only against a positive mint quantity #262</li> <li>Include burnt asset transactions in asset_txs #269</li> <li><code>tx_info</code> - Fix spend_redeemers CTE Join condition #269</li> </ul>"},{"location":"Build/grest-changelog/#110-for-all-networks","title":"[1.1.0] - For all networks.","text":"<p>This will be first major [breaking] release for Koios consumers in a while, and will be rolled out under new base prefix (<code>/api/v1</code>). The major work with this release was to start making use of newer flags in dbsync which help performance of queries under new endpoints. Please ensure to check out the release notes for <code>1.1.0rc</code> below. The list for this section is only a small addendum to <code>1.1.0rc</code>:</p>"},{"location":"Build/grest-changelog/#chores_2","title":"Chores:","text":"<ul> <li>Make use of asset-txo-cache for top assets on mainnet, and use this cache for <code>asset_addresses</code> and <code>policy_asset_addresses</code> #250</li> <li>Add v0 RPC redirectors to keep serve v0 endpoints from v1 #250</li> <li>Convert few simple RPC functions from PLPGSQL to SQL language to help with inline filtering #250</li> <li>Address linting results from SQLFluff #250</li> <li>Move db-scripts from guild-operators repository to koios-artifacts repository #250</li> <li>Drop stale db-scripts/genesis_table.sql file #250</li> <li>Add 3 additional indexes for collateral and reference inputs based on query times #250</li> <li>Add top 3 assets for preview/preprod to asset-txo-cache #250</li> <li>Bump schema version for koios-1.1.0 #250</li> <li>Minor patch for output data type (<code>pool_registrations</code> and <code>pool_retirements</code>) #249</li> </ul>"},{"location":"Build/grest-changelog/#110rc-for-all-networks","title":"[1.1.0rc] - For all networks.","text":"<p>This will be first major [breaking] release for Koios consumers in a while, and will be rolled out under new base prefix (<code>/api/v1</code>). The major work with this release was to start making use of newer flags in dbsync which help performance of queries under new endpoints. Also, you'd see quite a few new endpoint additions below, that'd be helping out with slightly lighter version of queries. To keep migration paths easier, we will ensure both v0 and v1 versions of the release is up for a month post release, before retiring v0.</p>"},{"location":"Build/grest-changelog/#new-endpoints-added_1","title":"New endpoints added:","text":"<ul> <li><code>/pool_registrations</code> - List of all pool registrations initiated in the requested epoch #239</li> <li><code>/pool_retirements</code> - List of all pool retirements initiated in the requested epoch #239</li> <li><code>/treasury_withdrawals</code> - List of withdrawals made from treasury #239</li> <li><code>/reserve_withdrawals</code> - List of withdrawals made from reserves (MIRs) #239</li> <li><code>/account_txs</code> - Transactions associated with a given stake address #239</li> <li><code>/address_utxos</code> - Get UTxO details for requested addresses #239</li> <li><code>/asset_utxos</code> - Get UTxO details for requested assets #239</li> <li><code>/script_utxos</code> - Get UTxO details for requested script hashes #239</li> <li><code>/utxo_info</code> - Details for requested UTxO arrays #239</li> <li><code>/script_info</code> - Information about a given script FROM script hashes #239</li> <li><code>/ogmios/</code> - Expose stateless ogmios endpoints #1690</li> </ul>"},{"location":"Build/grest-changelog/#data-inputoutput-changes_1","title":"Data Input/Output Changes:","text":"<ul> <li>Input - <code>/account_utxos</code> , <code>/credential_utxos</code> - Accept <code>extended</code> as an additional flag - which enables <code>asset_list</code>, <code>reference_script</code> and <code>inline_datum</code> to the output #239</li> <li>Output - <code>/block_txs</code> - Flatten output with transaction details (<code>tx_hash</code>, <code>epoch_no</code>, <code>block_height</code>, <code>block_time</code>) instead of <code>tx_hashes</code> array #239</li> <li>Output - <code>/epoch_params</code> - Update <code>cost_models</code> to JSON (upstream change in node) #239</li> <li>Output - <code>/account_assets</code> , <code>/address_assets</code> - Flatten the output result (instead of <code>asset_list</code> array) making it easier to apply horizontal filtering based on any of the fields</li> <li>Output - Align output fields for <code>/account_utxos</code> , <code>/address_utxos</code>, <code>/asset_utxos</code> , <code>/script_utxos</code> and <code>/utxo_info</code> to return same schema giving complete details about UTxOs involved, with few fields toggled based on <code>extended</code> input flag #239</li> <li>Output - <code>/pool_list</code> - Add various details to the endpoint for each pool (<code>pool_id_hex</code>,<code>active_epoch_no</code>,<code>margin</code>,<code>fixed_cost</code>,<code>pledge</code>,<code>reward_addr</code>,<code>owners</code>,<code>relays</code>,<code>ticker</code>,<code>meta_url</code>,<code>meta_hash</code>,<code>pool_status</code>,<code>retiring_epoch</code>) - this should mean some of the requests to <code>pool_info</code> should no longer be required #239</li> <li>Output - <code>/pool_updates</code> - In v0, <code>pool_updates</code> only provided pool registration updates, while <code>pool_status</code> corresponded to current status of pool. With v1, we will have registration as well as deregistration transactions, and each transaction will have <code>update_type</code> (enum of either <code>registration</code> or <code>deregistration</code>) instead of <code>pool_status</code>. As a side-effect, since a registration transaction only has <code>retiring_epoch</code> as metadata, all the other fields will show up as <code>null</code> for such a transaction #239</li> <li>Output - <code>/pool_metadata</code> , <code>/pool_relays</code> - Add <code>pool_status</code> field to denote whether pool is retired #239</li> <li>Output - <code>/datum_info</code> - Rename <code>hash</code> to <code>datum_hash</code> and add <code>creation_tx_hash</code> #239</li> <li>Output - <code>/native_script_list</code> - Remove <code>script</code> column (as it has pretty large output better queried against <code>script_info</code>), add <code>size</code> and change <code>type</code> to text #239</li> <li>Output - <code>/plutus_script_list</code> - Add <code>type</code> and <code>size</code> to output #239</li> <li>Output - <code>/asset_info</code> - Add <code>cip68_metadata</code> JSONB field #239</li> <li>Output - <code>/pool_history</code> - Add member_rewards #225</li> </ul>"},{"location":"Build/grest-changelog/#deprecations_1","title":"Deprecations:","text":"<ul> <li><code>/tx_utxos</code> - No longer required as replaced by <code>/utxo_info</code> #239</li> </ul>"},{"location":"Build/grest-changelog/#chores_3","title":"Chores:","text":"<ul> <li>Update base version to <code>v1</code> from <code>v0</code> #1690</li> <li>Allow Bearer Authentication against endpoints #239</li> <li>Cron job to apply corrections to epoch info #239</li> <li><code>epoch_info_cache</code> Remove protocol parameters, as they can be queried from live table. Accordingly update dependent queries #239</li> <li>Make use of new <code>consumed_by_tx_in_id</code> column in <code>tx_out</code> from dbsync 13.1.1.3 across endpoints #239</li> <li>Fix <code>_last_active_stake_validated_epoch</code> in active_stake_cache #222</li> <li>Typo for pool_history_cache.sql as well as add a check to ensure epoch_info_cache has run at least once prior to pool_history_cache #223</li> <li>Move control_table entry in cache tables to the end (instead of start) #226</li> <li>Fix Asset Info Cache (include mint/burn tx rather than sum for meta consideration) #226</li> <li>Update SQLs as per SQLFluff linting guidelines #226</li> <li>Fix for tip check in cron jobs #217</li> <li>Update cron jobs to exit if the database has not received block in 5 mins #200</li> <li>Update active stake cache to use control table #196</li> <li>Update Asset Info Cache entry whenever asset registry cache has an update #194</li> <li>Bump up margin for tx rollback lookup for asset_info_cache to 1000 , as 100 is too small a margin for 2-3 blocks , which can contain more than 100 transactions (of which if oldest transaction contains a mint, it will not get into the cache) #177</li> <li>Swap grestrpcs file to list exceptions and treat everything else as RPC #1690</li> <li>Update grest-poll.sh to have stricter spec validation and add health check for asset_registry #1690</li> <li>Update guild-deploy.sh to include latest pre-release for ogmios #1690</li> </ul>"},{"location":"Build/grest-changelog/#1010-for-all-networks","title":"[1.0.10] - For all networks.","text":"<p>The release is effectively same as <code>1.0.10rc</code> except with one minor modification below.</p>"},{"location":"Build/grest-changelog/#chores_4","title":"Chores:","text":"<ul> <li>Replace all RPC references for JSON endpoints with JSONB, this allows filtering child members of array elements using <code>cs.[{\"key\":\"value\"}]</code> in PostgREST #172</li> </ul>"},{"location":"Build/grest-changelog/#1010rc-for-non-mainnet-networks","title":"[1.0.10rc] - For non-mainnet networks","text":"<p>This release primarily focuses on ability to support better DeFi projects alongwith some value addition for existing clients by bringing in 10 new endpoints (paired with 2 deprecations), and few additional optional input parameters , and some additional output columns to existing endpoints. The only breaking change/fix is for output returned for <code>tx_info</code>.</p> <p>Also, dbsync 13.1.x.x has been released and is recommended to be used for this release</p>"},{"location":"Build/grest-changelog/#new-endpoints-added_2","title":"New endpoints added","text":"<ul> <li><code>/asset_addresses</code> -  Equivalent of deprecated <code>/asset_address_list</code> #149</li> <li><code>/asset_nft_address</code> - Returns address where the specified NFT sits on #149</li> <li><code>/account_utxos</code> - Returns brief details on non-empty UTxOs associated with a given stake address #149</li> <li><code>/asset_info_bulk</code> - Bulk version of <code>/asset_info</code> #142</li> <li><code>/asset_token_registry</code> - Returns assets registered via token registry on github #145</li> <li><code>/credential_utxos</code> - Returns UTxOs associated with a payment credential #149</li> <li><code>/param_updates</code> - Returns list of parameter update proposals applied to the network #149</li> <li><code>/policy_asset_addresses</code> - Returns addresses with quantity for each asset on a given policy #149</li> <li><code>/policy_asset_info</code> - Equivalent of deprecated <code>/asset_policy_info</code> but with more details in output #149</li> <li><code>/policy_asset_list</code> - Returns list of asset under the given policy (including supply) #142, #149</li> </ul>"},{"location":"Build/grest-changelog/#data-inputoutput-changes_2","title":"Data Input/Output Changes","text":"<ul> <li>Input - <code>/account_addresses</code> - Add optional <code>_first_only</code> and <code>_empty</code> flags to show only first address with tx or to include empty addresses to output #149</li> <li>Input - <code>/epoch_info</code> - Add optional <code>_include_next_epoch</code> field to show next epoch stats if available (eg: nonce, active stake) #143</li> <li>Output (addition) - <code>/account_assets</code> , <code>/address_assets</code> , <code>/address_info</code>, <code>/tx_info</code>, <code>/tx_utxos</code> - Add <code>decimals</code> to output #142</li> <li>Output (addition) - <code>/policy_asset_info</code> - Add <code>minting_tx_hash</code>, <code>total_supply</code>, <code>mint_cnt</code>, <code>burn_cnt</code> and <code>creation_time</code> fields to the output #149</li> <li>Output (breaking) - <code>/tx_info</code> - Change <code>_invalid_before</code> and <code>_invalid_after</code> to text field #141</li> <li>Output (breaking/removal) - <code>tx_info</code> - Remove the field <code>plutus_contracts</code> &gt; [array] &gt; <code>outputs</code> as there is no logic to connect it to inputs spending #163</li> </ul>"},{"location":"Build/grest-changelog/#deprecations_2","title":"Deprecations:","text":"<ul> <li><code>/asset_address_list</code> - Renamed to <code>asset_addresses</code> keeping naming line with other endpoints (old one still present, but will be deprecated in future release) #149</li> <li><code>/asset_policy_info</code> - Renamed to <code>policy_asset_info</code> keeping naming line with other endpoints (old one still present, but will be deprecated in future release) #149</li> </ul>"},{"location":"Build/grest-changelog/#chores_5","title":"Chores:","text":"<ul> <li><code>/epoch_info</code>, <code>/epoch_params</code> - Restrict output to current epoch #149</li> <li><code>/block_info</code> - Use <code>/previous_id</code> field to show previous/next blocks (previously was using block_id/height) #145</li> <li><code>/asset_info</code>/<code>asset_policy_info</code> - Fix mint tx data to be latest #141</li> <li>Support new guild scripts revamp #1572</li> <li>Add asset token registry check 1606</li> <li>New cache table <code>grest.asset_info_cache</code> to hold mint/burn counts alongwith first/last mint tx/keys #142</li> <li>Bump to Koios 1.0.10rc #149</li> <li>Fix typo in specs for <code>/pool_delegators</code> output column <code>latest_delegation_tx_hash</code> #149</li> <li>Add indexes for ones missing after configuring cardano-db-sync 13.1.0.0 #149</li> <li>Update PostgREST to be run as <code>authenticator</code> user, whose default <code>statement_timeout</code> is set to 65s and update configs accordingly #1606</li> </ul>"},{"location":"Build/grest-changelog/#109-for-all-networks","title":"[1.0.9] - For all networks","text":"<p>This release is effectively same as <code>1.0.9rc</code> below (please check out the notes accordingly), just with minor bug fix on <code>setup-grest.sh</code> itself.</p>"},{"location":"Build/grest-changelog/#109rc-for-non-mainnet-networks","title":"[1.0.9rc] - For non-mainnet networks","text":"<p>This release candidate is non-breaking for existing methods and inputs, but breaking for output objects for endpoints. The aim with release candidate version is to allow folks couple of weeks to test, adapt their libraries before applying to mainnet.</p>"},{"location":"Build/grest-changelog/#new-endpoints-added_3","title":"New endpoints added","text":"<ul> <li><code>datum_info</code> - List of datum information for given datum hashes</li> <li><code>account_info_cached</code> - Same as <code>account_info</code>, but serves cached information instead of live one</li> </ul>"},{"location":"Build/grest-changelog/#data-inputoutput-changes_3","title":"Data Input/Output changes","text":"<ul> <li><code>address_info</code>, <code>address_assets</code>, <code>account_assets</code>, <code>tx_info</code>, <code>asset_list</code> <code>asset_summary</code> to align output <code>asset_list</code> object to return array of <code>policy_id</code>, <code>asset_name</code>, <code>fingerprint</code> (and <code>quantity</code>, <code>minting_txs</code> where applicable) #120</li> <li><code>asset_history</code> - Fix metadata to wrap in array to refer to right object #122</li> <li><code>asset_txs</code> - Add optional boolean parameter <code>_history</code> (default: <code>false</code>) to toggle between querying current UTxO set vs entire history for asset #122</li> <li><code>pool_history</code> - <code>fixed_cost</code>, <code>pool_fees</code>, <code>deleg_rewards</code>, <code>epoch_ros</code> will be returned as 0 when null #122</li> <li><code>tx_info</code> - <code>plutus_contracts-&gt;outputs</code> can be null #122</li> </ul>"},{"location":"Build/grest-changelog/#changes-for-instance-providers","title":"Changes for Instance Providers","text":"<ul> <li>SQL queries have been moved from <code>guild-operators</code> repository to <code>koios-artifacts</code> repository. This is to ensure that the updates made to scripts and other tooling do not have a dependency on Koios query versioning #122</li> <li><code>block_info</code> - Use <code>block_no</code> instead of <code>id</code> to check for previous/next block hash #122</li> <li>Add topology for preprod and preview networks #122</li> </ul>"},{"location":"Build/grest-changelog/#108-for-all-networks","title":"[1.0.8] - For all networks","text":"<p>This release is contains minor bug-fixes that were discovered in koios-1.0.7. No major changes to output for this one.</p>"},{"location":"Build/grest-changelog/#changes-for-api","title":"Changes for API","text":""},{"location":"Build/grest-changelog/#new-endpoints-added_4","title":"New endpoints added","text":"<ul> <li>None</li> </ul>"},{"location":"Build/grest-changelog/#data-inputoutput-changes_4","title":"Data Input/Output changes","text":"<ul> <li><code>tx_info</code> and <code>tx_metadata</code> - Align metadata for JSON output format #1542</li> <li><code>blocks</code> - Query Output aligned to specs (<code>epoch</code> =&gt; <code>epoch_no</code>)</li> <li><code>epoch_block_protocols</code> - [ ** Specs only ** ] Fix Documentation schema , which was accidentally showing wrong output</li> <li><code>pool_delegators_history</code> - List all epochs instead of current, if no <code>_epoch_no</code> is specified #1545</li> </ul>"},{"location":"Build/grest-changelog/#changes-for-instance-providers_1","title":"Changes for Instance Providers","text":"<ul> <li><code>asset_info</code> - Fix metadata aggregaton for minting transactions with multiple metadata keys #1543</li> <li><code>stake_distribution_new_accounts</code> - Leftover reference for <code>account_info</code> which now accepts array, resulted in error to populate stake distribution cache for new accounts #1541</li> <li><code>grest-poll.sh</code> - Remove query view section from polling script, and remove grestrpcs re-creation per hour (it's already updated when <code>setup-grest.sh</code> is run) , in preparation for #1545</li> </ul>"},{"location":"Build/grest-changelog/#107-for-all-networks","title":"[1.0.7] - For all networks","text":"<p>This release continues updates from koios-1.0.6 to further utilise stake-snapshot cache tables which would be useful for SPOs as well as reduce downtime post epoch transition. One largely requested feature to accept bulk inputs for many block/address/account endpoints is now complete. Additionally, koios instance providers are now recommended to use cardano-node 1.35.3 with dbsync 13.0.5.</p>"},{"location":"Build/grest-changelog/#changes-for-api_1","title":"Changes for API","text":""},{"location":"Build/grest-changelog/#new-endpoints-added_5","title":"New endpoints added","text":"<ul> <li><code>pool_delegators_history</code> - Provides historical record for pool's delegators #1486</li> <li><code>pool_stake_snapshot</code> - Provides mark, set and go snapshot values for pool being queried. #1489</li> </ul>"},{"location":"Build/grest-changelog/#data-inputoutput-changes_5","title":"Data Input/Output changes","text":"<ul> <li><code>pool_delegators</code> - No longer accepts <code>_epoch_no</code> as parameter, as it only returns live delegators. Additionally provides <code>latest_delegation_hash</code> in output. #1486</li> <li><code>tx_info</code> - <code>epoch</code> =&gt; <code>epoch_no</code> #1494</li> <li><code>tx_info</code> - Change <code>collateral_outputs</code> (array) to <code>collateral_output</code> (object) as collateral output is only singular in current implementation #1496</li> <li><code>address_info</code> - Add <code>inline_datum</code> and <code>reference_script</code> to output #1500</li> <li><code>pool_info</code> - Add <code>sigma</code> field to output #1511</li> <li><code>pool_updates</code> - Add historical metadata information to output #1503</li> <li>Change block/address/account endpoints to accept bulk input where applicable. This resulted in GET requests changing to POST accepting payload of multiple blocks, addresses or accounts for respective endpoints as input (eg: <code>_stake_address text</code> becomes <code>_stake_addresses text[]</code>). The additional changes in output as below:</li> <li><code>block_txs</code> - Now returns <code>block_hash</code> and array of <code>tx_hashes</code></li> <li><code>address_info</code> - Additional field <code>address</code> returned in output</li> <li><code>address_assets</code> - Now returns <code>address</code> and an array of <code>assets</code> JSON</li> <li><code>account_addresses</code> - Accepts <code>stake_addresses</code> array and outputs <code>stake_address</code> and array of <code>addresses</code></li> <li><code>account_assets</code> - Accepts <code>stake_addresses</code> array and outputs <code>stake_address</code> and array of <code>assets</code> JSON</li> <li><code>account_history</code> - Accepts <code>stake_addresses</code> array alongwith <code>epoch_no</code> integer and outputs <code>stake_address</code> and array of <code>history</code> JSON</li> <li><code>account_info</code> - Accepts <code>stake_addresses</code> array and returns additional field <code>stake_address</code> to output</li> <li><code>account_rewards</code> - Now returns <code>stake_address</code> and an array of <code>rewards</code> JSON</li> <li><code>account_updates</code> - Now returns <code>stake_address</code> and an array of <code>updates</code> JSON</li> <li><code>asset_info</code> - Change <code>minting_tx_metadata</code> from array to object #1533</li> <li><code>account_addresses</code> - Sort results by oldest address first #1538</li> </ul>"},{"location":"Build/grest-changelog/#changes-for-instance-providers_2","title":"Changes for Instance Providers","text":"<ul> <li><code>epoch_info_cache</code> - Only update last_tx_id of previous epoch on epoch transition #1490 and #1502</li> <li><code>epoch_info_cache</code> / <code>stake_snapshot_cache</code> - Store total snapshot stake to epoch stake cache, and active pool stake to stake snapshot cache #1485</li> </ul>"},{"location":"Build/grest-changelog/#106106m-interim-release-for-all-networks-to-upgrade-to-dbsync-v13","title":"[1.0.6/1.0.6m] - Interim release for all networks to upgrade to dbsync v13","text":"<p>The backlog of items not being added to mainnet has been increasing due to delays with Vasil HFC event to Mainnet. As such we had to come up with a split update approach. The mainnet nodes are still not qualified to be Vasil-ready (in our opinion) for 1.35.x , but dbsync 13 can be used against node 1.34.1 fine. In order to cater for this split, we have added an intermediate koios-1.0.6m tag that brings dbsync updates while maintaining node-1.34.1.</p>"},{"location":"Build/grest-changelog/#changes-for-api_2","title":"Changes for API","text":""},{"location":"Build/grest-changelog/#data-output-changes","title":"Data Output Changes","text":"<ul> <li><code>pool_delegators</code> - <code>epoch_no</code> =&gt; <code>active_epoch_no</code> #1454</li> <li><code>asset_history</code> - Add <code>block_time</code> and <code>metadata</code> fields for all previous mint transactions #1468</li> <li><code>asset_info</code> - Retain latest mint transaction instead of first (in line with most CIPs as well as pool metadata - latest valid meta being live) #1468</li> <li>Ensure all output date formats is integer to keep in line with UNIX timestamps - to be revised in future if/when there are milliseconds #1460</li> <li><code>/tip</code> , <code>/blocks</code>, <code>/block_info</code> =&gt; <code>block_time</code></li> <li><code>/genesis</code> =&gt; <code>systemStart</code></li> <li><code>/epoch_info</code> =&gt; <code>start_time</code>, <code>first_block_time</code>, <code>last_block_time</code>, <code>end_time</code></li> <li><code>/tx_info</code> =&gt; <code>tx_timestamp</code></li> <li><code>/asset_info</code> =&gt; <code>creation_time</code></li> <li><code>tx_info</code> - Add Vasil data #1464</li> <li><code>collaterals</code> =&gt; <code>collateral_inputs</code></li> <li>Add <code>collateral_outputs</code>, <code>reference_inputs</code> to <code>tx_info</code></li> <li>Add <code>datum_hash</code>, <code>inline_datum</code>, <code>reference_script</code> to collateral input/outputs, reference inputs &amp; inputs/outputs JSON.</li> <li>Add complete <code>cost_model</code> instead of <code>cost_model_id</code> reference</li> <li><code>epoch_params</code> - Update leftover lovelace references to text for consistency: #1484</li> <li><code>key_deposit</code></li> <li><code>pool_deposit</code></li> <li><code>min_utxo_value</code></li> <li><code>min_pool_cost</code></li> <li><code>coins_per_utxo_size</code></li> </ul>"},{"location":"Build/grest-changelog/#changes-for-instance-providers_3","title":"Changes for Instance Providers","text":"<ul> <li><code>get-metrics.sh</code> - Add active/idle connections to database #1459</li> <li><code>grest-poll.sh</code>: Bump haproxy to 2.6.1 and set default value of API_STRUCT_DEFINITION to be dependent on network used. #1450</li> <li>Lighten <code>grest.account_active_stake_cache</code> - optimise code and delete historical view (beyond 4 epochs). [#1451(https://github.com/cardano-community/guild-operators/pull/1451)</li> <li><code>tx_metalabels</code> - Move metalabels from view to RPC using lose indexscan (much better performance) #1474</li> <li>Major re-work to artificially add last epoch's active stake cache data (brings in latest snapshot information without depending on node), not used in endpoints for this release #1452</li> <li><code>grest.stake_snapshot_cache</code> - Fix rewards for new accounts #1476</li> </ul>"},{"location":"Build/grest-changelog/#105-alpha-networks-only","title":"[1.0.5] - alpha networks only","text":"<p>Since there have been a few deviations wrt Vasil for testnet and mainnet, this version only targets networks except Mainnet!</p>"},{"location":"Build/grest-changelog/#changes-for-api_3","title":"Changes for API","text":""},{"location":"Build/grest-changelog/#data-output-changes_1","title":"Data Output Changes","text":"<ul> <li><code>/epoch_info</code> - Add <code>total_rewards</code> and <code>avg_block_reward</code> for a given epoch #43</li> <li>Update all date output formats to return UNIX timestamp (as per poll held in discussions group): #45</li> <li><code>/tip</code> , <code>/blocks</code>, <code>/block_info</code> =&gt; <code>block_time</code></li> <li><code>/genesis</code> =&gt; <code>systemStart</code></li> <li><code>/epoch_info</code> =&gt; <code>start_time</code>, <code>first_block_time</code>, <code>last_block_time</code>, <code>end_time</code></li> <li><code>/tx_info</code> =&gt; <code>tx_timestamp</code></li> <li><code>/asset_info</code> =&gt; <code>creation_time</code></li> <li><code>/blocks</code>, <code>/block_info</code> =&gt; Add <code>proto_major</code> and <code>proto_minor</code>  for a given block to output #55</li> </ul>"},{"location":"Build/grest-changelog/#changes-for-instance-providers_4","title":"Changes for Instance Providers","text":"<ul> <li>For consistency between date formats, we highly recommend you to upgrade your instance to use Postgres 14 (prolly a good time, since you would already need to recreate DB for dbsync v13). You can find sample instructions to do so here</li> <li>Various changes to backend scripts and performance optimisations that can be found here</li> </ul>"},{"location":"Build/grest-changelog/#101","title":"[1.0.1]","text":"<ul> <li>Modify <code>asset_registry_update.sh</code> script to rely on commit hash instead of POSIX timestamps, and performance bump. #1428</li> </ul>"},{"location":"Build/grest-changelog/#100","title":"[1.0.0]","text":"<ul> <li>First Production release for Koios gRest</li> </ul>"},{"location":"Build/grest-changelog/#100-rc1","title":"[1.0.0-rc1]","text":""},{"location":"Build/grest-changelog/#changes-for-api_4","title":"Changes for API","text":""},{"location":"Build/grest-changelog/#data-output-changes_2","title":"Data Output Changes","text":"<ul> <li>Improve: Add <code>epoch_no</code>, <code>block_no</code> to <code>/address_txs</code>, <code>/credential_txs</code> and <code>/asset_txs</code> endpoints. #1409</li> <li>Fix: Remove redundant policy_info for <code>/asset_txs</code>, returning transactions as an array - allows for leveraging native PostgREST filtering. #1409</li> <li>Fix: Pool Metadata sorting was incorrect for <code>/pool_info</code>. #1414</li> </ul>"},{"location":"Build/grest-changelog/#input-parameter-changes","title":"Input Parameter Changes","text":"<ul> <li>None</li> </ul>"},{"location":"Build/grest-changelog/#changes-for-instance-providers_5","title":"Changes for Instance Providers","text":""},{"location":"Build/grest-changelog/#added","title":"Added","text":"<ul> <li>Add setup-grest.sh versioning. When running setup-grest.sh against a branch/tag, it will now populate the version information on control table, the health checks will be able to use this versioning for downstream connections. #1403</li> </ul>"},{"location":"Build/grest-changelog/#fixed","title":"Fixed","text":"<ul> <li>Delete token token-registry folder when running <code>setup-grest.sh</code> with <code>-r</code> (reset flag), as the delta registry records to insert depends on file (POSIX) timestamps. #1410</li> <li>Remove duplicate tip check in <code>grest-poll.sh</code>. </li> </ul>"},{"location":"Build/grest-changelog/#100-rc0-2022-04-29","title":"[1.0.0-rc0] - 2022-04-29","text":"<ul> <li>Initial Release Candidate for Koios gRest API layer with 43 endpoints to query the chain.</li> </ul>"},{"location":"Build/grest/","title":"Koios gRest","text":"<p>Important</p> <ul> <li>An average pool operator may not require this component at all. Please verify if it is required for your use as mentioned here</li> <li>Ensure that you have setup DBSync and that it is in sync atleast to Mary fork before you proceed. IF you're participating in Koios services, ensure that you're using latest dbsync release</li> </ul>"},{"location":"Build/grest/#what-is-grest","title":"What is gRest","text":"<p>gRest is an open source implementation of a <code>query layer built over dbsync using PostgREST and HAProxy</code>. The package is built as part of Koios team's efforts to unite community individual stream of work together and give back a more aligned structure to query dbsync and adopt standardisation to queries utilising open-source tooling as well as collaboration. In addition to these, there are also accessibility features to deploy rules for failover, do healthchecks, set up priorities, have ability to prevent DDoS attacks, provide timeouts, report tips for analysis over a longer period, etc - which can prove to be really useful when performing any analysis for instances.</p> <p>Note</p> <p>Note that the scripts below do allow for provisioning ogmios integration too, but Ogmios - currently - is not designed to provide advanced session management for a server-client architecture in absence of a middleware. Thus, the availability for ogmios from monitoring instance is restricted to avoid ability to DDoS an instance.</p>"},{"location":"Build/grest/#components","title":"Components","text":"<ol> <li> <p>PostgREST:     An RPC JSON interface for any PostgreSQL database (in our case, database served via <code>cardano-db-sync</code>) to provide a RESTful Web Service. The endpoints of PostgREST in itself are essentially the table/functions defined in elected schema via grest config file. You can read more about advanced query syntax using PostgREST API here, but we will provide a simpler view using examples towards the end of the page. It is an easy alternative - with almost no overhead as it directly serves the underlying database as an API, as compared to <code>Cardano GraphQL</code> component (which may often have lags). Some of the other advantages of PostgREST over graphql based projects are also performance, being stateless, 0 overhead, support for JWT / native Postgres DB authentication against the Rest Interface as well.</p> </li> <li> <p>HAProxy:     An easy gateway proxy that automatically provides failover/basic DDoS protection, specify rules management for load balancing, setup multiple frontend/backends, provide easy means to have TLS enabled for public facing instances, etc. You may alter the settings for proxy layer as per your SecOps preferences. This component is optional (eg: if you prefer to expose your PostgREST server itself, you can do so using similar steps below).</p> </li> </ol>"},{"location":"Build/grest/#setup","title":"Setup gRest services","text":"<p>To start with you'd want to ensure your current shell session has access to Postgres credentials, continuing from examples from the above mentioned Sample Postgres deployment guide.</p> <pre><code>cd $CNODE_HOME/priv\nPGPASSFILE=$CNODE_HOME/priv/.pgpass\npsql cexplorer\n</code></pre> <p>Ensure that you can connect to your Postgres DB fine using above (quit from psql once validated using <code>\\q</code>). As part of <code>guild-deploy.sh</code> execution, you'd find setup-grest.sh file made available in <code>${CNODE_HOME}/scripts</code> folder, which will help you automate installation of PostgREST, HAProxy as well as brings in latest queries/functions provided via Koios to your instances.</p> <p>Warning</p> <p>As of now, gRest services are in alpha stage - while can be utilised, please remember there may be breaking changes and every collaborator is expected to work with the team to keep their instances up-to-date using alpha branch.</p> <p>Familiarise with the usage options for the setup script , the syntax can be viewed as below:</p> <pre><code>cd \"${CNODE_HOME}\"/scripts\n./setup-grest.sh -h\n#\n# Usage: setup-grest.sh [-f] [-i [p][r][m][c][d]] [-u] [-b &lt;branch&gt;]\n# \n# Install and setup haproxy, PostgREST, polling services and create systemd services for haproxy, postgREST and dbsync\n# \n# -f    Force overwrite of all files including normally saved user config sections\n# -i    Set-up Components individually. If this option is not specified, components will only be installed if found missing (eg: -i prcd)\n#     p    Install/Update PostgREST binaries by downloading latest release from github.\n#     r    (Re-)Install Reverse Proxy Monitoring Layer (haproxy) binaries and config\n#     m    Install/Update Monitoring agent scripts\n#     c    Overwrite haproxy, postgREST configs\n#     d    Overwrite systemd definitions\n# -u    Skip update check for setup script itself\n# -q    Run all DB Queries to update on postgres (includes creating grest schema, and re-creating views/genesis table/functions/triggers and setting up cron jobs)\n# -b    Use alternate branch of scripts to download - only recommended for testing/development (Default: master)\n#\n</code></pre> <p>To run the setup overwriting all standard deployment tasks from a branch (eg: <code>koios-1.0.9</code> branch), you may want to use: <pre><code>./setup-grest.sh -f -i prmcd -r -q -b koios-1.0.9\n</code></pre></p> <p>Similarly - if you'd like to re-install all components and force overwrite all configs but not reset cache tables, you may run: <pre><code>./setup-grest.sh -f -i prmcd -q\n</code></pre></p> <p>Another example could be to preserve your config, but only update queries using an alternate branch (eg: let's say you want to try the branch <code>alpha</code> prior to a tagged release). To do so, you may run: <pre><code>./setup-grest.sh -q -b alpha\n</code></pre></p> <p>Please ensure to follow the on-screen instructions, if any (for example restarting deployed services, or updating configs to specify correct target postgres URLs/enable TLS/add peers etc in <code>${CNODE_HOME}/priv/grest.conf</code> and <code>${CNODE_HOME}/files/haproxy.cfg</code>).</p> <p>The default ports used will make haproxy instance available at port 8053 or 8453 if TLS is enabled (you might want to enable firewall rule to open this port to services you would like to access). If you want to prevent unauthenticated access to grest schema, uncomment the jwt-secret and specify a custom <code>secret-token</code>.</p> <p>Reminder</p> <p>Once you've successfully deployed the grest instance, it will deploy certain cron jobs that will ensure the relevant cache tables are updated periodically. Until these have finished (especially on first run, it could take an hour or so on mainnet, your instance will likely not pass any tests from <code>grest-poll.sh</code> but that's expected.</p>"},{"location":"Build/grest/#tls","title":"Enable TLS on HAProxy","text":"<p>In order to enable SSL on your haproxy, all you need to do is edit the file <code>${CNODE_HOME}/files/haproxy.cfg</code> and update the frontend app section to uncomment ssl bind (and comment normal bind).</p> <p>Info</p> <ul> <li>server.pem referred below should be a chain containing server TLS certificate, signing certificates (intermediate/root) and private key.</li> <li>Make sure to replace the hostname to the CNAME/SAN used to create your TLS certificate.</li> </ul> <p>If you're not familiar with how to configure TLS OR would not like to buy one, you can find tips on how to create a TLS certificate for free via LetsEncrypt using tutorials here. Once you do have a TLS Certificate generated, you need to chain the private key and full chain cert together in a file - <code>/etc/ssl/server.pem</code> - which can be then referenced as below:</p> <p><pre><code>frontend app\n  #bind 0.0.0.0:8053\n  ## If using SSL, comment line above and uncomment line below\n  bind :8453 ssl crt /etc/ssl/server.pem no-sslv3\n  http-request set-log-level silent\n  acl srv_down nbsrv(grest_postgrest) eq 0\n  acl is_wss hdr(Upgrade) -i websocket\n  ...\n</code></pre> Restart haproxy service for changes to take effect.</p>"},{"location":"Build/grest/#validation","title":"Validation","text":"<p>With the setup, you also have a <code>checkstatus.sh</code> script, which will query the Postgres DB instance via haproxy (coming through postgREST), and only show an instance up if the latest block in your DB instance is within 180 seconds.</p> <p>Important</p> <p>If you'd like to participate in joining to the elastic cluster via Koios, please raise a PR request by editing topology files in this folder to do so!!</p> <p>If you were using <code>guild</code> network, you could do a couple of very basic sanity checks as per below:</p> <ol> <li> <p>To query active stake for pool <code>pool1z2ry6kxywgvdxv26g06mdywynvs7jj3uemnxv273mr5esukljsr</code> in epoch <code>122</code>, we can execute the below: <pre><code>curl -d _pool_bech32=pool1z2ry6kxywgvdxv26g06mdywynvs7jj3uemnxv273mr5esukljsr -d _epoch_no=122 -s http://localhost:8053/rpc/pool_active_stake\n## {\"active_stake_sum\" : 19409732875}\n</code></pre></p> </li> <li> <p>To check latest owner key(s) for a given pool <code>pool1z2ry6kxywgvdxv26g06mdywynvs7jj3uemnxv273mr5esukljsr</code>, you can execute the below: <pre><code>curl -d _pool_bech32=pool1z2ry6kxywgvdxv26g06mdywynvs7jj3uemnxv273mr5esukljsr -s http://localhost:8050/rpc/pool_owners\n## [{\"owner\" : \"stake_test1upx5p04dn3t6dvhfh27744su35vvasgaaq565jdxwlxfq5sdjwksw\"}, {\"owner\" : \"stake_test1uqak99cgtrtpean8wqwp7d9taaqkt9gkkxga05m5azcg27chnzfry\"}]\n</code></pre></p> </li> </ol> <p>You may want to explore what all endpoints come out of the box, and test them out, to do so - refer to API documentation for OpenAPI3 documentation. Each endpoint has a pre-filled example for mainnet and connects by default to primary Koios endpoint, allowing you to test endpoints and if needed - grab the <code>curl</code> commands to start testing yourself against your local or remote instances.</p>"},{"location":"Build/grest/#participating-in-koios-cluster-as-instance-provider","title":"Participating in Koios Cluster as instance Provider","text":"<p>If you're interested to participate in decentralised infrastructure by providing an instance, there are a few additional steps you'd need:</p> <ol> <li> <p>Enable ports for your HAProxy instance (default: 8053), gRest Exporter service (default: 8059) and (optionally) submit API instance (default: 8090) against the monitoring instance (do not need to open these ports to internet) of corresponding network.</p> </li> <li> <p>Ensure that each of the service above is listening on your public IP address (for instance, submitapi.sh might need to be edited to change HOSTADDR to <code>0.0.0.0</code> and restarted).</p> </li> <li> <p>Create a PR specifying connectivity information to your HAProxy port here.</p> </li> <li> <p>Make sure to join the telegram discussions group to participate in any discussions, actions, polls for new-features, etc. Feel free to give a shout in the group in case you have trouble following any of the above</p> </li> </ol>"},{"location":"Build/node-cli/","title":"Node & CLI","text":"<p>Reminder !!</p> <p>Unless there is a very particular reason you want to compile (eg: running on non-popular OS flavor), you DO NOT \"need\" to build node binaries - <code>guild-deploy.sh</code> already provides an option to download pre-compiled binaries. Ensure the Pre-Requisites are in place before you proceed.</p>"},{"location":"Build/node-cli/#build-instructions","title":"Build Instructions","text":""},{"location":"Build/node-cli/#clone-the-repository","title":"Clone the repository","text":"<p>Execute the below to clone the cardano-node repository to <code>$HOME/git</code> folder on your system:</p> <pre><code>cd ~/git\ngit clone https://github.com/intersectmbo/cardano-node\ncd cardano-node\n</code></pre>"},{"location":"Build/node-cli/#build-cardano-node","title":"Build Cardano Node","text":"<p>You can use the instructions below to build the latest release of cardano-node. </p> <pre><code>git fetch --tags --recurse-submodules --all\ngit pull\n# Replace tag against checkout if you do not want to build the latest released version, we recommend using battle tested node versions - which may not always be latest\ngit checkout $(curl -sLf https://api.github.com/repos/intersectmbo/cardano-node/releases/latest | jq -r .tag_name)\n\n# Use `-l` argument if you'd like to use system libsodium instead of IOG fork of libsodium while compiling\n$CNODE_HOME/scripts/cabal-build-all.sh\n</code></pre> <p>The above would copy the binaries built into <code>~/.local/bin</code> folder.</p>"},{"location":"Build/node-cli/#download-pre-compiled-binary-from-node-release","title":"Download pre-compiled Binary from Node release","text":"<p>While certain folks might want to build the node themselves (could be due to OS/arch compatibility, trust factor or customisations), for most it might not make sense to build the node locally. Instead, you can download the binaries using cardano-node release notes, where-in you can find the download links for every version. This is already taken care of by <code>guild-deploy.sh</code> if you used the option to download binaries (you can always re-run with specific arguments if unsure).</p>"},{"location":"Build/node-cli/#verify","title":"Verify","text":"<p>Execute <code>cardano-cli</code> and <code>cardano-node</code> to verify output as below (the exact version and git rev should depend on your checkout tag on github repository):</p> <pre><code>cardano-cli version\n# cardano-cli 9.x.x - linux-x86_64 - ghc-8.10\n# git rev &lt;...&gt;\ncardano-node version\n# cardano-node 9.x.x - linux-x86_64 - ghc-8.10\n# git rev &lt;...&gt;\n</code></pre>"},{"location":"Build/node-cli/#update-port-number-or-pool-name-for-relative-paths","title":"Update port number or pool name for relative paths","text":"<p>Before you go ahead with starting your node, you may want to update values for <code>CNODE_PORT</code> in <code>$CNODE_HOME/scripts/env</code>. Note that it is imperative for operational relays and pools to ensure that the port mentioned is opened via firewall to the destination your node is supposed to connect from. Update your network/firewall configuration accordingly. Future executions of <code>guild-deploy.sh</code> will preserve and not overwrite these values (or atleast back up if forced to overwrite).</p> <pre><code>CNODEBIN=\"${HOME}/.local/bin/cardano-node\"\nCCLI=\"${HOME}/.local/bin/cardano-cli\"\nCNODE_PORT=6000\nPOOL_NAME=\"GUILD\"\n</code></pre> <p>Important</p> <p>POOL_NAME is the name of folder that you will use when registering pools and starting node in core mode. This folder would typically contain your <code>hot.skey</code>,<code>vrf.skey</code> and <code>op.cert</code> files required. If the mentioned files are absent (expected if this is a fresh install), the node will automatically start in a relay mode.</p>"},{"location":"Build/node-cli/#start-the-node","title":"Start the node","text":"<p>To test starting the node in interactive mode, we will make use of pre-built script <code>cnode.sh</code>. This script automatically determines whether to start the node as a relay or block producer (if the required pool keys are present in the <code>$CNODE_HOME/priv/pool/&lt;POOL_NAME&gt;</code> as mentioned above). If the <code>&lt;MITHRIL_DOWNLOAD&gt;</code> variable is set to 'Y' it will download the latest snapshot from a Mithril aggregator to speed up the blockchain synchronization. The script contains a user-defined variable <code>CPU_CORES</code> which determines the number of CPU cores the node will use upon start-up:</p> <pre><code>######################################\n# User Variables - Change as desired #\n# Common variables set in env file   #\n######################################\n\n#CPU_CORES=4            # Number of CPU cores cardano-node process has access to (please don't set higher than physical core count, 4 recommended)\n</code></pre> <p>Now let's test starting the node in interactive mode.</p> <p>Note</p> <p>At this stage, upon executing <code>cnode.sh</code>, you are expected to see the live config and a line ending with <code>Listening on http://127.0.0.1:12798</code> - this is expected, as your logs are being written to <code>$CNODE_HOME/logs/node.json</code> . If so, you should be alright to return to your console by pressing Ctrl-C. The node will be started later using instructions below using systemd (Linux's service management). In case you receive any errors, please troubleshoot and fix those before proceeding.</p> <pre><code>cd \"${CNODE_HOME}\"/scripts\n./cnode.sh\n</code></pre> <p>Press Ctrl-C to exit node and return to console.</p>"},{"location":"Build/node-cli/#modify-the-nodes-config-files","title":"Modify the node's config files","text":"<p>Now that you've tested the basic node operation, you might want to customise your config files (assuming you are in top-level folder , i.e. <code>cd \"${CNODE_HOME}\"</code>) :</p> <ol> <li> <p>files/config.json : This file contains the logging configurations (tracers of to tune logging, paths for other genesis config files, address/ports on which the prometheus/EKG monitoring will listen, etc). Unless running more than one node on same machine (not recommended), you should be alright to use most of this file as-is. You might - however - want to double-check <code>PeerSharing</code> in this file, if using a relay node where you'd like connecting peers (marked as <code>\"advertise\": \"true\"</code> in topology.json) to be shared , you may turn this setting to <code>true</code>.</p> </li> <li> <p>files/topology.json : This file tells your node how to connect to other nodes (especially initially to start synching). You would want to update this file as below:</p> <ul> <li>Update the <code>localRoots</code> &gt; <code>accessPoints</code> section to include your local nodes that you want persistent connection against (eg: this could be your BP and own relay nodes) against definition where <code>trustable</code> is set to <code>true</code>.</li> <li>If you want specific peers to be advertised on the network for discovery, you may set <code>advertise</code> to <code>true</code> for that peer group. You do NOT want to do that on BP</li> <li>You'd want to update <code>localRoots</code> &gt; <code>valency</code> (<code>valency</code> is the same as <code>hotValency</code>, not yet replaced since the example in cardano-node-wiki repo still suggests <code>valency</code>) to number of connections from your localRoots that you always want to keep active connection to for that node.</li> <li>[Optional] - you can add/remove nodes from <code>publicRoots</code> section as well as <code>localRoots</code> &gt; <code>accessPoints</code> as desired, tho defaults populated should work fine. On mainnet, we did add a few additional nodes to help add more redundancy for initial sync.</li> <li><code>useLedgerAfterSlot</code> tells the node to establish networking with nodes from defined peers to sync the node initially until reaching an absolute slot number, after which - it can start attempting to connect to peers registered as pool relays on the network. You may want this number to be relatively recent (eg: not have it 50 epochs old).</li> <li>You can read further about topology file configuration here</li> </ul> </li> </ol> <p>Important</p> <p>On BP, You'd want to set <code>useLedgerAfterSlot</code> to <code>-1</code> for your Block Producing (Core) node - thereby, telling your Core node to remain in non-P2P mode, and ensure <code>PeerSharing</code> is to <code>false</code>.</p> <p>The resultant topology file on a relay could look something like below:</p> <pre><code>{\n\"bootstrapPeers\": [\n{\n\"address\": \"backbone.cardano.iog.io\",\n\"port\": 3001\n},\n{\n\"address\": \"backbone.mainnet.emurgornd.com\",\n\"port\": 3001\n},\n{\n\"address\": \"backbone.mainnet.cardanofoundation.org\",\n\"port\": 3001\n}\n],\n\"localRoots\": [\n{\n\"accessPoints\": [\n{\"address\": \"xx.xx.xx.xx\", \"port\": 6000 , \"description\": \"Core\"},\n{\"address\": \"zz.zz.zz.zz\", \"port\": 6000 , \"description\": \"Relay2\"}\n],\n\"advertise\": false,\n\"trustable\": true,\n\"hotValency\": 2\n},\n{\n\"accessPoints\": [\n{\"address\": \"node-dus.poolunder.com\",           \"port\": 6900, \"pool\": \"UNDR\",   \"location\": \"EU/DE/Dusseldorf\" },\n{\"address\": \"node-syd.poolunder.com\",           \"port\": 6900, \"pool\": \"UNDR\",   \"location\": \"OC/AU/Sydney\" },\n{\"address\": \"194.36.145.157\",                   \"port\": 6000, \"pool\": \"RDLRT\",  \"location\": \"EU/DE/Baden\" },\n{\"address\": \"95.216.38.251\",                    \"port\": 6000, \"pool\": \"RDLRT\",  \"location\": \"EU/FI/Helsinki\" },\n{\"address\": \"148.72.153.168\",                   \"port\": 16000, \"pool\": \"AAA\",   \"location\": \"NA/US/StLouis\" },\n{\"address\": \"78.47.99.41\",                      \"port\": 6000, \"pool\": \"AAA\",    \"location\": \"EU/DE/Nuremberg\" },\n{\"address\": \"relay1-pub.ahlnet.nu\",             \"port\": 2111, \"pool\": \"AHL\",    \"location\": \"EU/SE/Malmo\" },\n{\"address\": \"relay2-pub.ahlnet.nu\",             \"port\": 2111, \"pool\": \"AHL\",    \"location\": \"EU/SE/Malmo\" },\n{\"address\": \"relay1.clio.one\",                  \"port\": 6010, \"pool\": \"CLIO\",   \"location\": \"EU/IT/Milan\" },\n{\"address\": \"relay2.clio.one\",                  \"port\": 6010, \"pool\": \"CLIO\",   \"location\": \"EU/IT/Bozlano\" },\n{\"address\": \"relay3.clio.one\",                  \"port\": 6010, \"pool\": \"CLIO\",   \"location\": \"EU/IT/Bozlano\" }\n],\n\"advertise\": false,\n\"trustable\": false,\n\"hotValency\": 5,\n\"warmValency\": 10\n}\n],\n\"publicRoots\": [\n{\n\"accessPoints\": [],\n\"advertise\": false\n}\n],\n\"useLedgerAfterSlot\": 128908821\n}\n</code></pre> <p>Similarly, a typical topology file on a Core could look something like below:</p> <pre><code>{\n\"bootstrapPeers\": [],\n\"localRoots\": [\n{\n\"accessPoints\": [\n{\"address\": \"yy.yy.yy.yy\", \"port\": 6000, \"description\": \"Relay1\"},\n{\"address\": \"zz.zz.zz.zz\", \"port\": 6000, \"description\": \"Relay2\"}\n],\n\"advertise\": false,\n\"trustable\": true,\n\"hotValency\": 2\n}\n],\n\"publicRoots\": [\n{\n\"accessPoints\": [],\n\"advertise\": false\n}\n],\n\"useLedgerAfterSlot\": -1\n}\n</code></pre> <p>Once above two files are updated, since you modified the file manually - there is always a chance of human errors (eg: missing comma/quotes). Thus, we would recommend you to start the node interactively once again before proceeding.</p> <pre><code>cd \"${CNODE_HOME}\"/scripts\n./cnode.sh\n</code></pre> <p>As before, ensure you do not have any errors in the console. To stop the node, hit Ctrl-C - we will start the node as systemd later in the document.</p>"},{"location":"Build/node-cli/#start-the-submit-api","title":"Start the submit-api","text":"<p>Note</p> <p>An average pool operator may not require <code>cardano-submit-api</code> at all. Please verify if it is required for your use as mentioned here. If - however - you do run submit-api for accepting sizeable transaction load, you would want to override the default MEMPOOL_BYTES by uncommenting it in cnode.sh.</p> <p><code>cardano-submit-api</code> is one of the binaries built as part of <code>cardano-node</code> repository and allows you to submit transactions over a Web API. To run this service interactively, you can use the pre-built script below (<code>submitapi.sh</code>). Make sure to update <code>submitapi.sh</code> script to change listen IP or Port that you'd want to make this service available on.</p> <pre><code>cd $CNODE_HOME/scripts\n./submitapi.sh\n</code></pre> <p>To stop the process, hit Ctrl-C</p>"},{"location":"Build/node-cli/#systemd","title":"Run as systemd service","text":"<p>The preferred way to run the node (and submit-api) is through a service manager like systemd. This section explains how to setup a systemd service file.</p> <p>1. Deploy as a systemd service Execute the below command to deploy your node as a systemd service (from the respective scripts folder): <pre><code>cd $CNODE_HOME/scripts\n./cnode.sh -d\n# Deploying cnode.service as systemd service..\n# cnode.service deployed successfully!!\n\n./submitapi.sh -d\n# Deploying cnode-submit-api.service as systemd service..\n# cnode-submit-api deployed successfully!!\n</code></pre></p> <p>2. Start the service Run below commands to enable automatic start of service on startup and start it. <pre><code>sudo systemctl start cnode.service\nsudo systemctl start cnode-submit-api.service\n</code></pre></p> <p>3. Check status and stop/start commands  Replace <code>status</code> with <code>stop</code>/<code>start</code>/<code>restart</code> depending on what action to take. <pre><code>sudo systemctl status cnode.service\nsudo systemctl status cnode-submit-api.service\n</code></pre></p> <p>Important</p> <p>In case you see the node exit unsuccessfully upon checking status, please verify you've followed the transition process correctly as documented below, and that you do not have another instance of node already running. It would help to check your system logs, you can also check <code>journalctl -f -u cnode</code> to examine startup attempt for services, and scroll up until you see output for node startup attempt) for any errors while starting node.</p> <p>You can use gLiveView to monitor your node that was started as a systemd service.</p> <pre><code>cd $CNODE_HOME/scripts\n./gLiveView.sh\n</code></pre>"},{"location":"Build/offchain-metadata-tools/","title":"Offchain Metadata Tools","text":"<p>Important</p> <ul> <li>An average pool operator may not require offline-metadata-tools at all. Please verify if it is required for your use as mentioned here</li> <li>Ensure the Pre-Requisites are in place before you proceed.</li> </ul> <p>In the Cardano multi-asset era, this project helps you create and submit metadata describing your assets, storing them off-chain.</p>"},{"location":"Build/offchain-metadata-tools/#download-pre-built-binaries","title":"Download pre-built binaries","text":"<p>Go to input-output-hk/offchain-metadata-tools to download the binaries and place in a directory specified by <code>PATH</code>, e.g. <code>$HOME/.local/bin/</code>. </p>"},{"location":"Build/offchain-metadata-tools/#build-instructions","title":"Build Instructions","text":"<p>An alternative to pre-built binaries - instructions describe how to build the <code>token-metadata-creator</code> tool but the offchain-metadata-tools repository contains other tools as well. Build the ones needed for your installation.</p>"},{"location":"Build/offchain-metadata-tools/#clone-the-repository","title":"Clone the repository","text":"<p>Execute the below to clone the offchain-metadata-tools repository to $HOME/git folder on your system:</p> <pre><code>cd ~/git\ngit clone https://github.com/input-output-hk/offchain-metadata-tools.git\ncd offchain-metadata-tools/token-metadata-creator\n</code></pre>"},{"location":"Build/offchain-metadata-tools/#build-token-metadata-creator","title":"Build token-metadata-creator","text":"<p>You can use the instructions below to build <code>token-metadata-creator</code>, same steps can be executed in future to update the binaries (replacing appropriate tag) as well.</p> <p><pre><code>git fetch --tags --all\ngit pull\n# Replace master with appropriate tag if you'd like to avoid compiling against master\ngit checkout master\n$CNODE_HOME/scripts/cabal-build-all.sh\n</code></pre> The above would copy the binaries into <code>~/.local/bin</code> folder.</p>"},{"location":"Build/offchain-metadata-tools/#verify","title":"Verify","text":"<p>Verify that the tool is executable from anywhere by running:</p> <pre><code>token-metadata-creator -h\n</code></pre>"},{"location":"Build/wallet/","title":"Wallet","text":"<p>!&gt; - An average pool operator may not require <code>cardano-wallet</code> at all. Please verify if it is required for your use as mentioned here.</p> <p>Ensure the Pre-Requisites are in place before you proceed.</p>"},{"location":"Build/wallet/#build-instructions","title":"Build Instructions","text":"<p>Follow instructions below for building the cardano-wallet binary:</p>"},{"location":"Build/wallet/#clone-the-repository","title":"Clone the repository","text":"<p>Execute the below to clone the <code>cardano-wallet</code> repository to <code>$HOME/git</code> folder on your system:</p> <pre><code>cd ~/git\ngit clone https://github.com/cardano-foundation/cardano-wallet\ncd cardano-wallet\n</code></pre>"},{"location":"Build/wallet/#build-cardano-wallet","title":"Build Cardano Wallet","text":"<p>You can use the instructions below to build the latest release of cardano-wallet.</p> <p>!&gt; - Note that the latest release of <code>cardano-wallet</code> may not work with the latest release of <code>cardano-node</code>. Please check the compatibility of each <code>cardano-wallet</code> release yourself in the official docs, e.g. https://github.com/cardano-foundation/cardano-wallet/releases/latest.</p> <pre><code>git fetch --tags --all\ngit pull\n# Replace tag against checkout if you do not want to build the latest released version\ngit checkout $(curl -s https://api.github.com/repos/cardano-foundation/cardano-wallet/releases/latest | jq -r .tag_name)\n$CNODE_HOME/scripts/cabal-build-all.sh\n</code></pre> <p>The above would copy the binaries into <code>~/.local/bin</code> folder.</p>"},{"location":"Build/wallet/#start-the-wallet","title":"Start the wallet","text":"<p>You can run the below to connect to a <code>cardano-node</code> instance that is expected to be already running and the wallet will start syncing. <pre><code>cardano-wallet serve /\n    --node-socket $CNODE_HOME/sockets/node.socket /\n    --mainnet / # if using the testnet flag you also need to specify the testnet shelley-genesis.json file\n--database $CNODE_HOME/priv/wallet\n</code></pre></p>"},{"location":"Build/wallet/#verify-the-wallet-is-handling-requests","title":"Verify the wallet is handling requests","text":"<p><pre><code>cardano-wallet network information\n</code></pre> Expected output should be similar to the following <pre><code>Ok.\n{\n\"network_tip\": {\n\"time\": \"2021-06-01T17:31:05Z\",\n\"epoch_number\": 269,\n\"absolute_slot_number\": 31002374,\n\"slot_number\": 157574\n},\n\"node_era\": \"mary\",\n\"node_tip\": {\n\"height\": {\n\"quantity\": 5795127,\n\"unit\": \"block\"\n},\n\"time\": \"2021-06-01T17:31:00Z\",\n\"epoch_number\": 269,\n\"absolute_slot_number\": 31002369,\n\"slot_number\": 157569\n},\n\"sync_progress\": {\n\"status\": \"ready\"\n},\n\"next_epoch\": {\n\"epoch_start_time\": \"2021-06-04T21:44:51Z\",\n\"epoch_number\": 270\n}\n}\n</code></pre></p>"},{"location":"Build/wallet/#creatingrestoring-wallet","title":"Creating/Restoring Wallet","text":"<p>If you're creating a new wallet, you'd first want to generate a mnemonic for use (see below):</p> <p><pre><code>cardano-wallet recovery-phrase generate\n# false brother typical saddle settle phrase foster sauce ask sunset firm gate service render burger\n</code></pre> You can use the above mnemonic to then restore a wallet as per below: <pre><code>cardano-wallet wallet create from-recovery-phrase MyWalletName\n</code></pre></p>"},{"location":"Build/wallet/#expected-output","title":"Expected output:","text":"<pre><code>Please enter a 15\u201324 word recovery phrase: false brother typical saddle settle phrase foster sauce ask sunset firm gate service render burger\n(Enter a blank line if you do not wish to use a second factor.)\nPlease enter a 9\u201312 word second factor:\nPlease enter a passphrase: **********\nEnter the passphrase a second time: **********\nOk.\n{\n    ...\n}\n</code></pre>"},{"location":"Mithril/mithril-overview/","title":"Mithril Overview","text":"<p>Mithril Networks provide the ability to download and bootstrap cardano nodes via snapshots of the the Cardano blockchain. This is a great way to speed up the process of syncing a new node, especially for stake pool operators. The tools provided by Guild Operators are designed to facilitate the ease of use in setting up and managing the:</p> <ul> <li>Mithril Client to download a snapshot for the given network the node is attached to via the mithril-client.sh script.</li> <li>Mithril Signer to participate in the creation of stake based singatures of snapshots via the mithril-signer.sh script.</li> <li>Squid Mithril Relay to provide a relay for submitting the snapshots signatures to a Mithril Aggregator, as described in Run a Mithril signer as an SPO documentation via the mithril-relay.sh script.</li> </ul> <p>The <code>env</code> file contains a new environment variable <code>MITHRIL_DOWNLOAD</code> that when enabled allows the <code>cnode.sh</code> script to automatically download the latest Mithril snapshot if the local <code>db</code> directory is empty. This is useful for new nodes that need to be  bootstrapped with the latest snapshot to avoid synchronizing the entire blockchain from scratch. While also providing a high level of trust that the snapshot is valid since it is signed by multiple pool operators.</p>"},{"location":"Mithril/mithril-overview/#architecture","title":"Architecture","text":"<p>The architecture for Mithril Networks is described in detail at Mithril network architecture by CF/IOHK. However the architecture suggested and supported by the Guild Operators tools is not identical to the upstream documentation in that we provide a more simplified approach to the setup and management of the Mithril Network components and tools that allow setting up a Squid Mithril relays and an Nginx loadbalancer (aka sidecar) local to the Mithril signer. The Nginx sidecar provides the ability to loadbalance requests to multiple Squid based Mithril Relays running on each of the SPO's Cardano Relay nodes.</p>"},{"location":"Mithril/mithril-overview/#single-relay-architecture","title":"Single Relay Architecture","text":"<p>For SPO's who only have a single Cardano relay node, an Squid based Mithril relay can be run on the same node as the Cardano relay. This can be used by the Mithril signer to submit the snapshot signatures to the Mithril Aggregator.</p> <p></p>"},{"location":"Mithril/mithril-overview/#multi-relay-architecture","title":"Multi Relay Architecture","text":"<p>For SPO's who have multiple Cardano relay nodes, a Nginx relay sidecar can be run on the Block Producer and load balance requests over mutliple Cardano relay nodes, each running its own Nginx Mithril relay to pass the signature along to the Mithril aggregator. This can be used to avoid a single point of failure in case a Relay server is offline for any reason. This provides high availability for the Mithril signer through multiple relays as long as the local Nginx Mithril relay is running on the same server as the Cardano Block Producer node.</p> <p></p>"},{"location":"Mithril/mithril-overview/#installation","title":"Installation","text":"<p>The installation of the Mithril tools is automated via <code>guild-deploy.sh</code>. To participate in a Mithril network include the <code>-s m</code> flag which will install the Mithril Client and Mithril Signer release binaries to <code>\"${HOME}\"/.local/bin</code>.</p> <pre><code>guild-deploy.sh -s m\n</code></pre>"},{"location":"Mithril/mithril-overview/#bootstrapping-a-node-using-mithril-client","title":"Bootstrapping a node using Mithril Client","text":"<p>The Mithril client is used to download a snapshot of the Cardano blockchain from a Mithril Aggregator. The snapshot is then used to bootstrap a new Cardano node. The Mithril client can be used to download the latest snapshot, list all available snapshots, or show details of a specific snapshot.</p> <p>To bootstrap a Cardano node using the Mithril client, follow these steps:</p> <ol> <li> <p>Setup the Cardano Node: Use the guild tools to setup the Cardano node, either by building the binaries or using pre-compiled binaries. Follow the instructions in the guild-operators documentation.</p> </li> <li> <p>Create the Mithril environment file: Run the script with the <code>environment setup</code> command. This will create a new <code>mithril.env</code> file with all the necessary environment variables for the Mithril client.</p> </li> </ol> <pre><code>./mithril-client.sh environment setup\n</code></pre> <ol> <li>Download the latest Mithril snapshot: Once the environment file is set up, you can download the latest Mithril snapshot by running the script with the <code>snapshot download</code> command. This snapshot contains the latest state of the Cardano blockchain db from a Mithril Aggregator.</li> </ol> <pre><code>./mithril-client.sh snapshot download\n</code></pre>"},{"location":"Mithril/mithril-overview/#participating-in-mithril-network","title":"Participating in Mithril Network","text":"<p>The Mithril signer is used to participate in the creation of stake based signatures of snapshots. The Mithril signer can be used to sign a snapshots. The signed snapshot is then submitted to a Mithril Aggregator, via a Squid based Mithril Relay.</p> <p>The first step to participate in the Mithril network is to deploy your Squid based Mithril Relays. The Mithril relay is used to provide a private and highly available network for submitting the snapshots to a Mithril Aggregator.</p>"},{"location":"Mithril/mithril-overview/#deploying-the-squid-mithril-relay","title":"Deploying the Squid Mithril Relay","text":"<p>To deploy your Squid based Mithril Relays with your Cardano relay node, follow these steps:</p> <ol> <li> <p>Deploy the Squid Mithril Relay: Run the <code>mithril-relay.sh</code> script:</p> </li> <li> <p>Use the <code>-d</code> flag to deploy the Squid Mithril Relay.</p> </li> <li>Provide the IP address of your Block Producer when prompted to secure       the Mithril Relay to only accept traffic from your Block Producer.</li> <li>Optionally provide the relays listening port when prompted to use a port       other than the default 3132, or just press enter to use the default.</li> <li>Create the appropriate firewall rule to allow traffic from your Block       Producer to the Mithril Relay.</li> </ol> <pre><code>./mithril-relay.sh -d\n\nInstalling squid proxy\nEnter the IP address of your Block Producer: 1.2.3.4\nEnter the relays listening port (press Enter to use default 3132):\nUsing port 3132 for relays listening port.\nCreate the appropriate firewall rule: sudo ufw allow from 1.2.3.4 to any port 3132 proto tcp\n</code></pre> <ol> <li>Enable the Systemd Squid Mithril Relay service to start on boot.</li> </ol> <pre><code>   sudo systemctl enable --now squid\n</code></pre> <ol> <li>Repeat the process for each of your Cardano relay nodes.</li> </ol>"},{"location":"Mithril/mithril-overview/#deploying-the-mithril-signer","title":"Deploying the Mithril Signer","text":""},{"location":"Mithril/mithril-overview/#mithril-signer-with-single-relay","title":"Mithril Signer with Single Relay","text":"<ol> <li> <p>Deploy the Mithril Signer: Run the <code>mithril-signer.sh</code> script:</p> </li> <li> <p>Use the <code>-e</code> flag to update the <code>mithril.env</code> file with the Mithril       Signer environment variables.</p> </li> <li>Provide the IP address of your Mithril Relay when prompted.</li> <li> <p>Optionally provide the relays listening port when prompted to use a port.</p> <pre><code>  ./mithril-signer.sh -e\n  Enter the IP address of the relay endpoint: 4.5.6.7\n  Enter the port of the relay endpoint (press Enter to use default 3132):\n  Using RELAY_ENDPOINT=4.5.6.7:3132 for the Mithril signer relay endpoint.\n</code></pre> </li> <li> <p>Use the <code>-d</code> flag to deploy the Mithril Signer.</p> <pre><code>  ./mithril-signer.sh -d\n  Creating cnode-mithril-signer systemd service environment file..\n  Mithril signer service successfully deployed\n</code></pre> </li> <li> <p>Enable the Systemd service to start the Mithril Signer on boot.</p> <pre><code>  sudo systemctl enable cnode-mithril-signer\n</code></pre> </li> </ol>"},{"location":"Mithril/mithril-overview/#mithril-signer-with-multi-relay","title":"Mithril Signer with Multi Relay","text":"<ol> <li> <p>Deploy the Nginx sidecar loadbalancer: Run the <code>mithril-relay.sh</code> script:</p> </li> <li> <p>Use the <code>-l</code> flag to deploy the Nginx Mithril Relay.</p> </li> <li>Provide the IP address of your Mithril Relay(s) when prompted.</li> <li>Provide an IP address for your nginx loadbalancer (default 127.0.0.1)</li> <li>Optionally provide the relays listening port when prompted to use a port       other than the default 3132, or just press enter to use the default.</li> <li> <p>Create the appropriate firewall rule to allow traffic from your Block       Producer to the Mithril Relay(s).</p> <pre><code>./mithril-relay.sh -l\n\nInstalling nginx load balancer\nEnter the IP address of a relay: 4.5.6.7\nAre there more relays? (y/n) y\nEnter the IP address of a relay: 8.9.10.11\nAre there more relays? (y/n) n\nEnter the IP address of the load balancer (press Enter to use default 127.0.0.1):\nUsing IP address 127.0.0.1 for the load balancer configuration.\nEnter the relays listening port (press Enter to use default 3132):\nUsing port 3132 for relays listening port.\nStarting Mithril relay sidecar (nginx load balancer)\n</code></pre> </li> <li> <p>Enable the Systemd Nginx Mithril Relay service to start on boot.</p> <pre><code>sudo systemctl enable --now nginx\n</code></pre> </li> <li> <p>Deploy the Mithril Signer: Run the <code>mithril-signer.sh</code> script:</p> </li> <li> <p>Use the <code>-e</code> flag to update the <code>mithril.env</code> file with the Mithril       Signer environment variables.</p> </li> <li>Provide the IP address of your nginx loadbalancer when prompted.</li> <li> <p>Optionally provide the loadbalancer listening port when prompted to use a port.</p> <pre><code>    ./mithril-signer.sh -e\n    Enter the IP address of the relay endpoint: 127.0.0.1\n    Enter the port of the relay endpoint (press Enter to use default 3132):\n    Using RELAY_ENDPOINT=127.0.0.1:3132 for the Mithril signer relay endpoint.\n</code></pre> </li> <li> <p>Use the <code>-d</code> flag to deploy the Mithril Signer.</p> <pre><code>    ./mithril-signer.sh -d\n    Creating cnode-mithril-signer systemd service environment file..\n    Mithril signer service successfully deployed\n</code></pre> </li> <li> <p>Enable the Systemd service to start the Mithril Signer on boot.</p> <pre><code>    sudo systemctl enable cnode-mithril-signer\n</code></pre> </li> </ol>"},{"location":"Scripts/blockperf/","title":"BlockPerf","text":"<p>Reminder !!</p> <p>Ensure the Pre-Requisites are in place before you proceed.</p> <p><code>blockPerf.sh</code> is a script to monitor the network propagation of new blocks as seen by the local cardano-node.  </p>"},{"location":"Scripts/blockperf/#block-propagation-traces","title":"Block propagation traces","text":"<p>Although blockPerf can also run on the block producer, it makes the most sense to run it on the upstream relays. There it waits for each new block announced to the relay over the network by its remote peers. </p> <p>It looks for the delay times that result</p> <ul> <li>from the theoretical slot time of the block generator</li> <li>until the block header was offered to the local node</li> <li>the node requested the block </li> <li>the node downloaded the block</li> <li>the node has verified and adopted the block</li> </ul> <p>You can view this data locally as a console stream, or run it as a systemd service in background. </p> <p>BlockPerf also sends this data to the TopologyUpdater server, so that there is a possibility to compare this data (similar to sendtip to pooltool). As a contributing operator you get the possibility to see how your own relays compare to other nodes regarding receive quality, delay times and thus performance. </p> <p>There is no connection or constraint between the TopologyUpdater Relay subscription and the BlockPerf analysis. BlockPerf is even designed to work outside the cnTools suite. </p> <p>The results of these data are a good basis to make optimizations and to evaluate which changes were useful or might by required to improve the performance compared to other relay nodes.</p>"},{"location":"Scripts/blockperf/#installation","title":"Installation","text":"<p>The script is best run as a background process. This can be accomplished in many ways but the preferred method is to run it as a systemd service. A terminal multiplexer like tmux or screen could also be used but not covered here.</p>"},{"location":"Scripts/blockperf/#run-as-service","title":"Run as service","text":"<p>Use the <code>deploy-as-systemd.sh</code> script to create a systemd unit file. In this setup the script is started in \"service\" mode. Error/Warn level log output is handled by journald. <code>journalctl -f -u cnode-tu-blockperf.service</code> can be used to check service output (follow mode). </p> <p>Outside the cnTools environment call <code>blockPerf.sh -d</code> to install it as a systemd service. </p>"},{"location":"Scripts/blockperf/#console-view","title":"Console view","text":"<p>If you run blockPerf local in the console (<code>scripts/blockPerf.sh</code>) , immediately after the appearance of a new block it shows where it came from, how many slots away from the previous block it was, and how many milliseconds the individual steps took.</p> <pre><code>Block:.... 6860534\n Slot..... 52833850 (+59s)\n ......... 2022-02-09 09:49:01\n Header... 2022-02-09 09:49:02,780 (+1780 ms)\n Request.. 2022-02-09 09:49:02,780 (+0 ms)\n Block.... 2022-02-09 09:49:02,830 (+50 ms)\n Adopted.. 2022-02-09 09:49:02,900 (+70 ms)\n Size..... 79976 bytes\n delay.... 1.819971868 sec\n From..... 104.xxx.xxx.61:3001\n\nBlock:.... 6860535\n Slot..... 52833857 (+7s)\n ......... 2022-02-09 09:49:08\n Header... 2022-02-09 09:49:08,960 (+960 ms)\n Request.. 2022-02-09 09:49:08,970 (+10 ms)\n Block.... 2022-02-09 09:49:09,020 (+50 ms)\n Adopted.. 2022-02-09 09:49:09,090 (+70 ms)\n Size..... 64950 bytes\n delay.... 1.028341023 sec\n From..... 34.xxx.xxx.15:4001\n</code></pre>"},{"location":"Scripts/blockperf/#collaborative-web-view","title":"Collaborative web view","text":"<p>A further aim of the blockPerf project is to use the data that individual nodes send to the central TopologyUpdater database to produce graphical visualisations and evaluations that provide the participating node operators with useful insights into their performance compared to all others.</p> <p></p>"},{"location":"Scripts/cncli/","title":"CNCLI","text":"<p>Reminder !!</p> <p>Ensure the Pre-Requisites are in place before you proceed.</p> <p><code>cncli.sh</code> is a script to download and deploy CNCLI created and maintained by Andrew Westberg. It's a community-based CLI tool written in RUST for low-level <code>cardano-node</code> communication. Usage is optional and no script is dependent on it. The main features include:</p> <ul> <li>PING - Validates that the remote server is on the given network and returns its response time. Utilized by <code>gLiveView</code> for peer analysis if available. </li> <li>SYNC - Connects to a node (local or remote) and synchronizes blocks to a local <code>sqlite</code> database. </li> <li>VALIDATE - Validates that a block hash or partial block hash is on-chain.</li> <li>LEADERLOG - Calculates a stakepool's expected slot list. On the mainnet and the official testnet, the next epoch's leader schedule is available 1.5 days before the end of the epoch (<code>firstSlotOfNextEpoch - (3 * k / f)</code>).</li> <li>SENDTIP - Send node tip to PoolTool for network analysis and to show that your node is alive and well with a green badge.</li> <li>SENDSLOTS - Securely sends PoolTool the number of slots you have assigned for an epoch and validates the correctness of your past epochs.</li> </ul>"},{"location":"Scripts/cncli/#installation","title":"Installation","text":"<p><code>cncli.sh</code> script's main functions, <code>sync</code>, <code>leaderlog</code>, <code>validate</code> and PoolTool <code>sendslots</code>/<code>sendtip</code> are not meant to be run manually, but instead deployed as systemd services that run in the background to do the block scraping and validation automatically. Additional commands exist for manual execution to initiate the <code>sqlite</code> db, filling the blocklog DB with all blocks created by the pool known to the blockchain, migration of old cntoolsBlockCollector JSON blocklog, and re-validation of blocks and leaderlogs. See usage output below for a complete list of available commands.</p> <p>The script works in tandem with Log Monitor to provide faster adopted status but mainly to catch slots the node is leader for but are unable to create a block for. These are marked as invalid. Blocklog will however work fine without the <code>logMonitor</code> service and <code>CNCLI</code> is able to handle everything except catching invalid blocks.</p> <ol> <li>Run the latest version of <code>guild-deploy.sh</code> with <code>guild-deploy.sh -s c</code> to download and install RUST and CNCLI. IOG fork of libsodium required by CNCLI is automatically compiled by CNCLI build process. If a previous installation is found, RUST and CNCLI will be updated to the latest version.</li> <li>Run <code>deploy-as-systemd.sh</code> to deploy the systemd services that handle all the work in the background. Six systemd services in total are deployed whereof four are related to CNCLI. See above for the different purposes they serve.</li> <li> <p>If you want to disable some of the deployed services, run <code>sudo systemctl disable &lt;service&gt;</code></p> </li> <li> <p><code>cnode.service</code> (main <code>cardano-node</code> launcher)</p> </li> <li><code>cnode-cncli-sync.service</code></li> <li><code>cnode-cncli-leaderlog.service</code></li> <li><code>cnode-cncli-validate.service</code></li> <li><code>cnode-cncli-ptsendtip.service</code></li> <li><code>cnode-cncli-ptsendslots.service</code></li> <li><code>cnode-logmonitor.service</code> (see Log Monitor)</li> </ol>"},{"location":"Scripts/cncli/#configuration","title":"Configuration","text":"<p>You can override the values in the script at the User Variables section shown below. POOL_ID, POOL_VRF_SKEY and POOL_VRF_VKEY should automatically be detected if <code>POOL_NAME</code> is set in the common <code>env</code> file and can be left commented. PT_API_KEY and POOL_TICKER need to be set in the script if PoolTool <code>sendtip</code>/<code>sendslots</code> are to be used before starting the services. For the rest of the commented values, if the defaults do not provide the right values, uncomment and make adjustments.</p> <pre><code>#POOL_ID=\"\"                               # Automatically detected if POOL_NAME is set in env. Required for leaderlog calculation &amp; pooltool sendtip, lower-case hex pool id\n#POOL_VRF_SKEY=\"\"                         # Automatically detected if POOL_NAME is set in env. Required for leaderlog calculation, path to pool's vrf.skey file\n#POOL_VRF_VKEY=\"\"                         # Automatically detected if POOL_NAME is set in env. Required for block validation, path to pool's vrf.vkey file\n#PT_API_KEY=\"\"                            # POOLTOOL sendtip: set API key, e.g \"a47811d3-0008-4ecd-9f3e-9c22bdb7c82d\"\n#POOL_TICKER=\"\"                           # POOLTOOL sendtip: set the pools ticker, e.g. \"TCKR\"\n#PT_HOST=\"127.0.0.1\"                      # POOLTOOL sendtip: connect to a remote node, preferably block producer (default localhost)\n#PT_PORT=\"${CNODE_PORT}\"                  # POOLTOOL sendtip: port of node to connect to (default is CNODE_PORT from the env file)\n#CNCLI_DIR=\"${CNODE_HOME}/guild-db/cncli\" # path to the directory for cncli sqlite db\n#SLEEP_RATE=60                            # CNCLI leaderlog/validate: time to wait until next check (in seconds)\n#CONFIRM_SLOT_CNT=600                     # CNCLI validate: require at least these many slots to have passed before validating\n#CONFIRM_BLOCK_CNT=15                     # CNCLI validate: require at least these many blocks on top of minted before validating\n#TIMEOUT_LEDGER_STATE=300                 # CNCLI leaderlog: timeout in seconds for ledger-state query\n#BATCH_AUTO_UPDATE=N                      # Set to Y to automatically update the script if a new version is available without user interaction\n</code></pre>"},{"location":"Scripts/cncli/#run","title":"Run","text":"<p>Services are controlled by <code>sudo systemctl &lt;status|start|stop|restart&gt; &lt;service name&gt;</code> All services are configured as child services to <code>cnode.service</code> and as such, when an action is taken against this service it's replicated to all child services. E.g running <code>sudo systemctl start cnode.service</code> will also start all child services. </p> <p>Log output is handled by journald. <code>journalctl -f -u &lt;service&gt;</code> can be used to check service output (follow mode). Other logging configurations are not covered here. </p> <p>Recommended workflow to get started with CNCLI blocklog.</p> <ol> <li>Install and deploy services according to Installation section.</li> <li>Set required user variables according to Configuration section.</li> <li>(optional) If a previous blocklog db exist created by cntoolsBlockCollector, run this command to migrate json storage to new SQLite DB:</li> <li><code>$CNODE_HOME/scripts/cncli.sh migrate &lt;path&gt;</code> where  is the location to the directory containing all blocks_.json files. <li>Start deployed services with:</li> <li><code>sudo systemctl start cnode-cncli-sync.service</code> (starts <code>leaderlog</code>, <code>validate</code> &amp; <code>ptsendslots</code> automatically)</li> <li><code>sudo systemctl start cnode-logmonitor.service</code></li> <li><code>sudo systemctl start cnode-cncli-ptsendtip.service</code> (optional but recommended)</li> <li>alternatively restart the main service that will trigger a start of all services with:</li> <li><code>sudo systemctl restart cnode.service</code></li> <li>Run init command to fill the db with all blocks made by your pool known to the blockchain</li> <li><code>$CNODE_HOME/scripts/cncli.sh init</code></li> <li>Enjoy full blocklog automation and visit View Blocklog section for instructions on how to show blocks from the blocklog DB.</li> <pre><code>Usage: cncli.sh [operation &lt;sub arg&gt;]\nScript to run CNCLI, best launched through systemd deployed by 'deploy-as-systemd.sh'\n\nsync        Start CNCLI chainsync process that connects to cardano-node to sync blocks stored in SQLite DB (deployed as service)\nleaderlog   One-time leader schedule calculation for current epoch, then continuously monitors and calculates schedule for coming epochs, 1.5 days before epoch boundary on the mainnet (deployed as service)\n  force     Manually force leaderlog calculation and overwrite even if already done, exits after leaderlog is calculated\nvalidate    Continuously monitor and confirm that the blocks made actually was accepted and adopted by chain (deployed as service)\n  all       One-time re-validation of all blocks in blocklog db\n  epoch     One-time re-validation of blocks in blocklog db for the specified epoch \nptsendtip   Send node tip to PoolTool for network analysis and to show that your node is alive and well with a green badge (deployed as service)\nptsendslots Securely sends PoolTool the number of slots you have assigned for an epoch and validates the correctness of your past epochs (deployed as service)\ninit        One-time initialization adding all minted and confirmed blocks to blocklog\nmigrate     One-time migration from old blocklog (cntoolsBlockCollector) to new format (post cncli)\n  path      Path to the old cntoolsBlockCollector blocklog folder holding json files with blocks created\n</code></pre>"},{"location":"Scripts/cncli/#view-blocklog","title":"View Blocklog","text":"<p>Best and easiest viewed in CNTools and <code>gLiveView</code> but the blocklog database is a SQLite DB so if you are comfortable with SQL, the <code>sqlite3</code> command can be used to query the DB. </p> <p>Block status</p> <pre><code>- Leader    : Scheduled to make block at this slot\n- Ideal     : Expected/Ideal number of blocks assigned based on active stake (sigma)\n- Luck      : Leader slots assigned vs ideal slots for this epoch\n- Adopted   : Block created successfully\n- Confirmed : Block created validated to be on-chain with the certainty set in `cncli.sh` for `CONFIRM_BLOCK_CNT`\n- Missed    : Scheduled at slot but no record of it in CNCLI DB and no other pool has made a block for this slot\n- Ghosted   : Block created but marked as orphaned and no other pool has made a valid block for this slot -&gt; height battle or block propagation issue\n- Stolen    : Another pool has a valid block registered on-chain for the same slot\n- Invalid   : Pool failed to create block, base64 encoded error message can be decoded with `echo &lt;base64 hash&gt; | base64 -d | jq -r`\n</code></pre> CNTools <p>Open CNTools and select <code>[b] Blocks</code> to open the block viewer. Either select <code>Epoch</code> and enter the epoch you want to see a detailed view for or choose <code>Summary</code> to display blocks for last x epochs.</p> <p>If the node was elected to create blocks in the selected epoch it could look something like this:</p> Summary <pre><code> &gt;&gt; BLOCKS\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\nCurrent epoch: 96\n\n+--------+---------------------------+----------------------+--------------------------------------+\n| Epoch  | Leader | Ideal | Luck     | Adopted | Confirmed  | Missed | Ghosted | Stolen | Invalid  |\n+--------+---------------------------+----------------------+--------------------------------------+\n| 96     | 34     | 31.66 | 107.39%  | 18      | 18         | 0      | 0       | 0      | 0        |\n| 95     | 32     | 30.57 | 104.68%  | 32      | 32         | 0      | 0       | 0      | 0        |\n+--------+---------------------------+----------------------+--------------------------------------+\n\n[h] Home | [b] Block View | [i] Info | [*] Refresh\n</code></pre> Epoch <pre><code> &gt;&gt; BLOCKS\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\nCurrent epoch: 96\n\n+---------------------------+----------------------+--------------------------------------+\n| Leader | Ideal | Luck     | Adopted | Confirmed  | Missed | Ghosted | Stolen | Invalid  |\n+---------------------------+----------------------+--------------------------------------+\n| 34     | 31.66 | 107.39%  | 18      | 18         | 0      | 0       | 0      | 0        |\n+---------------------------+----------------------+--------------------------------------+\n\n+-----+------------+----------+---------------------+--------------------------+-------+-------------------------------------------------------------------+\n| #   | Status     | Block    | Slot | SlotInEpoch  | Scheduled At             | Size  | Hash                                                              |\n+-----+------------+----------+---------------------+--------------------------+-------+-------------------------------------------------------------------+\n| 1   | confirmed  | 2043444  | 11142827 | 40427    | 2020-11-16 08:34:03 CET  | 3     | ec216d3fb01e4a3cc3e85305145a31875d9561fa3bbcc6d0ee8297236dbb4115  |\n| 2   | confirmed  | 2044321  | 11165082 | 62682    | 2020-11-16 14:44:58 CET  | 3     | b75c33a5bbe49a74e4b4cc5df4474398bfb10ed39531fc65ec2acc51f89ddce5  |\n| 3   | confirmed  | 2044397  | 11166970 | 64570    | 2020-11-16 15:16:26 CET  | 3     | c1ea37fd72543779b6dab46e3e29e0e422784b5fd6188f828ace9eabcc87088f  |\n| 4   | confirmed  | 2044879  | 11178909 | 76509    | 2020-11-16 18:35:25 CET  | 3     | 35a116cec80c5dc295415e4fc8e6435c562b14a5d6833027006c988706c60307  |\n| 5   | confirmed  | 2046965  | 11232557 | 130157   | 2020-11-17 09:29:33 CET  | 3     | d566e5a1f6a3d78811acab4ae3bdcee6aa42717364f9afecd6cac5093559f466  |\n| 6   | confirmed  | 2047101  | 11235675 | 133275   | 2020-11-17 10:21:31 CET  | 3     | 3a638e01f70ea1c4a660fe4e6333272e6c61b11cf84dc8a5a107b414d1e057eb  |\n| 7   | confirmed  | 2047221  | 11238453 | 136053   | 2020-11-17 11:07:49 CET  | 3     | 843336f132961b94276603707751cdb9a1c2528b97100819ce47bc317af0a2d6  |\n| 8   | confirmed  | 2048692  | 11273507 | 171107   | 2020-11-17 20:52:03 CET  | 3     | 9b3eb79fe07e8ebae163870c21ba30460e689b23768d2e5f8e7118c572c4df36  |\n| 9   | confirmed  | 2049058  | 11282619 | 180219   | 2020-11-17 23:23:55 CET  | 3     | 643396ea9a1a2b6c66bb83bdc589fa19c8ae728d1f1181aab82e8dfe508d430a  |\n| 10  | confirmed  | 2049321  | 11289237 | 186837   | 2020-11-18 01:14:13 CET  | 3     | d93d305a955f40b2298247d44e4bc27fe9e3d1486ef3ef3e73b235b25247ccd7  |\n| 11  | confirmed  | 2049747  | 11299205 | 196805   | 2020-11-18 04:00:21 CET  | 3     | 19a43deb5014b14760c3e564b41027c5ee50e0a252abddbfcac90c8f56dc0245  |\n| 12  | confirmed  | 2050415  | 11316075 | 213675   | 2020-11-18 08:41:31 CET  | 3     | dd2cb47653f3bfb3ccc8ffe76906e07d96f1384bafd57a872ddbab3b352403e3  |\n| 13  | confirmed  | 2050505  | 11318274 | 215874   | 2020-11-18 09:18:10 CET  | 3     | deb834bc42360f8d39eefc5856bb6d7cabb6b04170c842dcbe7e9efdf9dbd2e1  |\n| 14  | confirmed  | 2050613  | 11320754 | 218354   | 2020-11-18 09:59:30 CET  | 3     | bf094f6fde8e8c29f568a253201e4b92b078e9a1cad60706285e236a91ec95ff  |\n| 15  | confirmed  | 2050807  | 11325239 | 222839   | 2020-11-18 11:14:15 CET  | 3     | 21f904346ba0fd2bb41afaae7d35977cb929d1d9727887f541782576fc6a62c9  |\n| 16  | confirmed  | 2050997  | 11330062 | 227662   | 2020-11-18 12:34:38 CET  | 3     | 109799d686fe3cad13b156a2d446a544fde2bf5d0e8f157f688f1dc30f35e912  |\n| 17  | confirmed  | 2051286  | 11336791 | 234391   | 2020-11-18 14:26:47 CET  | 3     | bb1beca7a1d849059110e3d7dc49ecf07b47970af2294fe73555ddfefb9561a8  |\n| 18  | confirmed  | 2051734  | 11348498 | 246098   | 2020-11-18 17:41:54 CET  | 3     | 87940b53c2342999c1ba4e185038cda3d8382891a16878a865f5114f540683de  |\n| 19  | leader     | -        | 11382001 | 279601   | 2020-11-19 03:00:17 CET  | -     | -                                                                 |\n| 20  | leader     | -        | 11419959 | 317559   | 2020-11-19 13:32:55 CET  | -     | -                                                                 |\n| 21  | leader     | -        | 11433174 | 330774   | 2020-11-19 17:13:10 CET  | -     | -                                                                 |\n| 22  | leader     | -        | 11434241 | 331841   | 2020-11-19 17:30:57 CET  | -     | -                                                                 |\n| 23  | leader     | -        | 11435289 | 332889   | 2020-11-19 17:48:25 CET  | -     | -                                                                 |\n| 24  | leader     | -        | 11440314 | 337914   | 2020-11-19 19:12:10 CET  | -     | -                                                                 |\n| 25  | leader     | -        | 11442361 | 339961   | 2020-11-19 19:46:17 CET  | -     | -                                                                 |\n| 26  | leader     | -        | 11443861 | 341461   | 2020-11-19 20:11:17 CET  | -     | -                                                                 |\n| 27  | leader     | -        | 11446997 | 344597   | 2020-11-19 21:03:33 CET  | -     | -                                                                 |\n| 28  | leader     | -        | 11453110 | 350710   | 2020-11-19 22:45:26 CET  | -     | -                                                                 |\n| 29  | leader     | -        | 11455323 | 352923   | 2020-11-19 23:22:19 CET  | -     | -                                                                 |\n| 30  | leader     | -        | 11505987 | 403587   | 2020-11-20 13:26:43 CET  | -     | -                                                                 |\n| 31  | leader     | -        | 11514983 | 412583   | 2020-11-20 15:56:39 CET  | -     | -                                                                 |\n| 32  | leader     | -        | 11516010 | 413610   | 2020-11-20 16:13:46 CET  | -     | -                                                                 |\n| 33  | leader     | -        | 11518958 | 416558   | 2020-11-20 17:02:54 CET  | -     | -                                                                 |\n| 34  | leader     | -        | 11533254 | 430854   | 2020-11-20 21:01:10 CET  | -     | -                                                                 |\n+-----+------------+----------+---------------------+--------------------------+-------+-------------------------------------------------------------------+\n</code></pre> gLiveView <p>Currently shows a block summary for current epoch. For full block details use CNTools for now. Invalid, missing, ghosted and stolen blocks only shown in case of a non-zero value.</p> <pre><code>\u2502--------------------------------------------------------------\u2502\n\u2502 BLOCKS   Leader  | Ideal  | Luck    | Adopted | Confirmed    \u2502\n\u2502          24        27.42    87.53%    1         1            \u2502\n\u2502          08:07:57 until leader XXXXXXXXX.....................\u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n</code></pre>"},{"location":"Scripts/cntools-changelog/","title":"Changelog","text":"<p>All notable changes to this tool will be documented in this file.</p> <p>Whenever you're updating between versions where format/hash of keys have changed , or you're changing networks - it is recommended to Backup your Wallet and Pool folders before you proceed with launching cntools on a fresh network.</p> <p>The format is based on Keep a Changelog, and this adheres to Semantic Versioning.</p>"},{"location":"Scripts/cntools-changelog/#1310-2024-08-01","title":"[13.1.0] - 2024-08-01","text":""},{"location":"Scripts/cntools-changelog/#added","title":"Added","text":"<ul> <li>New Vote menu option to hold everything related to voting on Cardano.</li> <li>Governance features according to CIP-1694 with Conway era.</li> <li>Catalyst vote registration.</li> <li>MultiSig wallet support through advanced menu item.</li> <li>Creation of new mnemonic(seed phrase) generated wallet.</li> <li>Custom mnemonic derivation path on mnemonic/hardware import.</li> <li>For a more in-depth list, see https://github.com/cardano-community/guild-operators/pull/1783</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed","title":"Changed","text":"<ul> <li>Removed CIP-0095 SPO poll</li> </ul>"},{"location":"Scripts/cntools-changelog/#1302-2024-06-07","title":"[13.0.2] - 2024-06-07","text":""},{"location":"Scripts/cntools-changelog/#fixed","title":"Fixed","text":"<ul> <li>Mnemonic import.</li> </ul>"},{"location":"Scripts/cntools-changelog/#1300-2024-06-07","title":"[13.0.0] - 2024-06-07","text":""},{"location":"Scripts/cntools-changelog/#added_1","title":"Added","text":"<ul> <li>New light mode using Koios API instead of local node for blockchain queries. Mode can be selected at runtime using -l (light), -n (node, default) or -o (offline).</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_1","title":"Changed","text":"<ul> <li>Improved submission handling</li> <li>Removed complicated file descriptor re-direction for logging. This simplified the code and improves screen printing speed as special hacks around FDs could be removed.</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_1","title":"Fixed","text":"<ul> <li>Update handling and reload of script.</li> </ul>"},{"location":"Scripts/cntools-changelog/#1210-2024-01-19","title":"[12.1.0] - 2024-01-19","text":""},{"location":"Scripts/cntools-changelog/#changed_2","title":"Changed","text":"<ul> <li>ADA price discovery through CoinGecko in set currency incl 24hr change. Shown in main UI, wallet list &amp; show as well as pool show. Disabled by default.</li> <li>Convert calculate-min-fee command references to function</li> </ul>"},{"location":"Scripts/cntools-changelog/#1202-2024-02-11","title":"[12.0.2] - 2024-02-11","text":""},{"location":"Scripts/cntools-changelog/#fixed_2","title":"Fixed","text":"<ul> <li>Retiring pools should be modifiable in case the user wants to cancel the pending retirement.</li> </ul>"},{"location":"Scripts/cntools-changelog/#1201-2024-01-26","title":"[12.0.1] - 2024-01-26","text":""},{"location":"Scripts/cntools-changelog/#fixed_3","title":"Fixed","text":"<ul> <li>Funds &gt; Send had a regression introduced in 12.0.0 thats now fixed (thanks Homer :)</li> </ul>"},{"location":"Scripts/cntools-changelog/#1200-2024-01-19","title":"[12.0.0] - 2024-01-19","text":""},{"location":"Scripts/cntools-changelog/#changed_3","title":"Changed","text":"<ul> <li>Minimum version bumps</li> <li>node 8.7.3</li> <li>cardano-hw-cli 1.14.0</li> <li>Ledger 5.0.1</li> <li>Trezor 2.6.4</li> <li>Send &gt; Funds flow updated to ask for Ada amount after tokens and destination to be able to calculate minimum amount valid to send correctly.</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_4","title":"Fixed","text":"<ul> <li>Minimum utxo calculation changed to use cardano-cli calculation tool (was broken)</li> </ul>"},{"location":"Scripts/cntools-changelog/#1102-2023-10-30","title":"[11.0.2] - 2023-10-30","text":""},{"location":"Scripts/cntools-changelog/#fixed_5","title":"Fixed","text":"<ul> <li>Fix additional Ada printing. Now omits trailing zeros from fraction part of Ada output.</li> </ul>"},{"location":"Scripts/cntools-changelog/#1101-2023-10-25","title":"[11.0.1] - 2023-10-25","text":""},{"location":"Scripts/cntools-changelog/#fixed_6","title":"Fixed","text":"<ul> <li>Fix display for Pool Cost and Pledge to accept integer as well as decimal format of ADA</li> </ul>"},{"location":"Scripts/cntools-changelog/#1100-2023-07-05","title":"[11.0.0] - 2023-07-05","text":""},{"location":"Scripts/cntools-changelog/#changed_4","title":"Changed","text":"<ul> <li>CNTools now part of Koios brand</li> </ul>"},{"location":"Scripts/cntools-changelog/#1040-2023-06-19","title":"[10.4.0] - 2023-06-19","text":""},{"location":"Scripts/cntools-changelog/#added_2","title":"Added","text":"<ul> <li>Support for SRV records</li> <li>Support for cardano-node 8.1.1</li> </ul>"},{"location":"Scripts/cntools-changelog/#1031-2023-06-03","title":"[10.3.1] - 2023-06-03","text":""},{"location":"Scripts/cntools-changelog/#fixed_7","title":"Fixed","text":"<ul> <li>Backup didn't properly exclude private keys</li> </ul>"},{"location":"Scripts/cntools-changelog/#1030-2023-05-18","title":"[10.3.0] - 2023-05-18","text":""},{"location":"Scripts/cntools-changelog/#added_3","title":"Added","text":"<ul> <li>Support for voting as per CIP-0094</li> </ul>"},{"location":"Scripts/cntools-changelog/#1023-2023-04-28","title":"[10.2.3] - 2023-04-28","text":""},{"location":"Scripts/cntools-changelog/#fixed_8","title":"Fixed","text":"<ul> <li>Additional HW signing fixes</li> </ul>"},{"location":"Scripts/cntools-changelog/#1022-2023-04-24","title":"[10.2.2] - 2023-04-24","text":""},{"location":"Scripts/cntools-changelog/#fixed_9","title":"Fixed","text":"<ul> <li>Add special case handling for hardware wallets to use stake keys as witness for registering stake address</li> </ul>"},{"location":"Scripts/cntools-changelog/#1021-2023-04-04","title":"[10.2.1] - 2023-04-04","text":""},{"location":"Scripts/cntools-changelog/#fixed_10","title":"Fixed","text":"<ul> <li>Moved <code>test_koios</code> call from cntools.library to cntools.sh</li> </ul>"},{"location":"Scripts/cntools-changelog/#1020-2023-03-13","title":"[10.2.0] - 2023-03-13","text":""},{"location":"Scripts/cntools-changelog/#fixed_11","title":"Fixed","text":"<ul> <li>HW signing fix due to deprecated cardano-hw-cli sign call.</li> <li>The check whether to use Koios API or not (env config) wasn't properly handled.</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_5","title":"Changed","text":"<ul> <li>Disabled Koios for balance lookup to prefer local node check. In most circumstances this will be faster due to low latency. If needed, set WALLET_SELECTION_FILTER_LIMIT in cntools.sh to a lower limit to skip balance lookup on wallet selection if you have many wallets and delay is too long.</li> </ul>"},{"location":"Scripts/cntools-changelog/#1011-2023-02-07","title":"[10.1.1] - 2023-02-07","text":""},{"location":"Scripts/cntools-changelog/#fixed_12","title":"Fixed","text":"<ul> <li>Disable <code>dialog</code> by default, it is an optional component - and no longer installed by default.</li> </ul>"},{"location":"Scripts/cntools-changelog/#1010-2023-01-17","title":"[10.1.0] - 2023-01-17","text":""},{"location":"Scripts/cntools-changelog/#added_4","title":"Added","text":"<ul> <li>Hardware Wallets: Allow signing using cold keys for a pool, use it for rotating KES keys.</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_6","title":"Changed","text":"<ul> <li>Keep deployment consistent with guild-deploy.sh</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_13","title":"Fixed","text":"<ul> <li>Fix parsing space in the name of assets</li> </ul>"},{"location":"Scripts/cntools-changelog/#1005-2022-11-07","title":"[10.0.5] - 2022-11-07","text":""},{"location":"Scripts/cntools-changelog/#changed_7","title":"Changed","text":"<ul> <li>Updated testnet token registry to be reused for each non-mainnet network</li> <li>Remove stale code for remote chain analysis</li> </ul>"},{"location":"Scripts/cntools-changelog/#1004-2022-08-26","title":"[10.0.4] - 2022-08-26","text":""},{"location":"Scripts/cntools-changelog/#changed_8","title":"Changed","text":"<ul> <li>Allow pool cost to use fraction of ADA</li> <li>Starts using koios-1.0.7 endpoints to fetch information</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_14","title":"Fixed","text":"<ul> <li>Fixes an issue with reusage of variable name and updated param name for cardano-cli.</li> <li>Fix token minting and burn assets</li> </ul>"},{"location":"Scripts/cntools-changelog/#1003-2022-08-16","title":"[10.0.3] - 2022-08-16","text":""},{"location":"Scripts/cntools-changelog/#fixed_15","title":"Fixed","text":"<ul> <li>env file was sourced after calling cntools.library, overriding test_koios result</li> </ul>"},{"location":"Scripts/cntools-changelog/#1002-2022-08-13","title":"[10.0.2] - 2022-08-13","text":""},{"location":"Scripts/cntools-changelog/#fixed_16","title":"Fixed","text":"<ul> <li>Bump min cardano-hw-cli version to 1.10.0</li> <li>Requires cardano-hw-cli to be present on online node for pool registration/modification to be able to transform tx if needed</li> <li>Transform tx if needed before any witnessing/signing is done.</li> <li>Wrong arguments in call to cardano-hw-cli for cddl-formatted tx</li> </ul>"},{"location":"Scripts/cntools-changelog/#1001-2022-07-14","title":"[10.0.1] - 2022-07-14","text":""},{"location":"Scripts/cntools-changelog/#changed_9","title":"Changed","text":"<ul> <li>Transactions now built using cddl-format to ensure that the formatting of transaction adheres the ledger specs.</li> <li>Default to mary era transaction building format for now.</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_17","title":"Fixed","text":"<ul> <li>Cold signing fix for pool registration / update. Last key was added twice when assemling witnesses.</li> </ul>"},{"location":"Scripts/cntools-changelog/#1000-2022-06-28","title":"[10.0.0] - 2022-06-28","text":""},{"location":"Scripts/cntools-changelog/#added_5","title":"Added","text":"<ul> <li>Support for Vasil Fork</li> <li>Preliminary support for Post HF updates (a short release will follow post fork in coming days)</li> <li>Minimum version for Node bumped to 1.35.0</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_10","title":"Changed","text":"<ul> <li>Pool &gt; Rotate code now uses kes-periodinfo CLI query to get counter from node (fallback for Koios)</li> <li>Pool &gt; Show Info updated to include current KES counter</li> <li>Update getEraIdentifier to include Babbage era</li> </ul>"},{"location":"Scripts/cntools-changelog/#910-2022-05-11","title":"[9.1.0] - 2022-05-11","text":""},{"location":"Scripts/cntools-changelog/#changed_11","title":"Changed","text":"<ul> <li>Harmonize flow for reusing old wallet configuration on pool modification vs setting new wallets.</li> <li>Remove the requirement for reward stake signing key in wallet registration/modification</li> <li>Reward wallet no longer auto-delegated on pool registration just like for multi-owners. </li> </ul>"},{"location":"Scripts/cntools-changelog/#9010-2022-05-03","title":"[9.0.10] - 2022-05-03","text":""},{"location":"Scripts/cntools-changelog/#fixed_18","title":"Fixed","text":"<ul> <li>Detect if cardano-hw-cli has execution permission</li> </ul>"},{"location":"Scripts/cntools-changelog/#909-2022-03-14","title":"[9.0.9] - 2022-03-14","text":""},{"location":"Scripts/cntools-changelog/#changed_12","title":"Changed","text":"<ul> <li>Add version (-v) argument to cntools script to print current version</li> </ul>"},{"location":"Scripts/cntools-changelog/#908-2022-03-07","title":"[9.0.8] - 2022-03-07","text":""},{"location":"Scripts/cntools-changelog/#changed_13","title":"Changed","text":"<ul> <li>Remove HASH_IDENTIFIER variable references (Ddz issue which required this seperation was resolved a while ago)</li> <li>Replace NETWORKID check with NWMAGIC variable</li> </ul>"},{"location":"Scripts/cntools-changelog/#907-2022-03-02","title":"[9.0.7] - 2022-03-02","text":""},{"location":"Scripts/cntools-changelog/#fixed_19","title":"Fixed","text":"<ul> <li>Call Test Koios function at start of CNTools, instead of calling by default every time env is sourced</li> </ul>"},{"location":"Scripts/cntools-changelog/#906-2022-02-20","title":"[9.0.6] - 2022-02-20","text":""},{"location":"Scripts/cntools-changelog/#fixed_20","title":"Fixed","text":"<ul> <li>Fix for update check if not executed from default scripts folder.</li> </ul>"},{"location":"Scripts/cntools-changelog/#905-2022-02-16","title":"[9.0.5] - 2022-02-16","text":""},{"location":"Scripts/cntools-changelog/#fixed_21","title":"Fixed","text":"<ul> <li>Script update code fixed to better handle in-app update. Would sometimes update but not source library correctly.</li> </ul>"},{"location":"Scripts/cntools-changelog/#904-2022-02-14","title":"[9.0.4] - 2022-02-14","text":""},{"location":"Scripts/cntools-changelog/#fixed_22","title":"Fixed","text":"<ul> <li>Update request for pool_info endpoint from Koios</li> </ul>"},{"location":"Scripts/cntools-changelog/#903-2022-02-01","title":"[9.0.3] - 2022-02-01","text":""},{"location":"Scripts/cntools-changelog/#added_6","title":"Added","text":"<ul> <li>Add a config variable TX_TTL to allow transaction to be valid (by default for 3600 slots) from the point of creation - previous default of 10 minutes on mainnet could be hit-and-miss with the state of network.</li> </ul>"},{"location":"Scripts/cntools-changelog/#902-2022-01-22","title":"[9.0.2] - 2022-01-22","text":""},{"location":"Scripts/cntools-changelog/#changed_14","title":"Changed","text":"<ul> <li>Add decimal param to token metadata creator and increase ticker max length to 9 chars according to spec changes.</li> </ul>"},{"location":"Scripts/cntools-changelog/#901-2022-01-17","title":"[9.0.1] - 2022-01-17","text":""},{"location":"Scripts/cntools-changelog/#changed_15","title":"Changed","text":"<ul> <li>Removing tool credits in offline metadata registry due to \"out of protocol\".</li> </ul>"},{"location":"Scripts/cntools-changelog/#900-2022-01-10","title":"[9.0.0] - 2022-01-10","text":""},{"location":"Scripts/cntools-changelog/#changed_16","title":"Changed","text":"<ul> <li>Due to changes in cardano-node 1.33.x -&gt; for utxo ledger lookup and previous heavy pool-params query, Koios API is now the default option for these lookups.</li> <li>You can update KOIOS_API env variable to connect to a local instance of koios (open source and incentivises all to build a decentralised query layer) if you'd not like to connect to remote instance.</li> <li>Visit the https://www.koios.rest/ for more information about Koios or check out the API documentation at https://api.koios.rest.</li> <li>If you'd like to revert to old behaviour (use CLI which could be slow to retrieve UTxOs), you can set ENABLE_KOIOS environment variable to N.</li> </ul>"},{"location":"Scripts/cntools-changelog/#882-2021-12-28","title":"[8.8.2] - 2021-12-28","text":""},{"location":"Scripts/cntools-changelog/#fixed_23","title":"Fixed","text":"<ul> <li>Transform txBody using canonical order before signing/witnessing in case of HW wallet.</li> <li>Bump minimum HW wallet versions:</li> <li>Ledger &gt;= 3.0.0</li> <li>Trezor &gt;= 2.4.3</li> <li>cardano-hw-cli &gt;= 1.9.0</li> </ul>"},{"location":"Scripts/cntools-changelog/#881-2021-12-18","title":"[8.8.1] - 2021-12-18","text":""},{"location":"Scripts/cntools-changelog/#fixed_24","title":"Fixed","text":"<ul> <li>Fallback to Mary era in build commands to keep ledger compatibility</li> </ul>"},{"location":"Scripts/cntools-changelog/#880-2021-12-15","title":"[8.8.0] - 2021-12-15","text":""},{"location":"Scripts/cntools-changelog/#fixed_25","title":"Fixed","text":"<ul> <li>Asset handling after cardano-node 1.32.1 version bump. ascii -&gt; hex change in cardano-cli.</li> </ul>"},{"location":"Scripts/cntools-changelog/#873-2021-11-30","title":"[8.7.3] - 2021-11-30","text":""},{"location":"Scripts/cntools-changelog/#fixed_26","title":"Fixed","text":"<ul> <li>Remove stale cntools.config comments</li> </ul>"},{"location":"Scripts/cntools-changelog/#872-2021-11-08","title":"[8.7.2] - 2021-11-08","text":""},{"location":"Scripts/cntools-changelog/#changed_17","title":"Changed","text":"<ul> <li>Remove check if pool reward wallet is a hw wallet, enforce that its also a multi-owner to the pool</li> </ul>"},{"location":"Scripts/cntools-changelog/#871-2021-11-04","title":"[8.7.1] - 2021-11-04","text":""},{"location":"Scripts/cntools-changelog/#fixed_27","title":"Fixed","text":"<ul> <li>Balance check of wrong wallet in certain circumstances for pool modify</li> </ul>"},{"location":"Scripts/cntools-changelog/#870-2021-10-05","title":"[8.7.0] - 2021-10-05","text":""},{"location":"Scripts/cntools-changelog/#changed_18","title":"Changed","text":"<ul> <li>CNTools configuration moved from cntools.config to cntools.sh</li> </ul>"},{"location":"Scripts/cntools-changelog/#866-2021-09-26","title":"[8.6.6] - 2021-09-26","text":""},{"location":"Scripts/cntools-changelog/#fixed_28","title":"Fixed","text":"<ul> <li>Pool rotation date calculation fix, 8.6.4 didn't properly fix it</li> </ul>"},{"location":"Scripts/cntools-changelog/#865-2021-09-15","title":"[8.6.5] - 2021-09-15","text":""},{"location":"Scripts/cntools-changelog/#fixed_29","title":"Fixed","text":"<ul> <li>Minimum utxo output calculation post Alonzo</li> </ul>"},{"location":"Scripts/cntools-changelog/#864-2021-09-14","title":"[8.6.4] - 2021-09-14","text":""},{"location":"Scripts/cntools-changelog/#fixed_30","title":"Fixed","text":"<ul> <li>Pool rotation date calculation fix (display only)</li> </ul>"},{"location":"Scripts/cntools-changelog/#863-2021-08-31","title":"[8.6.3] - 2021-08-31","text":""},{"location":"Scripts/cntools-changelog/#fixed_31","title":"Fixed","text":"<ul> <li>Pool retire fix</li> </ul>"},{"location":"Scripts/cntools-changelog/#862-2021-08-30","title":"[8.6.2] - 2021-08-30","text":""},{"location":"Scripts/cntools-changelog/#fixed_32","title":"Fixed","text":"<ul> <li>Revert <code>--whole-utxo</code> flag, as it returns all address and will not accept <code>--address</code></li> </ul>"},{"location":"Scripts/cntools-changelog/#861-2021-08-27","title":"[8.6.1] - 2021-08-27","text":""},{"location":"Scripts/cntools-changelog/#changed_19","title":"Changed","text":"<ul> <li>Alonzo related changes for era and minimum utxo.</li> </ul>"},{"location":"Scripts/cntools-changelog/#860-2021-08-27","title":"[8.6.0] - 2021-08-27","text":""},{"location":"Scripts/cntools-changelog/#changed_20","title":"Changed","text":"<ul> <li>Add <code>--whole-utxo</code> flag when query UTxO, as required by cardano-cli 1.28, to keep behaviour same as before.</li> <li>Baseline compatibility with 1.29</li> </ul>"},{"location":"Scripts/cntools-changelog/#8415-2021-07-15","title":"[8.4.15] - 2021-07-15","text":""},{"location":"Scripts/cntools-changelog/#changed_21","title":"Changed","text":"<ul> <li>Switch default to 'No' adding a message when sending funds</li> </ul>"},{"location":"Scripts/cntools-changelog/#8414-2021-07-14","title":"[8.4.14] - 2021-07-14","text":""},{"location":"Scripts/cntools-changelog/#fixed_33","title":"Fixed","text":"<ul> <li>Fix for upcoming unreleased dbsync rest endpoint </li> </ul>"},{"location":"Scripts/cntools-changelog/#8413-2021-07-08","title":"[8.4.13] - 2021-07-08","text":""},{"location":"Scripts/cntools-changelog/#changed_22","title":"Changed","text":"<ul> <li>Documentation references updated to new site layout</li> </ul>"},{"location":"Scripts/cntools-changelog/#8412-2021-06-28","title":"[8.4.12] - 2021-06-28","text":""},{"location":"Scripts/cntools-changelog/#fixed_34","title":"Fixed","text":"<ul> <li>Pre-source env in offline/online mode for checkUpdate depending on argument provided to cntools.sh</li> </ul>"},{"location":"Scripts/cntools-changelog/#8411-2021-06-25","title":"[8.4.11] - 2021-06-25","text":""},{"location":"Scripts/cntools-changelog/#changed_23","title":"Changed","text":"<ul> <li>KES calculation moved from CNTools &amp; gLiveView into a common function in env file. For online mode node metrics is used for KES expiration instead of static pool KES start period.</li> <li>General message metadata support added to 'funds &gt;&gt; send' according to CIP-0020.</li> </ul>"},{"location":"Scripts/cntools-changelog/#8410-2021-06-15","title":"[8.4.10] - 2021-06-15","text":""},{"location":"Scripts/cntools-changelog/#fixed_35","title":"Fixed","text":"<ul> <li>Fix display issue for CLI that were upgraded to Alonzo-Blue networks</li> </ul>"},{"location":"Scripts/cntools-changelog/#849-2021-06-15","title":"[8.4.9] - 2021-06-15","text":""},{"location":"Scripts/cntools-changelog/#changed_24","title":"Changed","text":"<ul> <li>Handle Various updates to grest queries [disabled] to make them independent of instances. Note: Version incremented thrice on PR branch itself</li> </ul>"},{"location":"Scripts/cntools-changelog/#846-2021-06-04","title":"[8.4.6] - 2021-06-04","text":""},{"location":"Scripts/cntools-changelog/#fixed_36","title":"Fixed","text":"<ul> <li>Add balance check for main pool owner, that there is at least one utxo available</li> <li>Allow utxo without lovelace (for future when we might have tokens on utxo without Ada, like on Alonzo TestNet)</li> <li>pctToFraction helper function didn't properly handle 0 value</li> </ul>"},{"location":"Scripts/cntools-changelog/#845-2021-05-31","title":"[8.4.5] - 2021-05-31","text":""},{"location":"Scripts/cntools-changelog/#fixed_37","title":"Fixed","text":"<ul> <li>Reset IFS at main loop, fixes invalid tip difference on home screen after going to Block &gt; Summary</li> </ul>"},{"location":"Scripts/cntools-changelog/#844-2021-05-19","title":"[8.4.4] - 2021-05-19","text":""},{"location":"Scripts/cntools-changelog/#fixed_38","title":"Fixed","text":"<ul> <li>Typo in Ledger ledger version requirement error and make it clearer that its the app version, not fw version.</li> </ul>"},{"location":"Scripts/cntools-changelog/#843-2021-05-17","title":"[8.4.3] - 2021-05-17","text":""},{"location":"Scripts/cntools-changelog/#fixed_39","title":"Fixed","text":"<ul> <li>Token Mint/Burn script file signing not completely removed in all places (1.27.0 change)</li> </ul>"},{"location":"Scripts/cntools-changelog/#842-2021-05-16","title":"[8.4.2] - 2021-05-16","text":""},{"location":"Scripts/cntools-changelog/#fixed_40","title":"Fixed","text":"<ul> <li>cardano-hw-cli version limited to 1.2.0 for current Trezor fw v2.3.6. Please manually downgrade version, available at https://github.com/vacuumlabs/cardano-hw-cli/releases , placing files in $HOME/bin/cardano-hw-cli</li> </ul>"},{"location":"Scripts/cntools-changelog/#841-2021-05-16","title":"[8.4.1] - 2021-05-16","text":""},{"location":"Scripts/cntools-changelog/#changed_25","title":"Changed","text":"<ul> <li>Wallet &gt;&gt; Show no longer require payment.vkey to be present, as long as either payment or base .addr file(s) exist</li> </ul>"},{"location":"Scripts/cntools-changelog/#840-2021-05-16","title":"[8.4.0] - 2021-05-16","text":""},{"location":"Scripts/cntools-changelog/#added_7","title":"Added","text":"<ul> <li>Compatibility with cardano-address 3.4.0 (while retaining support for 2.1.0)</li> </ul>"},{"location":"Scripts/cntools-changelog/#830-2021-05-15","title":"[8.3.0] - 2021-05-15","text":""},{"location":"Scripts/cntools-changelog/#added_8","title":"Added","text":"<ul> <li>New env variable called PGREST_API and if set and reachable, used instead of local node queries and for advanced modes</li> <li>New library function isPoolRegistered() for verifying if a pool is registered or not using either simple reg cert file detection (if REST API not set/reachable) or proper dbsync lookup using REST API. Used by Pool &gt;&gt; Show|List|Register|Modify</li> <li>Option to mint/burn assets in hybrid mode</li> <li>Transaction &gt;&gt; Sign now automatically tries to find the correct signing keys instead of having the user manually select the correct files</li> <li>** ADVANCED FEATURE ** - Chain Queries</li> <li>Menu is dynamically built based on queries(JSON files) in DBSYNC_QUERY_FOLDER (env variable, default files/dbsync/queries) three levels deep.</li> <li>A download option lets the user download the latest uploaded queries on Guild Operators GitHub site.</li> <li>Query files<ul> <li>Contains menu path, description, variables, and queries(multiple)</li> <li>Executes a predefined DBSync RPC/function through PostgREST API</li> <li>Variables used in RPC call can either be user input, CNTools variables like EKG metrics, or an item in the result from a previous query(in the same query file)</li> <li>Result from RPC call can either be printed or silent(only to be used for later query)</li> <li>Output is printed as JSON</li> </ul> </li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_26","title":"Changed","text":"<ul> <li>Minimum node version bumped to 1.27.0</li> <li>Menu has been re-designed with both back &amp; home options. Instead of returning to home menu after the completed operation user is returned to the last menu.</li> <li>Pool &gt;&gt; Show now use PostgREST API(if set), or the new pool-params cli query as fallback method.</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_41","title":"Fixed","text":"<ul> <li>1.27.0 introduced a few changes in CLI commands for assets minting/burning</li> </ul>"},{"location":"Scripts/cntools-changelog/#822-2021-05-02","title":"[8.2.2] - 2021-05-02","text":""},{"location":"Scripts/cntools-changelog/#fixed_42","title":"Fixed","text":"<ul> <li>KES expiration date fix</li> </ul>"},{"location":"Scripts/cntools-changelog/#821-2021-04-26","title":"[8.2.1] - 2021-04-26","text":""},{"location":"Scripts/cntools-changelog/#changed_27","title":"Changed","text":"<ul> <li>Make use of UPDATE_CHECK environment variable to skip any checks to internet by default</li> </ul>"},{"location":"Scripts/cntools-changelog/#820-2021-04-18","title":"[8.2.0] - 2021-04-18","text":""},{"location":"Scripts/cntools-changelog/#added_9","title":"Added","text":"<ul> <li>Ability to create &amp; update a Cardano Token Registry submission JSON file</li> <li>Requires 'token-metadata-creator' tool, instructions to download/build this tool added to Guild Operators documentation:</li> <li>https://cardano-community.github.io/guild-operators/Build/offchainMetadataTools</li> <li>Token Registry lookup in Wallet &gt;&gt; Show</li> <li>Token asset fingerprint generation according to https://github.com/cardano-foundation/CIPs/pull/64</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_28","title":"Changed","text":"<ul> <li>Redesigned input handling to be more flexible and improve output</li> </ul>"},{"location":"Scripts/cntools-changelog/#816-2021-04-14","title":"[8.1.6] - 2021-04-14","text":""},{"location":"Scripts/cntools-changelog/#changed_29","title":"Changed","text":"<ul> <li>Metadata creation now offer the choice to add a metadata JSON scaffold to see the required structure</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_43","title":"Fixed","text":"<ul> <li>Fixed metadata creation entering JSON metadata through text editor</li> </ul>"},{"location":"Scripts/cntools-changelog/#815-2021-04-09","title":"[8.1.5] - 2021-04-09","text":""},{"location":"Scripts/cntools-changelog/#fixed_44","title":"Fixed","text":"<ul> <li>Offline mode fix to ignore error when sourcing env</li> </ul>"},{"location":"Scripts/cntools-changelog/#814-2021-04-05","title":"[8.1.4] - 2021-04-05","text":""},{"location":"Scripts/cntools-changelog/#changed_30","title":"Changed","text":"<ul> <li>Enhanced minimum utxo calculation (credits to Martin providing this)</li> <li>based on calculations from https://github.com/intersectmbo/cardano-ledger/blob/master/doc/explanations/min-utxo-mary.rst</li> <li>Validation of wallet address balance on transactions improved</li> </ul>"},{"location":"Scripts/cntools-changelog/#813-2021-04-01","title":"[8.1.3] - 2021-04-01","text":""},{"location":"Scripts/cntools-changelog/#fixed_45","title":"Fixed","text":"<ul> <li>Alignment fix in blocks table</li> </ul>"},{"location":"Scripts/cntools-changelog/#812-2021-03-31","title":"[8.1.2] - 2021-03-31","text":""},{"location":"Scripts/cntools-changelog/#changed_31","title":"Changed","text":"<ul> <li>Manual CNTools update replaced with automatic by asking to update on startup like the rest of the scripts in the guild repository. </li> <li>Changelog truncated up to v6.0.0. Minor and patch version changelog entries merged with next major release changelog.</li> </ul>"},{"location":"Scripts/cntools-changelog/#811-2021-03-30","title":"[8.1.1] - 2021-03-30","text":""},{"location":"Scripts/cntools-changelog/#fixed_46","title":"Fixed","text":"<ul> <li>Relay registration condition</li> <li>Version number</li> </ul>"},{"location":"Scripts/cntools-changelog/#810-2021-03-26","title":"[8.1.0] - 2021-03-26","text":""},{"location":"Scripts/cntools-changelog/#added_10","title":"Added","text":"<ul> <li>IPv6 support in pool registration/modification</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_32","title":"Changed","text":"<ul> <li>Wallet delegation now lets you specify Pool ID in addition to local CNTools pool instead of previous cold.vkey cbor string</li> <li>A couple of functions regarding number validation moved to common env file</li> <li>Code adapted for changes in ledger-state dump used by 'Pool &gt;&gt; Show'</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_47","title":"Fixed","text":"<ul> <li>Backup &amp; restore now exclude gpg encrypted keys from online backup and suppression of false alarms</li> </ul>"},{"location":"Scripts/cntools-changelog/#802-2021-03-15","title":"[8.0.2] - 2021-03-15","text":""},{"location":"Scripts/cntools-changelog/#fixed_48","title":"Fixed","text":"<ul> <li>Bump cardano-hw-cli minimum version to 1.2.0</li> <li>Add Ledger Cardano app version check with minimum enforced version of 2.2.0</li> <li>Add Trezor firmware check with minimum enforced version of 2.3.6</li> </ul>"},{"location":"Scripts/cntools-changelog/#801-2021-03-05","title":"[8.0.1] - 2021-03-05","text":""},{"location":"Scripts/cntools-changelog/#fixed_49","title":"Fixed","text":"<ul> <li>Add BASH version check, version 4.4 or newer required</li> </ul>"},{"location":"Scripts/cntools-changelog/#800-2021-02-28","title":"[8.0.0] - 2021-02-28","text":""},{"location":"Scripts/cntools-changelog/#added_11","title":"Added","text":"<ul> <li>Multi Asset Token compatibility added throughout all CNTools operations. </li> <li>Sending Ada and custom tokens is done through the normal 'Funds &gt;&gt; Send' operation</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_33","title":"Changed","text":"<ul> <li>Metadata moved to a new Advanced section used for devs/advanced operations not normally used by SPOs.</li> <li>Accessed by enabling developer/advanced mode in cntools.config or by providing runtime flag '-a'</li> <li>Redesign of backup and restore.</li> <li>Deletion of private keys moved from backup to new section under <code>Advanced</code></li> <li>Backup now only contain content of priv folder (files &amp; scripts folders dropped)</li> <li>Restore operation now restore directly to priv folder instead of a random user selected folder to make restore easier and better. Before restore, a new full backup of priv folder is made and stored encrypted in priv/archive</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_50","title":"Fixed","text":"<ul> <li>JQ limitation workaround for large numbers</li> <li>Dialog compatibility improvement by preventing dialog launching a subshell on some systems causing dialog not to run</li> </ul>"},{"location":"Scripts/cntools-changelog/#716-2021-02-10","title":"[7.1.6] - 2021-02-10","text":"<ul> <li>Update curl commands when file isnt downloaded correctly (to give correct return code)</li> </ul>"},{"location":"Scripts/cntools-changelog/#715-2021-02-03","title":"[7.1.5] - 2021-02-03","text":""},{"location":"Scripts/cntools-changelog/#changed_34","title":"Changed","text":"<ul> <li>Guild Announcement/Support Telegram channel added to CNTools GUI</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_51","title":"Fixed","text":"<ul> <li>Fix for a special case using an incomplete wallet (missing stake keys) </li> </ul>"},{"location":"Scripts/cntools-changelog/#714-2021-02-01","title":"[7.1.4] - 2021-02-01","text":""},{"location":"Scripts/cntools-changelog/#fixed_52","title":"Fixed","text":"<ul> <li>Typo in function name after harmonization between scripts</li> </ul>"},{"location":"Scripts/cntools-changelog/#713-2021-01-30","title":"[7.1.3] - 2021-01-30","text":""},{"location":"Scripts/cntools-changelog/#fixed_53","title":"Fixed","text":"<ul> <li>Vacuumlabs cardano-hw-cli 1.1.3 support, now the minimum supported version</li> <li>Improved error handling</li> </ul>"},{"location":"Scripts/cntools-changelog/#711-2021-01-29","title":"[7.1.1] - 2021-01-29","text":""},{"location":"Scripts/cntools-changelog/#changed_35","title":"Changed","text":"<ul> <li>Minor change to future update notification for common env file</li> </ul>"},{"location":"Scripts/cntools-changelog/#710-2021-01-29","title":"[7.1.0] - 2021-01-29","text":""},{"location":"Scripts/cntools-changelog/#changed_36","title":"Changed","text":"<ul> <li>Remove ChainDB metrics references to align with cardano-node 1.25.1</li> <li>Moved some functions to env for reusability between tools</li> </ul>"},{"location":"Scripts/cntools-changelog/#702-2021-01-17","title":"[7.0.2] - 2021-01-17","text":""},{"location":"Scripts/cntools-changelog/#changed_37","title":"Changed","text":"<ul> <li>Re-add the option in offline workflow to use wallet folder that only contains stake keys for multi-owner pools</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_54","title":"Fixed","text":"<ul> <li>Verification of signing key in offline mode for extended signing keys (mnemonics imported wallets)</li> </ul>"},{"location":"Scripts/cntools-changelog/#701-2021-01-13","title":"[7.0.1] - 2021-01-13","text":""},{"location":"Scripts/cntools-changelog/#changed_38","title":"Changed","text":"<ul> <li>Add prompt before updating common env file, instead of auto-update</li> </ul>"},{"location":"Scripts/cntools-changelog/#700-2021-01-11","title":"[7.0.0] - 2021-01-11","text":"<p>Though mostly unchanged in the user interface, this is a major update with most of the code re-written/touched in the back-end. Only the most noticeable changes added to changelog. </p>"},{"location":"Scripts/cntools-changelog/#added_12","title":"Added","text":"<ul> <li>HW Wallet support through Vacuumlabs cardano-hw-cli (Ledger Nano X/S &amp; Trezor T)</li> <li>Vacuumlabs cardano-hw-cli added as build option to prereqs.sh, option '-w' incl Ledger udev rules. Software from Vacuumlabs and Ledger app still early in development and may contain limitations that require workarounds. Users are recommended to familiarise their usage using test wallets first.</li> <li>Because of HW wallet support, transaction signing has been re-designed. For CLI and HW wallet pool reg, raw tx is first witnessed by all signing keys separately and then assembled and signed instead of signing directly with all signing keys. But for all other HW wallet transactions, signing is done directly without first witnessing.</li> <li>Requires updated Cardano app in Ledger/Trezor set to be released in January 2021 to use in pool registration/modification.</li> <li>Option added to disable Dialog for file/dir input in cntools.config</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_39","title":"Changed","text":"<ul> <li>Logging completely re-designed from the ground. Previous selective logging wasn't very useful. All output(almost) now outputted to both the screen and to a timestamped log file. One log file is created per CNTools session. Old log file archived in logs/archive subfolder and last 10 log files kept, rest is pruned on CNTools startup.</li> <li>DEBUG    : Verbose output, most output printed on screen is logged as debug messages except explicitly disabled, like menu printing.</li> <li>INFO     : Informational and the most important output.</li> <li>ACTION   : e.g cardano-cli executions etc</li> <li>ERROR    : error messages and stderr output</li> <li>Verbosity setting in cntools.config removed.</li> <li>Offline workflow now use a single JSON transaction file holding all data needed. This allows us to bake in additional data in the JSON file in addition to the tx body to make it much more clear what the offline transaction do. Like signing key verification, transaction data like fee, source wallet, pool name etc. It also lets the user know on offline computer what signing keys is needed to sign the transaction.</li> <li>Sign Tx moved to Transaction &gt;&gt; Sign</li> <li>Submit Tx moved to Transaction &gt;&gt; Submit</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_55","title":"Fixed","text":"<ul> <li>Remove intermediate prompt for showing changelog, so that it's directly visible.</li> </ul>"},{"location":"Scripts/cntools-changelog/#631-2020-12-14","title":"[6.3.1] - 2020-12-14","text":""},{"location":"Scripts/cntools-changelog/#fixed_56","title":"Fixed","text":"<ul> <li>Array expansion not correctly handled for multi-owner signing keys</li> <li>KES rotation output fix in OFFLINE mode, op.cert should be copied, not cold.counter</li> <li>Output and file explorer workflow redesigned a bit for a better flow</li> <li>formatLovelace() thousand separator fix after forcing locale to C.UTF-8 in env</li> <li>formatAda() function added to pretty print pledge and cost w/o Lovelace</li> </ul>"},{"location":"Scripts/cntools-changelog/#630-2020-12-03","title":"[6.3.0] - 2020-12-03","text":""},{"location":"Scripts/cntools-changelog/#changed_40","title":"Changed","text":"<ul> <li>printTable function replaced with bash printf due to compatibility issues</li> <li>Improved workflow in pool registration/modification for relays and multi-owner.</li> <li>Standardized names for wallet and pool files/folders moved to env file from cntools.config</li> <li>Compatibility with 1.24.2 node (accomodate ledger schema and CLI changes), use 1.24.2 as baseline</li> <li>Move version check to env</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_57","title":"Fixed","text":"<ul> <li>Error output for prerequisite checks</li> </ul>"},{"location":"Scripts/cntools-changelog/#621-2020-11-28","title":"[6.2.1] - 2020-11-28","text":""},{"location":"Scripts/cntools-changelog/#changed_41","title":"Changed","text":"<ul> <li>Compatibility changes for cardano-node 1.23.0, now minimum version to run CNTools 6.2.1</li> <li>Cleanup of old code</li> </ul>"},{"location":"Scripts/cntools-changelog/#620-alpha-branch","title":"[6.2.0] - (alpha branch)","text":""},{"location":"Scripts/cntools-changelog/#added_13","title":"Added","text":"<ul> <li>Ability to post metadata on-chain, e.g. (but not limited to) Adams https://vote.crypto2099.io/</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_42","title":"Changed","text":"<ul> <li>Blocks view updated to adapt to the added CNCLI integration and changes made to block collector(logMonitor)</li> <li>CNCLI</li> <li>Log Monitor</li> <li>chattr file locking now optional to use, a new setting in cntools.config added for it.</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_58","title":"Fixed","text":"<ul> <li>unnecessary bech32 conversion in wallet import (non-breaking) </li> </ul>"},{"location":"Scripts/cntools-changelog/#610-2020-10-22","title":"[6.1.0] - 2020-10-22","text":""},{"location":"Scripts/cntools-changelog/#added_14","title":"Added","text":"<ul> <li>Wallet de-registration with key deposit refund (new cntools.config parameter, WALLET_STAKE_DEREG_FILENAME)</li> <li>Default values loaded for all config variables if omitted/missing in cntools.config</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_43","title":"Changed","text":"<ul> <li>Prometheus node metrics replaced with EKG</li> <li>Allow and handle missing pool.config in pool &gt;&gt; modify and show</li> <li>Cancel and return added in several helper functions if cardano-cli execution fails</li> <li>Various tweaks to the output</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_59","title":"Fixed","text":"<ul> <li>Script execution permissions after internal update</li> <li>Handle redirect in curl metadata fetch</li> </ul>"},{"location":"Scripts/cntools-changelog/#603-2020-10-16","title":"[6.0.3] - 2020-10-16","text":""},{"location":"Scripts/cntools-changelog/#fixed_60","title":"Fixed","text":"<ul> <li>Shelley epoch transition calculation used the wrong byron metric in the calculation</li> </ul>"},{"location":"Scripts/cntools-changelog/#602-2020-10-16","title":"[6.0.2] - 2020-10-16","text":""},{"location":"Scripts/cntools-changelog/#fixed_61","title":"Fixed","text":"<ul> <li>Internal update had the wrong path to env file breaking automatic update, please use prereqs.sh to update</li> <li>Fix in 6.0.1 broke pool id retrieval, now compatible with both pre and post cardano-node 1.21.2 syntax.</li> </ul>"},{"location":"Scripts/cntools-changelog/#601-2020-10-16","title":"[6.0.1] - 2020-10-16","text":""},{"location":"Scripts/cntools-changelog/#fixed_62","title":"Fixed","text":"<ul> <li>As per change to cardano-cli syntax, pool ID requires <code>--cold-verification-key-file</code> instead of <code>--verification-key-file</code></li> </ul>"},{"location":"Scripts/cntools-changelog/#600-2020-10-15","title":"[6.0.0] - 2020-10-15","text":"<p>This is a major release with a lot of changes. It is highly recommended that you familiarise yourself with the usage for Hybrid or Online v/s Offline mode on a testnet environment before doing it on production. Please visit https://cardano-community.github.io/guild-operators/upgrade for details.</p>"},{"location":"Scripts/cntools-changelog/#added_15","title":"Added","text":"<ul> <li>Allow CNTools to operate in offline mode. Offline features include:</li> <li>Simplified Walet Show/List menu</li> <li>Wallet delete without balance check option</li> <li>Pool KES Rotation</li> <li>Sign a staging transaction.</li> <li>Allow creation of staging tx files using ttl as input in an online/offline-hybrid mode, that can be sent to offline server to sign.</li> <li>To Transfer Funds</li> <li>Withdraw Rewards</li> <li>Delegate</li> <li>Register/Modify/Retire pool</li> <li>Allow import of a signed transaction to submit in online mode</li> <li>Allow import of 15/24 words based wallets. Note that you'd need <code>cardano-address</code> and <code>bech32</code> in yout $PATH to use this feature (available if you rebuild <code>cardano-node</code> using updated <code>cabal-build-all.sh</code>), reusing guide from @ilap.</li> <li>Backup now offer the ability to create an online backup without wallet payment/stake and pool cold sign keys</li> <li>Regular(offline) backup now display a warning if wallet payment/stake and pool cold sign keys are missing due to being deleted manually or by previous backup</li> <li>Retire notification in pool &gt;&gt; show</li> <li>Sanity check before launching a second instance of cnode.sh</li> <li>Doc update to run cnode.sh as a systemd service</li> <li>Use secure remove (<code>srm</code>) when available when deleting files.</li> <li>Balance check notification added before wallet selection menus are shown to know that work is done in the background</li> <li>Ability to select a different pool owner and reward wallet</li> <li>Multi-owner support using stake vkey/skey files</li> <li>Added TIMEOUT_LEDGER_STATE(default 300s) in cntools.config to be used instead of static 60 seconds for querying shelley ledger-state.</li> <li>Option to delete private keys after successful backup</li> <li>itnRewards.sh script to claim ITN rewards incl docs update</li> <li>More explicit error messages at startup</li> <li>Basic sanity checks for socket file</li> <li>Backup &amp; Restore of wallets, pools and configuration files</li> <li>External KES rotation script using CNTools library</li> <li>Add few flags to control prereqs to allow skipping overwriting files, deploying OS packages, etc</li> <li>cntools.sh: Drop an error if log not found, indicating config with no JSON being used</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_44","title":"Changed","text":"<ul> <li>Improved trap/exit handling</li> <li>Allow thousand separator(<code>,</code>) in user input for sending ADA and pledge/cost at pool registration to make it easier to count the zeros</li> <li>User input for files and directories now open a dialog gui to make it easier to find the correct path</li> <li>CNTools now uses and works with <code>cardano-node 1.19.0</code>, please upgrade if you're not using this version.</li> <li>Use manual calculation based on slot tip to get KES period</li> <li>Removed ledger dump dependency from Pool Register, Modify, Retire and List.</li> <li>The cost of the ledger dump is too high, replaced with a simple check if pools registration certificate exist in pool folder</li> <li>Pool &gt;&gt; Show|Delegators are now the only options dumping the ledger-state</li> <li>Wallet vkeys no longer encrypted as skeys are the only ones in need of protection</li> <li>Update command change (change applied after this release is active):</li> <li>Minor/Patch release: it will warn, backup and replace CNTools script files including cntools.config</li> <li>Major release: No change, prompt user to backup and run prereqs.sh according to instructions.</li> <li>Troubleshooting improvements:</li> <li>Split 'config in json format' and 'hasPrometheus' checks</li> <li>Output node sync stats if Shelley transition epoch is to be calculated</li> <li>Protocol parameters output check to give an improved error message</li> <li>Pool &gt;&gt; Show view updated to show modified pool values if Pool &gt;&gt; Modify has been used to update pool parameters</li> <li>The section has also been updated to make it a little bit easier to read</li> <li>Pool &gt;&gt; Delegators view also use updated pledge value if a pool modification has been registered to check if pledge is met</li> <li>Use mainnet as default, since other testnets are either retired or not being maintained :(</li> <li>Backup original files when doing upgrades, so that users do not lose their changes.</li> <li>Major update description updated</li> <li>env file update removed from minor update</li> <li>Prometheus metrics used for various functions and now required to run CNTools, enabled by default</li> <li>Changed references to ptn0 to generalize the usage</li> <li>Change CNTools changelog heading format - +1 sublevels for headings (used by newer documentation)</li> <li>Delegators previously displayed in <code>Pool &gt;&gt; Show</code> now moved to its own menu option   This is to de-clutter and because it takes time to parse this data from ledger-state</li> <li>stake.cert no longer encrypted in wallet</li> <li>Meta description now has a limit of 255 chars to match smash server limit</li> <li>ledger-state timeout increased to 60s</li> <li>Update ptn0 config to align with hydra config as much as possible, while keeping trace options on</li> <li>moved update check to be one of the first things CNTools does after start to be able to show critical changes before anything else runs.</li> <li>Parse node logs to check the transition from Byron to shelley era, and save the epoch for transition in db folder. This is required for calculating KES keys.</li> <li>Please make sure to use config files created by the prereqs.sh, or enable JSON loggers for your config.</li> <li>Update cnode.sh.templ to archive logs every time node is restart, this ensures that we're not searching for previous log history when network was changed. Network being changed would automatically deduce db folder was deleted.</li> <li>Update default network to MC3</li> </ul>"},{"location":"Scripts/cntools-changelog/#removed","title":"Removed","text":"<ul> <li><code>Pool &gt;&gt; Delegators</code> removed.</li> <li>If/when a better option than dumping and parsing ledger-state dump arise re-adding it will be considered. </li> <li>Utilize the community explorers listed at https://cardano-community.github.io/support-faq/explorers </li> <li>POOL_PLEDGECERT_FILENAME removed from config, WALLET_DELEGCERT_FILENAME is used instead for delegation cert to pool, no need to keep a separate cert in pool folder for this, its the wallet that is delegated.</li> <li>Redundant sections in guide</li> <li>Stale delegate.counter</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_63","title":"Fixed","text":"<ul> <li>Check <code>pool &gt;&gt; show</code> stake distribution showing up as always 0.</li> <li>KES expiration calculation</li> <li>Slot interval calculation</li> <li>Custom vname replacement(when using <code>prereqs.sh -t</code>) fix for internal update</li> <li>Pool registration and de-registration certificates removed in case of retire/re-registration</li> <li>KES Expiry to use KES Period instead of Epoch duration</li> <li>Block Collector script adapted for cardano-node 1.19.0.</li> <li>Block hash is now truncated in log, issue https://github.com/intersectmbo/cardano-node/issues/1738</li> <li>High cpu usage reported in a few cases when running Block Collector</li> <li>Depending on log level, parsing and byte64 enc each entry with jq could potentially put high load on weaker systems. Replaced with grep to only parse entries containing specific traces.</li> <li>Docs for creating systemd block collector service file updated to include user env in run command</li> <li>cardano-node 1.19.0 introduced an issue that required us to use KES as current - 1 while rotating.</li> <li>A new getPoolID helper function added to extract both hex and bech32 pool ID</li> <li>Added <code>--output-format hex</code> when extracting pool ID in hex format</li> <li>A new pool.id-bech32 file gets created if cold.vkey is available and decrypted</li> <li>Added error check to see if cardano-cli is in $PATH before continuing.</li> <li>Backup &amp; Restore paths were failing on machines due to alnum class availability on certain interpreters.</li> <li>Rewards were not counted in stake and pledge</li> <li>Removed +i file locking on .addr files when using <code>Wallet &gt;&gt; Encrypt</code> as these are re-generated from keys and need to be writable</li> <li>Balance check added to <code>Funds &gt;&gt; Withdraw</code> for base address as this is used to pay the withdraw transaction fee</li> <li>Resolve issue with Multi Owner causing an error with new pool registration (error was due to quotes)</li> <li>Mainnet uses dedicated condition for slot checks</li> <li>Timeout moved to a variable in cntools.library</li> <li>KES Calculation for current KES period and KES expiration date   Please re-check expiration date using Pool &gt;&gt; Show</li> <li>calc_slots to be network independent</li> <li>prom_host should be calculated from config file, instead of having to update a config</li> <li>Minor typo in menu</li> <li>Parse Config for virtual forks, which adds supports for MC4</li> <li>CNTools block collector fix</li> <li>column application added as a prereq, bsdmainutils/util-linux</li> <li>cntoolsBlockCollector.sh get epoch using function</li> <li>KES count was not showing up in Katip</li> <li>Funds -&gt; Delegation was broken as per recent changes in 1.17, corrected key type for delegation certificate</li> <li><code>Pool &gt;&gt; Show</code> delegator rewards parsing from ledger-state</li> <li>Slot sync format improvement</li> <li>kesExpiration function to use 17 fixed byron transition epochs </li> </ul>"},{"location":"Scripts/cntools-changelog/#500-2020-07-20","title":"[5.0.0] - 2020-07-20","text":""},{"location":"Scripts/cntools-changelog/#added_16","title":"Added","text":"<ul> <li>HASH_IDENTIFIER where applicable to differentiate between network modes for commands used, required due to legacy Byron considerations</li> <li>add ptn0-praos.json and ptn0-combinator.json to reduce confusion between formats, make prereqs default to combinator, and accept p argument to indicate praos mode.</li> <li>cardano-node 1.16.0 refers to txhash using quotes, sed them out</li> <li>show what's new at startup after update</li> <li>file size check for pool metadata file</li> <li>Add nonce in pool metadata JSON to keep registration attempts unique, avoiding one hash pointing to multiple URLs</li> <li>Change default network to <code>mainnet_candidate</code>, and add second argument (g) to run prereqs against guild network</li> <li>allow the use of pre-existing metadata from URL when registering or modifying pool</li> <li>minimum pool cost check against protocol</li> <li>Refresh option to home screen</li> <li>Ability to register multiple relay DNS A records as well as a mix of DNS A and IPv4</li> <li>Valid for pool registration and modification</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_45","title":"Changed","text":"<ul> <li>Default config switched to combinator instead of testnet</li> <li>Start maintaining seperate versions of praos and combinator config files.</li> <li>Add 10s timeout to wget commmands in case of issue</li> <li>timestamp added to pool metadata file to make every creation unique</li> <li>Cancel shortcut changed from <code>[c]</code> to <code>[Esc]</code></li> <li>Default pool cost from 256 -&gt; 400</li> <li>slotinterval calculation to include decentralisation parameter</li> <li>mainnet candidate compatible slot calculation, 17 fixed byron transition epochs (needs to be fixed for mainnet)</li> <li>Pool metadata information to copy file as-is as well as wait for keypress to make sure file is copied before proceeding with registration.</li> <li>Now use internal table builder to display previous relays</li> <li>Instead of giving relays from previous registration as default values it will now ask if you want to re-register relays exactly as before to minimize steps and complexity</li> </ul>"},{"location":"Scripts/cntools-changelog/#removed_1","title":"Removed","text":"<ul> <li>Delete cntools-updater script</li> <li>NODE_SOCKET_PATH config parameter(replaced by CARDANO_NODE_SOCKET_PATH)</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_64","title":"Fixed","text":"<ul> <li>Slots reference was mixing up for shelley testnet in absence of a combinator network</li> <li>numfmt dependency removed in favor of printf formatting</li> <li>Vkey delegation fix due to json format switch</li> <li>ADA not displayed in a couple of the wallet selection menus</li> <li>KES calculation support for both MC and Shelley Testnet</li> <li>Slot tip reference calculation for shelley testnet</li> </ul>"},{"location":"Scripts/cntools-changelog/#400-2020-07-13","title":"[4.0.0] - 2020-07-13","text":""},{"location":"Scripts/cntools-changelog/#added_17","title":"Added","text":"<ul> <li>Add PROTOCOL_IDENTIFIER and NETWORK_IDENTIFIER instead of harcoded entries for combinator v/s TPraos &amp; testnet v/s magic differentiators respectively.</li> <li>Keep both ptn0.yaml and ptn0-combinator.yaml to keep validity with mainnet-combinator</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_46","title":"Changed","text":"<ul> <li>Revert back default for Public network to Shelley_Testnet as per https://t.me/CardanoStakePoolWorkgroup/282606</li> </ul>"},{"location":"Scripts/cntools-changelog/#300-2020-07-12","title":"[3.0.0] - 2020-07-12","text":""},{"location":"Scripts/cntools-changelog/#added_18","title":"Added","text":"<ul> <li>Basic health check data on main menu</li> <li>Epoch, time until next epoch, node tip vs calculated reference tip and a warning if node is lagging behind.</li> <li>Address era and encoding to <code>Wallet &gt;&gt; Show</code></li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_47","title":"Changed","text":"<ul> <li>Release <code>2.1.1</code> included a change to env file and thus require a major version bump.</li> <li>Modified output on Update screen slightly.</li> <li>KES calculation, now take into account the byron era and the transition period until shelley start</li> <li>Credit to Martin @ ATADA for inspiration on how to calculate this</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_65","title":"Fixed","text":"<ul> <li>Version fix to include patch version</li> </ul>"},{"location":"Scripts/cntools-changelog/#200-2020-07-12","title":"[2.0.0] - 2020-07-12","text":""},{"location":"Scripts/cntools-changelog/#added_19","title":"Added","text":"<ul> <li>Support for cardano-node 1.15.x  </li> <li>calculate-min-fee update to reflect change in 1.15.     change was required to support byron witnesses.</li> <li>gettip update as output is now json formatted</li> <li>bech32 addressing in 1.15 required changes to delegator lookup in <code>Pool &gt;&gt; Show</code></li> <li>add --cardano-mode to query parameters</li> <li>--mainnet flag for address generation</li> <li>Output verbosity   A new config parameter added for output verbosity using say function.   0 = Minimal - Show relevant information (default)   1 = Normal  - More information about whats going on behind the scene   2 = Maximal - Debug level for troubleshooting</li> <li>Improve delegators list in <code>Pool &gt;&gt; Show</code></li> <li>Identify owners delegations</li> <li>Display owner stake in red if <code>(stake + reward)</code> is below pledge (single-owner only for now)</li> <li>Display all lovelace values in floating point ADA with 6 decimals (lovelaces) using locales</li> <li>Block Collector summary view</li> <li>KES rotation notification/warning on startup and in pool list/show views</li> <li>Live stake and delegators in <code>Pool &gt;&gt; Show</code></li> <li>Changelog </li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_48","title":"Changed","text":"<ul> <li>op-cert creation moved from <code>Pool &gt;&gt; New</code> to <code>Pool &gt;&gt; Register</code>.</li> <li>Output changed in various places throughout.</li> <li>Include reward in delegators stake.</li> <li>Release now include patch version in addition to major and minor version.   In-app update modified to reflect this change.</li> <li>Block Collector table view</li> <li>Various minor code improvements</li> </ul>"},{"location":"Scripts/cntools-changelog/#removed_2","title":"Removed","text":"<ul> <li>Enterprise wallet upgrade option in <code>Wallet &gt;&gt; List</code> </li> <li><code>Not a registered wallet on chain</code> information from Wallet listing</li> <li>en_US.UTF-8 locale dependency</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_66","title":"Fixed","text":"<ul> <li>meta_json_url check</li> <li>Invalid tx_in when registering stake wallet</li> <li>Delegators rewards in <code>Pool &gt;&gt; Show</code></li> <li>Work-around awk versions that only support 32-bit integers</li> <li>Sometimes cardano-node log contain duplicate traces for the same slot at log file rollover, now filtered</li> <li>Correct nwmagic - was hardcoded to 42</li> <li>Set script locale to fix format issue</li> </ul>"},{"location":"Scripts/cntools-changelog/#100-2020-07-07","title":"[1.0.0] - 2020-07-07","text":"<ul> <li>First official major release</li> </ul>"},{"location":"Scripts/cntools-common/","title":"Common Tasks","text":"<p>Important</p> <p>Familiarize yourself with the Online workflow of creating wallets and pools on the Preview/Preprod/Guild network first. You can then move on to test the Offline Workflow. The Offline workflow means that the private keys never touch the Online node. When comfortable with both the online and offline CNTools workflow, it's time to deploy what you learnt on the mainnet.</p> <p>This chapter describes some common use-cases for wallet and pool creation when running CNTools in Online mode. CNTools contains much more functionality not described here.</p> Create Wallet <p>A wallet is needed for pledge and to pay for pool registration fee.</p> <ol> <li>Choose <code>[w] Wallet</code> and you will be presented with the following menu:   <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; WALLET\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n Wallet Management\n\n ) New         - create a new wallet\n ) Import      - import a Daedalus/Yoroi 24/25 mnemonic or Ledger/Trezor HW wallet\n ) Register    - register a wallet on chain\n ) De-Register - De-Register (retire) a registered wallet\n ) List        - list all available wallets in a compact view\n ) Show        - show detailed view of a specific wallet\n ) Remove      - remove a wallet\n ) Decrypt     - remove write protection and decrypt wallet\n ) Encrypt     - encrypt wallet keys and make all files immutable\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n Select Wallet Operation\n\n  [n] New\n  [i] Import\n  [r] Register\n  [z] De-Register\n  [l] List\n  [s] Show\n  [x] Remove\n  [d] Decrypt\n  [e] Encrypt\n  [h] Home\n</code></pre></li> <li>Choose <code>[n] New</code> to create a new wallet. <code>[i] Import</code> can also be used to import a Daedalus/Yoroi based 15 or 24 word wallet seed  </li> <li>Give the wallet a name  </li> <li>CNTools will give you the wallet address.  For example:   <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; WALLET &gt;&gt; NEW\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nName of new wallet: Test\n\nNew Wallet         : Test\nAddress            : addr_test1qpq5qjr774cyc6kxcwp060k4t4hwp42q43v35lmcg3gcycu5uwdwld5yr8m8fgn7su955zf5qahtrgljqfjfa4nr8jfsj4alxk\nEnterprise Address : addr_test1vpq5qjr774cyc6kxcwp060k4t4hwp42q43v35lmcg3gcyccuxhdka\n\nYou can now send and receive Ada using the above addresses.\nNote that Enterprise Address will not take part in staking.\nWallet will be automatically registered on chain if you\nchoose to delegate or pledge wallet when registering a stake pool.\n</code></pre></li> <li>Send some money to this wallet. Either through the faucet or have a friend send you some.</li> </ol> <ul> <li>The wallet must have funds in it before you can proceed.</li> <li>The Wallet created from here is not derived from mnemonics, please use next tab if you'd like to use a wallet that can also be accessed from Daedalus/Yoroi</li> </ul> Import Daedalus/Yoroi/HW Wallet <p>The <code>Import</code> feature of CNTools is originally based on this guide from Ilap.</p> <p>If you would like to use <code>Import</code> function to import a Daedalus/Yoroi based 15 or 24 word wallet seed, please ensure that <code>cardano-address</code> and <code>bech32</code> bineries are available in your <code>$PATH</code> environment variable:   <pre><code>bech32 --version\n1.1.0\n\ncardano-address --version\n3.5.0\n</code></pre></p> <p>If the version is not as per above, please run the latest <code>guild-deploy.sh</code> from here and rebuild <code>cardano-node</code> as instructed here.</p> <p>To import a Daedalus/Yoroi wallet to CNTools, open CNTools and select the <code>[w] Wallet</code> option, and then select the <code>[i] Import</code>, the following menu will appear:   <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; WALLET &gt;&gt; IMPORT\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n Wallet Import\n\n ) Mnemonic  - Daedalus/Yoroi 24 or 25 word mnemonic\n ) HW Wallet - Ledger/Trezor hardware wallet\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n Select Wallet operation\n\n  [m] Mnemonic\n  [w] HW Wallet\n  [h] Home\n</code></pre></p> <p>Note</p> <p>You can import Hardware wallet using <code>[w] HW Wallet</code> above, but please note that before you are able to use hardware wallet in CNTools, you need to ensure you can detect your hardware device at OS level using <code>cardano-hw-cli</code></p> <p>Select the wallet you want to import, for Daedalus / Yoroi wallets select <code>[m] Mnemonic</code>:   <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; WALLET &gt;&gt; IMPORT &gt;&gt; MNEMONIC\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nName of imported wallet: TEST\n\n24 or 15 word mnemonic(space separated):\n</code></pre> Give your wallet a name (in this case 'TEST'), and enter your mnemonic phrase. Please ensure that you **READ* through the complete notes presented by CNTools before proceeding.</p> Create Pool <p>Create the necessary pool keys.</p> <ol> <li>From the main menu select <code>[p] Pool</code> <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; POOL\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n Pool Management\n\n ) New      - create a new pool\n ) Register - register created pool on chain using a stake wallet (pledge wallet)\n ) Modify   - change pool parameters and register updated pool values on chain\n ) Retire   - de-register stake pool from chain in specified epoch\n ) List     - a compact list view of available local pools\n ) Show     - detailed view of specified pool\n ) Rotate   - rotate pool KES keys\n ) Decrypt  - remove write protection and decrypt pool\n ) Encrypt  - encrypt pool cold keys and make all files immutable\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n Select Pool Operation\n\n  [n] New\n  [r] Register\n  [m] Modify\n  [x] Retire\n  [l] List\n  [s] Show\n  [o] Rotate\n  [d] Decrypt\n  [e] Encrypt\n  [h] Home\n</code></pre></li> <li>Select <code>[n] New</code> to create a new pool  </li> <li>Give the pool a name. In our case, we call it TEST. The result should look something like this:   <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; POOL &gt;&gt; NEW\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nPool Name: TEST\n\nPool: TEST\nID (hex)    : 8d5a3510f18ce241115da38a1b2419ed82d308599c16e98caea1b4c0\nID (bech32) : pool134dr2y833n3yzy2a5w9pkfqeakpdxzzenstwnr9w5x6vqtnclue\n</code></pre></li> </ol> Register Pool <p>Register the pool on-chain.</p> <ol> <li>From the main menu select <code>[p] Pool</code> </li> <li>Select <code>[r] Register</code> </li> <li>Select the pool you just created  </li> <li>CNTools will give you prompts to set pledge, margin, cost, metadata, and relays. Enter values that are useful to you.  </li> </ol> <p>Make sure you set your pledge low enough to insure your funds in your wallet will cover pledge plus pool registration fees.  </p> <ol> <li>Select wallet to use as pledge wallet, <code>Test</code> in our case. As this is a newly created wallet, you will be prompted to continue with wallet registration. When complete and if successful, both wallet and pool will be registered on-chain.</li> </ol> <p>It will look something like this:   <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; POOL &gt;&gt; REGISTER\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nOnline mode  -  The default mode to use if all keys are available\n\nHybrid mode  -  1) Go through the steps to build a transaction file\n                2) Copy the built tx file to an offline node\n                3) Sign it using 'Sign Tx' with keys on offline node\n                   (CNTools started in offline mode '-o' without node connection)\n                4) Copy the signed tx file back to the online node and submit using 'Submit Tx'\n\nSelected value: [o] Online\n\n# Select pool\nSelected pool: TEST\n\n# Pool Parameters\npress enter to use default value\n\nPledge (in Ada, default: 50,000):\nMargin (in %, default: 7.5):\nCost (in Ada, minimum: 340, default: 340):\n\n# Pool Metadata\n\nEnter Pool's JSON URL to host metadata file - URL length should be less than 64 chars (default: https://foo.bat/poolmeta.json):\n\nEnter Pool's Name (default: TEST):\nEnter Pool's Ticker , should be between 3-5 characters (default: TEST):\nEnter Pool's Description (default: No Description):\nEnter Pool's Homepage (default: https://foo.com):\n\nOptionally set an extended metadata URL?\nSelected value: [n] No\n{\n  \"name\": \"TEST\",\n  \"ticker\": \"TEST\",\n  \"description\": \"No Description\",\n  \"homepage\": \"https://foo.com\",\n  \"nonce\": \"1613146429\"\n}\n\nPlease host file /opt/cardano/guild/priv/pool/TEST/poolmeta.json as-is at https://foo.bat/poolmeta.json\n\n# Pool Relay Registration\nSelected value: [d] A or AAAA DNS record (single)\nEnter relays's DNS record, only A or AAAA DNS records: relay.foo.com\nEnter relays's port: 6000\nAdd more relay entries?\nSelected value: [n] No\n\n# Select main owner/pledge wallet (normal CLI wallet)\nSelected wallet: Test (100,000.000000 Ada)\nWallet Test3 not registered on chain\n\nWaiting for new block to be created (timeout = 600 slots, 600s)\nINFO: press any key to cancel and return (won't stop transaction)\n\nOwner #1 : Test added!\n\nRegister a multi-owner pool (you need to have stake.vkey of any additional owner in a seperate wallet folder under $CNODE_HOME/priv/wallet)?\nSelected value: [n] No\n\nUse a separate rewards wallet from main owner?\nSelected value: [n] No\n\nWaiting for new block to be created (timeout = 600 slots, 600s)\nINFO: press any key to cancel and return (won't stop transaction)\n\nPool TEST successfully registered!\nOwner #1      : Test\nReward Wallet : Test\nPledge        : 50,000 Ada\nMargin        : 7.5 %\nCost          : 340 Ada\n\nUncomment and set value for POOL_NAME in ./env with 'TEST'\n\nINFO: Total balance in 1 owner/pledge wallet(s) are: 99,497.996518 Ada\n</code></pre></p> <ol> <li>As mentioned in the above output: Uncomment and set value for <code>POOL_NAME</code> in <code>./env</code> with 'TEST' (in our case, the <code>POOL_NAME</code> is <code>TEST</code>). The <code>cnode.sh</code> script will automatically detect whether the files required to run as a block producing node are present in the <code>$CNODE_HOME/priv/pool/&lt;POOL_NAME&gt;</code> directory.</li> </ol> Rotate KES Keys <p>The node runs with an operational certificate, generated using the KES hot key. For security reasons, the protocol asks to re-generate (or rotate) your KES key once reaching expiry. On mainnet, this expiry is in 62 cycles of 18 hours (thus, to ask for rotation quarterly), after which your node will not be able to forge valid blocks unless rotated. To be able to rotate KES keys, your cold keys files (<code>cold.skey</code>, <code>cold.vkey</code> and <code>cold.counter</code>) need to be present on the machine where you run CNTools to rotate your KES key.</p> <ol> <li> <p>To Rotate KES keys and generate the operational certificate - <code>op.cert</code>.</p> </li> <li> <p>From the main menu select <code>[p] Pool</code> </p> </li> <li>Select <code>[o] Rotate</code> </li> <li>Select the pool you just created  </li> </ol> <p>The output should look like:</p> <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; POOL &gt;&gt; ROTATE KES\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nSelect pool to rotate KES keys on\nSelected pool: TEST\n\nPool KES keys successfully updated\nNew KES start period  : 240\nKES keys will expire  : 302 - 2021-09-04 11:24:31 UTC\n\nRestart your pool node for changes to take effect\n\npress any key to return to home menu\n</code></pre> <ol> <li>Start or restart your <code>cardano-node</code>. If deployed as a <code>systemd</code> service as shown here, you can run <code>sudo systemctl restart cnode</code>.</li> <li>Ensure the node is running as a block producing (core) node.</li> </ol> <p>You can use gLiveView - the output at the top should say <code>&gt; Cardano Node - (Core - Guild)</code>.</p> <p>Alternatively, you can check the node logs in <code>$CNODE_HOME/logs/</code> to see whether the node is performing leadership checks (<code>TraceStartLeadershipCheck</code>, <code>TraceNodeIsNotLeader</code>, etc.) </p>"},{"location":"Scripts/cntools/","title":"Overview","text":"<p>Important</p> <ul> <li>Ensure the Pre-Requisites are in place before you proceed.</li> <li>The active testers for this script use Fedora/CentOS/RHEL/Ubuntu operating systems, other OS may require customisations.</li> <li>The tool uses the folder structure defined here. Everyone is free to customise, but while doing so beware that you may introduce changes that may not be tested during updates.</li> <li>Always use Preview/Preprod/Guild network first to familiarise, read the warning/messages in full, maintain your keys/backups with passwords (no one other than yourself can retrieve the funds if you make an accident), before performing actions on mainnet.</li> </ul> <p>Koios CNTools is like a swiss army knife for pool operators to simplify typical operations regarding their wallet keys and pool management. Please note that this tool only aims to simplify usual tasks for its users, but it should NOT act as an excuse to skip understanding how to manually work through things or basics of Linux operations. The skills highlighted on the home page are paramount for a stake pool operator, and so is the understanding of configuration files and network. Please ensure you've read and understood the disclaimers before proceeding.</p> <p>Visit the Changelog section to see progress and current release.</p>"},{"location":"Scripts/cntools/#overview","title":"Overview","text":"<p>The tool consist of three files.  </p> <ul> <li><code>cntools.sh</code> - the main script to launch cntools.</li> <li><code>cntools.library</code> - internal script with helper functions.</li> </ul> <p>In addition to the above files, there is also a dependency on the common <code>env</code> file. CNTools connects to your node through the configuration in the <code>env</code> file located in the same directory as the script. Customize <code>env</code> and <code>cntools.sh</code> files to your needs.</p> <p>Additionally, CNTools can integrate and enable optional functionalities based on external components:</p> <ul> <li><code>cncli.sh</code> is a companion script with optional functionalities to run on the core node (block producer) such as monitoring created blocks, calculating leader schedules and block validation.</li> <li><code>logMonitor.sh</code> is another companion script meant to be run together with the <code>cncli.sh</code> script to give a more complete picture.</li> </ul> <p>See CNCLI and Log Monitor sections for more details.</p> <p>Koios CNTools can operate in following modes:</p> <ul> <li>Online - The default mode using either a local node or Koios API to query the blockchain.</li> <li>Local <code>-n</code> - The local node is used to query the blockchain for needed data. This is the default mode when you start CNTools without parameters.</li> <li>Light <code>-l</code> - Koios query layer is used and removes the need for a local node deployment. This mode is both quicker and lighter on resources but comes with a third party dependency.</li> <li>Hybrid - When running in online mode, this option can be used in menus to create offline transaction files that can be passed to Offline CNTools to sign.</li> <li>Offline <code>-o</code> - Launches CNTools with a limited set of features. This mode does not require access to cardano-node or access to an internet connection. It is mainly used to create Wallet/Pool and access <code>Transaction &gt;&gt; Sign</code> to sign an offline transaction file created in Hybrid mode.</li> <li>Advanced <code>-a</code> - Exposes a new <code>Advanced</code> menu, which allows users to manage (create/mint/burn) new assets.</li> </ul> <p>In addition to above mentioned runtime arguments to launch CNTools in different modes, it can also be persisted by editing User Variables section within <code>cntools.sh</code> script.</p>"},{"location":"Scripts/cntools/#download-and-update","title":"Download and Update","text":"<p>The update functionality is provided from within CNTools. In case of breaking changes, please follow the prompts post-upgrade. If stuck, it's always best to re-run the latest <code>guild-deploy.sh</code> before proceeding.</p> <p>If you have not updated in a while, it is possible that you might come from a release with breaking changes. If so, please be sure to check out the upgrade instructions.</p>"},{"location":"Scripts/cntools/#navigation","title":"Navigation","text":"<p>The scripts menu supports both arrow key navigation and shortcut key selection. The character within the square brackets is the shortcut to press for quick navigation. For other selections like wallet and pool menu that don't contain shortcuts, there is a third way to navigate. Key pressed is compared to the first character of the menu option and if there is a match the selection jumps to this location. A handy way to quickly navigate a large menu. </p>"},{"location":"Scripts/cntools/#hardware-wallet","title":"Hardware Wallet","text":"<p>CNTools includes hardware wallet support since version <code>7.0.0</code> through Vacuumlabs <code>cardano-hw-cli</code> application. Initialize and update firmware/app on the device to the latest version before usage following the manufacturer instructions.</p> <p>To enable hardware support run <code>guild-deploy.sh -s w</code>. This downloads and installs Vacuumlabs <code>cardano-hw-cli</code> including <code>udev</code> configuration. When a new version of Vacuumlabs <code>cardano-hw-cli</code> is released, run <code>guild-deploy.sh -s w</code> again to update. For additional runtime options, run <code>guild-deploy.sh -h</code>.</p> Ledger <ul> <li>Supported devices: Nano S / Nano X  </li> <li>Make sure the latest cardano app is installed on the device.</li> </ul> Trezor <ul> <li>Supported devices: Model T  </li> <li>Make sure the latest firmware is installed on the device. In addition to this, install <code>Trezor Bridge</code> for your system before trying to use your Trezor device in CNTools. You can find the latest version of the bridge at https://wallet.trezor.io/#/bridge</li> </ul>"},{"location":"Scripts/cntools/#offline-workflow","title":"Offline Workflow","text":"<p>CNTools can be run in online and offline mode. At a very high level, for working with offline devices, remember that you need to use CNTools in an online node to generate a staging transaction for the desired type of transaction, and then move the staging transaction to an offline node to sign (authorize) using the signing keys on your offline node - and then bring back the signed transaction to the online node for submission to the chain. </p> <p>For the offline workflow, all the wallet and pool keys should be kept on the offline node. The backup function in CNTools has an option to create a backup without private keys (sensitive signing keys) to be transferred to online node. All other files are included in the backup to be transferred to the online node. </p> <p>Keys excluded from backup when created without private keys: Wallet - <code>payment.skey</code>, <code>stake.skey</code> Pool   - <code>cold.skey</code></p> <p>Note that setting up an offline server requires good SysOps background (you need to be aware of how to set up your server with offline mirror repository, how to transfer files across and be fairly familiar with the disk layout presented in the documentation). The <code>guild-deploy.sh</code> in its current state is not expected to run on an offline machine. Essentially, you simply need the <code>cardano-cli</code>, <code>bech32</code>, <code>cardano-address</code> binaries in your <code>$PATH</code>, OS level dependency packages [<code>jq</code>, <code>coreutils</code>, <code>pkgconfig</code>, <code>gcc-c++</code> and <code>bc</code> ], and perhaps a copy from your online <code>cnode</code> directory (to ensure you have the right <code>genesis</code>/<code>config</code> files on your offline server). We strongly recommend you to familiarise yourself with the workflow on the preview / preprod / guild networks first, before attempting on mainnet.</p> <p>Example workflow for creating a wallet and pool:</p>  sequenceDiagram     Note over Offline: Create/Import a wallet     Note over Offline: Create a new pool     Note over Offline: Rotate KES keys to generate op.cert     Note over Offline: Create a backup w/o private keys     Offline-&gt;&gt;Online: Transfer backup to online node     Note over Online: Fund the wallet base address with enough Ada     Note over Online: Register wallet using ' Wallet \u00bb Register ' in hybrid mode     Online-&gt;&gt;Offline: Transfer built tx file back to offline node     Note over Offline: Use ' Transaction &gt;&gt; Sign ' with payment.skey from wallet to sign transaction     Offline-&gt;&gt;Online: Transfer signed tx back to online node     Note over Online: Use ' Transaction &gt;&gt; Submit ' to send signed transaction to blockchain     Note over Online: Register pool in hybrid mode     loop         Offline--&gt;Online: Repeat steps to sign and submit built pool registration transaction     end     Note over Online: Verify that pool was successfully registered with ' Pool \u00bb Show '   Online mode <p>To start CNTools in Online (advanced) Mode, execute the script from the <code>$CNODE_HOME/scripts/</code> directory: <pre><code>cd $CNODE_HOME/scripts\n./cntools.sh -a\n</code></pre></p> <p>You should get a screen that looks something like this:</p> <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; Koios CNTools vX.X.X - Guild - CONNECTED &lt;&lt;\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n Main Menu    Telegram Announcement / Support channel: t.me/CardanoKoios/9759\n\n ) Wallet      - create, show, remove and protect wallets\n ) Funds       - send, withdraw and delegate\n ) Pool        - pool creation and management\n ) Transaction - Sign and Submit a cold transaction (hybrid/offline mode)\n ) Blocks      - show core node leader schedule &amp; block production statistics\n ) Backup      - backup &amp; restore of wallet/pool/config\n ) Advanced    - Developer and advanced features: metadata, multi-assets, ...\n ) Refresh     - reload home screen content\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n                                                  Epoch 276 - 3d 19:08:27 until next\n What would you like to do?                                         Node Sync: 12 :)\n\n  [w] Wallet\n  [f] Funds\n  [p] Pool\n  [t] Transaction\n  [b] Blocks\n  [u] Update\n  [z] Backup &amp; Restore\n  [a] Advanced\n  [r] Refresh\n  [q] Quit\n</code></pre> Offline mode <p>To start CNTools in Offline Mode, execute the script from the <code>$CNODE_HOME/scripts/</code> directory using the <code>-o</code> flag: <pre><code>cd $CNODE_HOME/scripts\n./cntools.sh -o\n</code></pre></p> <p>The main menu header should let you know that node is started in offline mode: <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; Koios CNTools vX.X.X - Guild - OFFLINE &lt;&lt;\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n Main Menu    Telegram Announcement / Support channel: t.me/CardanoKoios/9759\n\n ) Wallet      - create, show, remove and protect wallets\n ) Funds       - send, withdraw and delegate\n ) Pool        - pool creation and management\n ) Transaction - Sign and Submit a cold transaction (hybrid/offline mode)\n\n ) Backup      - backup &amp; restore of wallet/pool/config\n\n ) Refresh     - reload home screen content\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n                                                  Epoch 276 - 3d 19:03:46 until next\n What would you like to do?\n\n  [w] Wallet\n  [f] Funds\n  [p] Pool\n  [t] Transaction\n  [z] Backup &amp; Restore\n  [r] Refresh\n  [q] Quit\n</code></pre></p>"},{"location":"Scripts/env/","title":"Common env","text":"<p>A common environment file called <code>env</code> is sourced by most scripts in the Guild Operators repository. This file holds common variables and functions needed by other scripts. There are several benefits to this, not having to specify duplicate settings and being able to reuse functions decreasing the risk of misconfiguration and inconsistency.</p>"},{"location":"Scripts/env/#installation","title":"Installation","text":"<p><code>env</code> file is downloaded together with the rest of the scripts when Pre-Requisites if followed and located in the <code>$CNODE_HOME/scripts/</code> directory. The file is also automatically downloaded/updated by some of the individual scripts if missing, like <code>cntools.sh</code>, <code>gLiveView.sh</code> and <code>topologyUpdater.sh</code>. All custom changes in User Variables section are untouched on updates unless a forced overwrite is selected when running <code>guild-deploy.sh</code>.</p>"},{"location":"Scripts/env/#configuration","title":"Configuration","text":"<p>Most variables can be left commented to use the automatically detected or default value. But there are some that need to be set as explained below.</p> <ul> <li><code>CNODE_PORT</code> - This is the most important variable and needs to be set. Used when launching the node through <code>cnode.sh</code> and to identify the correct process of the node.</li> <li><code>CNODE_HOME</code> - The root directory of the Cardano node holding all the files needed. Can be left commented if <code>guild-deploy.sh</code> has been run as this variable is then exported and added as a system environment variable.</li> <li><code>POOL_NAME</code> - If the node is to be started as a block producer by <code>cnode.sh</code> this variable needs to be uncommented and set. This is the name given to the pool in CNTools (not ticker), i.e. the pool directory name under <code>$CNODE_HOME/priv/pool/&lt;POOL_NAME&gt;</code></li> </ul> <p>Take your time and look through the different variables and their explanations and decide if you need/want to change the default setting. For a default deployment using <code>guild-deploy.sh</code>, the <code>CNODE_PORT</code> (all installs) and <code>POOL_NAME</code> (only block producer) should be the only variables needed to be set.</p> <pre><code>######################################\n# User Variables - Change as desired #\n# Leave as is if unsure              #\n######################################\n\n#CCLI=\"${HOME}/.local/bin/cardano-cli\"                  # Override automatic detection of path to cardano-cli executable\n#CNCLI=\"${HOME}/.local/bin/cncli\"                       # Override automatic detection of path to cncli executable (https://github.com/AndrewWestberg/cncli)\n#CNODE_HOME=\"/opt/cardano/cnode\"                        # Override default CNODE_HOME path (defaults to /opt/cardano/cnode)\nCNODE_PORT=6000                                         # Set node port\n#CONFIG=\"${CNODE_HOME}/files/config.json\"               # Override automatic detection of node config path\n#SOCKET=\"${CNODE_HOME}/sockets/node.socket\"             # Override automatic detection of path to socket\n#TOPOLOGY=\"${CNODE_HOME}/files/topology.json\"           # Override default topology.json path\n#LOG_DIR=\"${CNODE_HOME}/logs\"                           # Folder where your logs will be sent to (must pre-exist)\n#DB_DIR=\"${CNODE_HOME}/db\"                              # Folder to store the cardano-node blockchain db\n#UPDATE_CHECK=\"Y\"                                       # Check for updates to scripts, it will still be prompted before proceeding (Y|N).\n#TMP_DIR=\"/tmp/cnode\"                                   # Folder to hold temporary files in the various scripts, each script might create additional subfolders\n#EKG_HOST=127.0.0.1                                     # Set node EKG host IP\n#EKG_PORT=12788                                         # Override automatic detection of node EKG port\n#PROM_HOST=127.0.0.1                                    # Set node Prometheus host IP\n#PROM_PORT=12798                                        # Override automatic detection of node Prometheus port\n#EKG_TIMEOUT=3                                          # Maximum time in seconds that you allow EKG request to take before aborting (node metrics)\n#CURL_TIMEOUT=10                                        # Maximum time in seconds that you allow curl file download to take before aborting (GitHub update process)\n#BLOCKLOG_DIR=\"${CNODE_HOME}/guild-db/blocklog\"         # Override default directory used to store block data for core node\n#BLOCKLOG_TZ=\"UTC\"                                      # TimeZone to use when displaying blocklog - https://en.wikipedia.org/wiki/List_of_tz_database_time_zones\n#SHELLEY_TRANS_EPOCH=208                                # Override automatic detection of shelley epoch start, e.g 208 for mainnet\n#TG_BOT_TOKEN=\"\"                                        # Uncomment and set to enable telegramSend function. To create your own BOT-token and Chat-Id follow guide at:\n#TG_CHAT_ID=\"\"                                          # https://cardano-community.github.io/guild-operators/Scripts/sendalerts\n#USE_EKG=\"N\"                                            # Use EKG metrics from the node instead of Promethus. Promethus metrics(default) should yield slightly better performance\n#TIMEOUT_LEDGER_STATE=300                               # Timeout in seconds for querying and dumping ledger-state\n#IP_VERSION=4                                           # The IP version to use for push and fetch, valid options: 4 | 6 | mix (Default: 4)\n\n#WALLET_FOLDER=\"${CNODE_HOME}/priv/wallet\"              # Root folder for Wallets\n#POOL_FOLDER=\"${CNODE_HOME}/priv/pool\"                  # Root folder for Pools\n# Each wallet and pool has a friendly name and subfolder containing all related keys, certificates, ...\n#POOL_NAME=\"\"                                           # Set the pool's name to run node as a core node (the name, NOT the ticker, ie folder name)\n\n#WALLET_PAY_VK_FILENAME=\"payment.vkey\"                  # Standardized names for all wallet related files\n#WALLET_PAY_SK_FILENAME=\"payment.skey\"\n#WALLET_HW_PAY_SK_FILENAME=\"payment.hwsfile\"\n#WALLET_PAY_ADDR_FILENAME=\"payment.addr\"\n#WALLET_BASE_ADDR_FILENAME=\"base.addr\"\n#WALLET_STAKE_VK_FILENAME=\"stake.vkey\"\n#WALLET_STAKE_SK_FILENAME=\"stake.skey\"\n#WALLET_HW_STAKE_SK_FILENAME=\"stake.hwsfile\"\n#WALLET_STAKE_ADDR_FILENAME=\"reward.addr\"\n#WALLET_STAKE_CERT_FILENAME=\"stake.cert\"\n#WALLET_STAKE_DEREG_FILENAME=\"stake.dereg\"\n#WALLET_DELEGCERT_FILENAME=\"delegation.cert\"\n\n#POOL_ID_FILENAME=\"pool.id\"                             # Standardized names for all pool related files\n#POOL_HOTKEY_VK_FILENAME=\"hot.vkey\"\n#POOL_HOTKEY_SK_FILENAME=\"hot.skey\"\n#POOL_COLDKEY_VK_FILENAME=\"cold.vkey\"\n#POOL_COLDKEY_SK_FILENAME=\"cold.skey\"\n#POOL_OPCERT_COUNTER_FILENAME=\"cold.counter\"\n#POOL_OPCERT_FILENAME=\"op.cert\"\n#POOL_VRF_VK_FILENAME=\"vrf.vkey\"\n#POOL_VRF_SK_FILENAME=\"vrf.skey\"\n#POOL_CONFIG_FILENAME=\"pool.config\"\n#POOL_REGCERT_FILENAME=\"pool.cert\"\n#POOL_CURRENT_KES_START=\"kes.start\"\n#POOL_DEREGCERT_FILENAME=\"pool.dereg\"\n\n#ASSET_FOLDER=\"${CNODE_HOME}/priv/asset\"                # Root folder for Multi-Assets containing minted assets and subfolders for Policy IDs\n#ASSET_POLICY_VK_FILENAME=\"policy.vkey\"                 # Standardized names for all multi-asset related files\n#ASSET_POLICY_SK_FILENAME=\"policy.skey\"\n#ASSET_POLICY_SCRIPT_FILENAME=\"policy.script\"           # File extension '.script' mandatory\n#ASSET_POLICY_ID_FILENAME=\"policy.id\"\n</code></pre>"},{"location":"Scripts/gliveview/","title":"gLiveView","text":"<p>Reminder !!</p> <p>Ensure the Pre-Requisites are in place before you proceed.</p> <p>Koios gLiveView is a local monitoring tool to use in addition to remote monitoring tools like Prometheus/Grafana, Zabbix or IOG's RTView. This is especially useful when moving to a systemd deployment - if you haven't done so already - as it offers an intuitive UI to monitor the node status.</p>"},{"location":"Scripts/gliveview/#configuration-startup","title":"Configuration &amp; Startup","text":"<p>For most setups, it's enough to set <code>CNODE_PORT</code> in the <code>env</code> file. The rest of the variables should automatically be detected. If required, modify User Variables in <code>env</code> and <code>gLiveView.sh</code> to suit your environment (if the environment is customised). This should lead you to a stage where you can now start running <code>./gLiveView.sh</code> in the folder you downloaded the script (the default location would be <code>$CNODE_HOME/scripts</code>). Note that the script is smart enough to automatically detect when you're running as a Core or Relay and will show fields accordingly.</p> <p>The tool can be run in legacy mode with only standard ASCII characters for terminals with trouble displaying the box-drawing characters. Run <code>./gLiveView.sh -h</code> to show available command-line parameters or permanently set it directly in script.</p> <p>Note !!</p> <p>Keeping gLiveView to it's intent of being a dashboard and not a full-fledged monitoring tool, we intend to keep most relevant information for a node operator in a minimalistic dashboard, accordingly - gLiveView runs by default in compact mode. One can enable verbose mode by pressing 'v' to unhide additional fields.</p> <p>A sample output from both core and relay together with peer analysis:</p> Core <p></p> Relay <p></p> Peer Analysis <p></p>"},{"location":"Scripts/gliveview/#upper-main-section","title":"Upper main section","text":"<p>Displays live metrics from cardano-node gathered through the nodes EKG/Prometheus(env setting) endpoint.</p> <ul> <li>Epoch Progress - Epoch number and progress is live from the node while date calculation until epoch boundary is based on offline genesis parameters.  </li> <li>Block - The nodes current block height since genesis start.  </li> <li>Slot - The nodes current slot height since current epoch start.  </li> <li>Density - With the current chain parameters(MainNet), a block is created roughly every 20 seconds(<code>activeSlotsCoeff</code>). A slot on MainNet happens every 1 second(<code>slotLength</code>), thus the max chain density can be calculated as <code>slotLength/activeSlotsCoeff = 5%</code>. Normally, the value should fluctuate around this value.  </li> <li>Total Tx - The total number of transactions processed since node start.  </li> <li>Pending Tx - The number of transactions and the bytes(total, in kb) currently in mempool to be included in upcoming blocks.  </li> <li>Tip (ref) - Reference tip is an offline calculation based on genesis values for current slot height since genesis start.  </li> <li>Tip (diff) / Status - Will either show node status as <code>starting|sync xx.x%</code> or if close to reference tip, the tip difference <code>Tip (ref) - Tip (node)</code> to see how far of the tip (diff value) the node is. With current parameters a slot diff up to 40 from reference tip is considered good but it should usually stay below 30. It's perfectly normal to see big differences in slots between blocks. It's the built in randomness at play. To see if a node is really healthy and staying on tip you would need to compare the tip between multiple nodes.  </li> <li>Forks - The number of forks since node start. Each fork means the blockchain evolved in a different direction, thereby discarding blocks. A high number of forks means there is a higher chance of orphaned blocks.  </li> <li>Peers In / Out - Shows how many connections the node has established in and out. See Peer analysis section for how to get more details of incoming and outgoing connections.  </li> <li>P2P Mode </li> <li><code>Cold</code> peers indicate the number of inactive but known peers to the node.</li> <li><code>Warm</code> peers tell how many established connections the node has.</li> <li><code>Hot</code> peers how many established connections are actually active.</li> <li><code>Bi-Dir</code>(bidirectional) and <code>Uni-Dir</code>(unidirectional) indicate how the handshake protocol negotiated the connection. The connection between p2p nodes will always be bidirectional, but it will be unidirectional between p2p nodes and non-p2p nodes. </li> <li><code>Duplex</code> shows the connections that are actually used in both directions, only bidirectional connections have this potential.</li> <li>Mem (RSS) - RSS is the Resident Set Size and shows how much memory is allocated to cardano-node and that is in RAM. It does not include memory that is swapped out. It does include memory from shared libraries as long as the pages from those libraries are actually in memory. It does include all stack and heap memory.  </li> <li>Mem (Live) / (Heap) - GC (Garbage Collector) values that show how much memory is used for live/heap data. A large difference between them (or the heap approaching the physical memory limit) means the node is struggling with the garbage collector and/or may begin swapping.  </li> <li>GC Minor / Major - Collecting garbage from \"Young space\" is called a Minor GC. Major (Full) GC is done more rarily and is a more expensive operation. Explaining garbage collection is a topic outside the scope of this documentation and google is your friend for this.  </li> <li>Block propagation - Last Block measures the duration between when the last block was scheduled to be produced and when the node learned about it. Late blocks are blocks whose delay is larger than 5s. If the node is not synching, the number of late blocks needs to stay low. Within \u2153/5s estimates the chance of observing a delay of \u2153/5s (based on the delays observed for previous blocks). A healthy node needs to stay above 95% of blocks within 3s. Finally, served blocks counts how many blocks were fetched by \"in\" peers. If this does not increase for a long time, it means the \"in\" peers are learning about new blocks from somewhere else (and therefore this node is not contributing towards accelerating the propagation). Overall, these metrics are helpful in tweaking the topology and/or performance of the network links.  </li> </ul>"},{"location":"Scripts/gliveview/#core-section","title":"Core section","text":"<p>If the node is run as a core, identified by the 'forge-about-to-lead' parameter, a second core section is displayed.</p> <ul> <li>KES period / expiration - This section contain the current and remaining KES periods as well as a calculated date for the expiration. When getting close to expire date the values will change color.  </li> <li> <p>Missed slot checks - A value that show if the node have missed slots for attempting leadership checks (as absolute value and percentage since node startup).   !!! info \"Missed Slot Leadership Check\"  </p> <p>Note that while this counter should ideally be close to zero, you would often see a higher value if the node is busy (e.g. paused for garbage collection or busy with reward calculations). A consistently high percentage of missed slots would need further investigation (assistance for troubleshooting can be seeked here ), as in extremely remote cases - it can overlap with a slot that your node could be a leader for.</p> </li> <li> <p>Blocks - If CNCLI is activated to store blocks created in a blocklog DB, data from this blocklog is displayed. See linked CNCLI documentation for details regarding the different block metrics. If CNCLI is not deployed, block metrics displayed are taken from node metrics and show blocks created by the node since node start.</p> </li> </ul>"},{"location":"Scripts/gliveview/#peer-analysis","title":"Peer analysis","text":"<p>A manual peer analysis can be triggered by key press <code>p</code>. A latency test will be done on incoming and outgoing connections to the node.</p> <p>Note</p> <p>Note that with P2P enabled, an incoming/outgoing connection can be reused for bi-directional traffic. There isnt a way to distinctly identify the P2P peer's direction yet for a given IP.</p> <p>Outgoing connections(peers in topology file), ping type used is done in this order: 1. cncli - If available, this gives the most accurate measure as it checks the entire handshake process against the remote peer. 2. ss - Sends a TCP SYN package to ping the remote peer on the <code>cardano-node</code> port. Should give ~100% success rate. 2. tcptraceroute - Same as ss. 3. ping - fallback method using ICMP ping against IP. Will only work if firewall of remote peer accept ICMP traffic.  </p> <p>For incoming connections, only ICMP ping is used as remote peer port is unknown. It's not uncommon to see many undetermined peers for incoming connections as it's a good security practice to disable ICMP in firewall.</p> <p>Once the analysis is finished, it will display the RTTs (return-trip times) for the peers and group them in ranges 0-50, 50-100, 100-200, 200&lt;. The analysis is NOT live. Press <code>[h] Home</code> to go back to default view or <code>[i] Info</code> to show in-script help text. <code>Up</code> and <code>Down</code> arrow keys is used to select incoming or outgoing detailed list of IPs and their RTT value. <code>Left (&lt;)</code> and <code>Right (&gt;)</code> arrow keys can be used to navigate the pages in the selected list.</p>"},{"location":"Scripts/gliveview/#troubleshootingcustomisations","title":"Troubleshooting/Customisations","text":"<p>In case you run into trouble while running the script, you might want to edit <code>env</code> &amp; <code>gLiveView.sh</code> and look at User Variables section. You can override the values if the automatic detection do not provide the right information, but we would appreciate if you could also notify us by raising an issue against the GitHub repository:</p> <p>gLiveView.sh <pre><code>######################################\n# User Variables - Change as desired #\n######################################\n\nNODE_NAME=\"Cardano Node\"                  # Change your node's name prefix here, keep at or below 19 characters!\nREFRESH_RATE=2                            # How often (in seconds) to refresh the view (additional time for processing and output may slow it down)\nLEGACY_MODE=false                         # (true|false) If enabled unicode box-drawing characters will be replaced by standard ASCII characters\nRETRIES=3                                 # How many attempts to connect to running Cardano node before erroring out and quitting\nPEER_LIST_CNT=6                           # Number of peers to show on each in/out page in peer analysis view\nTHEME=\"dark\"                              # dark  = suited for terminals with a dark background\n# light = suited for terminals with a bright background\nENABLE_IP_GEOLOCATION=\"Y\"                 # Enable IP geolocation on outgoing and incoming connections using ip-api.com\n</code></pre></p>"},{"location":"Scripts/itnrewards/","title":"Itnrewards","text":""},{"location":"Scripts/itnrewards/#concept","title":"Concept","text":"<p>To claim rewards earned during the Incentivized TestNet the private and public keys from ITN must be converted to Shelley stake keys. A script called <code>itnRewards.sh</code> has been created to guide you through the process of converting the keys and to create a CNTools compatible wallet from were the rewards can be withdrawn. </p> graph TB     A([\"itnRewards.sh\"])     A --x B([\"ITN Owner skey (ed25519[e]_sk)..\"]) --x D([\"cardano-cli shelley key convert-itn-key ..\"])     A --x C([\"ITN Owner vkey (ed25519_pk)..\"]) --x D     D --x E([\"Stake skey/vkey\"]) --x L     A --x F([\"cardano-cli shelley ..\"])     F --x G([\"Payment skey/vkey/addr\"]) --x L     F --x H([\"Reward addr\"]) --x L     F --x I([\"Base addr\"]) --x L     L[CNTools Wallet]     ;"},{"location":"Scripts/itnrewards/#steps","title":"Steps","text":"<ul> <li>If the secret key used for <code>jcli</code> account in ITN was ed25519_sk (not extended), you can run the <code>itnRewards.sh</code> script providing the name for the CNTools wallet and ITN owner public/secret keys that were used to register your pool as below.   <pre><code>cd $CNODE_HOME/scripts\n./itnRewards.sh MyITNWallet ~/jormu/account/priv/owner.sk ~/jormu/account/priv/owner.pk\n</code></pre></li> <li>Start CNTools and verify that the correct balance is shown in the wallet reward address</li> <li>Fund base address of the wallet with enough funds to pay the withdraw tx fee</li> <li>Use <code>FUNDS &gt;&gt; WITHDRAW</code> to move rewards to the base address of wallet</li> <li>You can now spend/move funds as you see fit</li> </ul>"},{"location":"Scripts/itnwitness/","title":"Itnwitness","text":"<p>Disclaimer</p> <p>Currently this is to protect the existing pools from the ITN who already have a delegator base against spoofing - to avoid scammers building on results of ITN from known pools. There would be a solution in the future for Mainnet nodes too - but it doesn't apply to those in its current form.</p>"},{"location":"Scripts/itnwitness/#concept","title":"Concept","text":"<p>Due to the expected ticker spoofing attack for pools that were famous during ITN, some of the community members have proposed an interim solution to verify the legitimacy of a pool for delegators. You can check the high-level workflow below:</p> graph TB     A(\"ITN Owner skey (ed25519/ed25519e) ..\") --x C([\"jcli key sign ..\"])     B(\"Haskell Pool ID (pool.id) ..\") --x C     C --x D(\"Signature key, (pool.sig) ..\")     E(\"ITN Owner vkey (ed25519_pk) ..\") --x F(\"Extended Metadata JSON (poolmeta_extended.json) ..\")     D --x F     F --x G(\"Pool Meta JSON (poolmeta.json) ..\")     ;"},{"location":"Scripts/itnwitness/#steps","title":"Steps","text":"<p>The actual implementation is pretty straightforward, we will keep it brisk - as we assume ones participating are fairly familiar with <code>jcli</code> usage.</p> <ul> <li>You need to use your owner keys that were used to register your pool, and it should match the owner public key you presented on official cardano-foundation github while registering metadata.</li> <li>Store your pool ID in a file (eg: <code>mainnet_pool.id</code>)</li> <li>Sign the file using your owner secret key from ITN (eg: <code>owner_skey</code>) as per below:   <pre><code>jcli key sign --secret-key ~/jormu/account/priv/owner.sk $CNODE_HOME/priv/pool/TEST/pool.id --output mainnet_pool.sig\ncat mainnet_pool.sig\n# ed25519_sig1sn32v3z...d72rg7rc6gs\n</code></pre></li> <li>Add this signature and owner public key to the extended pool JSON , so that it looks like below:   <pre><code>{\n\"itn\": {\n\"owner\": \"ed25519_pk1...\",\n\"witness\": \"ed25519_sig1...\"\n}\n}\n</code></pre></li> <li>Host this signature file online at a URL with raw contents easily accessible on the internet (eg: https://my.pool.com/extended-metadata.json)</li> <li>When you register/modify a pool using CNTools, use the above mentioned URL to add to your pool metadata.</li> </ul> <p>If the process is approved to appear for wallets, we may consider providing easier alternatives. If any queries about the process, or any additions please create a git issue/PR against guild repository - to capture common queries and update instructions/help text where appropriate.</p>"},{"location":"Scripts/itnwitness/#sample-output-of-json-files-generated","title":"Sample output of JSON files generated","text":"<ul> <li> <p>Metadata JSON used for registering pool (one that will be hosted URL used to define pool, eg: https://hosting.site/poolmeta.json)   <pre><code>{\n\"name\":\"Test\",\n\"ticker\":\"TEST\",\n\"description\":\"For demo purposes only\",\n\"homepage\":\"https://hosting.site\",\n\"nonce\":\"1595816423\",\n\"extended\":\"https://hosting.site/poolmeta_extended.json\"\n}\n</code></pre></p> </li> <li> <p>Extended Metadata JSON used for hosting additional metadata  (hosted at URL referred in <code>extended</code> field above, thus - eg : https://hosting.site/poolmeta_extended.json)</p> </li> </ul> <pre><code>{\n\"itn\": {\n\"owner\": \"ed25519_pk1...\",\n\"witness\": \"ed25519_sig1...\"\n}\n}\n</code></pre>"},{"location":"Scripts/logmonitor/","title":"Log Monitor","text":"<p>Reminder !!</p> <p>Ensure the Pre-Requisites are in place before you proceed.</p> <p><code>logMonitor.sh</code> is a general purpose JSON log monitoring script for traces created by <code>cardano-node</code>. Currently, it looks for traces related to leader slots and block creation but other uses could be added in the future. </p>"},{"location":"Scripts/logmonitor/#block-traces","title":"Block traces","text":"<p>For the core node (block producer) the <code>logMonitor.sh</code> script can be run to monitor the JSON log file created by <code>cardano-node</code> for traces related to leader slots and block creation.   </p> <p>For optimal coverage, it's best run together with CNCLI scripts as they provide different functionalities. Together, they create a complete picture of blocks assigned, created, validated or invalidated due to node issues. </p>"},{"location":"Scripts/logmonitor/#installation","title":"Installation","text":"<p>The script is best run as a background process. This can be accomplished in many ways but the preferred method is to run it as a systemd service. A terminal multiplexer like tmux or screen could also be used but not covered here.</p> <p>Use the <code>deploy-as-systemd.sh</code> script to create a systemd unit file (deployed together with CNCLI). Log output is handled by journald. <code>journalctl -f -u cnode-logmonitor.service</code> can be used to check service output (follow mode). Other logging configurations are not covered here.  </p>"},{"location":"Scripts/logmonitor/#view-blocklog","title":"View Blocklog","text":"<p>Best viewed in CNTools or gLiveView. See CNCLI for example output.</p>"},{"location":"Scripts/mithril-client/","title":"Client","text":"<p><code>mithril-client.sh</code> is a script to manage the Mithril client, a tool used to set up the Mithril client environment and manage downloading Mithril snapshots and stake distributions. The main features include:</p> <ul> <li>environment - Creates a new <code>mithril.env</code> file with all the necessary environment variables for the Mithril client.</li> <li>cardano-db - Download, list all or show a specific available Mithril snapshot.</li> <li>stake-distribution - Download or list available Mithril stake distributions.</li> <li>-u - Skip script update check.</li> </ul>"},{"location":"Scripts/mithril-client/#usage","title":"Usage","text":"<pre><code>Usage: bash [-u] &lt;command&gt; &lt;subcommand&gt; [&lt;sub arg&gt;]\nA script to run Cardano Mithril Client\n\n-u          Skip script update check overriding UPDATE_CHECK value in env (must be first argument to script)\n\nCommands:\nenvironment           Manage mithril environment file\n  setup               Setup mithril environment file\n  override            Override default variable in the mithril environment file\n  update              Update mithril environment file\ncardano-db            Interact with Cardano DB\n  download            Download Cardano DB from Mithril snapshot\n  snapshot            Interact with Mithril snapshots\n    list              List available Mithril snapshots\n      json            List availble Mithril snapshots in JSON format\n    show              Show details of a Mithril snapshot\n      json            Show details of a Mithril snapshot in JSON format\nstake-distribution    Interact with Mithril stake distributions\n  download            Download latest stake distribution\n  list                List available stake distributions\n    json              Output latest Mithril snapshot in JSON format\n</code></pre>"},{"location":"Scripts/mithril-client/#preparing-a-relay-or-block-producer-node","title":"Preparing a Relay or Block Producer Node","text":"<p>To prepare a relay or block producer node, you should follow these steps:</p> <ol> <li>Create the Mithril environment file: Run the script with the <code>environment setup</code> command. This will create a new <code>mithril.env</code> file with all the necessary environment variables for the Mithril client.</li> </ol> <pre><code>./mithril-client.sh environment setup\n</code></pre> <ol> <li>Download the latest Mithril snapshot: Once the environment file is set up, you can download the latest Mithril snapshot by running the script with the <code>snapshot download</code> command. This snapshot contains the latest state of the Cardano blockchain db from a Mithril Aggregator.</li> </ol> <pre><code>./mithril-client.sh cardano-db download\n</code></pre>"},{"location":"Scripts/mithril-client/#investigating-available-snapshots","title":"Investigating Available Snapshots","text":"<p>You can investigate the available snapshots by using the <code>snapshot list</code> and <code>snapshot show</code> commands:</p> <ul> <li>List all available Mithril snapshots: You can list all available Mithril snapshots by running the script with the <code>snapshot list</code> command. Add <code>json</code> at the end to get the output in JSON format.</li> </ul> <pre><code>./mithril-client.sh cardano-dbsnapshot list\n./mithril-client.sh cardano-dbsnapshot list json\n</code></pre> <ul> <li>Show details of a specific Mithril snapshot: You can show details of a specific Mithril snapshot by running the script with the <code>snapshot show &lt;DIGEST&gt;</code> command, where <code>&lt;DIGEST&gt;</code> is the digest of the snapshot. Add <code>json</code> at the end to get the output in JSON format.</li> </ul> <pre><code>./mithril-client.sh cardano-dbsnapshot show &lt;DIGEST&gt;\n./mithril-client.sh cardano-dbsnapshot show &lt;DIGEST&gt; json\n./mithril-client.sh cardano-dbsnapshot show json &lt;DIGEST&gt;\n</code></pre>"},{"location":"Scripts/mithril-client/#managing-stake-distributions","title":"Managing Stake Distributions","text":"<p>You can manage stake distributions by using the <code>stake-distribution download</code> and <code>stake-distribution list</code> commands:</p> <ul> <li>Download the latest Mithril stake distribution: You can download the latest Mithril stake distribution by running the script with the <code>stake-distribution download</code> command.</li> </ul> <pre><code>./mithril-client.sh stake-distribution download\n</code></pre> <ul> <li>List all available Mithril stake distributions: You can list all available Mithril stake distributions by running the script with the <code>stake-distribution list</code> command. Add <code>json</code> at the end to get the output in JSON format.</li> </ul> <pre><code>./mithril-client.sh stake-distribution list\n./mithril-client.sh stake-distribution list json\n</code></pre>"},{"location":"Scripts/mithril-relay/","title":"Relay","text":"<p><code>mithril-relay.sh</code> is a bash script for deployment of Squid Mithril Relays and a Nginx loadbalancer. It provides functionalities such as:</p> <ul> <li>Installing and configuring Squid as a relay for a Cardano Block Producer.</li> <li>Installing and configuring Nginx as a load balancer for multiple Mithril Relays.</li> </ul>"},{"location":"Scripts/mithril-relay/#usage","title":"Usage","text":"<pre><code>bash [-d] [-l] [-u] [-h]\nA script to setup Cardano Mithril relays\n\n-d  Install squid and configure as a relay\n-l  Install nginx and configure as a load balancer\n-u  Skip update check\n-h  Show this help text\n</code></pre>"},{"location":"Scripts/mithril-relay/#description","title":"Description","text":"<p>The <code>mithril-relay.sh</code> script is a bash script for managing the Mithril Relay Server. It provides functionalities such as installing and configuring Squid as a relay, installing and configuring Nginx as a load balancer.</p>"},{"location":"Scripts/mithril-relay/#environment-variables","title":"Environment Variables","text":"<p>The script uses the following environment variable:</p> <ul> <li><code>RELAY_LISTENING_PORT</code>: The port on which the relay server listens.</li> </ul>"},{"location":"Scripts/mithril-relay/#execution","title":"Execution","text":"<p>The script parses command line options and performs the corresponding actions based on the options provided. If the <code>-d</code> option is provided, it installs Squid and configures it as a relay. If the <code>-l</code> option is provided, it installs Nginx and configures it as a load balancer. If no options are provided, it displays the usage message.</p>"},{"location":"Scripts/mithril-signer/","title":"Signer","text":"<p><code>mithril-signer.sh</code> is a bash script for managing the Mithril Signer Server. It provides functionalities such as deploying the server as a systemd service and updating the environment file to contain variables specific to the Mithril Signer.</p>"},{"location":"Scripts/mithril-signer/#usage","title":"Usage","text":"<pre><code>Usage: bash [-d] [-D] [-e] [-k] [-r] [-s] [-u] [-h]\nA script to setup, run and verify Cardano Mithril Signer\n\n-d    Deploy mithril-signer as a systemd service\n-D    Run mithril-signer as a daemon\n-e    Update mithril environment file\n-k    Stop signer using SIGINT\n-r    Verify signer registration\n-s    Verify signer signature\n-u    Skip update check\n-h    Show this help text\n</code></pre>"},{"location":"Scripts/mithril-signer/#description","title":"Description","text":"<p>This script is a bash script for managing the Mithril Signer Server. It provides functionalities such as deploying the server as a systemd service, updating the environment file, and running the server.</p>"},{"location":"Scripts/mithril-signer/#environment-variables","title":"Environment Variables","text":"<p>The script uses several environment variables, some of which are:</p> <ul> <li><code>MITHRILBIN</code>: Path for mithril-signer binary, if not in <code>$PATH</code>.</li> <li><code>HOSTADDR</code>: Default Listen IP/Hostname for Mithril Signer Server.</li> <li><code>POOL_NAME</code>: The name of the pool.</li> <li><code>NETWORK_NAME</code>: The name of the network.</li> </ul>"},{"location":"Scripts/mithril-signer/#execution","title":"Execution","text":"<p>The script parses command line options, sources the environment file, sets default values, and performs basic sanity checks. It then checks if the <code>-d</code> or <code>-u</code> options were specified and performs the corresponding actions. If no options were specified, it runs the Mithril Signer Server.</p>"},{"location":"Scripts/sendalerts/","title":"Sendalerts","text":"<p>!&gt; Ensure the Pre-Requisites are in place before you proceed.</p> <p>This section describes the ways in which CNTools can send important messages to the operator.</p>"},{"location":"Scripts/sendalerts/#telegram-alerts","title":"Telegram alerts","text":"<p>If known but unwanted errors occur on your node, or if characteristic values indicate an unusual status , CNTools can send you Telegram alert messages. </p> <p>To do this, you first have to activate your own bot and link it to your own Telegram user. Here is an explanation of how this works:</p> <ol> <li> <p>Open Telegram and search for \"botfather\".</p> </li> <li> <p>Write him your wish: <code>/newbot</code>.</p> </li> <li> <p>Define a name for your bot, such as <code>cntools_[POOLNAME]_alerts</code>.</p> </li> <li> <p>Botfather will confirm the creation of your bot by giving you the unique bot access token. Keep it safe and private.</p> </li> <li> <p>Now send at least one direct message to your new bot.</p> </li> <li> <p>Open this URL in your browser by using your own, just created bot access token:</p> </li> </ol> <pre><code>https://api.telegram.org/bot&lt;your-access-token&gt;/getUpdates\n</code></pre> <ol> <li>the result is a JSON. Look for the value of <code>result.message.chat.id</code>.     This chat id should be a large integer number.</li> </ol> <p>This is all you need to enable your Telegram alerts in the <code>scripts/env</code> file - uncomment and add the chat ID to the <code>TG_CHAT_ID</code> user variable in the <code>env</code> file: <pre><code>...\nTG_CHAT_ID=\"&lt;YOUR_TG_CHAT_ID&gt;\"\n...  \n</code></pre></p>"},{"location":"Scripts/topologyupdater/","title":"Topology Updater","text":"<p>Reminder !!</p> <ul> <li>Starting cardano-node release 8.9.2, it is recommended to run node with P2P config enabled. If running node with P2P, one does not require to run topology updater anymore.</li> <li>Since the network had to get along without the P2P network module for a while, it needed static topology files. This \"TopologyUpdater\" service, which was far from being perfect due to its centralization factor, was intended to be a temporary solution to allow everyone to activate their relay nodes without having to postpone and wait for manual topology completion requests.</li> <li>You should NOT set up topologyUpdater for your block producing nodes.</li> </ul> <p>The topologyUpdater shell script must be executed on the relay node as a cronjob exactly every 60 minutes. After 4 consecutive requests (3 hours) the node is considered a new relay node in listed in the topology file. If the node is turned off, it's automatically delisted after 3 hours.</p>"},{"location":"Scripts/topologyupdater/#download","title":"Download and Configure","text":"<p>If you have run guild-deploy.sh, this should already be available in your scripts folder and make this step unnecessary.</p> <p>Before the updater can make a valid request to the central topology service, it must query the current tip/blockNo from the well-synced local node. It connects to your node through the configuration in the script as well as the common <code>env</code> configuration file. Customize these files for your needs.</p> <p>To download <code>topologyUpdater.sh</code> manually, you can execute the commands below and test executing Topology Updater once (it's OK if first execution gives back an error): <pre><code>cd $CNODE_HOME/scripts\ncurl -s -o topologyUpdater.sh https://raw.githubusercontent.com/cardano-community/guild-operators/master/scripts/cnode-helper-scripts/topologyUpdater.sh\ncurl -s -o env https://raw.githubusercontent.com/cardano-community/guild-operators/master/scripts/cnode-helper-scripts/env\nchmod 750 topologyUpdater.sh\n./topologyUpdater.sh\n</code></pre></p>"},{"location":"Scripts/topologyupdater/#modify","title":"Examine and modify the variables within topologyUpdater.sh script","text":"<p>Out of the box, the scripts might come with some assumptions, that may or may not be valid for your environment. One of the common changes as an SPO would be to the complete CUSTOM_PEERS section as below to include your local relays/BP nodes (described in the How do I add my own nodes section), and any additional peers you'd like to be always available at minimum. Please do take time to update the variables in User Variables section in  <code>env</code> &amp; <code>topologyUpdater.sh</code>:</p> <pre><code>### topologyUpdater.sh\n\n######################################\n# User Variables - Change as desired #\n######################################\n\nCNODE_HOSTNAME=\"CHANGE ME\"                                # (Optional) Must resolve to the IP you are requesting from\nCNODE_VALENCY=1                                           # (Optional) for multi-IP hostnames\nMAX_PEERS=15                                              # Maximum number of peers to return on successful fetch\n#CUSTOM_PEERS=\"None\"                                      # Additional custom peers to (IP,port[,valency]) to add to your target topology.json\n# eg: \"10.0.0.1,3001|10.0.0.2,3002|relays.mydomain.com,3003,3\"\n#BATCH_AUTO_UPDATE=N                                      # Set to Y to automatically update the script if a new version is available without user interaction\n</code></pre> <p>Any customisations you add above, will be saved across future <code>guild-deploy.sh</code> executions, unless you specify the <code>-f</code> flag to overwrite completely.</p>"},{"location":"Scripts/topologyupdater/#deploy","title":"Deploy the script","text":"<p>systemd service The script can be deployed as a background service in different ways but the recommended and easiest way if guild-deploy.sh was used, is to utilize the <code>deploy-as-systemd.sh</code> script to setup and schedule the execution. This will deploy both push &amp; fetch service files as well as timers for a scheduled 60 min node alive message and cnode restart at the user set interval (default: 24 hours) when running the deploy script.</p> <ul> <li><code>cnode-tu-push.service</code>    : pushes a node alive message to Topology Updater API</li> <li><code>cnode-tu-push.timer</code>      : schedules the push service to execute once every hour</li> <li><code>cnode-tu-fetch.service</code>   : fetches a fresh topology file before the <code>cnode.service</code> file is started/restarted</li> <li><code>cnode-tu-restart.service</code> : handles the restart of <code>cardano-node</code> (<code>cnode.sh</code>)</li> <li><code>cnode-tu-restart.timer</code>   : schedules the <code>cardano-node</code> restart service, default every 24h</li> </ul> <p><code>systemctl list-timers</code> can be used to to check the push and restart service schedule.</p> <p>crontab job Another way to deploy the <code>topologyUpdater.sh</code> script is as a <code>crontab</code> job. Add the script to be executed once per hour at a minute of your choice (eg xx:25 o'clock in the example below). The example below will handle both the fetch and push in a single call to the script once an hour. In addition to the below crontab job for topologyUpdater, it's expected that you also add a scheduled restart of the relay node to pick up a fresh topology file fetched by topologyUpdater script with relays that are alive and well.</p> <pre><code>25 * * * * /opt/cardano/cnode/scripts/topologyUpdater.sh\n</code></pre>"},{"location":"Scripts/topologyupdater/#logs","title":"Logs","text":"<p>You can check the last result of push message in <code>logs/topologyUpdater_lastresult.json</code>. If deployed as systemd service, use <code>sudo journalctl -u &lt;service&gt;</code> to check output from service.</p> <p>If one of the parameters is outside the allowed ranges, invalid or missing the returned JSON will tell you what needs to be fixed.</p> <p>Don't try to execute the script more often than once per hour. It's completely useless and may lead to a temporary blacklisting.</p>"},{"location":"Scripts/topologyupdater/#why-does-my-topology-file-only-contain-iog-peers","title":"Why does my topology file only contain IOG peers?","text":"<p>Each subscribed node (4 consecutive requests) is allowed to fetch a subset of other nodes to prove loyalty/stability of the relay. Until reaching this point, your fetch calls will only return IOG peers combined with any custom peers added in USER VARIABLES section of <code>topologyUpdater.sh</code> script</p> <p>The engineers of <code>cardano-node</code> network stack suggested to use around 20 peers. More peers create unnecessary and unwanted system load and delays.</p> <p>In its default setting, topologyUpdater returns a list of 15 remote peers. </p> <p>Note that the change in topology is only effective upon restart of your node. Make sure you account for some scheduled restarts on your relays, to help onboard newer relays onto the network (as described in the systemd section).</p>"},{"location":"Scripts/topologyupdater/#how-do-i-add-my-own-relaysstatic-nodes-in-addition-to-dynamic-list-generated-by-topologyupdater","title":"How do I add my own relays/static nodes in addition to dynamic list generated by topologyUpdater?","text":"<p>Most of the Stake Pool Operators may have few preferences (own relays, close friends, etc) that they would like to add to their topology by default. This is where the <code>CUSTOM_PEERS</code> variable in <code>topologyUpdater.sh</code> comes in. You can add a list of peers in the format of: <code>hostname/IP:port[:valency]</code> here and the output <code>topology.json</code> formed will already include the custom peers that you supplied. Every custom peer is defined in the form <code>[address]:[port]</code> and optional <code>:[valency]</code> (if not specified, the valency defaults to <code>1</code>). Multiple custom peers are separated by <code>|</code>. An example of a valid <code>CUSTOM_PEERS</code> variable would be:</p> <p><pre><code>CUSTOM_PEERS=\"foo.bar.io,3001,2|198.175.21.197,6001|36.233.3.89,6000\n</code></pre> The list above would add three custom peers with the specified addresses and ports, with the first one additionally specifying the optional valency parameter (in this case <code>2</code>).</p>"},{"location":"Scripts/topologyupdater/#how-are-the-peers-for-my-topology-file-selected","title":"How are the peers for my topology file selected?","text":"<p>We calculate the distance on the Earth's surface from your node's IP to all subscribed peers. We then order the peers by distance (closest first) and start by selecting one peer. We then skip some, pick the next, skip, pick, skip, pick ... until we reach the end of the list (furthest away). The number of skipped records is calculated in a way to have the desired number of peers at the end.</p> <p>Every requesting node has its personal distance to all other nodes.</p> <p>We assume this should result in a well-distributed and interconnected peering network.</p>"},{"location":"docker/build/","title":"Build","text":""},{"location":"docker/build/#intro","title":"Intro","text":"<p>\ud83d\udca1 Docker containers are the fastest way to run a Cardano node in both \"Relay\" and \"Block-Producing\" (Pool) mode.</p>"},{"location":"docker/build/#how-to-build","title":"How to build","text":"<pre><code>docker build -t cardanocommunity/cardano-node:latest - &lt; dockerfile_bin\n</code></pre>"},{"location":"docker/build/#for-windows-users","title":"For Windows Users","text":"<p>With Powershell on Windows, you can run docker by typing the following command:</p> <pre><code>Get-Content dockerfile_bin  | docker build -t guild-operators/cardano-node:latest -\n</code></pre>"},{"location":"docker/build/#see-also","title":"See also","text":"<p>Docker Tips</p> <p>Docker Official Docs</p>"},{"location":"docker/docker/","title":"Overview","text":"<p>Running your own Cardano node has never been so fast and easy.</p> <p>But first, a kind reminder to the security aspects of running docker containers.</p>"},{"location":"docker/docker/#external-resources","title":"External resources","text":"<ul> <li>DockerHub Guild's images</li> <li>YouTube Guild's Videos</li> </ul>"},{"location":"docker/docker/#built-in-cardano-software","title":"\ud83d\udd14 Built-in Cardano software","text":"<ul> <li>cardano-address</li> <li>cardano-cli</li> <li>cardano-hw-cli</li> <li>cardano-node</li> <li>cardano-submit-api</li> <li>mithril-client</li> <li>mithril-signer</li> </ul>"},{"location":"docker/docker/#mithril","title":"Mithril","text":""},{"location":"docker/docker/#built-in-tools","title":"\ud83d\udd14 Built-in tools","text":"<ul> <li>CNTools</li> <li>gLiveView</li> <li>CNCLI</li> <li>Ogmios</li> <li>Cardano Hardware CLI</li> <li>Cardano Signer</li> <li>Monitoring ready (with EKG and Prometheus)</li> </ul>"},{"location":"docker/docker/#docker-splash-screen","title":"Docker Splash screen","text":""},{"location":"docker/docker/#cntools","title":"Cntools","text":""},{"location":"docker/docker/#gliveview","title":"gLiveView","text":""},{"location":"docker/docker/#gliveview-peers-analyzer","title":"gLiveView Peers analyzer","text":""},{"location":"docker/docker/#cncli","title":"CNCLI","text":""},{"location":"docker/docker/#strategy","title":"Guild Operators Docker strategy ( mainnet/ preview / preprod / guild)","text":"<p>Modular docker images based on Debian.</p> <p>Based on the Guild's work the Cardano Node image is built in a single stage: -&gt; dockerfile_bin</p> <ul> <li>Uses <code>guild-deploy.sh</code> to:</li> <li>Install the os prerequisites</li> <li>Add the cardano software from release binaries</li> <li>Add the guild's SPO tools and the node's configuration files.</li> </ul>"},{"location":"docker/docker/#additional-docs","title":"Additional docs","text":"<p>If you prefer to build the images your own than you can check:</p> <ul> <li>Docker Build Documentation</li> <li>Docker Tips</li> </ul>"},{"location":"docker/docker/#port-mapping","title":"Port mapping","text":"<p>The dockerfiles are located in ./files/docker/</p> Node Ports Wallet Ports Flavor Node  (6000) Wallet (8090) Debian Prometheus (12798) Prometheus (12798) EKG (12781)"},{"location":"docker/run/","title":"Run","text":""},{"location":"docker/run/#os-requirements","title":"OS Requirements","text":"<ul> <li><code>docker-ce</code> installed - Get Docker.</li> </ul> Private mode Public mode <p>Note</p> <p>1) <code>--entrypoint=bash</code> # This option won't start the node's container but only the OS running (the node software wont actually start, you'll need to manually execute entrypoint.sh ), ready to get in (trough the command <code>docker exec -it &lt; container name or hash &gt;  /bin/bash</code>) and play/explore around with it in command line mode. 2) all guild tools env variable can be used to start a new container using custom values by using the \"-e\" option. 3) CPU and RAM and Shared Memory allocation option for the container can be used when you start the container (i.e. --shm-size or --memory or --cpus official docker resource docs) 4) <code>--env MITHRIL_DOWNLOAD=Y</code> # This option will allow Mithril client to download the latest Mithril snapshot of the blockchain when the container starts and does not have a copy of the blockchain yet. This is useful when you want to start a new node from scratch and don't want to wait for the node to sync from the network. This option is not currently available for the guild network. 5) <code>--env ENTRYPOINT_PROCESS=mithril-signer.sh</code> # This option will allow the container to start the Mithril signer process instead of the node process. This is useful when you want to run a Mithril signer node and have the container setup the configuration files based on the NETWORK environment varaible.</p>"},{"location":"docker/run/#use-cases","title":"Use Cases","text":"<ul> <li>Pool Management</li> <li>Wallet Management</li> <li>Node testing</li> </ul> <pre><code>docker run --init -dit\n--name &lt;YourCName&gt;\n--security-opt=no-new-privileges\n-e NETWORK=mainnet\n-v &lt;your_custom_path&gt;:/opt/cardano/cnode/priv\n-v &lt;your_custom_db_path&gt;:/opt/cardano/cnode/db\ncardanocommunity/cardano-node\n</code></pre>"},{"location":"docker/run/#use-cases_1","title":"Use Cases:","text":"<ul> <li>Node Relay</li> </ul> <pre><code>docker run --init -dit\n--name &lt;YourCName&gt;\n--security-opt=no-new-privileges\n-e NETWORK=mainnet\n-p 6000:6000\n-v &lt;your_custom_path&gt;:/opt/cardano/cnode/priv\n-v &lt;your_custom_db_path&gt;:/opt/cardano/cnode/db\ncardanocommunity/cardano-node\n</code></pre> <ul> <li>Node Relay with custom permanent cfg by passing the env variable CONFIG    (Mapping your configuration folder as below will allow you to retain configurations if you update or delete your container)</li> </ul> <pre><code>docker run --init -dit\n--name &lt;YourCName&gt;\n--security-opt=no-new-privileges\n-e NETWORK=mainnet\n-e CONFIG=/opt/cardano/cnode/priv/&lt;your own configuration files&gt;.yml\n-p 6000:6000\n-v &lt;your_custom_path&gt;:/opt/cardano/cnode/priv\n-v &lt;your_custom_db_path&gt;:/opt/cardano/cnode/db\ncardanocommunity/cardano-node\n</code></pre>"},{"location":"docker/security/","title":"Security","text":""},{"location":"docker/security/#docker-security-best-practices","title":"Docker Security best practices","text":""},{"location":"docker/security/#intro","title":"Intro","text":"<p>On the security front, Docker developers are faced with different types of security attacks such as:</p> <ul> <li>Kernel exploits: Since the host\u2019s kernel is shared in the container, a compromised container can attack the entire host.</li> <li>Container breakouts: Caused when the user is able to escape the container namespace and interact with other processes on the host.</li> <li>Denial-of-service attacks: Occur when some containers take up enough resources to hamper the functioning of other applications.</li> <li>Poisoned images: Caused when an untrusted image is being run and a hacker is able to access application data and, potentially, the host itself.</li> </ul> <p>Docker containers are now being exploited to covertly mine for cryptocurrency, marking a shift from ransomware to cryptocurrency malware.  As with all things in security, also Docker security is a moving target \u2014 so it\u2019s helpful to have access to up-to-date information, including experience-based best practices, for securing your containerized environments.</p>"},{"location":"docker/security/#here-below-some-key-concepts","title":"Here below some key concepts:","text":"<ol> <li> <p>Use a Third-Party Security Tool Docker allows you to use containers from untrusted public repositories, which increases the need to scrutinize whether the container was created securely and whether it is free of any corrupt or malicious files. For this, use a multi-purpose security tool that gives extensive dev-to-production security controls.(keep reading below)</p> </li> <li> <p>Manage Vulnerability It is best to have a sound vulnerability management program that has multiple checks throughout the container lifecycle. Vulnerability management should incorporate quality gates to detect access issues and weaknesses for a potential exploit from dev-to-production environments.</p> </li> <li> <p>Monitor and Audit Container Activity It is vital to monitor the container ecosystem and detect suspicious activity. Container monitoring activities provide real-time reports that can help you react promptly to a security breach.</p> </li> <li> <p>Enable Docker Content Trust Docker Content Trustis a new feature incorporated into Docker 1.8. It is disabled by default, but once enabled, allows you to verify the integrity, authenticity, and publication date of all Docker images from the Docker Hub Registry.</p> </li> <li> <p>Use Docker Bench for Security You should consider Docker Bench for Security as your must-use script. Once the script is run, you will notice a lot of information regarding configuration best practices for deploying Docker containers that can be used to further secure your Docker server and containers.</p> </li> <li> <p>Resource Utilization To reduce performance impacts and denial-of-service attacks, it is a good practice to implement limits on the system resources that the containers can consume. If, for example, a web server is compromised, it helps to limit the impact to the other processes that are running on a host.</p> </li> <li> <p>RBAC RBAC is role-based access control. If you have multiple users accessing you enviroment, this is a must-have. It can be quite expensive to implement but portainer makes it super easy.</p> </li> </ol>"},{"location":"docker/security/#security-docker-best-practices","title":"Security Docker best practices:","text":""},{"location":"docker/security/#the-guild-docker-images-are-not-using-all-the-following-tips-due-to-functional-purpose","title":"The Guild Docker images are not using all the following tips due to functional purpose","text":"<p>Guild tips:</p> <ul> <li> <p><code>NEVER NEVER NEVER expose Docker API publicly!!!</code> (disabled by default)</p> </li> <li> <p>Keep Docker Host Up-to-date</p> </li> <li>Reverse uptime: containers that are frequently shut down and replaced by new container are more difficult for hackers to attack.</li> <li>Use a Firewall or Expose only the ports you need to be public.</li> <li>Use a *<code>Reverse Proxy</code></li> <li>Do not Change **<code>Docker Socket Ownership</code></li> <li>Do not <code>Run Docker Containers as Root</code></li> <li><code>Use Trusted Docker Images</code></li> <li><code>Use Privileged Mode Carefully</code> (This is usually done by adding --privileged you can use <code>--security-opt=no-new-privileges</code> instead)</li> </ul> <p>Some more general tips:</p> <ul> <li>Restrict container capabilities: <code>\"--cap-drop ALL\"</code></li> <li>Use Docker Secrets</li> <li>Change DOCKER_OPTS to ***Respect IP Table Firewall </li> <li>Control Docker Resource Usage</li> <li>Rate Limit: is quite common to mitigate brute force or denial of service attacks.</li> <li>Fail2ban: Fail2ban scans your log files and bans IP address that shows malicious intent</li> <li>Container Vulnerability Scanner</li> </ul>"},{"location":"docker/security/#notes","title":"Notes:","text":"<ul> <li>*Nginx is a very good choice as load balancer and/or reverse proxy.</li> <li>**By default the socket is owned by root user and docker group.</li> <li>*** On Ubuntu/Debian based systems, edit /etc/default/docker and add the following line: <code>DOCKER_OPTS= \"--iptables=false\"</code></li> </ul>"},{"location":"docker/tips/","title":"Tips","text":""},{"location":"docker/tips/#how-to-run-a-cardano-node-with-docker","title":"How to run a Cardano Node with Docker","text":"<p>With this quick guide you will be able to run a cardano node in seconds and also have the powerfull Koios SPO scripts built-in.</p>"},{"location":"docker/tips/#how-to-operate-interactively-within-the-container","title":"How to operate interactively within the container","text":"<p>Once executed the container as a deamon with attached tty you are then able to enter the container by using the flag <code>-dit</code> .</p> <p>While if you have a hook within the container console, use the following command (change <code>CN</code> with your container name):</p> <pre><code>docker exec -it CN bash </code></pre> <p>This command will bring you within the container bash env ready to use the Koios tools.</p>"},{"location":"docker/tips/#docker-flags-explained","title":"Docker flags explained","text":"<pre><code>\"docker build\" options explained:\n -t : option is to \"tag\" the image you can name the image as you prefer as long as you maintain the references between dockerfiles.\n\n\"docker run\" options explained:\n -d : for detach the container\n -i : interactive enabled -t : terminal session enabled\n -e : set an Env Variable\n -p : set exposed ports (by default if not specified the ports will be reachable only internally)\n--hostname : Container's hostname\n --name : Container's name\n</code></pre>"},{"location":"docker/tips/#custom-container-with-your-own-cfg","title":"Custom container with your own cfg","text":"<pre><code>docker run --init -itd  \n-name Relay                                   # Optional (recommended for quick access): set a name for your newly created container.\n-p 9000:6000                                  # Optional: to expose the internal container's port (6000) to the host &lt;IP&gt; port 9000\n-e NETWORK=mainnet                            # Mandatory: mainnet / preprod / guild-mainnet / guild\n--security-opt=no-new-privileges              # Option to prevent privilege escalations\n-v &lt;YourNetPath&gt;:/opt/cardano/cnode/sockets   # Optional: useful to share the node socket with other containers\n-v &lt;YourCfgPath&gt;:/opt/cardano/cnode/priv      # Optional: if used has to contain all the sensitive keys needed to run a node as core\n-v &lt;YourDBbk&gt;:/opt/cardano/cnode/db           # Optional: if not set a fresh DB will be downloaded from scratch\ncardanocommunity/cardano-node:latest          # Mandatory: image to run\n</code></pre> <p>Note</p> <p>To be able to use the CNTools encryption key feature you need to manually change in \"cntools.config\" ENABLE_CHATTR to \"true\" and not use the <code>--security-opt=no-new-privileges</code> docker run option.</p>"},{"location":"docker/tips/#docker-cli-managment","title":"Docker CLI managment","text":""},{"location":"docker/tips/#official","title":"Official","text":"<ul> <li>docker inspect</li> <li>docker ps</li> <li>docker ls</li> <li>docker stop</li> </ul>"},{"location":"docker/tips/#un-official-docker-managment-cli-tool","title":"Un-Official Docker managment cli tool","text":"<ul> <li>Lazydocker</li> </ul>"},{"location":"docker/tips/#docker-backups-and-restores","title":"Docker backups and restores","text":"<p>The docker container has an optional backup and restore functionality that can be used to backup the <code>/opt/cardano/cnode/db</code> directory. To have the  backup persist longer than the countainer, the backup directory should be mounted as a volume.</p> <p>[!NOTE] The backup and restore functionality is disabled by default.</p> <p>[!WARNING] Make sure adequate space exists on the host as the backup will double the space consumed by the database. </p>"},{"location":"docker/tips/#creating-a-backup","title":"Creating a Backup","text":"<p>When the container is started with the ENABLE_BACKUP environment variable set to Y the container will automatically create a backup in the <code>/opt/cardano/cnode/backup/$NETWORK-db</code> directory. The backup will be created when the container is started and if the backup directory is smaller than the db directory.</p>"},{"location":"docker/tips/#restoring-from-a-backup","title":"Restoring from a Backup","text":"<p>When the container is started with the ENABLE_RESTORE environment variable set to Y the container will automatically restore the latest backup from the <code>/opt/cardano/cnode/backup/$NETWORK-db</code> directory. The database will be restored when the container is started and if the backup directory is larger than the db directory.</p>"},{"location":"docker/tips/#configuration-update-check-functionality","title":"Configuration Update Check Functionality","text":"<p>The container now includes a static copy of each network's configuration files (Mainnet, Preprod, Preview, Sanchonet, and Guild networks). The <code>NETWORK</code> environment variable passed into the container determines which configuration files are copied into <code>$CNODE_HOME/files</code>.</p> <p>The <code>UPDATE_CHECK</code> environment variable controls whether the container updates these configuration files from GitHub before starting. By default, the container has the environment variable set to <code>UPDATE_CHECK=N</code>, meaning the container uses the configuration files it was built with. This can be overriden either persistently or dynamically.</p>"},{"location":"docker/tips/#persistently-updating-configuration-files","title":"Persistently updating configuration files","text":"<p>To always update the configuration files from GitHub, set the <code>UPDATE_CHECK</code> environment variable when creating the container by using the <code>--env</code> option, for example <code>--env UPDATE_CHECK=Y</code>.</p> <p>To always update the configuration files from a specific GitHub account, set the <code>G_ACCOUNT</code> environment variable when creating the container by using the <code>--env</code> option, for example <code>--env G_ACCOUNT=gh-fork-user</code>.</p> <p>[!NOTE] There is no way to change the environment variable of an already running container. To rollback the configuration files and scripts stop and remove the container and start it without setting the environment variable.</p>"},{"location":"docker/tips/#dynamically-updating-configuration-files","title":"Dynamically updating configuration files","text":"<p>Set an environment file during create/run using <code>--env-file=file</code>, for example <code>--env-file=/opt/cardano/cnode/.env</code>.</p> <ul> <li>When <code>UPDATE_CHECK</code> is not defined in the environment file, the container will use the built-in configs.</li> <li>When <code>UPDATE_CHECK=Y</code> is defined in the environment file the container will update configs and scripts from the   <code>cardano-community</code> GitHub repository.</li> <li>When <code>G_ACCOUNT</code> is defined in the environment file, the container will update configs and scripts from the GitHub   repository of the specified account.</li> </ul> <p>To rollback the configuration files to the built-in versions, remove the <code>UPDATE_CHECK=Y</code> or set it to <code>UPDATE_CHECK=N</code> in the environment file. The static configuration files in the container will be used, however the scripts will remain updated. If you want both the configuration files and scripts to be rolled back, you will need to stop and remove the container and create a new one.</p>"},{"location":"docker/tips/#building-images-from-forked-repositories","title":"Building Images from Forked Repositories","text":"<p>Run the Docker Image GitHub Action to build and push images to the <code>ghcr.io</code> registry.</p> <ul> <li>The <code>G_ACCOUNT</code> will be inherited from the <code>GITHUB_REPOSITORY_OWNER</code>.</li> <li>It will be all lowercase so it matches container image name requirements.</li> <li>All images not from master branch or when Testing workflow is checked will be pushed to <code>ghcr.io</code>.</li> <li>Images from the master branch will also be pushed to the <code>ghcr.io</code> registry as long as the Testing workflow remains checked.</li> </ul>"}]}
\ No newline at end of file
+{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":"<p>This documentation site (rather the repository itself) is created by some of the well known and experienced community members and contains instructions/information about various guild tools which simplify various stake-ops (setting up, managing and monitoring pools) for operators. Note that the guides are present to help you simplify your tasks - but as an entity responsible for creating blocks on a financial platform, we expect some basic pre-requisite skill sets - at professional level - before entering the portal:</p> <ul> <li>Learn about working with architecture, setup and essentials (SysOps);</li> <li>Understand how to secure your server (SecOps);</li> <li>Be comfortable with working with <code>cardano-cli</code>, and have worked on preview/preprod/guild networks for pool operations without use of wrapper scripts - as an education exercise;</li> <li>Read the documentation and disclaimers. The guide and tools only aim to simplify your tasks, it will not try to do everything for you - neither scope best practices;</li> <li>We recommend anyone using these instructions to use offline workflow, and NEVER have their wallet keys and pool keys to be made available to online servers, as supported by the tools.</li> </ul> <p>Everyone is welcome to contribute to the repository (via documentation, testing, code, videos, etc). Our aim is to work together and reduce confusion rather than hosting 100 versions of documentation - each marketing their pool in a way.</p>"},{"location":"#support","title":"Support","text":"<p>The Telegram Support channel is used to announce new releases and changes to the code base. This is also the place to ask general questions regarding the documentation and scripts on this site.  </p> <p>To report bugs and issues with scripts and documentation please open a GitHub Issue. Feature requests are best opened as a discussion thread.</p>"},{"location":"#getting-started","title":"Getting Started","text":"<p>Use the sidebar to navigate through the topics. Note that the instructions assume the folder structure as per here.</p> <p>Again, Feedback/Contribution and ownership of tasks is always welcome. If you're interested in collaborating regularly, make a start - and you should be part of the guild already .</p>"},{"location":"basics/","title":"Basics","text":""},{"location":"basics/#architecture","title":"Architecture","text":"<p>The architecture for various components are already described at docs.cardano.org by CF/IOHK. We will not reinvent the wheel </p>"},{"location":"basics/#manual-software-pre-requirements","title":"Manual Software Pre-Requirements","text":"<p>While we do not intend to hand out step-by-step instructions, the tools are often misused as a shortcut to avoid ensuring base skillsets mentioned on home page. Some of the common gotchas that we often find SPOs to miss out on:</p> <ul> <li>It is imperative that pools operate with highly accurate system time, in order to propogate blocks to network in a timely manner and avoid penalties to own (or at times other competing) blocks. Please refer to sample guidance here  for details - the precise steps may depend on your OS.</li> <li>Ensure your Firewall rules at Network as well as OS level are updated according to the usage of your system, you'd want to whitelist the rules that you really need to open to world (eg: You might need node and SSH ports to be open to relays and perhaps home workstation on core, while open node to internet on relays, depending on your topology and configuration that you run).</li> <li>Update your SSH Configuration to prevent password-based logon.</li> <li>Ensure that you use offline workflow, you should never require to have your offline keys on online nodes. The tools provide you backup/restore functionality to only pass online keys to online nodes.</li> </ul>"},{"location":"basics/#pre-requisites","title":"Pre-Requisites","text":"<p>Reminder !!</p> <p>You're expected to run the commands below from same session, using same working directories as indicated and using a <code>non-root user with sudo access</code>. You are expected to be familiar with this as part of pre-requisite skill sets for stake pool operators.</p>"},{"location":"basics/#os-prereqs","title":"Set up OS packages, folder structure and fetch files from repo","text":"<p>The pre-requisites for Linux systems are automated to be executed as a single script. This script uses opt-in election of what you'd like the script to do. The defaults without any arguments will only update static part of script contents for you. To download the pre-requisites scripts, execute the below:</p> <pre><code>mkdir \"$HOME/tmp\";cd \"$HOME/tmp\"\n# Install curl\n# CentOS / RedHat - sudo dnf -y install curl\n# Ubuntu / Debian - sudo apt -y install curl\ncurl -sS -o guild-deploy.sh https://raw.githubusercontent.com/cardano-community/guild-operators/master/scripts/cnode-helper-scripts/guild-deploy.sh\nchmod 755 guild-deploy.sh\n</code></pre> <p>Important !!</p> <p>Please familiarise with the syntax of <code>guild-deploy.sh</code> before proceeding (using -h as below). The exact parameters you want to run with is dependent on your, below are only sample instructions</p> <p>The usage syntax can be checked using <code>./guild-deploy.sh -h</code> , sample output below:</p> <pre><code>Usage: guild-deploy.sh [-n &lt;mainnet|guild|preprod|preview|sanchonet&gt;] [-p path] [-t &lt;name&gt;] [-b &lt;branch&gt;] [-u] [-s [p][b][l][m][d][c][o][w][x][f][s]]\nSet up dependencies for building/using common tools across cardano ecosystem.\nThe script will always update dynamic content from existing scripts retaining existing user variables\n\n-n    Connect to specified network instead of mainnet network (Default: connect to cardano mainnet network) eg: -n guild\n-p    Parent folder path underneath which the top-level folder will be created (Default: /opt/cardano)\n-t    Alternate name for top level folder - only alpha-numeric chars allowed (Default: cnode)\n-b    Use alternate branch of scripts to download - only recommended for testing/development (Default: master)\n-u    Skip update check for script itself\n-s    Selective Install, only deploy specific components as below:\n  p   Install common pre-requisite OS-level Dependencies for most tools on this repo (Default: skip)\nb   Install OS level dependencies for tools required while building cardano-node/cardano-db-sync components (Default: skip)\nl   Build and Install libsodium fork from IO repositories (Default: skip)\nm   Download latest (released) binaries for mithril-signer, mithril-client (Default: skip)\nd   Download latest (released) binaries for bech32, cardano-address, cardano-node, cardano-cli, cardano-db-sync and cardano-submit-api (Default: skip)\nc   Download latest (released) binaries for CNCLI (Default: skip)\no   Download latest (released) binaries for Ogmios (Default: skip)\nw   Download latest (released) binaries for Cardano Hardware CLI (Default: skip)\nx   Download latest (released) binaries for Cardano Signer binary (Default: skip)\nf   Force overwrite config files (backups of existing ones will be created) (Default: skip)\ns   Force overwrite entire content [including user variables] of scripts (Default: skip)\n</code></pre> <ol> <li>If you receive an error for <code>glibc</code>, it would likely be due to the build mismatch between pre-compiled binary and your OS, which is not uncommon. You may need to compile cncli manually on your OS as per instructions here - make sure to copy the output binary to <code>\"${HOME}/.local/bin\"</code> folder.</li> </ol> <p>A typical example install to install most components but not overwrite static part of existing files for preview network would be:</p> <pre><code>./guild-deploy.sh -b master -n preview -t cnode -s pdlcowx\n. \"${HOME}/.bashrc\"\n</code></pre> <p>If instead of download, you'd want to build the components yourself, you could use:</p> <pre><code>./guild-deploy.sh -b master -n preview -t cnode -s pblcowx\n. \"${HOME}/.bashrc\"\n</code></pre> <p>Lastly, if you'd want to update your scripts but not install any additional dependencies, you may simply run:</p> <pre><code>./guild-deploy.sh -b master -n preview -t cnode\n</code></pre>"},{"location":"basics/#folder-structure","title":"Folder structure","text":"<p>Running the script above will create the folder structure as per below, for your reference. You do NOT require <code>CNODE_HOME</code> to be set on shell level as scripts will derive the parent folder and assign it at runtime, the addition in <code>~/.bashrc</code> is only for your ease to switch to right folder:</p> <pre><code>/opt/cardano/cnode            # Top-Level Folder\n\u251c\u2500\u2500 ...\n\u251c\u2500\u2500 files                     # Config, genesis and topology files\n\u2502   \u251c\u2500\u2500 ...\n\u2502   \u251c\u2500\u2500 byron-genesis.json    # Byron Genesis file referenced in config.json\n\u2502   \u251c\u2500\u2500 shelley-genesis.json  # Genesis file referenced in config.json\n\u2502   \u251c\u2500\u2500 alonzo-genesis.json    # Alonzo Genesis file referenced in config.json\n\u2502   \u251c\u2500\u2500 config.json           # Config file used by cardano-node\n\u2502   \u2514\u2500\u2500 topology.json         # Map of chain for cardano-node to boot from\n\u251c\u2500\u2500 db                        # DB Store for cardano-node\n\u251c\u2500\u2500 guild-db                  # DB Store for guild-specific tools and additions (eg: cncli, cardano-db-sync's schema)\n\u251c\u2500\u2500 logs                      # Logs for cardano-node\n\u251c\u2500\u2500 priv                      # Folder to store your keys (permission: 600)\n\u251c\u2500\u2500 scripts                   # Scripts to start and interact with cardano-node\n\u2514\u2500\u2500 sockets                   # Socket files created by cardano-node\n</code></pre>"},{"location":"build/","title":"Overview","text":"<p>The documentation here uses instructions from Intersect MBO repositories as foundation, with additional info which we can contribute to where appropriate. Note that not everyone needs to build each component. You can refer to architecture to understand and qualify which of the components built by IO you want to run.</p>"},{"location":"build/#components","title":"Components","text":"<p>For most Pool Operators, simply building cardano-node should be enough. Use the below to decide whether you need other components:</p> graph TB   A([Interact with HD Walletslocally])   B([Explore blockchainlocally])   C([Easy pool-ops andfund management])   D([Create Custom Assets])   E([Monitor node using Terminal UI])   F([Sign/verify any datausing crypto keys])   N(Node)   O(Ogmios)   P(gRest/Koios)   Q(DBSync)   R(Wallet)   S(CNTools)   T(Tx Submit API)   U(GraphQL)   V(OfflineMetadataTools)   X(gLiveView)   Y(cardano-signer)   Z[(PostgreSQL)]  N --x C --x S N --x D --x S &amp; V N --x E --x X N --x B B --x U --x Q B --x P --x Q P --x O P --x T F ---x Y N --x A --x R Q --x Z  <p>Important</p> <p>We strongly prefer use of gRest over GraphQL components due to performance, security, simplicity, control and most importantly - consistency benefits. Please refer to official documentations if you're interested in <code>GraphQL</code> or <code>Cardano-Rest</code> components instead.</p> <p>Note</p> <p>The instructions are intentionally limited to stack/cabal** to avoid wait times/availability of nix/docker files on a rapidly developing codebase - this also helps us prevent managing multiple versions of instructions.</p>"},{"location":"build/#description-for-components-built-by-community","title":"Description for components built by community","text":""},{"location":"build/#cntools","title":"CNTools","text":"<p>A swiss army knife for pool operators, primarily built by Ola, to simplify typical operations regarding their wallet keys and pool management. You can read more about it here</p>"},{"location":"build/#gliveview","title":"gLiveView","text":"<p>A local node monitoring tool, primarily built by Ola, to use in addition to remote monitoring tools like Prometheus/Grafana, Zabbix or IOG's RTView. This is especially useful when moving to a systemd deployment - if you haven't done so already - as it offers an intuitive UI to monitor the node status. You can read more about it here</p>"},{"location":"build/#topology-updater","title":"Topology Updater","text":"<p>A temporary node-to-node discovery solution, run by Markus, that was started initially to bridge the gap created while awaiting completion of P2P on cardano network, but has since become an important lifeline to the network health - to allow everyone to activate their relay nodes without having to postpone and wait for manual topology completion requests. You can read more about it here</p>"},{"location":"build/#koiosgrest","title":"Koios/gRest","text":"<p>A full-featured local query layer node to explore blockchain data (via dbsync) using standardised pre-built queries served via API as per standard from Koios - for which user can opt to participate in elastic query layer. You can read more about build steps here and reference API endpoints here</p>"},{"location":"build/#ogmios","title":"Ogmios","text":"<p>A lightweight bridge interface for cardano-node. It offers a WebSockets API that enables local clients to speak Ouroboros' mini-protocols via JSON/RPC. You can read more about it here</p>"},{"location":"build/#cncli","title":"CNCLI","text":"<p>A CLI tool written in Rust by Andrew Westberg for low-level communication with cardano-node. It is commonly used by SPOs to check their leader logs (integrates with CNTools as well as gLiveView) or to send their pool's health information to https://pooltool.io. You can read more about it here</p>"},{"location":"build/#cardano-signer","title":"Cardano Signer","text":"<p>A tool written by Martin to sign/verify data (hex, text or binary) using cryptographic keys to generate data as per CIP-8 or CIP-36 standards. You can read more about it here</p>"},{"location":"catalystf11/","title":"Catalystf11","text":""},{"location":"catalystf11/#marlowehub-unifying-platform-for-marlowe-smart-contracts-phase-1-smart-contracts","title":"MarloweHub: Unifying Platform for Marlowe Smart Contracts - Phase 1 - Smart Contracts","text":"<p>Category: Concept   Applicant: mike (pooltool.io)   Requested funds: \u20b3100,000.00   Links: Ideascale, Lidonation </p>"},{"location":"catalystf11/#adastat-cardano-explorer-open-source-improved-reboot-towards-a-first-class-community-blockchain-explorer","title":"AdaStat Cardano Explorer - Open Source Improved Reboot towards a first-class community blockchain explorer","text":"<p>Category: Product   Applicant: Dmytro Stashenko (adastat.net)   Requested funds: \u20b3180,300.00   Links: Ideascale, Lidonation </p>"},{"location":"catalystf11/#adahold-decentralized-price-can-only-go-up-token-solution-for-true-ada-hodlers-smart-contract","title":"AdaHold: Decentralized Price-Can-Only-Go-Up Token, Solution For TRUE Ada Hodlers - Smart Contract","text":"<p>Category: Concept   Applicant: Dmytro Stashenko (adastat.net)   Requested funds: \u20b399,700.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#sundae-labs-next-gen-uplc-debugger-with-aiken-integration","title":"Sundae Labs Next-Gen UPLC Debugger with Aiken Integration","text":"<p>Category: Developers   Applicant: Dan Gonzalez (sundae.fi)   Requested funds: \u20b3140,000.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#margin-pool-enhanced-liquidity-for-margin-trading-on-sundaeswap","title":"Margin Pool: Enhanced Liquidity for Margin Trading on SundaeSwap","text":"<p>Category: Solution   Applicant: Dan Gonzalez (sundae.fi)   Requested funds: \u20b3300,000.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#regulated-and-permissioned-defi-with-sundae-and-kora-labs","title":"Regulated and Permissioned DeFi with Sundae and Kora Labs","text":"<p>Category: Concept   Applicant: Dan Gonzalez (sundae.fi)   Requested funds: \u20b3100,000.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#sundae-labs-comprehensive-specification-development-for-gummiworm-protocol-on-cardano","title":"Sundae Labs Comprehensive Specification Development for Gummiworm Protocol on Cardano","text":"<p>Category: Concept   Applicant: Dan Gonzalez (sundae.fi)   Requested funds: \u20b3100,000.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#nftcdnio-universal-nft-viewer-api-musicvideoweb3d","title":"[nftcdn.io] Universal NFT Viewer API (Music+Video+Web+3D+\u2026)","text":"<p>Category: Product   Applicant: Smaug (pool.pm)   Requested funds: \u20b3299,999.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#nftcdnio-nsfw-nft-detection-for-marketplaces-wallets-explorers","title":"[nftcdn.io] NSFW NFT Detection for Marketplaces, Wallets &amp; Explorers","text":"<p>Category: Product   Applicant: Smaug (pool.pm)   Requested funds: \u20b3149,999.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#cardanoscan-analytics-charts","title":"Cardanoscan Analytics Charts","text":"<p>Category: Product   Applicant: Strica (cardanoscan.io)   Requested funds: \u20b344,000.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#cardanoscan-api-javascript-sdk","title":"Cardanoscan API Javascript SDK","text":"<p>Category: Developers   Applicant: Strica (cardanoscan.io)   Requested funds: \u20b364,000.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#integrate-keystone-hardware-wallet-into-typhon","title":"Integrate Keystone Hardware Wallet into Typhon","text":"<p>Category: Product   Applicant: Strica (cardanoscan.io)   Requested funds: \u20b384,000.00   Links: Ideascale, Lidonation </p>"},{"location":"catalystf11/#add-support-for-marlowe-on-cardanoscan","title":"Add support for Marlowe on Cardanoscan","text":"<p>Category: Product   Applicant: Strica (cardanoscan.io)   Requested funds: \u20b3133,000.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#cardanoscan-data-info-bubbles","title":"Cardanoscan data info bubbles","text":"<p>Category: Ecosystem   Applicant: Strica (cardanoscan.io)   Requested funds: \u20b335,000.00   Links: Ideascale, Lidonation</p>"},{"location":"catalystf11/#create-a-cnt-marketplace-on-norwegian-block-exchange-nbxcom","title":"Create a CNT marketplace on Norwegian Block Exchange (NBX.COM)","text":"<p>Category: Product   Applicant: Eystein Hansen (nbx.com)   Requested funds: \u20b3145,000.00   Links: Ideascale, Lidonation</p>"},{"location":"contributors/","title":"Contributors","text":"<p>Everyone is welcome to contribute to the guide, as well as the repository. Below is just a thank you to people who have been contributing consistently:</p> <p>Adam Chris Damjan Homer Markus OCG Ola Ahlman Pal Dorogi Papacarp PegasusPool Psychomb RdLrT RedOracle SmaugPool </p> <p>To start contributing, simply hit the github repository and raise Issue/Pull Request</p>"},{"location":"grest-meets/","title":"GRest Meeting summaries","text":"<p>Thank you all for joining and contributing to the project </p> <p>Below you can find a short summary of every GRest meeting held, both for logging purposes and for those who were not able to attend.</p>"},{"location":"grest-meets/#participants","title":"Participants:","text":"Participant 16Sep2021 02Sep2021 26Aug2021 19Aug2021 12Aug2021 29Jul2021 22Jul2021 15Jul2021 09Jul2021 02Jul2021 25Jun2021 Damjan Homer Markus Ola RdLrT Red Papacarp Paddy GimbaLabs 16Sep2021 02Sep2021 26Aug2021 19Aug2021 12Aug2021 29Jul2021 22Jul2021 15Jul2021 09Jul2021 02Jul2021 <p>After the initial stand-up updates from participants, we went through the entire Trello board, updating/deleting existing tickets and creating some new ones.</p> 25Jun2021"},{"location":"grest-meets/#scheduling-running-update-queries","title":"Scheduling running update queries","text":"<ul> <li>Postgres triggers are synchronous so they slow down db-sync</li> <li>Decided to explore crontab for query scheduling instead</li> </ul>"},{"location":"grest-meets/#refactor-of-queries","title":"Refactor of queries","text":"<ul> <li>Discussed how to structure RPC endpoints and what each should include</li> <li>Details have been captured in the Trello board</li> </ul>"},{"location":"grest-meets/#postgres-tuning","title":"postgres tuning","text":"<ul> <li>Discussed possible tunings to the postgres config</li> <li>Probably reducing WAL usage</li> </ul>"},{"location":"grest-meets/#updates","title":"Updates","text":"<ul> <li>making good progress on the website (koios.rest) - great job Markus!</li> <li>query tickets are now well structured and put into sections (Account, Pool, Transactions...) on the Trello board - nice work Priyank/Ola!</li> </ul>"},{"location":"grest-meets/#queries","title":"Queries","text":"<ul> <li>Transaction cache table:</li> <li>We would like to avoid 'handling' rollbacks in that table, instead simply dump if multiple entries for a transaction, as it can have much higher combination and volume to process - especially post a node/postgres/dbsync restart.</li> <li> <p>Solution being tested:</p> <ul> <li>Use an md5 hash of concatenated tx_id, tx_index and block hash to generate unique serves as primary key in that table</li> <li>On RPC built off cache tables query layer, we would add a validation for block hash being in the public.block table and exclude those who are not as part of result</li> <li>This way we don't handle rollbacks, and also keep a record if in future we require to cross check/re-run delta</li> </ul> </li> <li> <p>Pool cache table:</p> </li> <li>Need to check if transaction cache method is useful here too</li> <li>Using strace we could verify the order in which tables are touched, and avoid trigger-check to run on every block.</li> </ul>"},{"location":"grest-meets/#problems","title":"Problems","text":"<ul> <li>Priyank noticed that if including any foreign keys for tx cache, it can cause a spike in load resulting in crash of db-sync instance (with locks). There arent any visible advantages maintaining a constraint on cache table anyways, as it decreases performance. Thus, we'd keep the cache tables simple and not include any foreign keys.</li> <li>Infrastructure upgrades unlikely to help for such cases (though we may need to increase baseline specs back to initial discussion anyways, but that would be during performance testing stages).</li> </ul>"},{"location":"grest-meets/#actions","title":"Actions","text":"<ul> <li>POST/GET endpoint rules:</li> <li>Use GET for endpoints that take no input parameters (PostgREST native parameters can still be applied via URL)</li> <li>Any endpoint that we accept consumer to provide parameter for should be POST</li> <li>Switch cardanoqueries.ha to api.koios.rest on the API docs</li> <li>Load balancing:</li> <li>Until we see additional instances, the 4 trusted instances serve as Monitoring layer.</li> <li>If/when we start having community instances, we could start splitting topologies to be geographically balanced - using github as source.</li> </ul>"},{"location":"grest-meets/#queries_1","title":"Queries","text":"<ul> <li>stake distribution:</li> <li> <p>we will run the full query on regular intervals, ready for review for first iteration, will see about delta post tx cache query</p> </li> <li> <p>transaction history:</p> </li> <li>transaction history query needs to be switched to populate a cached table instead</li> <li> <p>need to think about how to approach inputs/outputs in the cached table (1 row per transaction with json objects for inputs/outputs or multiple rows for tx hash)</p> </li> <li> <p>address_txs:</p> </li> <li> <p>this endpoint should bring back list of txs, and have provision to use after and before block hash - lightweight against public schema</p> </li> <li> <p>pool cache table:</p> </li> <li>cached table to aggregate info from all the pool tables together (pool_metadata, pool_hash, pool_update...)</li> <li>the cached data should include the full history of all pools as well as the current state (latest pool update)</li> <li>will then be used for (most likely) all pool related endpoints without the need for joins</li> </ul>"},{"location":"grest-meets/#transaction-submission-feature","title":"Transaction submission feature","text":"<ul> <li>a post endpoint separate from gREST ones (different port), proxy'd over haproxy using same health check script appended for node</li> <li>will receive signed transactions in 2 formats (file and cbor) and use cardano-submit-api or CLI to submit them to the blockchain respectively</li> <li>use cases are mostly light wallets, and third-party wallets or CNTools could implement such light features with it (no need for cardano-node with CNTools)</li> </ul>"},{"location":"grest-meets/#db-replication-presentation-by-redoracle","title":"DB replication presentation by Redoracle","text":"<ul> <li>Proposition to move the gRest schema and tables required by the API to smaller instances that can be scaled more easily</li> <li>Pros and Cons to the approach discussed, worth investigating based on performance comparisons</li> </ul>"},{"location":"grest-meets/#process-for-upgrading-our-instances","title":"Process for upgrading our instances:","text":"<ul> <li>Collaboration between trusted peers will be needed, to upgrade sequentially (3 off 6 instances, for example)</li> <li>Use DNS subdomain for upgraded nodes for testing</li> <li>Ideally upgrade processes to be done between 2nd-4th day of epoch to avoid overloading smaller subset in peak hours</li> <li>Enhance grest-poll to use arguments and seperate haproxy backends, allowing for test based reduction</li> </ul>"},{"location":"grest-meets/#queries_2","title":"Queries:","text":""},{"location":"grest-meets/#stake-distribution","title":"Stake distribution","text":"<ul> <li>we need to implement triggers</li> <li>dealing with block rollbacks is tricky</li> <li>Priyank will make an example of his idea of how to deal with it that others can use/build upon</li> </ul>"},{"location":"grest-meets/#tx-history","title":"Tx History","text":"<ul> <li>Current PR to be split into two (to include value/assets and not have to return JSON that is resource intensive to generate/parse):</li> <li>Addr to Tx Hash list using start and end blocks</li> <li>Bulk Tx Hash (limit 100) query to get as much details about tx sent as possible</li> <li>Consider if cache table makes sense after above change. If yes, we also need triggers that can handle rollbacks</li> </ul>"},{"location":"grest-meets/#problems_1","title":"PROBLEMS","text":"<ul> <li>stake distribution query needs to be completed</li> <li>it's hard to use docker to replicate our current setup</li> </ul>"},{"location":"grest-meets/#actions_1","title":"ACTIONS","text":"<ul> <li>additional things to add to stake_distribution query</li> <li>Add logic to record and check tx based on block.id for last but 3rd block in existing query</li> <li>Add a control table in grest schema to record block_hash used for last update, start time, and end time. This will act as a checkpoint for polling of queries that are not live (separate backend in haproxy)</li> <li> <p>create a trigger every 2 minutes (or similar) to run stake_distribution query</p> </li> <li> <p>docker:</p> </li> <li>problems with performance due to nature of IOPs and throughput usage for resources being isolated and can only access each other through sockets.</li> <li>still useful to test whether fully dockerized (each component isolated) can keep up to chain tip</li> <li>consider dockerizing all resources in one container to give new joiners a simple one liner to get up and running - this still doesn't ensure optimal performance, Tuning will still be an additional task for any infrastructure to customise setup to best results achievable</li> </ul>"},{"location":"grest-meets/#problems_2","title":"PROBLEMS","text":"<ul> <li>Not everyone reporting to the monitoring dashboard</li> <li>We don't fully understand the execution time deviations of the stake distribution query</li> <li>catalyst rewards are hard to isolate</li> <li>branch 10.1.x has been deleted on the db-sync repo</li> <li>people have a hard time catching up with the project after being away for a while</li> </ul>"},{"location":"grest-meets/#actions_2","title":"ACTIONS","text":"<ul> <li>missing instances start reporting to monitoring</li> <li>run stake_distribution query on multiple instances, report output of <code>EXPLAIN (ANALYZE, BUFFERS)</code></li> <li>catalyst rewards can be ignored until there is a clear path to get them: Fix underway using open PR</li> <li>if someone needs help getting the right db-sync commit, message Priyank for help as the branch is now deleted</li> <li>add project metadata (requirements) to grest doc header in a checklist format that folks can use to ensure their setup is up-to-date with the current project state</li> <li>Discussed long-term plans (will be added separately in group)</li> </ul>"},{"location":"grest-meets/#problems_3","title":"PROBLEMS","text":"<ul> <li>how to sync live stake between instances (or is there need for it?)</li> </ul>"},{"location":"grest-meets/#actions_3","title":"ACTIONS","text":"<ol> <li> <p>Team</p> <ul> <li>catch live stake distributions in a separate table (in our <code>grest</code> schema)<ul> <li>these queries can run on a schedule</li> <li>response comes from the instance with the latest data</li> </ul> </li> <li>other approaches:<ul> <li>possibly distribute pools between instances (complex approach)</li> <li>run full query once and only check for new/leaving delegators (probably impossible because of existing delegator UTXO movements)</li> </ul> </li> <li>implement monitoring of execution times for all the queries</li> <li>come up with a timeline for launch (next call)</li> <li>stress test before launch</li> <li>start building queries listed on Trello board</li> </ul> </li> <li> <p>Individual</p> <ul> <li>sync db-sync instances to commit <code>84226d33eed66be8e61d50b7e1dacebdc095cee9</code> on <code>release/10.1.x</code></li> <li>update setups to reflect recent directory restructuring and updated instructions</li> </ul> </li> </ol>"},{"location":"grest-meets/#introduction-for-new-joiner-paddy","title":"Introduction for new joiner - Paddy","text":"<ul> <li>from Shamrock stake pool / poolpeek</li> <li>gRest project could be helpful for pool peek</li> <li>Paddy will probably run an individual instance</li> </ul>"},{"location":"grest-meets/#problems_4","title":"Problems","text":"<ul> <li>there is a problem with extremely high CPU usage haproxy, tuning underway.</li> <li>live stake query has multiple variations, and we need to figure out what is the correct one.</li> </ul>"},{"location":"grest-meets/#action-items","title":"Action Items","text":"<ul> <li>Everyone should add monitoring to their instances</li> <li>restructure RPC query files (separate metadata in <code>&lt;query&gt;.json</code> and sql in <code>&lt;query&gt;.sql</code>), also remove <code>get_</code> prefix</li> <li>Add new queries from the list</li> <li>fix haproxy CPU usage (use <code>nbthreads</code> in config, tune maxconn, switch to http mode)</li> <li>gather multiple variations of the live stake query and ask Erik for clarification on which one is correct</li> <li>Start working on other queries listed on trello</li> </ul>"},{"location":"grest-meets/#deployment-scripts","title":"Deployment scripts","text":"<p>Ola added automatic deployment of services to the scripts last week. We added new tasks on Trello ticket, including flags for multiple networks (guild, testnet, mainnet), haproxy service dynamically creating hosts and doc updates. Overall, the script works well with some manual interaction still required at the moment.</p>"},{"location":"grest-meets/#supported-networks","title":"Supported Networks","text":"<p>Just for the record here, a 16GB (or even 8GB) instance is enough to support both testnet and guild networks.</p>"},{"location":"grest-meets/#db-sync-versioning","title":"db-sync versioning","text":"<p>We agreed to use the <code>release/10.1.x</code> branch which is not yet released but built to include Alonzo migrations to avoid rework later. This version does require Alonzo config and hash to be in the node's <code>config.json</code>. This has to be done manually and the files are available here. Once fully released, all members should rebuild the released version to ensure each instance is running the same code.</p>"},{"location":"grest-meets/#dns-naming","title":"DNS naming","text":"<p>For the DNS setup ticket, we started to think about the instance names for the 2 DNS instances (orange in the graph). Submissions for names will be made in the Telegram group, and will probably make a poll once we have the entries finalised.</p>"},{"location":"grest-meets/#monitoring-system","title":"Monitoring System","text":"<p>Priyank started setting up the monitoring on his instance which can then easily be switched to a separate monitoring instance. We agreed to use Prometheus / Grafana combo for data source / visualisation. We'll probably need to create some custom archiving of data to keep it long term as Prometheus stores only the last 30 days of data.</p>"},{"location":"grest-meets/#next-meeting","title":"Next meeting","text":"<p>We would like to make Friday @ 07:00 UTC the standard time and keep meetings at weekly frequency. A poll will still be created for next weeks, but if there are no objections / requests for switching the time around (which we have not had so far) we can go ahead with the making Friday the standard with polls no longer required and only reminders / Google invites sent every week.</p>"},{"location":"grest-meets/#deployment-scripts_1","title":"Deployment scripts","text":"<p>During the last week, work has been done on deployment scripts for all services (db-sync, gRest and haproxy) -&gt; this is now in testing with updated instructions on trello. Everybody can put their name down on the ticket to signify when the setup is complete and note down any comments for bugs/improvements. This is the main priority at the moment as it would allow us to start transferring our setups to mainnet.</p>"},{"location":"grest-meets/#switch-to-mainnet","title":"Switch to Mainnet","text":"<p>Following on from that, we created a ticket for starting to set up mainnet instances -&gt; we can use 32GB RAM to start and increase later. While making sure everything works against the guild network is priority, people are free to start on this as well as we anticipate we are almost ready for the switch.</p>"},{"location":"grest-meets/#supported-networks_1","title":"Supported Networks","text":"<p>This brings me to another discussion point which is on which networks are to be supported. After some discussion, it was agreed to keep beefy servers for mainnet, and have small independent instances for testnet maintained by those interested, while guild instance is pretty lightweight and useful to keep.</p>"},{"location":"grest-meets/#monitoring-system_1","title":"Monitoring System","text":"<p>The ticket for creating a centralised monitoring system was discussed and updated. I would say it would be good to have at least a basic version of the system in place around the time we switch to mainnet. The system could eventually serve for:  - analysis of instance  - performances and subsequent tuning  - endpoints usage  - anticipation of system requirements increases  - etc.</p> <p>I would say that this should be an important topic of the next meeting to come up with an approach on how we will structure this system so that we can start building it in time for mainnet switch.</p>"},{"location":"grest-meets/#handling-ssl","title":"Handling SSL","text":"<p>Enabling SSL was agreed to not be required by each instance, but is optional and documentation should be created for how to automate the process of renewing SSL certificates for those wishing to add it to their instance. The end user facing endpoints \"Instance Checker\" will of course be SSL-enabled.</p>"},{"location":"grest-meets/#next-meeting_1","title":"Next meeting","text":"<p>We somewhat agreed to another meeting next week again at the same time, but some participants aren't 100% for availability. Friday at 07:00 UTC might be a good standard time we hold on to, but I will make a poll like last time so that we can get more info before confirming the meeting.</p>"},{"location":"grest-meets/#meeting-structure","title":"Meeting Structure","text":"<p>As this was the first meeting, at the start we discussed about the meeting structure. In general, we agreed to something like listed below, but this can definitely change in the future:</p> <p>1) 2-liner (60s) round the table stand-ups by everyone to sync up on what they were doing / are planning to do / mention struggles etc. This itself often sparks discussions. 2) going through the Trello board tasks with the intention of discussing and possbily assigning them to individuals / smaller groups (maybe 1-2-3 people choose to work together on a single task)</p>"},{"location":"grest-meets/#stand-ups","title":"Stand-ups","text":"<p>We then proceeded to give a status of where we are individually in terms of what's been done, a summary below:</p> <ul> <li>Homer, Ola, Markus, Priyank and Damjan have all set up their dbsync + gRest endpoints against guild network and added to topology.</li> <li>Ola laid down the groundwork for CNTools to integrate with API endpoints created so far.</li> <li>Markus has created the systemd scripts and will add them soon to repo</li> <li>Damjan is tracking live stake query that includes payment + stake address, but is awaiting fix on dbsync for pool refund (contextual reserves -&gt; account) , also need to validate reserve -&gt; MIR certs</li> <li>Priyank created initial haproxy settings for polls done, need to complete agent based on design finalisation</li> </ul>"},{"location":"grest-meets/#main-discussion-points","title":"Main discussion points","text":"<ol> <li>Directory structure on the repo -&gt; General agreement is to have anything related to db-sync/postgREST separated from the current cnode-helper-scripts directory. We can finalise the end locations of files a bit later, for now intent should be to simply add them all to /files/dbsync folder. <code>prereqs.sh</code> addendum can be done once artifacts are finalised (added a Trello ticket for tracking).</li> <li>DNS/haproxy configurations:   We have two options:   a. controlled approach for endpoints - wherein there is a layer of haproxy that will load balance and ensure tip being in sync for individual providers (individuals can provide haproxy OR gRest instances).   b. completely decentralised - each client to maintain haproxy endpoint, and fails over to other node if its not up to recent tip.   I think that in general, it was agreed to use a hybrid approach. Details are captured in diagram here. DNS endpoint can be reserved post initial testing of haproxy-agent against mainnet nodes.</li> <li>Internal monitoring system   This would be important and useful and has not been mentioned before this meeting (as far as I know). Basically, a system for monitoring all of our instances together and also handling alerts. Not only for ensuring good quality of service, but also for logging and inspection of short- and long-term trends to better understand what's happening. A ticket is added to trello board</li> </ol>"},{"location":"grest-meets/#next-meeting_2","title":"Next meeting","text":"<p>All in all, I think we saw that there is need for these meetings as there are a lot of things to discuss and new ideas come up (like the monitoring system). We went for over an hour (~1h15min) and still didn't have enough time to go through the board, we basically only touched the DNS/haproxy part of the board. This tells me that we are in a stage where more frequent meetings are required, weekly instead of biweekly, as we are in the initial stage and it's important to build things right from the start rather than having to refactor later on. With that, the participants in general agreed to another meeting next week, but this will be confirmed in the TG chat and the times can be discussed then.</p>"},{"location":"sidebar/","title":"Tree","text":"<ul> <li>Home: index.md</li> <li>Basics: basics.md</li> <li>Build and Run:<ul> <li>Overview: build.md</li> <li>Node &amp; CLI: Build/node-cli.md</li> <li>DBSync: Build/dbsync.md</li> <li>Koios gRest: Build/grest.md</li> <li>Offchain Metadata Tools: Build/offchain-metadata-tools.md</li> </ul> </li> <li>Scripts:<ul> <li>Common env: Scripts/env.md</li> <li>BlockPerf: Scripts/blockperf.md</li> <li>Koios SPO Tools:</li> <li>CNTools:<ul> <li>Overview: Scripts/cntools.md</li> <li>Common Tasks: Scripts/cntools-common.md</li> <li>Changelog: Scripts/cntools-changelog.md</li> <li>Log Monitor: Scripts/logmonitor.md</li> </ul> </li> <li>gLiveView: Scripts/gliveview.md</li> <li>Topology Updater: Scripts/topologyupdater.md</li> <li>CNCLI: Scripts/cncli.md</li> <li>Mithril:</li> <li>Overview: Mithril/mithril-overview.md</li> <li>Client: Scripts/mithril-client.md</li> <li>Signer: Scripts/mithril-signer.md</li> <li>Relay: Scripts/mithril-relay.md</li> </ul> </li> <li>Docker:<ul> <li>Overview: docker/docker.md</li> <li>Build: docker/build.md</li> <li>Run: docker/run.md</li> <li>Tips: docker/tips.md</li> </ul> </li> <li>Appendix:<ul> <li>Sample Postgres Setup: Appendix/postgres.md</li> <li>GRest meet summaries: grest-meets.md</li> </ul> </li> <li>Contributors: contributors.md</li> </ul>"},{"location":"upgrade/","title":"Upgrade","text":""},{"location":"upgrade/#steps-for-upgrading","title":"Steps for Upgrading","text":"<p>Warning</p> <p>Make sure you go through upgrade steps for your setup in a non-mainnet environment first!!</p> <ul> <li>Download the latest <code>guild-deploy.sh</code> (always double check syntax of the script with <code>guild-deploy.sh -h</code>). The scripts modified with user content (<code>env</code>, <code>gLiveView.sh</code>, <code>topologyUpdater.sh</code>, <code>cnode.sh</code>, etc) will be backed up before overwriting. The backed up files will be in the same folder as the original files, and will be named as <code>${filename}_bkp&lt;timestamp&gt;</code>. More static files (genesis files or some of the scripts themselves) will not be backed up, as they're not expected to be modified.</li> </ul> <p>Remember</p> <p>You are expected to provide appropriate environment-specific parameters (eg: custom top level folder [-p], alternate name for top level folder [-t], network flag [-n], etc) to the examples that pertain to your use case</p> <ul> <li> <p>Depending on node release, you may be able to simply perform an update-in-place of scripts and node, or for some cases, you may need to overwrite configs as well. Some Examples below:</p> <ul> <li>A typical run of the guild-deploy script that will perform update in place of current scripts on mainnet and update binaries (alongwith re-compiling libsodium dependencies). In this scenario no config files or user variables in scripts are overwritten. This is typically relevant on minor patch releases of node upgrade:</li> </ul> <pre><code>mkdir \"$HOME/tmp\";cd \"$HOME/tmp\"\ncurl -sfS -o guild-deploy.sh https://raw.githubusercontent.com/cardano-community/guild-operators/master/scripts/cnode-helper-scripts/guild-deploy.sh &amp;&amp; chmod 700 guild-deploy.sh\n./guild-deploy.sh -s dl -b master -n mainnet -t cnode -p /opt/cardano\n</code></pre> <ul> <li>Another scenario would be when you're required to overwrite configs (eg: node-8.1.2 to node-9.1.0 introduced change in genesis/config/topology file formats). In this case, you'd want to overwrite your config files as well. You should follow changelog in node release notes to verify if you'd need to overwrite configs. Note that every time you do this, you may need to re-add your customisations - if any - to the relevant config files (typically - almost always, you'd have to update the topology.json when overwriting configs). There are backups created of original file in <code>\"${CNODE_HOME}\"/files</code> folder if you'd like to compare/reuse previous version.</li> </ul> <pre><code>mkdir \"$HOME/tmp\";cd \"$HOME/tmp\"\ncurl -sfS -o guild-deploy.sh https://raw.githubusercontent.com/cardano-community/guild-operators/master/scripts/cnode-helper-scripts/guild-deploy.sh &amp;&amp; chmod 700 guild-deploy.sh\n./guild-deploy.sh -s dlf -b master -n mainnet -t cnode -p /opt/cardano\n</code></pre> </li> </ul> <p>Beware</p> <p>When upgrading node, depending on node versions (especially for major release) - you'd likely have to wait for node to revalidate/replay ledger. This can take a few hours. Please always plan ahead, do it first on a relay to ensure you've got \"${CNODE_HOME}/db\" folder ready to copy over (while source and target node have been shutdown) - prior to starting on upgrade on new machine. If mithril for target node version is ready, you can also use mithril-client to download snapshot instead of replaying, which may save you some time</p> <ul> <li>Once guild-deploy script has been run, source your bashrc file again (or restart session), and ensure <code>\"${HOME}\"/.local/bin</code> is part of your \\(PATH environment variable. If your shell does not auto-run bashrc, you may want to set it a call to \"\\)\"/.bashrc in your <code>.profile</code></li> </ul> <pre><code>source \"${HOME}\"/.bashrc\necho \"${PATH}\"\n</code></pre>"},{"location":"upgrade/#troubleshooting","title":"Troubleshooting","text":"<ul> <li> <p>We've found users often confuse between \\(PATH variable resolution between multiple shell sessions, systemd, etc. While if you only used this guide, the binaries should be in \"\\)/.local/bin\", you may have manually downloaded to another location before. To avoid this, you can edit the following files and uncomment and set the following variables to the appropriate paths as per your deployment (eg: <code>CCLI=\"${HOME}\"/.local/bin/cardano-cli</code> if following above):</p> <ul> <li>env : CCLI, CNCLI, CNODEBIN</li> <li>[If applicable] dbsync.sh: DBSYNCBIN</li> <li>[If applicable] submitapi.sh: SUBMITAPIBIN</li> <li>[If applicable] ogmios.sh: OGMIOSBIN</li> </ul> </li> <li> <p>The above should take care of tools and services. However, you might still have duplicate binaries in your $PATH (previous artifacts, re-build using old scripts, etc) - it is best that you remove any old binary files from alternate folders. You can do so by executing the below:</p> </li> </ul> <pre><code>whereis bech32 cardano-address cardano-cli cardano-db-sync cardano-hw-cli cardano-node cardano-submit-api cncli ogmios\n</code></pre> <p>For some cases - you might have no values (eg: you may not use <code>cardano-db-sync</code>, <code>cncli</code>, <code>ogmios</code> and/or <code>cardano-hw-cli</code>. You need not take any actions for the binaries you do not use.</p> <ul> <li>If you are having trouble connecting to node post upgrade from gLiveView, typically the first issue you'd want to eliminate is  whether you missed that node might need ledger replay/revalidation (which could take hours as indicated earlier on the page). You can check the node status via <code>sudo systemctl status cnode</code>. If the node shows as up, you can monitor logs via <code>tail -100f \"${CNODE_HOME}/logs/node.json</code>. If you're unable to start node using systemd itself, you can check the systemd output via <code>sudo journalctl -xeu cnode.service</code> and scroll back until you see a startup attempt with reason (typically this could be a couple of pages back). If nothing obvious (eg: showing files used as <code>/files/config.json</code> instead of <code>$CNODE_HOME/files/config.json</code> )comes up from there, you'd want to view node logs to see if there is something recorded in <code>node.json</code> instead.</li> </ul>"},{"location":"upgrade/#unintended","title":"Unintended update-in-place","text":"<p>Let's say you accidentally did an update of the files that you didnt intend to and want to revert all your scripts to previous state. While you have the backups of the scripts created while doing in-place update, you can always make use of branch flag that most scripts that perform update-in-place provide (eg: for gLiveView.sh that you want to restore to 8.1.2 state), this would be <code>-b node-8.1.2</code>. The available tags that can be used can be visited here</p>"},{"location":"upgrade/#support","title":"Support/Improvements","text":"<p>Hope the guide above helps you with the migration, but again - we could've missed some edge cases. If so, please report via chat in Koios Discussions channel or open an issue on github. Please DO NOT make edits to the script content based on forum/alternate guide/channels, while done with best intentions - there have been solutions put online that modify files unnecessarily instead of correcting configs and disabling updates, such actions will only cause trouble for future updates.</p>"},{"location":"Appendix/RecoverByronWallet/","title":"Unofficial Instructions for recovering your Byron Era funds on the new Incentivized Shelley Testnet","text":""},{"location":"Appendix/RecoverByronWallet/#1-grab-and-install-haskell","title":"1.  Grab and install Haskell","text":"<pre><code>curl -sSL https://get.haskellstack.org/ | sh\n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#2-get-the-wallet","title":"2.  Get the wallet","text":"<p>note:  you must build from source as of today as there are changes that just got into master you need <pre><code>git clone https://github.com/cardano-foundation/cardano-wallet.git\n</code></pre></p>"},{"location":"Appendix/RecoverByronWallet/#3-go-into-the-wallet-directory","title":"3.  Go into the wallet directory","text":"<pre><code>cd cardano-wallet\n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#4-build-the-wallet","title":"4. Build the wallet","text":"<p><pre><code>stack build --test --no-run-tests\n</code></pre> If it fails there are a few reasons we have found: - The cardano build instructions reference a few things that may be missing.  Check those. - or maybe one of these would help:</p>"},{"location":"Appendix/RecoverByronWallet/#libssl","title":"Libssl:","text":"<pre><code>sudo apt install libssl-dev\n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#sqlite","title":"Sqlite :","text":"<pre><code>sudo apt-get install sqlite3 libsqlite3-dev \n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#gmp","title":"gmp:","text":"<pre><code>sudo apt-get install libgmp3-dev \n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#systemd-dev","title":"systemd dev:","text":"<pre><code>sudo apt install libsystemd-dev\n</code></pre> <p>get coffee...  It takes awhile</p>"},{"location":"Appendix/RecoverByronWallet/#5-when-its-done-install-executables-to-your-path","title":"5.  When its done, install executables to your path","text":"<pre><code>stack install\n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#6-test-to-make-sure-cardano-wallet-jormungandr-works-fine","title":"6.  Test to make sure cardano-wallet-jormungandr works fine.","text":"<p>Generate your new mnemonics you will need below.  Note that this generates 15 words as opposed to your byron era mnemnomics which were only 12 words.  </p> <pre><code>cardano-wallet-jormungandr mnemonic generate\n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#7-launch-the-wallet-as-a-service","title":"7.  Launch the wallet as a service.","text":"<p>you can either open another terminal window or use screen or something.  anyway, wherever you run this next command you won't be able to use anymore for a terminal until you stop the wallet </p> <p>change --node-port 3001 to wherever you have your jormungandr rest interface running.  for me it was 5001..  so</p> <p>change --port 3002 to wherever you want to access the wallet interface at.  If you have other things running avoid those ports.  for most, 3002 should be free</p> <p>just to future proof these instructions.  genesis should be whatever genesis you are on.</p> <pre><code>cardano-wallet-jormungandr serve --node-port 3001 --port 3002 --genesis-block-hash e03547a7effaf05021b40dd762d5c4cf944b991144f1ad507ef792ae54603197\n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#8-restore-your-byron-wallet","title":"8.  Restore your byron wallet:","text":"<p>---&gt;in another window</p> <p>replace foo, foo, foo with all your mnemnomics from the byron wallet you are restoring</p> <p>Also, if you put your wallet on a different port than 3002, fix that too</p> <p><pre><code>curl -X POST -H \"Content-Type: application/json\" -d '{ \"name\": \"legacy_wallet\", \"mnemonic_sentence\": [\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\"], \"passphrase\": \"areallylongpassword\"}' http://localhost:3002/v2/byron-wallets\n</code></pre> Thats going to spit out some information about a wallet it creates, you should see the value of your wallet - hopefully its not zero.  And you need the wallet ID for the next step</p>"},{"location":"Appendix/RecoverByronWallet/#9-create-your-shelley-wallet","title":"9.  Create your shelley wallet:","text":"<p>Remember all those mnemnomics you made above.. put them here instead of all the foo's.</p> <p><pre><code>curl -X POST -H \"Content-Type: application/json\" -d '{ \"name\": \"pool_wallet\", \"mnemonic_sentence\": [\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\",\"foo\"], \"passphrase\": \"areallylongpasswordagain\"}' http://localhost:3002/v2/wallets\n</code></pre> Important thing to get is the wallet id from this command</p>"},{"location":"Appendix/RecoverByronWallet/#10-migrate-your-funds","title":"10.  Migrate your funds","text":"<p>Now you are ready to migrate your wallet.  replace the <code>&lt;old wallet id&gt;</code> and <code>&lt;new wallet id&gt;</code> with the values you got above</p> <pre><code>curl -X POST -H \"Content-Type: application/json\" -d '{\"passphrase\": \"areallylongpassword\"}' http://localhost:3002/v2/byron-wallets/&lt;old wallet id&gt;/migrations/&lt;new wallet id&gt;\n</code></pre>"},{"location":"Appendix/RecoverByronWallet/#11-congratulations-your-funds-are-now-in-your-new-wallet","title":"11.  Congratulations.  your funds are now in your new wallet.","text":"<p>From here we recommend you send them to a new address entirely owned and created by jcli or whatever method you have been using for the testnet process.</p> <p>This technically may not be required.  But a lot of us did it and we know it works for setting up pools and stuff.</p> <p>send a small amount first just to make sure you are in control of the transaction and don't send your funds to la la land.</p> <p>If you want to send to another address use the command below, but replace the address that you want to send it to, the amount, and your <code>&lt;new wallet id&gt;</code> <pre><code>curl -X POST -H \"Content-Type: application/json\" -d '{\"payments\": [ { \"address\": \"&lt;address to send to&gt;\"\", \"amount\": { \"quantity\": 83333330000000, \"unit\": \"lovelace\" } } ], \"passphrase\": \"areallylongpasswordagain\"}' http://localhost:3002/v2/wallets/&lt;new wallet id&gt;/transactions\n</code></pre></p>"},{"location":"Appendix/monitoring/","title":"Monitoring","text":"<p>Ensure the Pre-Requisites are in place before you proceed.</p> <p>This is an easy-to-use script to automate setting up of monitoring tools. Tasks automates the following tasks: - Installs Prometheus, Node Exporter and Grafana Servers for your respective Linux architecture. - Configure Prometheus to connect to cardano node and node exporter jobs. - Provisions the installed prometheus server to be automatically available as data source in Grafana. - Provisions two of the common grafana dashboards used to monitor <code>cardano-node</code> by SkyLight and IOHK to be readily consumed from Grafana. - Deploy <code>prometheus</code>,<code>node_exporter</code> and <code>grafana-server</code> as systemd service on Linux. - Start and enable those services.</p> <p>Note that securing prometheus/grafana servers via TLS encryption and other security best practices are out of scope for this document, and its mainly aimed to help you get started with monitoring without much fuss.</p> <p>!&gt; Ensure that you've opened the firewall port for grafana server (default used in this script is 5000)</p>"},{"location":"Appendix/monitoring/#download-setup_monsh","title":"Download setup_mon.sh","text":"<p>If you have run <code>guild-deploy.sh</code>, you can skip this step. To download monitoring script, you can execute the commands below: <pre><code>cd $CNODE_HOME/scripts\nwget https://raw.githubusercontent.com/cardano-community/guild-operators/master/scripts/cnode-helper-scripts/setup_mon.sh\nchmod 750 setup_mon.sh\n</code></pre></p>"},{"location":"Appendix/monitoring/#customise-any-environment-variables","title":"Customise any Environment Variables","text":"<p>The default selection may not always be usable for everyone. You can customise further environment variable settings by opening in editor (eg: <code>vi setup_mon.sh</code> ), and updating variables below to your liking:</p> <pre><code>#!/usr/bin/env bash\n# shellcheck disable=SC2209,SC2164\n\n######################################################################\n#### Environment Variables\n######################################################################\nCNODE_IP=127.0.0.1\nCNODE_PORT=12798\nGRAFANA_HOST=0.0.0.0\nGRAFANA_PORT=5000\nPROJ_PATH=/opt/cardano/monitoring\nPROM_HOST=127.0.0.1\nPROM_PORT=9090\nNEXP_PORT=$(( PROM_PORT + 1 ))\n````\n\n#### Set up Monitoring\n\nExecute setup_mon.sh with full path to destination folder you want to setup monitoring in. If you're following guild folder structure, you do not need to specify `-d`. Read the usage comments below before you run the actual script.\n\nNote that to deploy services as systemd, the script expect sudo access is available to the user running the script.\n\n``` bash\ncd $CNODE_HOME/scripts\n# To check Usage parameters:\n# ./setup_mon.sh -h\n#Usage: setup_mon.sh [-d directory] [-h hostname] [-p port]\n#Setup monitoring using Prometheus and Grafana for Cardano Node\n#-d directory      Directory where you'd like to deploy the packages for prometheus , node exporter and grafana\n#-i IP/hostname    IPv4 address or a FQDN/DNS name where your cardano-node (relay) is running (check for hasPrometheus in config.json; eg: 127.0.0.1 if same machine as cardano-node)\n#-p port           Port at which your cardano-node is exporting stats (check for hasPrometheus in config.json; eg: 12798)\n./setup_mon.sh\n# \n# Downloading prometheus v2.18.1...\n# Downloading grafana v7.0.0...\n# Downloading exporter v0.18.1...\n# Downloading grafana dashboard(s)...\n#   - SKYLight Monitoring Dashboard\n#   - IOHK Monitoring Dashboard\n# \n# NOTE: Could not create directory as rdlrt, attempting sudo ..\n# NOTE: No worries, sudo worked !! Moving on ..\n# Configuring components\n# Registering Prometheus as datasource in Grafana..\n# Creating service files as root..\n# \n# =====================================================\n# Installation is completed\n# =====================================================\n# \n# - Prometheus (default): http://127.0.0.1:9090/metrics\n#     Node metrics:       http://127.0.0.1:12798\n#     Node exp metrics:   http://127.0.0.1:9091\n# - Grafana (default):    http://0.0.0.0:5000\n# \n# \n# You need to do the following to configure grafana:\n# 0. The services should already be started, verify if you can login to grafana, and prometheus. If using 127.0.0.1 as IP, you can check via curl\n# 1. Login to grafana as admin/admin (http://0.0.0.0:5000)\n# 2. Add \"prometheus\" (all lowercase) datasource (http://127.0.0.1:9090)\n# 3. Create a new dashboard by importing dashboards (left plus sign).\n#   - Sometimes, the individual panel's \"prometheus\" datasource needs to be refreshed.\n# \n# Enjoy...\n# \n# Cleaning up...\n</code></pre>"},{"location":"Appendix/monitoring/#view-dashboards","title":"View Dashboards","text":"<p>You should now be able to Login to grafana dashboard, using the public IP of your server, at port 5000. The initial credentials to login would be admin/admin, and you will be asked to update your password upon first login. Once logged on, you should be able to go to <code>Manage &gt; Dashboards</code> and select the dashboard you'd like to view. Note that if you've just started the server, you might see graphs as empty, as initial interval for dashboards is 12 hours. You can change it to 5 minutes by looking at top right section of the page.</p> <p>Thanks to Pal Dorogi for the original setup instructions used for modifying.</p>"},{"location":"Appendix/postgres/","title":"Sample Postgres Setup","text":"<p>These deployment instructions used for reference while building cardano-db-sync tool, with the scope being ease of set up, and some tuning baselines for those who are new to Postgres DB. It is recommended to customise these as per your needs for Production builds.</p> <p>Important</p> <p>You'd find it pretty useful to set up ZFS on your system prior to setting up Postgres, to help with your IOPs throughput requirements. You can find sample install instructions here. You can set up your entire root mount to be on ZFS, or you can opt to mount a file as ZFS on \"${CNODE_HOME}\"</p>"},{"location":"Appendix/postgres/#install-postgresql-server","title":"Install PostgreSQL Server","text":"<p>Execute commands below to set up Postgres Server</p> <pre><code># Determine OS platform\nOS_ID=$( (grep -i ^ID_LIKE= /etc/os-release || grep -i ^ID= /etc/os-release) | cut -d= -f 2)\nDISTRO=$(grep -i ^NAME= /etc/os-release | cut -d= -f 2)\n\nif [ -z \"${OS_ID##*debian*}\" ]; then\n#Debian/Ubuntu\nwget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -\n  RELEASE=$(lsb_release -cs)\necho \"deb [arch=amd64] http://apt.postgresql.org/pub/repos/apt/ ${RELEASE}\"-pgdg main | sudo tee  /etc/apt/sources.list.d/pgdg.list\n  sudo apt-get update\n  sudo apt-get -y install postgresql-16 postgresql-server-dev-16 postgresql-contrib libghc-hdbc-postgresql-dev\n  sudo systemctl restart postgresql\n  sudo systemctl enable postgresql\nelse\necho \"We have no automated procedures for this ${DISTRO} system\"\nfi\n</code></pre>"},{"location":"Appendix/postgres/#create-user-in-postgres","title":"Create User in Postgres","text":"<p>Login to Postgres instance as superuser:</p> <pre><code>echo $(whoami)\n# &lt;user&gt;\nsudo su postgres\npsql\n</code></pre> <p>Note the  returned as the output of <code>echo $(whoami)</code> command. Replace all instance of  in the documentation below. Execute the below in psql prompt. Replace  and PasswordYouWant with your OS user (output of <code>echo $(whoami)</code> command executed above) and a password you'd like to authenticate to Postgres with: <p><pre><code>CREATE ROLE &lt;user&gt; SUPERUSER LOGIN;\nALTER USER &lt;user&gt; PASSWORD 'PasswordYouWant';\n\\q\n</code></pre> Type <code>exit</code> at shell to return to your user from postgres</p>"},{"location":"Appendix/postgres/#verify-login-to-postgres-instance","title":"Verify Login to postgres instance","text":"<pre><code>export PGPASSFILE=$CNODE_HOME/priv/.pgpass\necho \"/var/run/postgresql:5432:cexplorer:*:*\" &gt; $PGPASSFILE\nchmod 0600 $PGPASSFILE\npsql postgres\n# psql (15.0)\n# Type \"help\" for help.\n# \n# postgres=#\n</code></pre>"},{"location":"Appendix/postgres/#tuning-your-instance","title":"Tuning your instance","text":"<p>Before you start populating your DB instance using dbsync data, now might be a good time to put some thought on to baseline configuration of your postgres instance by editing <code>/etc/postgresql/15/main/postgresql.conf</code>. Typically, you might find a lot of common standard practices parameters available in tuning guides. For our consideration, it would be nice to start with some baselines - for which we will use inputs from example here, which would need to be customised further to your environment and resources.</p> <p>In a typical Koios [gRest] setup, we use below for minimum viable specs (i.e. 64GB RAM, &gt; 8 CPUs, &gt;16K IOPs for <code>ioping -q -S512M -L -c 10 -s8k .</code> output when postgres data directory is on ZFS configured with max arc of 4GB), we find the below configuration to be the best common setup:</p> Parameter Value Comment data_directory '/opt/cardano/cnode/guild-db/pgdb/15' Move postgres data directory to ZFS mount at /opt/cardano/cnode, ensure it's writable by postgres user effective_cache_size 8GB Be conservative as Node and DBSync by themselves will need ~32-40GB of RAM if ledger-state is enabled effective_io_concurrency 4 Can go higher if you have substantially higher IOPs/IO throughputs lc_time 'en_US.UTF-8' Just to use standard server-side time formatting between instances, can adapt to your preferences log_timezone 'UTC' For consistency, to avoid timezone confusions maintenance_work_mem 512MB Helps with vacuum/index/foreign key maintainance (with 4 workers, it's set to max 2GB) max_connections 200 Allow maximum of 200 connections, the koios connections are still controlled via postgrest db-pool max_parallel_maintenance_workers 4 Max workers postgres will use for maintainance max_parallel_workers 4 Max workers postgres will use across the system max_parallel_workers_per_gather 2 Parallel threads per query, do not increase to higher values as it will multiply memory usage max_wal_size 4GB Used for WAL automatic checkpoints (disabled later) max_worker_processes 4 Maximum number of background processes system can support min_wal_size 1GB Used for WAL automatic checkpoints (disabled later) random_page_cost 1.1 Use higher value if IOPs has trouble catching up (you can use 4 instead of 1.1) shared_buffers 4GB Conservative limit to allow for node/dbsync/zfs memory usage timezone 'UTC' For consistency, to avoid timezone confusions wal_buffers 16MB WAL consumption in shared buffer (disabled later) work_mem 16MB Base memory size before writing to temporary disk files <p>In addition to above, due to the nature of usage by dbsync (synching from node and restart traversing back to last saved ledger-state snapshot), we leverage data retention on blockchain - as we're not affected by loss of volatile information upon a restart of instance. Thus, we can relax some of the data retention and protection against corruption related settings, as those are IOPs/CPU Load Average impacts that the instance does not need to spend. We'd recommend setting 3 of those below in your <code>/etc/postgresql/15/main/postgresql.conf</code>:</p> Parameter Value wal_level minimal max_wal_senders 0 synchronous_commit off <p>Once your changes are done, ensure to restart postgres service using <code>sudo systemctl restart postgresql</code>.</p>"},{"location":"Build/dbsync/","title":"DBSync","text":"<p>Important</p> <p>An average pool operator may not require cardano-db-sync at all. Please verify if it is required for your use as mentioned here.  </p> <ul> <li>Ensure the Pre-Requisites are in place before you proceed.</li> <li>The Cardano DB Sync relies on an existing PostgreSQL server. To keep the focus on building dbsync tool, and not how to setup postgres itself, you can refer to Sample Local PostgreSQL Server Deployment instructions for setting up a Postgres instance. Specifically, we expect the <code>PGPASSFILE</code> environment variable is set as per the instructions in the sample guide, for <code>db-sync</code> to be able to connect.</li> <li>One of the biggest obstacles for user experience when running dbsync is ensuring you satisfy EACH of the points mentioned in System Requirements here. Also, note that we do not advise running dbsync on mainnet if your RAM is below 48GB.</li> </ul>"},{"location":"Build/dbsync/#build-instructions","title":"Build Instructions","text":""},{"location":"Build/dbsync/#clone-the-repository","title":"Clone the repository","text":"<p>Execute the below to clone the <code>cardano-db-sync</code> repository to <code>$HOME/git</code> folder on your system:</p> <pre><code>cd ~/git\ngit clone https://github.com/intersectmbo/cardano-db-sync\ncd cardano-db-sync\n</code></pre>"},{"location":"Build/dbsync/#build-cardano-db-sync","title":"Build Cardano DB Sync","text":"<p>You can use the instructions below to build the latest release of <code>cardano-db-sync</code>.</p> <p><pre><code>git fetch --tags --all\ngit pull\n# Include the cardano-crypto-praos and libsodium components for db-sync\n# On CentOS 7 (GCC 4.8.5) we should also do\n# echo -e \"package cryptonite\\n  flags: -use_target_attributes\" &gt;&gt; cabal.project.local\n# Replace tag against checkout if you do not want to build the latest released version\ngit checkout $(curl -sLf https://api.github.com/repos/intersectmbo/cardano-db-sync/releases/latest | jq -r .tag_name)\n# Use `-l` argument if you'd like to use system libsodium instead of IOG fork of libsodium while compiling\n$CNODE_HOME/scripts/cabal-build-all.sh\n</code></pre> The above would copy the <code>cardano-db-sync</code> binary into <code>~/.local/bin</code> folder.</p>"},{"location":"Build/dbsync/#prepare-db-for-sync","title":"Prepare DB for sync","text":"<p>Now that binaries are available, let's create our database (when going through breaking changes, you may need to use <code>--recreatedb</code> instead of <code>--createdb</code> used for the first time. Again, we expect that <code>PGPASSFILE</code> environment variable is already set (refer to the top of this guide for sample instructions):</p> <pre><code>cd ~/git/cardano-db-sync\n# scripts/postgresql-setup.sh --dropdb #if exists already, will fail if it doesnt - thats OK\nscripts/postgresql-setup.sh --createdb\n# Password:\n# Password:\n# All good!\n</code></pre> <p>Verify you can see \"All good!\" as above!</p>"},{"location":"Build/dbsync/#create-symlink-to-schema-folder","title":"Create Symlink to schema folder","text":"<p>DBSync instance requires the schema files from the git repository to be present and available to the dbsync instance. You can either clone the <code>~/git/cardano-db-sync/schema</code> folder OR create a symlink to the folder and make it available to the startup command we will be using. We will use the latter in sample below:</p> <pre><code>ln -s ~/git/cardano-db-sync/schema $CNODE_HOME/guild-db/schema\n</code></pre>"},{"location":"Build/dbsync/#restore-using-snapshot","title":"Restore using Snapshot","text":"<p>If you're running a mainnet/preview/preprod instance of dbsync, you might want to consider use of dbsync snapshots as documented here. The snapshot files from IO for their default configs as of recent epoch are available via links in release notes. Note that the snapshots should only be used pertaining to their specific configs, if using configs from Koios - you'd want to look at snapshots here instead.</p> <p>At high-level, this would involve steps as below (read and update paths as per your environment):</p> <pre><code># Replace the actual link below with the latest one from release notes\nwget https://update-cardano-mainnet.iohk.io/cardano-db-sync/13/db-sync-snapshot-schema-13-block-7622755-x86_64.tgz\nrm -rf ${CNODE_HOME}/guild-db/ledger-state ; mkdir -p ${CNODE_HOME}/guild-db/ledger-state\ncd -; cd ~/git/cardano-db-sync\nscripts/postgresql-setup.sh --restore-snapshot /tmp/dbsyncsnap.tgz ${CNODE_HOME}/guild-db/ledger-state\n# The restore may take a while, please be patient and do not interrupt the restore process. Once restore is successful, you may delete the downloaded snapshot as below:\n#   rm -f /tmp/dbsyncsnap.tgz\n</code></pre>"},{"location":"Build/dbsync/#test-running-dbsync-manually-at-terminal","title":"Test running dbsync manually at terminal","text":"<p>In order to verify that you can run dbsync, before making a start - you'd want to ensure that you can run it interactively once. To do so, try the commands below:</p> <pre><code>cd $CNODE_HOME/scripts\nexport PGPASSFILE=$CNODE_HOME/priv/.pgpass\n./dbsync.sh\n</code></pre> <p>You can monitor logs if needed via parallel session using <code>tail -10f $CNODE_HOME/logs/dbsync.json</code>. If there are no error, you would want to press Ctrl-C to stop the dbsync.sh execution and deploy it as a systemd service. To do so, use the commands below (the creation of file is done using <code>sudo</code> permissions, but you can always deploy it manually):</p> <pre><code>cd $CNODE_HOME/scripts\n./dbsync.sh -d\n# Deploying cnode-dbsync.service as systemd service..\n# cnode-dbsync.service deployed successfully!!\n</code></pre> <p>Now to start dbsync instance, you can run <code>sudo systemctl start cnode-dbsync</code></p> <p>Note</p> <p>Note that dbsync while syncs, it might defer creation of indexes/constraints to speed up initial catch up. Once relatively closer to tip, this will initiate creation of indexes - which can take a while in background. Thus, you might notice the query timings right after reaching to tip might not be as good.</p>"},{"location":"Build/dbsync/#update-dbsync","title":"Update DBSync","text":"<p>Updating dbsync can have different tasks depending on the versions involved. We attempt to briefly explain the tasks involved:</p> <ul> <li>Shutdown dbsync (eg: <code>sudo systemctl stop cnode-dbsync</code>)</li> <li>Update binaries (either download pre-compiled binaries via guild-deploy.sh or using build instructions above)</li> <li> <p>Go to your git folder, pull and checkout to latest version as in example below (if you were to switch to <code>13.1.1.3</code>):</p> <pre><code>cd ~/git/cardano-db-sync\ngit pull\ngit checkout 13.1.1.3\n</code></pre> </li> <li> <p>If going through major version update (eg: 13.x.x.x to 14.x.x.x), you might need to rebuild and resync db from scratch, you may still follow the section to restore using snapshot to save some time (as long as you use a compatible snapshot).</p> </li> <li>If the underlying <code>cardano-node</code> version has changed (specifically if it's <code>ledger-state</code> schema is different), you'd also need to clear the ledger-state directory (eg: <code>rm -rf $CNODE_HOME/guild-db/ledger-state</code>)</li> <li>Test that <code>dbsync.sh</code> starts up fine manually as described above. If it does, stop it and go ahead with startup of systemd service (i.e. <code>sudo systemctl start cnode-dbsync</code>)</li> </ul>"},{"location":"Build/dbsync/#validation","title":"Validation","text":"<p>To validate, connect to your <code>postgres</code> instance and execute commands as per below:</p> <pre><code>export PGPASSFILE=$CNODE_HOME/priv/.pgpass\npsql cexplorer\n</code></pre> <p>You should be at the <code>psql</code> prompt, you can check the tables and verify they're populated:</p> <pre><code>\\dt\nselect * from meta;\n</code></pre> <p>A sample output of the above two commands may look like below (the number of tables and names may vary between versions):</p> <pre><code>cexplorer=# \\dt\nList of relations\n Schema |           Name            | Type  | Owner\n--------+---------------------------+-------+-------\n public | ada_pots                  | table | centos\n public | admin_user                | table | centos\n public | block                     | table | centos\n public | delegation                | table | centos\n public | delisted_pool             | table | centos\n public | epoch                     | table | centos\n public | epoch_param               | table | centos\n public | epoch_stake               | table | centos\n public | ma_tx_mint                | table | centos\n public | ma_tx_out                 | table | centos\n public | meta                      | table | centos\n public | orphaned_reward           | table | centos\n public | param_proposal            | table | centos\n public | pool_hash                 | table | centos\n public | pool_meta_data            | table | centos\n public | pool_metadata             | table | centos\n public | pool_metadata_fetch_error | table | centos\n public | pool_metadata_ref         | table | centos\n public | pool_owner                | table | centos\n public | pool_relay                | table | centos\n public | pool_retire               | table | centos\n public | pool_update               | table | centos\n public | pot_transfer              | table | centos\n public | reserve                   | table | centos\n public | reserved_ticker           | table | centos\n public | reward                    | table | centos\n public | schema_version            | table | centos\n public | slot_leader               | table | centos\n public | stake_address             | table | centos\n public | stake_deregistration      | table | centos\n public | stake_registration        | table | centos\n public | treasury                  | table | centos\n public | tx                        | table | centos\n public | tx_in                     | table | centos\n public | tx_metadata               | table | centos\n public | tx_out                    | table | centos\n public | withdrawal                | table | centos\n(37 rows)\n\n\n\nselect * from meta;\n id |     start_time      | network_name\n----+---------------------+--------------\n  1 | 2017-09-23 21:44:51 | mainnet\n(1 row)\n</code></pre>"},{"location":"Build/grest-changelog/","title":"Koios gRest Changelog","text":""},{"location":"Build/grest-changelog/#112-for-all-networks","title":"[1.1.2] - For all networks.","text":"<p>This release is minor bugfix for data consistency changes behind the scenes. It has no impact to any of the API endpoints.</p>"},{"location":"Build/grest-changelog/#chores","title":"Chores:","text":"<ul> <li>Performance optimisation for <code>pool_info</code> : Add a pool_delegators_list RPC that brings across only minimal information required by pool_info, thereby improving it's performance #281</li> <li>Fix <code>pool_history</code> stats for latest epochs by restricting cache to current - 3 epoch while calculating the subsequent information using live status from the database #282</li> <li>Fix rewards/treasury/reserves calculation in <code>account_info</code> , <code>account_info_cached</code> and <code>stake_distribution_cache</code>.</li> </ul>"},{"location":"Build/grest-changelog/#111-for-all-networks","title":"[1.1.1] - For all networks.","text":"<p>This release primarily focuses on backend performance fixes and work with dbsync 13.2.0.2 - while also, we have started preparing compatibility with upcoming koios lite release, to make it a seamless swap for specific endpoints without any impact to consumers. There are no breaking (impact to existing columns or inputs) changes with this release, but we have retired 2 deprecated endpoints that were almost unused on mainnet. Due to the amount of backend changes in queries, there is a chance that we may have missed some data accuracy checks, and - hence - would like to test in non-mainnet networks first before marking final release. Accordingly, any testing/reports of data inconsistency would be welcome.</p>"},{"location":"Build/grest-changelog/#new-endpoints-added","title":"New endpoints added:","text":"<ul> <li><code>/asset_policy_mints</code> - List of mint/burn count for all assets minted under a policy #269</li> <li><code>/block_tx_info</code> - Equivalent of tx_info but uses blocks as inputs to fetch tx_info against all tx in the block[s] requested, also contains additional flags to control performance and output #255</li> <li><code>/cli_protocol_params</code> - Return protocl-parameters as returned by <code>cardano-cli</code> from Koios servers #269</li> </ul>"},{"location":"Build/grest-changelog/#data-inputoutput-changes","title":"Data Input/Output Changes:","text":"<ul> <li>Output - <code>/reserve_withdrawals</code> , <code>/treasury_withdrawals</code> - Add <code>earned_epoch</code> and <code>spendable_epoch</code> fields #269</li> <li>Output - <code>/block</code> - Add <code>parent_hash</code> field #263</li> <li>Output - <code>/account_list</code> - Add <code>stake_address_hex</code> and <code>script_hash</code> fields #263</li> <li>Output - <code>/asset_list</code> - Add <code>script_hash</code> field #263</li> <li>Output - <code>/asset_summary</code> - Add <code>addresses</code> field #263</li> <li>Output - <code>/asset_addresses</code> , <code>/asset_nft_address</code> and <code>/policy_asset_addresses</code> - Add <code>stake_address</code> field #262</li> <li>Output - Fix <code>/script_utxos</code> as it was incorrectly returning object instead of array for asset_list #269</li> <li>Output - <code>/tx_info</code> - Add <code>plutus_contract</code> -&gt; <code>spends_input</code> to <code>plutus_contracts</code> to point the input transaction being consumed by the script #269</li> </ul>"},{"location":"Build/grest-changelog/#deprecations","title":"Deprecations:","text":"<ul> <li>None</li> </ul>"},{"location":"Build/grest-changelog/#retirements","title":"Retirements:","text":"<ul> <li><code>asset_address_list</code> and <code>asset_policy_info</code> endpoints are now retired, as they were marked as deprecated in Koios 1.0.10 , and we have seen it's usage to be negligible (only a single hit in 48 hours on mainnet while marking this release). #269</li> </ul>"},{"location":"Build/grest-changelog/#chores_1","title":"Chores:","text":"<ul> <li>Retire <code>stake_distribution_new_accounts</code> and <code>stake_snapshot_cache</code> cache, as we directly perform lookup on live tables for newly registered accounts #269</li> <li>Active stake cache no longer reads the logs, but instead relies on newly added <code>epoch_sync_progress table</code> #269</li> <li>Reduce asset_info_cache rollback lookup from 1000 to 250 #269</li> <li>Replace <code>consumed_by_tx_in_id</code> references in SQL by <code>consumed_by_tx_id</code> #269</li> <li><code>pool_history_cache</code> now breaks into populating 500 epochs at a time (on guildnet, this query used to run for hours against ~20K epochs) #269</li> <li>Accomodate splitting of <code>reward</code> table into <code>instant_reward</code> #269</li> <li>Add a check in <code>stake_distribution_cache</code> to ensure that epoch info cache was run for current - 1 epoch. #269</li> <li>Change return type for internal function <code>grest.cip67_strip_label</code> from <code>text</code> to <code>bytea</code> #269</li> <li>Remove any references to <code>tx_in</code> as it is no longer required #269</li> <li>Remove references to <code>pool_offline_data</code> with <code>off_chain_pool_data</code> #269</li> <li>Disable running <code>asset-txo-cache-update</code> as the endpoints leveraging asset-txo will be moved to koios-lite</li> <li>Convert <code>block</code>, <code>account_list</code>, <code>asset_list</code> <code>asset_token_registry</code> from view to function #263</li> <li><code>asset_info_cache</code> - ensure mint tx is only against a positive mint quantity #262</li> <li>Include burnt asset transactions in asset_txs #269</li> <li><code>tx_info</code> - Fix spend_redeemers CTE Join condition #269</li> </ul>"},{"location":"Build/grest-changelog/#110-for-all-networks","title":"[1.1.0] - For all networks.","text":"<p>This will be first major [breaking] release for Koios consumers in a while, and will be rolled out under new base prefix (<code>/api/v1</code>). The major work with this release was to start making use of newer flags in dbsync which help performance of queries under new endpoints. Please ensure to check out the release notes for <code>1.1.0rc</code> below. The list for this section is only a small addendum to <code>1.1.0rc</code>:</p>"},{"location":"Build/grest-changelog/#chores_2","title":"Chores:","text":"<ul> <li>Make use of asset-txo-cache for top assets on mainnet, and use this cache for <code>asset_addresses</code> and <code>policy_asset_addresses</code> #250</li> <li>Add v0 RPC redirectors to keep serve v0 endpoints from v1 #250</li> <li>Convert few simple RPC functions from PLPGSQL to SQL language to help with inline filtering #250</li> <li>Address linting results from SQLFluff #250</li> <li>Move db-scripts from guild-operators repository to koios-artifacts repository #250</li> <li>Drop stale db-scripts/genesis_table.sql file #250</li> <li>Add 3 additional indexes for collateral and reference inputs based on query times #250</li> <li>Add top 3 assets for preview/preprod to asset-txo-cache #250</li> <li>Bump schema version for koios-1.1.0 #250</li> <li>Minor patch for output data type (<code>pool_registrations</code> and <code>pool_retirements</code>) #249</li> </ul>"},{"location":"Build/grest-changelog/#110rc-for-all-networks","title":"[1.1.0rc] - For all networks.","text":"<p>This will be first major [breaking] release for Koios consumers in a while, and will be rolled out under new base prefix (<code>/api/v1</code>). The major work with this release was to start making use of newer flags in dbsync which help performance of queries under new endpoints. Also, you'd see quite a few new endpoint additions below, that'd be helping out with slightly lighter version of queries. To keep migration paths easier, we will ensure both v0 and v1 versions of the release is up for a month post release, before retiring v0.</p>"},{"location":"Build/grest-changelog/#new-endpoints-added_1","title":"New endpoints added:","text":"<ul> <li><code>/pool_registrations</code> - List of all pool registrations initiated in the requested epoch #239</li> <li><code>/pool_retirements</code> - List of all pool retirements initiated in the requested epoch #239</li> <li><code>/treasury_withdrawals</code> - List of withdrawals made from treasury #239</li> <li><code>/reserve_withdrawals</code> - List of withdrawals made from reserves (MIRs) #239</li> <li><code>/account_txs</code> - Transactions associated with a given stake address #239</li> <li><code>/address_utxos</code> - Get UTxO details for requested addresses #239</li> <li><code>/asset_utxos</code> - Get UTxO details for requested assets #239</li> <li><code>/script_utxos</code> - Get UTxO details for requested script hashes #239</li> <li><code>/utxo_info</code> - Details for requested UTxO arrays #239</li> <li><code>/script_info</code> - Information about a given script FROM script hashes #239</li> <li><code>/ogmios/</code> - Expose stateless ogmios endpoints #1690</li> </ul>"},{"location":"Build/grest-changelog/#data-inputoutput-changes_1","title":"Data Input/Output Changes:","text":"<ul> <li>Input - <code>/account_utxos</code> , <code>/credential_utxos</code> - Accept <code>extended</code> as an additional flag - which enables <code>asset_list</code>, <code>reference_script</code> and <code>inline_datum</code> to the output #239</li> <li>Output - <code>/block_txs</code> - Flatten output with transaction details (<code>tx_hash</code>, <code>epoch_no</code>, <code>block_height</code>, <code>block_time</code>) instead of <code>tx_hashes</code> array #239</li> <li>Output - <code>/epoch_params</code> - Update <code>cost_models</code> to JSON (upstream change in node) #239</li> <li>Output - <code>/account_assets</code> , <code>/address_assets</code> - Flatten the output result (instead of <code>asset_list</code> array) making it easier to apply horizontal filtering based on any of the fields</li> <li>Output - Align output fields for <code>/account_utxos</code> , <code>/address_utxos</code>, <code>/asset_utxos</code> , <code>/script_utxos</code> and <code>/utxo_info</code> to return same schema giving complete details about UTxOs involved, with few fields toggled based on <code>extended</code> input flag #239</li> <li>Output - <code>/pool_list</code> - Add various details to the endpoint for each pool (<code>pool_id_hex</code>,<code>active_epoch_no</code>,<code>margin</code>,<code>fixed_cost</code>,<code>pledge</code>,<code>reward_addr</code>,<code>owners</code>,<code>relays</code>,<code>ticker</code>,<code>meta_url</code>,<code>meta_hash</code>,<code>pool_status</code>,<code>retiring_epoch</code>) - this should mean some of the requests to <code>pool_info</code> should no longer be required #239</li> <li>Output - <code>/pool_updates</code> - In v0, <code>pool_updates</code> only provided pool registration updates, while <code>pool_status</code> corresponded to current status of pool. With v1, we will have registration as well as deregistration transactions, and each transaction will have <code>update_type</code> (enum of either <code>registration</code> or <code>deregistration</code>) instead of <code>pool_status</code>. As a side-effect, since a registration transaction only has <code>retiring_epoch</code> as metadata, all the other fields will show up as <code>null</code> for such a transaction #239</li> <li>Output - <code>/pool_metadata</code> , <code>/pool_relays</code> - Add <code>pool_status</code> field to denote whether pool is retired #239</li> <li>Output - <code>/datum_info</code> - Rename <code>hash</code> to <code>datum_hash</code> and add <code>creation_tx_hash</code> #239</li> <li>Output - <code>/native_script_list</code> - Remove <code>script</code> column (as it has pretty large output better queried against <code>script_info</code>), add <code>size</code> and change <code>type</code> to text #239</li> <li>Output - <code>/plutus_script_list</code> - Add <code>type</code> and <code>size</code> to output #239</li> <li>Output - <code>/asset_info</code> - Add <code>cip68_metadata</code> JSONB field #239</li> <li>Output - <code>/pool_history</code> - Add member_rewards #225</li> </ul>"},{"location":"Build/grest-changelog/#deprecations_1","title":"Deprecations:","text":"<ul> <li><code>/tx_utxos</code> - No longer required as replaced by <code>/utxo_info</code> #239</li> </ul>"},{"location":"Build/grest-changelog/#chores_3","title":"Chores:","text":"<ul> <li>Update base version to <code>v1</code> from <code>v0</code> #1690</li> <li>Allow Bearer Authentication against endpoints #239</li> <li>Cron job to apply corrections to epoch info #239</li> <li><code>epoch_info_cache</code> Remove protocol parameters, as they can be queried from live table. Accordingly update dependent queries #239</li> <li>Make use of new <code>consumed_by_tx_in_id</code> column in <code>tx_out</code> from dbsync 13.1.1.3 across endpoints #239</li> <li>Fix <code>_last_active_stake_validated_epoch</code> in active_stake_cache #222</li> <li>Typo for pool_history_cache.sql as well as add a check to ensure epoch_info_cache has run at least once prior to pool_history_cache #223</li> <li>Move control_table entry in cache tables to the end (instead of start) #226</li> <li>Fix Asset Info Cache (include mint/burn tx rather than sum for meta consideration) #226</li> <li>Update SQLs as per SQLFluff linting guidelines #226</li> <li>Fix for tip check in cron jobs #217</li> <li>Update cron jobs to exit if the database has not received block in 5 mins #200</li> <li>Update active stake cache to use control table #196</li> <li>Update Asset Info Cache entry whenever asset registry cache has an update #194</li> <li>Bump up margin for tx rollback lookup for asset_info_cache to 1000 , as 100 is too small a margin for 2-3 blocks , which can contain more than 100 transactions (of which if oldest transaction contains a mint, it will not get into the cache) #177</li> <li>Swap grestrpcs file to list exceptions and treat everything else as RPC #1690</li> <li>Update grest-poll.sh to have stricter spec validation and add health check for asset_registry #1690</li> <li>Update guild-deploy.sh to include latest pre-release for ogmios #1690</li> </ul>"},{"location":"Build/grest-changelog/#1010-for-all-networks","title":"[1.0.10] - For all networks.","text":"<p>The release is effectively same as <code>1.0.10rc</code> except with one minor modification below.</p>"},{"location":"Build/grest-changelog/#chores_4","title":"Chores:","text":"<ul> <li>Replace all RPC references for JSON endpoints with JSONB, this allows filtering child members of array elements using <code>cs.[{\"key\":\"value\"}]</code> in PostgREST #172</li> </ul>"},{"location":"Build/grest-changelog/#1010rc-for-non-mainnet-networks","title":"[1.0.10rc] - For non-mainnet networks","text":"<p>This release primarily focuses on ability to support better DeFi projects alongwith some value addition for existing clients by bringing in 10 new endpoints (paired with 2 deprecations), and few additional optional input parameters , and some additional output columns to existing endpoints. The only breaking change/fix is for output returned for <code>tx_info</code>.</p> <p>Also, dbsync 13.1.x.x has been released and is recommended to be used for this release</p>"},{"location":"Build/grest-changelog/#new-endpoints-added_2","title":"New endpoints added","text":"<ul> <li><code>/asset_addresses</code> -  Equivalent of deprecated <code>/asset_address_list</code> #149</li> <li><code>/asset_nft_address</code> - Returns address where the specified NFT sits on #149</li> <li><code>/account_utxos</code> - Returns brief details on non-empty UTxOs associated with a given stake address #149</li> <li><code>/asset_info_bulk</code> - Bulk version of <code>/asset_info</code> #142</li> <li><code>/asset_token_registry</code> - Returns assets registered via token registry on github #145</li> <li><code>/credential_utxos</code> - Returns UTxOs associated with a payment credential #149</li> <li><code>/param_updates</code> - Returns list of parameter update proposals applied to the network #149</li> <li><code>/policy_asset_addresses</code> - Returns addresses with quantity for each asset on a given policy #149</li> <li><code>/policy_asset_info</code> - Equivalent of deprecated <code>/asset_policy_info</code> but with more details in output #149</li> <li><code>/policy_asset_list</code> - Returns list of asset under the given policy (including supply) #142, #149</li> </ul>"},{"location":"Build/grest-changelog/#data-inputoutput-changes_2","title":"Data Input/Output Changes","text":"<ul> <li>Input - <code>/account_addresses</code> - Add optional <code>_first_only</code> and <code>_empty</code> flags to show only first address with tx or to include empty addresses to output #149</li> <li>Input - <code>/epoch_info</code> - Add optional <code>_include_next_epoch</code> field to show next epoch stats if available (eg: nonce, active stake) #143</li> <li>Output (addition) - <code>/account_assets</code> , <code>/address_assets</code> , <code>/address_info</code>, <code>/tx_info</code>, <code>/tx_utxos</code> - Add <code>decimals</code> to output #142</li> <li>Output (addition) - <code>/policy_asset_info</code> - Add <code>minting_tx_hash</code>, <code>total_supply</code>, <code>mint_cnt</code>, <code>burn_cnt</code> and <code>creation_time</code> fields to the output #149</li> <li>Output (breaking) - <code>/tx_info</code> - Change <code>_invalid_before</code> and <code>_invalid_after</code> to text field #141</li> <li>Output (breaking/removal) - <code>tx_info</code> - Remove the field <code>plutus_contracts</code> &gt; [array] &gt; <code>outputs</code> as there is no logic to connect it to inputs spending #163</li> </ul>"},{"location":"Build/grest-changelog/#deprecations_2","title":"Deprecations:","text":"<ul> <li><code>/asset_address_list</code> - Renamed to <code>asset_addresses</code> keeping naming line with other endpoints (old one still present, but will be deprecated in future release) #149</li> <li><code>/asset_policy_info</code> - Renamed to <code>policy_asset_info</code> keeping naming line with other endpoints (old one still present, but will be deprecated in future release) #149</li> </ul>"},{"location":"Build/grest-changelog/#chores_5","title":"Chores:","text":"<ul> <li><code>/epoch_info</code>, <code>/epoch_params</code> - Restrict output to current epoch #149</li> <li><code>/block_info</code> - Use <code>/previous_id</code> field to show previous/next blocks (previously was using block_id/height) #145</li> <li><code>/asset_info</code>/<code>asset_policy_info</code> - Fix mint tx data to be latest #141</li> <li>Support new guild scripts revamp #1572</li> <li>Add asset token registry check 1606</li> <li>New cache table <code>grest.asset_info_cache</code> to hold mint/burn counts alongwith first/last mint tx/keys #142</li> <li>Bump to Koios 1.0.10rc #149</li> <li>Fix typo in specs for <code>/pool_delegators</code> output column <code>latest_delegation_tx_hash</code> #149</li> <li>Add indexes for ones missing after configuring cardano-db-sync 13.1.0.0 #149</li> <li>Update PostgREST to be run as <code>authenticator</code> user, whose default <code>statement_timeout</code> is set to 65s and update configs accordingly #1606</li> </ul>"},{"location":"Build/grest-changelog/#109-for-all-networks","title":"[1.0.9] - For all networks","text":"<p>This release is effectively same as <code>1.0.9rc</code> below (please check out the notes accordingly), just with minor bug fix on <code>setup-grest.sh</code> itself.</p>"},{"location":"Build/grest-changelog/#109rc-for-non-mainnet-networks","title":"[1.0.9rc] - For non-mainnet networks","text":"<p>This release candidate is non-breaking for existing methods and inputs, but breaking for output objects for endpoints. The aim with release candidate version is to allow folks couple of weeks to test, adapt their libraries before applying to mainnet.</p>"},{"location":"Build/grest-changelog/#new-endpoints-added_3","title":"New endpoints added","text":"<ul> <li><code>datum_info</code> - List of datum information for given datum hashes</li> <li><code>account_info_cached</code> - Same as <code>account_info</code>, but serves cached information instead of live one</li> </ul>"},{"location":"Build/grest-changelog/#data-inputoutput-changes_3","title":"Data Input/Output changes","text":"<ul> <li><code>address_info</code>, <code>address_assets</code>, <code>account_assets</code>, <code>tx_info</code>, <code>asset_list</code> <code>asset_summary</code> to align output <code>asset_list</code> object to return array of <code>policy_id</code>, <code>asset_name</code>, <code>fingerprint</code> (and <code>quantity</code>, <code>minting_txs</code> where applicable) #120</li> <li><code>asset_history</code> - Fix metadata to wrap in array to refer to right object #122</li> <li><code>asset_txs</code> - Add optional boolean parameter <code>_history</code> (default: <code>false</code>) to toggle between querying current UTxO set vs entire history for asset #122</li> <li><code>pool_history</code> - <code>fixed_cost</code>, <code>pool_fees</code>, <code>deleg_rewards</code>, <code>epoch_ros</code> will be returned as 0 when null #122</li> <li><code>tx_info</code> - <code>plutus_contracts-&gt;outputs</code> can be null #122</li> </ul>"},{"location":"Build/grest-changelog/#changes-for-instance-providers","title":"Changes for Instance Providers","text":"<ul> <li>SQL queries have been moved from <code>guild-operators</code> repository to <code>koios-artifacts</code> repository. This is to ensure that the updates made to scripts and other tooling do not have a dependency on Koios query versioning #122</li> <li><code>block_info</code> - Use <code>block_no</code> instead of <code>id</code> to check for previous/next block hash #122</li> <li>Add topology for preprod and preview networks #122</li> </ul>"},{"location":"Build/grest-changelog/#108-for-all-networks","title":"[1.0.8] - For all networks","text":"<p>This release is contains minor bug-fixes that were discovered in koios-1.0.7. No major changes to output for this one.</p>"},{"location":"Build/grest-changelog/#changes-for-api","title":"Changes for API","text":""},{"location":"Build/grest-changelog/#new-endpoints-added_4","title":"New endpoints added","text":"<ul> <li>None</li> </ul>"},{"location":"Build/grest-changelog/#data-inputoutput-changes_4","title":"Data Input/Output changes","text":"<ul> <li><code>tx_info</code> and <code>tx_metadata</code> - Align metadata for JSON output format #1542</li> <li><code>blocks</code> - Query Output aligned to specs (<code>epoch</code> =&gt; <code>epoch_no</code>)</li> <li><code>epoch_block_protocols</code> - [ ** Specs only ** ] Fix Documentation schema , which was accidentally showing wrong output</li> <li><code>pool_delegators_history</code> - List all epochs instead of current, if no <code>_epoch_no</code> is specified #1545</li> </ul>"},{"location":"Build/grest-changelog/#changes-for-instance-providers_1","title":"Changes for Instance Providers","text":"<ul> <li><code>asset_info</code> - Fix metadata aggregaton for minting transactions with multiple metadata keys #1543</li> <li><code>stake_distribution_new_accounts</code> - Leftover reference for <code>account_info</code> which now accepts array, resulted in error to populate stake distribution cache for new accounts #1541</li> <li><code>grest-poll.sh</code> - Remove query view section from polling script, and remove grestrpcs re-creation per hour (it's already updated when <code>setup-grest.sh</code> is run) , in preparation for #1545</li> </ul>"},{"location":"Build/grest-changelog/#107-for-all-networks","title":"[1.0.7] - For all networks","text":"<p>This release continues updates from koios-1.0.6 to further utilise stake-snapshot cache tables which would be useful for SPOs as well as reduce downtime post epoch transition. One largely requested feature to accept bulk inputs for many block/address/account endpoints is now complete. Additionally, koios instance providers are now recommended to use cardano-node 1.35.3 with dbsync 13.0.5.</p>"},{"location":"Build/grest-changelog/#changes-for-api_1","title":"Changes for API","text":""},{"location":"Build/grest-changelog/#new-endpoints-added_5","title":"New endpoints added","text":"<ul> <li><code>pool_delegators_history</code> - Provides historical record for pool's delegators #1486</li> <li><code>pool_stake_snapshot</code> - Provides mark, set and go snapshot values for pool being queried. #1489</li> </ul>"},{"location":"Build/grest-changelog/#data-inputoutput-changes_5","title":"Data Input/Output changes","text":"<ul> <li><code>pool_delegators</code> - No longer accepts <code>_epoch_no</code> as parameter, as it only returns live delegators. Additionally provides <code>latest_delegation_hash</code> in output. #1486</li> <li><code>tx_info</code> - <code>epoch</code> =&gt; <code>epoch_no</code> #1494</li> <li><code>tx_info</code> - Change <code>collateral_outputs</code> (array) to <code>collateral_output</code> (object) as collateral output is only singular in current implementation #1496</li> <li><code>address_info</code> - Add <code>inline_datum</code> and <code>reference_script</code> to output #1500</li> <li><code>pool_info</code> - Add <code>sigma</code> field to output #1511</li> <li><code>pool_updates</code> - Add historical metadata information to output #1503</li> <li>Change block/address/account endpoints to accept bulk input where applicable. This resulted in GET requests changing to POST accepting payload of multiple blocks, addresses or accounts for respective endpoints as input (eg: <code>_stake_address text</code> becomes <code>_stake_addresses text[]</code>). The additional changes in output as below:</li> <li><code>block_txs</code> - Now returns <code>block_hash</code> and array of <code>tx_hashes</code></li> <li><code>address_info</code> - Additional field <code>address</code> returned in output</li> <li><code>address_assets</code> - Now returns <code>address</code> and an array of <code>assets</code> JSON</li> <li><code>account_addresses</code> - Accepts <code>stake_addresses</code> array and outputs <code>stake_address</code> and array of <code>addresses</code></li> <li><code>account_assets</code> - Accepts <code>stake_addresses</code> array and outputs <code>stake_address</code> and array of <code>assets</code> JSON</li> <li><code>account_history</code> - Accepts <code>stake_addresses</code> array alongwith <code>epoch_no</code> integer and outputs <code>stake_address</code> and array of <code>history</code> JSON</li> <li><code>account_info</code> - Accepts <code>stake_addresses</code> array and returns additional field <code>stake_address</code> to output</li> <li><code>account_rewards</code> - Now returns <code>stake_address</code> and an array of <code>rewards</code> JSON</li> <li><code>account_updates</code> - Now returns <code>stake_address</code> and an array of <code>updates</code> JSON</li> <li><code>asset_info</code> - Change <code>minting_tx_metadata</code> from array to object #1533</li> <li><code>account_addresses</code> - Sort results by oldest address first #1538</li> </ul>"},{"location":"Build/grest-changelog/#changes-for-instance-providers_2","title":"Changes for Instance Providers","text":"<ul> <li><code>epoch_info_cache</code> - Only update last_tx_id of previous epoch on epoch transition #1490 and #1502</li> <li><code>epoch_info_cache</code> / <code>stake_snapshot_cache</code> - Store total snapshot stake to epoch stake cache, and active pool stake to stake snapshot cache #1485</li> </ul>"},{"location":"Build/grest-changelog/#106106m-interim-release-for-all-networks-to-upgrade-to-dbsync-v13","title":"[1.0.6/1.0.6m] - Interim release for all networks to upgrade to dbsync v13","text":"<p>The backlog of items not being added to mainnet has been increasing due to delays with Vasil HFC event to Mainnet. As such we had to come up with a split update approach. The mainnet nodes are still not qualified to be Vasil-ready (in our opinion) for 1.35.x , but dbsync 13 can be used against node 1.34.1 fine. In order to cater for this split, we have added an intermediate koios-1.0.6m tag that brings dbsync updates while maintaining node-1.34.1.</p>"},{"location":"Build/grest-changelog/#changes-for-api_2","title":"Changes for API","text":""},{"location":"Build/grest-changelog/#data-output-changes","title":"Data Output Changes","text":"<ul> <li><code>pool_delegators</code> - <code>epoch_no</code> =&gt; <code>active_epoch_no</code> #1454</li> <li><code>asset_history</code> - Add <code>block_time</code> and <code>metadata</code> fields for all previous mint transactions #1468</li> <li><code>asset_info</code> - Retain latest mint transaction instead of first (in line with most CIPs as well as pool metadata - latest valid meta being live) #1468</li> <li>Ensure all output date formats is integer to keep in line with UNIX timestamps - to be revised in future if/when there are milliseconds #1460</li> <li><code>/tip</code> , <code>/blocks</code>, <code>/block_info</code> =&gt; <code>block_time</code></li> <li><code>/genesis</code> =&gt; <code>systemStart</code></li> <li><code>/epoch_info</code> =&gt; <code>start_time</code>, <code>first_block_time</code>, <code>last_block_time</code>, <code>end_time</code></li> <li><code>/tx_info</code> =&gt; <code>tx_timestamp</code></li> <li><code>/asset_info</code> =&gt; <code>creation_time</code></li> <li><code>tx_info</code> - Add Vasil data #1464</li> <li><code>collaterals</code> =&gt; <code>collateral_inputs</code></li> <li>Add <code>collateral_outputs</code>, <code>reference_inputs</code> to <code>tx_info</code></li> <li>Add <code>datum_hash</code>, <code>inline_datum</code>, <code>reference_script</code> to collateral input/outputs, reference inputs &amp; inputs/outputs JSON.</li> <li>Add complete <code>cost_model</code> instead of <code>cost_model_id</code> reference</li> <li><code>epoch_params</code> - Update leftover lovelace references to text for consistency: #1484</li> <li><code>key_deposit</code></li> <li><code>pool_deposit</code></li> <li><code>min_utxo_value</code></li> <li><code>min_pool_cost</code></li> <li><code>coins_per_utxo_size</code></li> </ul>"},{"location":"Build/grest-changelog/#changes-for-instance-providers_3","title":"Changes for Instance Providers","text":"<ul> <li><code>get-metrics.sh</code> - Add active/idle connections to database #1459</li> <li><code>grest-poll.sh</code>: Bump haproxy to 2.6.1 and set default value of API_STRUCT_DEFINITION to be dependent on network used. #1450</li> <li>Lighten <code>grest.account_active_stake_cache</code> - optimise code and delete historical view (beyond 4 epochs). [#1451(https://github.com/cardano-community/guild-operators/pull/1451)</li> <li><code>tx_metalabels</code> - Move metalabels from view to RPC using lose indexscan (much better performance) #1474</li> <li>Major re-work to artificially add last epoch's active stake cache data (brings in latest snapshot information without depending on node), not used in endpoints for this release #1452</li> <li><code>grest.stake_snapshot_cache</code> - Fix rewards for new accounts #1476</li> </ul>"},{"location":"Build/grest-changelog/#105-alpha-networks-only","title":"[1.0.5] - alpha networks only","text":"<p>Since there have been a few deviations wrt Vasil for testnet and mainnet, this version only targets networks except Mainnet!</p>"},{"location":"Build/grest-changelog/#changes-for-api_3","title":"Changes for API","text":""},{"location":"Build/grest-changelog/#data-output-changes_1","title":"Data Output Changes","text":"<ul> <li><code>/epoch_info</code> - Add <code>total_rewards</code> and <code>avg_block_reward</code> for a given epoch #43</li> <li>Update all date output formats to return UNIX timestamp (as per poll held in discussions group): #45</li> <li><code>/tip</code> , <code>/blocks</code>, <code>/block_info</code> =&gt; <code>block_time</code></li> <li><code>/genesis</code> =&gt; <code>systemStart</code></li> <li><code>/epoch_info</code> =&gt; <code>start_time</code>, <code>first_block_time</code>, <code>last_block_time</code>, <code>end_time</code></li> <li><code>/tx_info</code> =&gt; <code>tx_timestamp</code></li> <li><code>/asset_info</code> =&gt; <code>creation_time</code></li> <li><code>/blocks</code>, <code>/block_info</code> =&gt; Add <code>proto_major</code> and <code>proto_minor</code>  for a given block to output #55</li> </ul>"},{"location":"Build/grest-changelog/#changes-for-instance-providers_4","title":"Changes for Instance Providers","text":"<ul> <li>For consistency between date formats, we highly recommend you to upgrade your instance to use Postgres 14 (prolly a good time, since you would already need to recreate DB for dbsync v13). You can find sample instructions to do so here</li> <li>Various changes to backend scripts and performance optimisations that can be found here</li> </ul>"},{"location":"Build/grest-changelog/#101","title":"[1.0.1]","text":"<ul> <li>Modify <code>asset_registry_update.sh</code> script to rely on commit hash instead of POSIX timestamps, and performance bump. #1428</li> </ul>"},{"location":"Build/grest-changelog/#100","title":"[1.0.0]","text":"<ul> <li>First Production release for Koios gRest</li> </ul>"},{"location":"Build/grest-changelog/#100-rc1","title":"[1.0.0-rc1]","text":""},{"location":"Build/grest-changelog/#changes-for-api_4","title":"Changes for API","text":""},{"location":"Build/grest-changelog/#data-output-changes_2","title":"Data Output Changes","text":"<ul> <li>Improve: Add <code>epoch_no</code>, <code>block_no</code> to <code>/address_txs</code>, <code>/credential_txs</code> and <code>/asset_txs</code> endpoints. #1409</li> <li>Fix: Remove redundant policy_info for <code>/asset_txs</code>, returning transactions as an array - allows for leveraging native PostgREST filtering. #1409</li> <li>Fix: Pool Metadata sorting was incorrect for <code>/pool_info</code>. #1414</li> </ul>"},{"location":"Build/grest-changelog/#input-parameter-changes","title":"Input Parameter Changes","text":"<ul> <li>None</li> </ul>"},{"location":"Build/grest-changelog/#changes-for-instance-providers_5","title":"Changes for Instance Providers","text":""},{"location":"Build/grest-changelog/#added","title":"Added","text":"<ul> <li>Add setup-grest.sh versioning. When running setup-grest.sh against a branch/tag, it will now populate the version information on control table, the health checks will be able to use this versioning for downstream connections. #1403</li> </ul>"},{"location":"Build/grest-changelog/#fixed","title":"Fixed","text":"<ul> <li>Delete token token-registry folder when running <code>setup-grest.sh</code> with <code>-r</code> (reset flag), as the delta registry records to insert depends on file (POSIX) timestamps. #1410</li> <li>Remove duplicate tip check in <code>grest-poll.sh</code>. </li> </ul>"},{"location":"Build/grest-changelog/#100-rc0-2022-04-29","title":"[1.0.0-rc0] - 2022-04-29","text":"<ul> <li>Initial Release Candidate for Koios gRest API layer with 43 endpoints to query the chain.</li> </ul>"},{"location":"Build/grest/","title":"Koios gRest","text":"<p>Important</p> <ul> <li>An average pool operator may not require this component at all. Please verify if it is required for your use as mentioned here</li> <li>Ensure that you have setup DBSync and that it is in sync atleast to Mary fork before you proceed. IF you're participating in Koios services, ensure that you're using latest dbsync release</li> </ul>"},{"location":"Build/grest/#what-is-grest","title":"What is gRest","text":"<p>gRest is an open source implementation of a <code>query layer built over dbsync using PostgREST and HAProxy</code>. The package is built as part of Koios team's efforts to unite community individual stream of work together and give back a more aligned structure to query dbsync and adopt standardisation to queries utilising open-source tooling as well as collaboration. In addition to these, there are also accessibility features to deploy rules for failover, do healthchecks, set up priorities, have ability to prevent DDoS attacks, provide timeouts, report tips for analysis over a longer period, etc - which can prove to be really useful when performing any analysis for instances.</p> <p>Note</p> <p>Note that the scripts below do allow for provisioning ogmios integration too, but Ogmios - currently - is not designed to provide advanced session management for a server-client architecture in absence of a middleware. Thus, the availability for ogmios from monitoring instance is restricted to avoid ability to DDoS an instance.</p>"},{"location":"Build/grest/#components","title":"Components","text":"<ol> <li> <p>PostgREST:     An RPC JSON interface for any PostgreSQL database (in our case, database served via <code>cardano-db-sync</code>) to provide a RESTful Web Service. The endpoints of PostgREST in itself are essentially the table/functions defined in elected schema via grest config file. You can read more about advanced query syntax using PostgREST API here, but we will provide a simpler view using examples towards the end of the page. It is an easy alternative - with almost no overhead as it directly serves the underlying database as an API, as compared to <code>Cardano GraphQL</code> component (which may often have lags). Some of the other advantages of PostgREST over graphql based projects are also performance, being stateless, 0 overhead, support for JWT / native Postgres DB authentication against the Rest Interface as well.</p> </li> <li> <p>HAProxy:     An easy gateway proxy that automatically provides failover/basic DDoS protection, specify rules management for load balancing, setup multiple frontend/backends, provide easy means to have TLS enabled for public facing instances, etc. You may alter the settings for proxy layer as per your SecOps preferences. This component is optional (eg: if you prefer to expose your PostgREST server itself, you can do so using similar steps below).</p> </li> </ol>"},{"location":"Build/grest/#setup","title":"Setup gRest services","text":"<p>To start with you'd want to ensure your current shell session has access to Postgres credentials, continuing from examples from the above mentioned Sample Postgres deployment guide.</p> <pre><code>cd $CNODE_HOME/priv\nPGPASSFILE=$CNODE_HOME/priv/.pgpass\npsql cexplorer\n</code></pre> <p>Ensure that you can connect to your Postgres DB fine using above (quit from psql once validated using <code>\\q</code>). As part of <code>guild-deploy.sh</code> execution, you'd find setup-grest.sh file made available in <code>${CNODE_HOME}/scripts</code> folder, which will help you automate installation of PostgREST, HAProxy as well as brings in latest queries/functions provided via Koios to your instances.</p> <p>Warning</p> <p>As of now, gRest services are in alpha stage - while can be utilised, please remember there may be breaking changes and every collaborator is expected to work with the team to keep their instances up-to-date using alpha branch.</p> <p>Familiarise with the usage options for the setup script , the syntax can be viewed as below:</p> <pre><code>cd \"${CNODE_HOME}\"/scripts\n./setup-grest.sh -h\n#\n# Usage: setup-grest.sh [-f] [-i [p][r][m][c][d]] [-u] [-b &lt;branch&gt;]\n# \n# Install and setup haproxy, PostgREST, polling services and create systemd services for haproxy, postgREST and dbsync\n# \n# -f    Force overwrite of all files including normally saved user config sections\n# -i    Set-up Components individually. If this option is not specified, components will only be installed if found missing (eg: -i prcd)\n#     p    Install/Update PostgREST binaries by downloading latest release from github.\n#     r    (Re-)Install Reverse Proxy Monitoring Layer (haproxy) binaries and config\n#     m    Install/Update Monitoring agent scripts\n#     c    Overwrite haproxy, postgREST configs\n#     d    Overwrite systemd definitions\n# -u    Skip update check for setup script itself\n# -q    Run all DB Queries to update on postgres (includes creating grest schema, and re-creating views/genesis table/functions/triggers and setting up cron jobs)\n# -b    Use alternate branch of scripts to download - only recommended for testing/development (Default: master)\n#\n</code></pre> <p>To run the setup overwriting all standard deployment tasks from a branch (eg: <code>koios-1.0.9</code> branch), you may want to use: <pre><code>./setup-grest.sh -f -i prmcd -r -q -b koios-1.0.9\n</code></pre></p> <p>Similarly - if you'd like to re-install all components and force overwrite all configs but not reset cache tables, you may run: <pre><code>./setup-grest.sh -f -i prmcd -q\n</code></pre></p> <p>Another example could be to preserve your config, but only update queries using an alternate branch (eg: let's say you want to try the branch <code>alpha</code> prior to a tagged release). To do so, you may run: <pre><code>./setup-grest.sh -q -b alpha\n</code></pre></p> <p>Please ensure to follow the on-screen instructions, if any (for example restarting deployed services, or updating configs to specify correct target postgres URLs/enable TLS/add peers etc in <code>${CNODE_HOME}/priv/grest.conf</code> and <code>${CNODE_HOME}/files/haproxy.cfg</code>).</p> <p>The default ports used will make haproxy instance available at port 8053 or 8453 if TLS is enabled (you might want to enable firewall rule to open this port to services you would like to access). If you want to prevent unauthenticated access to grest schema, uncomment the jwt-secret and specify a custom <code>secret-token</code>.</p> <p>Reminder</p> <p>Once you've successfully deployed the grest instance, it will deploy certain cron jobs that will ensure the relevant cache tables are updated periodically. Until these have finished (especially on first run, it could take an hour or so on mainnet, your instance will likely not pass any tests from <code>grest-poll.sh</code> but that's expected.</p>"},{"location":"Build/grest/#tls","title":"Enable TLS on HAProxy","text":"<p>In order to enable SSL on your haproxy, all you need to do is edit the file <code>${CNODE_HOME}/files/haproxy.cfg</code> and update the frontend app section to uncomment ssl bind (and comment normal bind).</p> <p>Info</p> <ul> <li>server.pem referred below should be a chain containing server TLS certificate, signing certificates (intermediate/root) and private key.</li> <li>Make sure to replace the hostname to the CNAME/SAN used to create your TLS certificate.</li> </ul> <p>If you're not familiar with how to configure TLS OR would not like to buy one, you can find tips on how to create a TLS certificate for free via LetsEncrypt using tutorials here. Once you do have a TLS Certificate generated, you need to chain the private key and full chain cert together in a file - <code>/etc/ssl/server.pem</code> - which can be then referenced as below:</p> <p><pre><code>frontend app\n  #bind 0.0.0.0:8053\n  ## If using SSL, comment line above and uncomment line below\n  bind :8453 ssl crt /etc/ssl/server.pem no-sslv3\n  http-request set-log-level silent\n  acl srv_down nbsrv(grest_postgrest) eq 0\n  acl is_wss hdr(Upgrade) -i websocket\n  ...\n</code></pre> Restart haproxy service for changes to take effect.</p>"},{"location":"Build/grest/#validation","title":"Validation","text":"<p>With the setup, you also have a <code>checkstatus.sh</code> script, which will query the Postgres DB instance via haproxy (coming through postgREST), and only show an instance up if the latest block in your DB instance is within 180 seconds.</p> <p>Important</p> <p>If you'd like to participate in joining to the elastic cluster via Koios, please raise a PR request by editing topology files in this folder to do so!!</p> <p>If you were using <code>guild</code> network, you could do a couple of very basic sanity checks as per below:</p> <ol> <li> <p>To query active stake for pool <code>pool1z2ry6kxywgvdxv26g06mdywynvs7jj3uemnxv273mr5esukljsr</code> in epoch <code>122</code>, we can execute the below: <pre><code>curl -d _pool_bech32=pool1z2ry6kxywgvdxv26g06mdywynvs7jj3uemnxv273mr5esukljsr -d _epoch_no=122 -s http://localhost:8053/rpc/pool_active_stake\n## {\"active_stake_sum\" : 19409732875}\n</code></pre></p> </li> <li> <p>To check latest owner key(s) for a given pool <code>pool1z2ry6kxywgvdxv26g06mdywynvs7jj3uemnxv273mr5esukljsr</code>, you can execute the below: <pre><code>curl -d _pool_bech32=pool1z2ry6kxywgvdxv26g06mdywynvs7jj3uemnxv273mr5esukljsr -s http://localhost:8050/rpc/pool_owners\n## [{\"owner\" : \"stake_test1upx5p04dn3t6dvhfh27744su35vvasgaaq565jdxwlxfq5sdjwksw\"}, {\"owner\" : \"stake_test1uqak99cgtrtpean8wqwp7d9taaqkt9gkkxga05m5azcg27chnzfry\"}]\n</code></pre></p> </li> </ol> <p>You may want to explore what all endpoints come out of the box, and test them out, to do so - refer to API documentation for OpenAPI3 documentation. Each endpoint has a pre-filled example for mainnet and connects by default to primary Koios endpoint, allowing you to test endpoints and if needed - grab the <code>curl</code> commands to start testing yourself against your local or remote instances.</p>"},{"location":"Build/grest/#participating-in-koios-cluster-as-instance-provider","title":"Participating in Koios Cluster as instance Provider","text":"<p>If you're interested to participate in decentralised infrastructure by providing an instance, there are a few additional steps you'd need:</p> <ol> <li> <p>Enable ports for your HAProxy instance (default: 8053), gRest Exporter service (default: 8059) and (optionally) submit API instance (default: 8090) against the monitoring instance (do not need to open these ports to internet) of corresponding network.</p> </li> <li> <p>Ensure that each of the service above is listening on your public IP address (for instance, submitapi.sh might need to be edited to change HOSTADDR to <code>0.0.0.0</code> and restarted).</p> </li> <li> <p>Create a PR specifying connectivity information to your HAProxy port here.</p> </li> <li> <p>Make sure to join the telegram discussions group to participate in any discussions, actions, polls for new-features, etc. Feel free to give a shout in the group in case you have trouble following any of the above</p> </li> </ol>"},{"location":"Build/node-cli/","title":"Node & CLI","text":"<p>Reminder !!</p> <p>Unless there is a very particular reason you want to compile (eg: running on non-popular OS flavor), you DO NOT \"need\" to build node binaries - <code>guild-deploy.sh</code> already provides an option to download pre-compiled binaries. Ensure the Pre-Requisites are in place before you proceed.</p>"},{"location":"Build/node-cli/#build-instructions","title":"Build Instructions","text":""},{"location":"Build/node-cli/#clone-the-repository","title":"Clone the repository","text":"<p>Execute the below to clone the cardano-node repository to <code>$HOME/git</code> folder on your system:</p> <pre><code>cd ~/git\ngit clone https://github.com/intersectmbo/cardano-node\ncd cardano-node\n</code></pre>"},{"location":"Build/node-cli/#build-cardano-node","title":"Build Cardano Node","text":"<p>You can use the instructions below to build the latest release of cardano-node. </p> <pre><code>git fetch --tags --recurse-submodules --all\ngit pull\n# Replace tag against checkout if you do not want to build the latest released version, we recommend using battle tested node versions - which may not always be latest\ngit checkout $(curl -sLf https://api.github.com/repos/intersectmbo/cardano-node/releases/latest | jq -r .tag_name)\n\n# Use `-l` argument if you'd like to use system libsodium instead of IOG fork of libsodium while compiling\n$CNODE_HOME/scripts/cabal-build-all.sh\n</code></pre> <p>The above would copy the binaries built into <code>~/.local/bin</code> folder.</p>"},{"location":"Build/node-cli/#download-pre-compiled-binary-from-node-release","title":"Download pre-compiled Binary from Node release","text":"<p>While certain folks might want to build the node themselves (could be due to OS/arch compatibility, trust factor or customisations), for most it might not make sense to build the node locally. Instead, you can download the binaries using cardano-node release notes, where-in you can find the download links for every version. This is already taken care of by <code>guild-deploy.sh</code> if you used the option to download binaries (you can always re-run with specific arguments if unsure).</p>"},{"location":"Build/node-cli/#verify","title":"Verify","text":"<p>Execute <code>cardano-cli</code> and <code>cardano-node</code> to verify output as below (the exact version and git rev should depend on your checkout tag on github repository):</p> <pre><code>cardano-cli version\n# cardano-cli 9.x.x - linux-x86_64 - ghc-8.10\n# git rev &lt;...&gt;\ncardano-node version\n# cardano-node 9.x.x - linux-x86_64 - ghc-8.10\n# git rev &lt;...&gt;\n</code></pre>"},{"location":"Build/node-cli/#update-port-number-or-pool-name-for-relative-paths","title":"Update port number or pool name for relative paths","text":"<p>Before you go ahead with starting your node, you may want to update values for <code>CNODE_PORT</code> in <code>$CNODE_HOME/scripts/env</code>. Note that it is imperative for operational relays and pools to ensure that the port mentioned is opened via firewall to the destination your node is supposed to connect from. Update your network/firewall configuration accordingly. Future executions of <code>guild-deploy.sh</code> will preserve and not overwrite these values (or atleast back up if forced to overwrite).</p> <pre><code>CNODEBIN=\"${HOME}/.local/bin/cardano-node\"\nCCLI=\"${HOME}/.local/bin/cardano-cli\"\nCNODE_PORT=6000\nPOOL_NAME=\"GUILD\"\n</code></pre> <p>Important</p> <p>POOL_NAME is the name of folder that you will use when registering pools and starting node in core mode. This folder would typically contain your <code>hot.skey</code>,<code>vrf.skey</code> and <code>op.cert</code> files required. If the mentioned files are absent (expected if this is a fresh install), the node will automatically start in a relay mode.</p>"},{"location":"Build/node-cli/#start-the-node","title":"Start the node","text":"<p>To test starting the node in interactive mode, we will make use of pre-built script <code>cnode.sh</code>. This script automatically determines whether to start the node as a relay or block producer (if the required pool keys are present in the <code>$CNODE_HOME/priv/pool/&lt;POOL_NAME&gt;</code> as mentioned above). If the <code>&lt;MITHRIL_DOWNLOAD&gt;</code> variable is set to 'Y' it will download the latest snapshot from a Mithril aggregator to speed up the blockchain synchronization. The script contains a user-defined variable <code>CPU_CORES</code> which determines the number of CPU cores the node will use upon start-up:</p> <pre><code>######################################\n# User Variables - Change as desired #\n# Common variables set in env file   #\n######################################\n\n#CPU_CORES=4            # Number of CPU cores cardano-node process has access to (please don't set higher than physical core count, 4 recommended)\n</code></pre> <p>Now let's test starting the node in interactive mode.</p> <p>Note</p> <p>At this stage, upon executing <code>cnode.sh</code>, you are expected to see the live config and a line ending with <code>Listening on http://127.0.0.1:12798</code> - this is expected, as your logs are being written to <code>$CNODE_HOME/logs/node.json</code> . If so, you should be alright to return to your console by pressing Ctrl-C. The node will be started later using instructions below using systemd (Linux's service management). In case you receive any errors, please troubleshoot and fix those before proceeding.</p> <pre><code>cd \"${CNODE_HOME}\"/scripts\n./cnode.sh\n</code></pre> <p>Press Ctrl-C to exit node and return to console.</p>"},{"location":"Build/node-cli/#modify-the-nodes-config-files","title":"Modify the node's config files","text":"<p>Now that you've tested the basic node operation, you might want to customise your config files (assuming you are in top-level folder , i.e. <code>cd \"${CNODE_HOME}\"</code>) :</p> <ol> <li> <p>files/config.json : This file contains the logging configurations (tracers of to tune logging, paths for other genesis config files, address/ports on which the prometheus/EKG monitoring will listen, etc). Unless running more than one node on same machine (not recommended), you should be alright to use most of this file as-is. You might - however - want to double-check <code>PeerSharing</code> in this file, if using a relay node where you'd like connecting peers (marked as <code>\"advertise\": \"true\"</code> in topology.json) to be shared , you may turn this setting to <code>true</code>.</p> </li> <li> <p>files/topology.json : This file tells your node how to connect to other nodes (especially initially to start synching). You would want to update this file as below:</p> <ul> <li>Update the <code>localRoots</code> &gt; <code>accessPoints</code> section to include your local nodes that you want persistent connection against (eg: this could be your BP and own relay nodes) against definition where <code>trustable</code> is set to <code>true</code>.</li> <li>If you want specific peers to be advertised on the network for discovery, you may set <code>advertise</code> to <code>true</code> for that peer group. You do NOT want to do that on BP</li> <li>You'd want to update <code>localRoots</code> &gt; <code>valency</code> (<code>valency</code> is the same as <code>hotValency</code>, not yet replaced since the example in cardano-node-wiki repo still suggests <code>valency</code>) to number of connections from your localRoots that you always want to keep active connection to for that node.</li> <li>[Optional] - you can add/remove nodes from <code>publicRoots</code> section as well as <code>localRoots</code> &gt; <code>accessPoints</code> as desired, tho defaults populated should work fine. On mainnet, we did add a few additional nodes to help add more redundancy for initial sync.</li> <li><code>useLedgerAfterSlot</code> tells the node to establish networking with nodes from defined peers to sync the node initially until reaching an absolute slot number, after which - it can start attempting to connect to peers registered as pool relays on the network. You may want this number to be relatively recent (eg: not have it 50 epochs old).</li> <li>You can read further about topology file configuration here</li> </ul> </li> </ol> <p>Important</p> <p>On BP, You'd want to set <code>useLedgerAfterSlot</code> to <code>-1</code> for your Block Producing (Core) node - thereby, telling your Core node to remain in non-P2P mode, and ensure <code>PeerSharing</code> is to <code>false</code>.</p> <p>The resultant topology file on a relay could look something like below:</p> <pre><code>{\n\"bootstrapPeers\": [\n{\n\"address\": \"backbone.cardano.iog.io\",\n\"port\": 3001\n},\n{\n\"address\": \"backbone.mainnet.emurgornd.com\",\n\"port\": 3001\n},\n{\n\"address\": \"backbone.mainnet.cardanofoundation.org\",\n\"port\": 3001\n}\n],\n\"localRoots\": [\n{\n\"accessPoints\": [\n{\"address\": \"xx.xx.xx.xx\", \"port\": 6000 , \"description\": \"Core\"},\n{\"address\": \"zz.zz.zz.zz\", \"port\": 6000 , \"description\": \"Relay2\"}\n],\n\"advertise\": false,\n\"trustable\": true,\n\"hotValency\": 2\n},\n{\n\"accessPoints\": [\n{\"address\": \"node-dus.poolunder.com\",           \"port\": 6900, \"pool\": \"UNDR\",   \"location\": \"EU/DE/Dusseldorf\" },\n{\"address\": \"node-syd.poolunder.com\",           \"port\": 6900, \"pool\": \"UNDR\",   \"location\": \"OC/AU/Sydney\" },\n{\"address\": \"194.36.145.157\",                   \"port\": 6000, \"pool\": \"RDLRT\",  \"location\": \"EU/DE/Baden\" },\n{\"address\": \"95.216.38.251\",                    \"port\": 6000, \"pool\": \"RDLRT\",  \"location\": \"EU/FI/Helsinki\" },\n{\"address\": \"148.72.153.168\",                   \"port\": 16000, \"pool\": \"AAA\",   \"location\": \"NA/US/StLouis\" },\n{\"address\": \"78.47.99.41\",                      \"port\": 6000, \"pool\": \"AAA\",    \"location\": \"EU/DE/Nuremberg\" },\n{\"address\": \"relay1-pub.ahlnet.nu\",             \"port\": 2111, \"pool\": \"AHL\",    \"location\": \"EU/SE/Malmo\" },\n{\"address\": \"relay2-pub.ahlnet.nu\",             \"port\": 2111, \"pool\": \"AHL\",    \"location\": \"EU/SE/Malmo\" },\n{\"address\": \"relay1.clio.one\",                  \"port\": 6010, \"pool\": \"CLIO\",   \"location\": \"EU/IT/Milan\" },\n{\"address\": \"relay2.clio.one\",                  \"port\": 6010, \"pool\": \"CLIO\",   \"location\": \"EU/IT/Bozlano\" },\n{\"address\": \"relay3.clio.one\",                  \"port\": 6010, \"pool\": \"CLIO\",   \"location\": \"EU/IT/Bozlano\" }\n],\n\"advertise\": false,\n\"trustable\": false,\n\"hotValency\": 5,\n\"warmValency\": 10\n}\n],\n\"publicRoots\": [\n{\n\"accessPoints\": [],\n\"advertise\": false\n}\n],\n\"useLedgerAfterSlot\": 128908821\n}\n</code></pre> <p>Similarly, a typical topology file on a Core could look something like below:</p> <pre><code>{\n\"bootstrapPeers\": [],\n\"localRoots\": [\n{\n\"accessPoints\": [\n{\"address\": \"yy.yy.yy.yy\", \"port\": 6000, \"description\": \"Relay1\"},\n{\"address\": \"zz.zz.zz.zz\", \"port\": 6000, \"description\": \"Relay2\"}\n],\n\"advertise\": false,\n\"trustable\": true,\n\"hotValency\": 2\n}\n],\n\"publicRoots\": [\n{\n\"accessPoints\": [],\n\"advertise\": false\n}\n],\n\"useLedgerAfterSlot\": -1\n}\n</code></pre> <p>Once above two files are updated, since you modified the file manually - there is always a chance of human errors (eg: missing comma/quotes). Thus, we would recommend you to start the node interactively once again before proceeding.</p> <pre><code>cd \"${CNODE_HOME}\"/scripts\n./cnode.sh\n</code></pre> <p>As before, ensure you do not have any errors in the console. To stop the node, hit Ctrl-C - we will start the node as systemd later in the document.</p>"},{"location":"Build/node-cli/#start-the-submit-api","title":"Start the submit-api","text":"<p>Note</p> <p>An average pool operator may not require <code>cardano-submit-api</code> at all. Please verify if it is required for your use as mentioned here. If - however - you do run submit-api for accepting sizeable transaction load, you would want to override the default MEMPOOL_BYTES by uncommenting it in cnode.sh.</p> <p><code>cardano-submit-api</code> is one of the binaries built as part of <code>cardano-node</code> repository and allows you to submit transactions over a Web API. To run this service interactively, you can use the pre-built script below (<code>submitapi.sh</code>). Make sure to update <code>submitapi.sh</code> script to change listen IP or Port that you'd want to make this service available on.</p> <pre><code>cd $CNODE_HOME/scripts\n./submitapi.sh\n</code></pre> <p>To stop the process, hit Ctrl-C</p>"},{"location":"Build/node-cli/#systemd","title":"Run as systemd service","text":"<p>The preferred way to run the node (and submit-api) is through a service manager like systemd. This section explains how to setup a systemd service file.</p> <p>1. Deploy as a systemd service Execute the below command to deploy your node as a systemd service (from the respective scripts folder): <pre><code>cd $CNODE_HOME/scripts\n./cnode.sh -d\n# Deploying cnode.service as systemd service..\n# cnode.service deployed successfully!!\n\n./submitapi.sh -d\n# Deploying cnode-submit-api.service as systemd service..\n# cnode-submit-api deployed successfully!!\n</code></pre></p> <p>2. Start the service Run below commands to enable automatic start of service on startup and start it. <pre><code>sudo systemctl start cnode.service\nsudo systemctl start cnode-submit-api.service\n</code></pre></p> <p>3. Check status and stop/start commands  Replace <code>status</code> with <code>stop</code>/<code>start</code>/<code>restart</code> depending on what action to take. <pre><code>sudo systemctl status cnode.service\nsudo systemctl status cnode-submit-api.service\n</code></pre></p> <p>Important</p> <p>In case you see the node exit unsuccessfully upon checking status, please verify you've followed the transition process correctly as documented below, and that you do not have another instance of node already running. It would help to check your system logs, you can also check <code>journalctl -f -u cnode</code> to examine startup attempt for services, and scroll up until you see output for node startup attempt) for any errors while starting node.</p> <p>You can use gLiveView to monitor your node that was started as a systemd service.</p> <pre><code>cd $CNODE_HOME/scripts\n./gLiveView.sh\n</code></pre>"},{"location":"Build/offchain-metadata-tools/","title":"Offchain Metadata Tools","text":"<p>Important</p> <ul> <li>An average pool operator may not require offline-metadata-tools at all. Please verify if it is required for your use as mentioned here</li> <li>Ensure the Pre-Requisites are in place before you proceed.</li> </ul> <p>In the Cardano multi-asset era, this project helps you create and submit metadata describing your assets, storing them off-chain.</p>"},{"location":"Build/offchain-metadata-tools/#download-pre-built-binaries","title":"Download pre-built binaries","text":"<p>Go to input-output-hk/offchain-metadata-tools to download the binaries and place in a directory specified by <code>PATH</code>, e.g. <code>$HOME/.local/bin/</code>. </p>"},{"location":"Build/offchain-metadata-tools/#build-instructions","title":"Build Instructions","text":"<p>An alternative to pre-built binaries - instructions describe how to build the <code>token-metadata-creator</code> tool but the offchain-metadata-tools repository contains other tools as well. Build the ones needed for your installation.</p>"},{"location":"Build/offchain-metadata-tools/#clone-the-repository","title":"Clone the repository","text":"<p>Execute the below to clone the offchain-metadata-tools repository to $HOME/git folder on your system:</p> <pre><code>cd ~/git\ngit clone https://github.com/input-output-hk/offchain-metadata-tools.git\ncd offchain-metadata-tools/token-metadata-creator\n</code></pre>"},{"location":"Build/offchain-metadata-tools/#build-token-metadata-creator","title":"Build token-metadata-creator","text":"<p>You can use the instructions below to build <code>token-metadata-creator</code>, same steps can be executed in future to update the binaries (replacing appropriate tag) as well.</p> <p><pre><code>git fetch --tags --all\ngit pull\n# Replace master with appropriate tag if you'd like to avoid compiling against master\ngit checkout master\n$CNODE_HOME/scripts/cabal-build-all.sh\n</code></pre> The above would copy the binaries into <code>~/.local/bin</code> folder.</p>"},{"location":"Build/offchain-metadata-tools/#verify","title":"Verify","text":"<p>Verify that the tool is executable from anywhere by running:</p> <pre><code>token-metadata-creator -h\n</code></pre>"},{"location":"Build/wallet/","title":"Wallet","text":"<p>!&gt; - An average pool operator may not require <code>cardano-wallet</code> at all. Please verify if it is required for your use as mentioned here.</p> <p>Ensure the Pre-Requisites are in place before you proceed.</p>"},{"location":"Build/wallet/#build-instructions","title":"Build Instructions","text":"<p>Follow instructions below for building the cardano-wallet binary:</p>"},{"location":"Build/wallet/#clone-the-repository","title":"Clone the repository","text":"<p>Execute the below to clone the <code>cardano-wallet</code> repository to <code>$HOME/git</code> folder on your system:</p> <pre><code>cd ~/git\ngit clone https://github.com/cardano-foundation/cardano-wallet\ncd cardano-wallet\n</code></pre>"},{"location":"Build/wallet/#build-cardano-wallet","title":"Build Cardano Wallet","text":"<p>You can use the instructions below to build the latest release of cardano-wallet.</p> <p>!&gt; - Note that the latest release of <code>cardano-wallet</code> may not work with the latest release of <code>cardano-node</code>. Please check the compatibility of each <code>cardano-wallet</code> release yourself in the official docs, e.g. https://github.com/cardano-foundation/cardano-wallet/releases/latest.</p> <pre><code>git fetch --tags --all\ngit pull\n# Replace tag against checkout if you do not want to build the latest released version\ngit checkout $(curl -s https://api.github.com/repos/cardano-foundation/cardano-wallet/releases/latest | jq -r .tag_name)\n$CNODE_HOME/scripts/cabal-build-all.sh\n</code></pre> <p>The above would copy the binaries into <code>~/.local/bin</code> folder.</p>"},{"location":"Build/wallet/#start-the-wallet","title":"Start the wallet","text":"<p>You can run the below to connect to a <code>cardano-node</code> instance that is expected to be already running and the wallet will start syncing. <pre><code>cardano-wallet serve /\n    --node-socket $CNODE_HOME/sockets/node.socket /\n    --mainnet / # if using the testnet flag you also need to specify the testnet shelley-genesis.json file\n--database $CNODE_HOME/priv/wallet\n</code></pre></p>"},{"location":"Build/wallet/#verify-the-wallet-is-handling-requests","title":"Verify the wallet is handling requests","text":"<p><pre><code>cardano-wallet network information\n</code></pre> Expected output should be similar to the following <pre><code>Ok.\n{\n\"network_tip\": {\n\"time\": \"2021-06-01T17:31:05Z\",\n\"epoch_number\": 269,\n\"absolute_slot_number\": 31002374,\n\"slot_number\": 157574\n},\n\"node_era\": \"mary\",\n\"node_tip\": {\n\"height\": {\n\"quantity\": 5795127,\n\"unit\": \"block\"\n},\n\"time\": \"2021-06-01T17:31:00Z\",\n\"epoch_number\": 269,\n\"absolute_slot_number\": 31002369,\n\"slot_number\": 157569\n},\n\"sync_progress\": {\n\"status\": \"ready\"\n},\n\"next_epoch\": {\n\"epoch_start_time\": \"2021-06-04T21:44:51Z\",\n\"epoch_number\": 270\n}\n}\n</code></pre></p>"},{"location":"Build/wallet/#creatingrestoring-wallet","title":"Creating/Restoring Wallet","text":"<p>If you're creating a new wallet, you'd first want to generate a mnemonic for use (see below):</p> <p><pre><code>cardano-wallet recovery-phrase generate\n# false brother typical saddle settle phrase foster sauce ask sunset firm gate service render burger\n</code></pre> You can use the above mnemonic to then restore a wallet as per below: <pre><code>cardano-wallet wallet create from-recovery-phrase MyWalletName\n</code></pre></p>"},{"location":"Build/wallet/#expected-output","title":"Expected output:","text":"<pre><code>Please enter a 15\u201324 word recovery phrase: false brother typical saddle settle phrase foster sauce ask sunset firm gate service render burger\n(Enter a blank line if you do not wish to use a second factor.)\nPlease enter a 9\u201312 word second factor:\nPlease enter a passphrase: **********\nEnter the passphrase a second time: **********\nOk.\n{\n    ...\n}\n</code></pre>"},{"location":"Mithril/mithril-overview/","title":"Mithril Overview","text":"<p>Mithril Networks provide the ability to download and bootstrap cardano nodes via snapshots of the the Cardano blockchain. This is a great way to speed up the process of syncing a new node, especially for stake pool operators. The tools provided by Guild Operators are designed to facilitate the ease of use in setting up and managing the:</p> <ul> <li>Mithril Client to download a snapshot for the given network the node is attached to via the mithril-client.sh script.</li> <li>Mithril Signer to participate in the creation of stake based singatures of snapshots via the mithril-signer.sh script.</li> <li>Squid Mithril Relay to provide a relay for submitting the snapshots signatures to a Mithril Aggregator, as described in Run a Mithril signer as an SPO documentation via the mithril-relay.sh script.</li> </ul> <p>The <code>env</code> file contains a new environment variable <code>MITHRIL_DOWNLOAD</code> that when enabled allows the <code>cnode.sh</code> script to automatically download the latest Mithril snapshot if the local <code>db</code> directory is empty. This is useful for new nodes that need to be  bootstrapped with the latest snapshot to avoid synchronizing the entire blockchain from scratch. While also providing a high level of trust that the snapshot is valid since it is signed by multiple pool operators.</p>"},{"location":"Mithril/mithril-overview/#architecture","title":"Architecture","text":"<p>The architecture for Mithril Networks is described in detail at Mithril network architecture by CF/IOHK. However the architecture suggested and supported by the Guild Operators tools is not identical to the upstream documentation in that we provide a more simplified approach to the setup and management of the Mithril Network components and tools that allow setting up a Squid Mithril relays and an Nginx loadbalancer (aka sidecar) local to the Mithril signer. The Nginx sidecar provides the ability to loadbalance requests to multiple Squid based Mithril Relays running on each of the SPO's Cardano Relay nodes.</p>"},{"location":"Mithril/mithril-overview/#single-relay-architecture","title":"Single Relay Architecture","text":"<p>For SPO's who only have a single Cardano relay node, an Squid based Mithril relay can be run on the same node as the Cardano relay. This can be used by the Mithril signer to submit the snapshot signatures to the Mithril Aggregator.</p> <p></p>"},{"location":"Mithril/mithril-overview/#multi-relay-architecture","title":"Multi Relay Architecture","text":"<p>For SPO's who have multiple Cardano relay nodes, a Nginx relay sidecar can be run on the Block Producer and load balance requests over mutliple Cardano relay nodes, each running its own Nginx Mithril relay to pass the signature along to the Mithril aggregator. This can be used to avoid a single point of failure in case a Relay server is offline for any reason. This provides high availability for the Mithril signer through multiple relays as long as the local Nginx Mithril relay is running on the same server as the Cardano Block Producer node.</p> <p></p>"},{"location":"Mithril/mithril-overview/#installation","title":"Installation","text":"<p>The installation of the Mithril tools is automated via <code>guild-deploy.sh</code>. To participate in a Mithril network include the <code>-s m</code> flag which will install the Mithril Client and Mithril Signer release binaries to <code>\"${HOME}\"/.local/bin</code>.</p> <pre><code>guild-deploy.sh -s m\n</code></pre>"},{"location":"Mithril/mithril-overview/#bootstrapping-a-node-using-mithril-client","title":"Bootstrapping a node using Mithril Client","text":"<p>The Mithril client is used to download a snapshot of the Cardano blockchain from a Mithril Aggregator. The snapshot is then used to bootstrap a new Cardano node. The Mithril client can be used to download the latest snapshot, list all available snapshots, or show details of a specific snapshot.</p> <p>To bootstrap a Cardano node using the Mithril client, follow these steps:</p> <ol> <li> <p>Setup the Cardano Node: Use the guild tools to setup the Cardano node, either by building the binaries or using pre-compiled binaries. Follow the instructions in the guild-operators documentation.</p> </li> <li> <p>Create the Mithril environment file: Run the script with the <code>environment setup</code> command. This will create a new <code>mithril.env</code> file with all the necessary environment variables for the Mithril client.</p> </li> </ol> <pre><code>./mithril-client.sh environment setup\n</code></pre> <ol> <li>Download the latest Mithril snapshot: Once the environment file is set up, you can download the latest Mithril snapshot by running the script with the <code>snapshot download</code> command. This snapshot contains the latest state of the Cardano blockchain db from a Mithril Aggregator.</li> </ol> <pre><code>./mithril-client.sh snapshot download\n</code></pre>"},{"location":"Mithril/mithril-overview/#participating-in-mithril-network","title":"Participating in Mithril Network","text":"<p>The Mithril signer is used to participate in the creation of stake based signatures of snapshots. The Mithril signer can be used to sign a snapshots. The signed snapshot is then submitted to a Mithril Aggregator, via a Squid based Mithril Relay.</p> <p>The first step to participate in the Mithril network is to deploy your Squid based Mithril Relays. The Mithril relay is used to provide a private and highly available network for submitting the snapshots to a Mithril Aggregator.</p>"},{"location":"Mithril/mithril-overview/#deploying-the-squid-mithril-relay","title":"Deploying the Squid Mithril Relay","text":"<p>To deploy your Squid based Mithril Relays with your Cardano relay node, follow these steps:</p> <ol> <li> <p>Deploy the Squid Mithril Relay: Run the <code>mithril-relay.sh</code> script:</p> </li> <li> <p>Use the <code>-d</code> flag to deploy the Squid Mithril Relay.</p> </li> <li>Provide the IP address of your Block Producer when prompted to secure       the Mithril Relay to only accept traffic from your Block Producer.</li> <li>Optionally provide the relays listening port when prompted to use a port       other than the default 3132, or just press enter to use the default.</li> <li>Create the appropriate firewall rule to allow traffic from your Block       Producer to the Mithril Relay.</li> </ol> <pre><code>./mithril-relay.sh -d\n\nInstalling squid proxy\nEnter the IP address of your Block Producer: 1.2.3.4\nEnter the relays listening port (press Enter to use default 3132):\nUsing port 3132 for relays listening port.\nCreate the appropriate firewall rule: sudo ufw allow from 1.2.3.4 to any port 3132 proto tcp\n</code></pre> <ol> <li>Enable the Systemd Squid Mithril Relay service to start on boot.</li> </ol> <pre><code>   sudo systemctl enable --now squid\n</code></pre> <ol> <li>Repeat the process for each of your Cardano relay nodes.</li> </ol>"},{"location":"Mithril/mithril-overview/#deploying-the-mithril-signer","title":"Deploying the Mithril Signer","text":""},{"location":"Mithril/mithril-overview/#mithril-signer-with-single-relay","title":"Mithril Signer with Single Relay","text":"<ol> <li> <p>Deploy the Mithril Signer: Run the <code>mithril-signer.sh</code> script:</p> </li> <li> <p>Use the <code>-e</code> flag to update the <code>mithril.env</code> file with the Mithril       Signer environment variables.</p> </li> <li>Provide the IP address of your Mithril Relay when prompted.</li> <li> <p>Optionally provide the relays listening port when prompted to use a port.</p> <pre><code>  ./mithril-signer.sh -e\n  Enter the IP address of the relay endpoint: 4.5.6.7\n  Enter the port of the relay endpoint (press Enter to use default 3132):\n  Using RELAY_ENDPOINT=4.5.6.7:3132 for the Mithril signer relay endpoint.\n</code></pre> </li> <li> <p>Use the <code>-d</code> flag to deploy the Mithril Signer.</p> <pre><code>  ./mithril-signer.sh -d\n  Creating cnode-mithril-signer systemd service environment file..\n  Mithril signer service successfully deployed\n</code></pre> </li> <li> <p>Enable the Systemd service to start the Mithril Signer on boot.</p> <pre><code>  sudo systemctl enable cnode-mithril-signer\n</code></pre> </li> </ol>"},{"location":"Mithril/mithril-overview/#mithril-signer-with-multi-relay","title":"Mithril Signer with Multi Relay","text":"<ol> <li> <p>Deploy the Nginx sidecar loadbalancer: Run the <code>mithril-relay.sh</code> script:</p> </li> <li> <p>Use the <code>-l</code> flag to deploy the Nginx Mithril Relay.</p> </li> <li>Provide the IP address of your Mithril Relay(s) when prompted.</li> <li>Provide an IP address for your nginx loadbalancer (default 127.0.0.1)</li> <li>Optionally provide the relays listening port when prompted to use a port       other than the default 3132, or just press enter to use the default.</li> <li> <p>Create the appropriate firewall rule to allow traffic from your Block       Producer to the Mithril Relay(s).</p> <pre><code>./mithril-relay.sh -l\n\nInstalling nginx load balancer\nEnter the IP address of a relay: 4.5.6.7\nAre there more relays? (y/n) y\nEnter the IP address of a relay: 8.9.10.11\nAre there more relays? (y/n) n\nEnter the IP address of the load balancer (press Enter to use default 127.0.0.1):\nUsing IP address 127.0.0.1 for the load balancer configuration.\nEnter the relays listening port (press Enter to use default 3132):\nUsing port 3132 for relays listening port.\nStarting Mithril relay sidecar (nginx load balancer)\n</code></pre> </li> <li> <p>Enable the Systemd Nginx Mithril Relay service to start on boot.</p> <pre><code>sudo systemctl enable --now nginx\n</code></pre> </li> <li> <p>Deploy the Mithril Signer: Run the <code>mithril-signer.sh</code> script:</p> </li> <li> <p>Use the <code>-e</code> flag to update the <code>mithril.env</code> file with the Mithril       Signer environment variables.</p> </li> <li>Provide the IP address of your nginx loadbalancer when prompted.</li> <li> <p>Optionally provide the loadbalancer listening port when prompted to use a port.</p> <pre><code>    ./mithril-signer.sh -e\n    Enter the IP address of the relay endpoint: 127.0.0.1\n    Enter the port of the relay endpoint (press Enter to use default 3132):\n    Using RELAY_ENDPOINT=127.0.0.1:3132 for the Mithril signer relay endpoint.\n</code></pre> </li> <li> <p>Use the <code>-d</code> flag to deploy the Mithril Signer.</p> <pre><code>    ./mithril-signer.sh -d\n    Creating cnode-mithril-signer systemd service environment file..\n    Mithril signer service successfully deployed\n</code></pre> </li> <li> <p>Enable the Systemd service to start the Mithril Signer on boot.</p> <pre><code>    sudo systemctl enable cnode-mithril-signer\n</code></pre> </li> </ol>"},{"location":"Scripts/blockperf/","title":"BlockPerf","text":"<p>Reminder !!</p> <p>Ensure the Pre-Requisites are in place before you proceed.</p> <p><code>blockPerf.sh</code> is a script to monitor the network propagation of new blocks as seen by the local cardano-node.  </p>"},{"location":"Scripts/blockperf/#block-propagation-traces","title":"Block propagation traces","text":"<p>Although blockPerf can also run on the block producer, it makes the most sense to run it on the upstream relays. There it waits for each new block announced to the relay over the network by its remote peers. </p> <p>It looks for the delay times that result</p> <ul> <li>from the theoretical slot time of the block generator</li> <li>until the block header was offered to the local node</li> <li>the node requested the block </li> <li>the node downloaded the block</li> <li>the node has verified and adopted the block</li> </ul> <p>You can view this data locally as a console stream, or run it as a systemd service in background. </p> <p>BlockPerf also sends this data to the TopologyUpdater server, so that there is a possibility to compare this data (similar to sendtip to pooltool). As a contributing operator you get the possibility to see how your own relays compare to other nodes regarding receive quality, delay times and thus performance. </p> <p>There is no connection or constraint between the TopologyUpdater Relay subscription and the BlockPerf analysis. BlockPerf is even designed to work outside the cnTools suite. </p> <p>The results of these data are a good basis to make optimizations and to evaluate which changes were useful or might by required to improve the performance compared to other relay nodes.</p>"},{"location":"Scripts/blockperf/#installation","title":"Installation","text":"<p>The script is best run as a background process. This can be accomplished in many ways but the preferred method is to run it as a systemd service. A terminal multiplexer like tmux or screen could also be used but not covered here.</p>"},{"location":"Scripts/blockperf/#run-as-service","title":"Run as service","text":"<p>Use the <code>deploy-as-systemd.sh</code> script to create a systemd unit file. In this setup the script is started in \"service\" mode. Error/Warn level log output is handled by journald. <code>journalctl -f -u cnode-tu-blockperf.service</code> can be used to check service output (follow mode). </p> <p>Outside the cnTools environment call <code>blockPerf.sh -d</code> to install it as a systemd service. </p>"},{"location":"Scripts/blockperf/#console-view","title":"Console view","text":"<p>If you run blockPerf local in the console (<code>scripts/blockPerf.sh</code>) , immediately after the appearance of a new block it shows where it came from, how many slots away from the previous block it was, and how many milliseconds the individual steps took.</p> <pre><code>Block:.... 6860534\n Slot..... 52833850 (+59s)\n ......... 2022-02-09 09:49:01\n Header... 2022-02-09 09:49:02,780 (+1780 ms)\n Request.. 2022-02-09 09:49:02,780 (+0 ms)\n Block.... 2022-02-09 09:49:02,830 (+50 ms)\n Adopted.. 2022-02-09 09:49:02,900 (+70 ms)\n Size..... 79976 bytes\n delay.... 1.819971868 sec\n From..... 104.xxx.xxx.61:3001\n\nBlock:.... 6860535\n Slot..... 52833857 (+7s)\n ......... 2022-02-09 09:49:08\n Header... 2022-02-09 09:49:08,960 (+960 ms)\n Request.. 2022-02-09 09:49:08,970 (+10 ms)\n Block.... 2022-02-09 09:49:09,020 (+50 ms)\n Adopted.. 2022-02-09 09:49:09,090 (+70 ms)\n Size..... 64950 bytes\n delay.... 1.028341023 sec\n From..... 34.xxx.xxx.15:4001\n</code></pre>"},{"location":"Scripts/blockperf/#collaborative-web-view","title":"Collaborative web view","text":"<p>A further aim of the blockPerf project is to use the data that individual nodes send to the central TopologyUpdater database to produce graphical visualisations and evaluations that provide the participating node operators with useful insights into their performance compared to all others.</p> <p></p>"},{"location":"Scripts/cncli/","title":"CNCLI","text":"<p>Reminder !!</p> <p>Ensure the Pre-Requisites are in place before you proceed.</p> <p><code>cncli.sh</code> is a script to download and deploy CNCLI created and maintained by Andrew Westberg. It's a community-based CLI tool written in RUST for low-level <code>cardano-node</code> communication. Usage is optional and no script is dependent on it. The main features include:</p> <ul> <li>PING - Validates that the remote server is on the given network and returns its response time. Utilized by <code>gLiveView</code> for peer analysis if available. </li> <li>SYNC - Connects to a node (local or remote) and synchronizes blocks to a local <code>sqlite</code> database. </li> <li>VALIDATE - Validates that a block hash or partial block hash is on-chain.</li> <li>LEADERLOG - Calculates a stakepool's expected slot list. On the mainnet and the official testnet, the next epoch's leader schedule is available 1.5 days before the end of the epoch (<code>firstSlotOfNextEpoch - (3 * k / f)</code>).</li> <li>SENDTIP - Send node tip to PoolTool for network analysis and to show that your node is alive and well with a green badge.</li> <li>SENDSLOTS - Securely sends PoolTool the number of slots you have assigned for an epoch and validates the correctness of your past epochs.</li> </ul>"},{"location":"Scripts/cncli/#installation","title":"Installation","text":"<p><code>cncli.sh</code> script's main functions, <code>sync</code>, <code>leaderlog</code>, <code>validate</code> and PoolTool <code>sendslots</code>/<code>sendtip</code> are not meant to be run manually, but instead deployed as systemd services that run in the background to do the block scraping and validation automatically. Additional commands exist for manual execution to initiate the <code>sqlite</code> db, filling the blocklog DB with all blocks created by the pool known to the blockchain, migration of old cntoolsBlockCollector JSON blocklog, and re-validation of blocks and leaderlogs. See usage output below for a complete list of available commands.</p> <p>The script works in tandem with Log Monitor to provide faster adopted status but mainly to catch slots the node is leader for but are unable to create a block for. These are marked as invalid. Blocklog will however work fine without the <code>logMonitor</code> service and <code>CNCLI</code> is able to handle everything except catching invalid blocks.</p> <ol> <li>Run the latest version of <code>guild-deploy.sh</code> with <code>guild-deploy.sh -s c</code> to download and install RUST and CNCLI. IOG fork of libsodium required by CNCLI is automatically compiled by CNCLI build process. If a previous installation is found, RUST and CNCLI will be updated to the latest version.</li> <li>Run <code>deploy-as-systemd.sh</code> to deploy the systemd services that handle all the work in the background. Six systemd services in total are deployed whereof four are related to CNCLI. See above for the different purposes they serve.</li> <li> <p>If you want to disable some of the deployed services, run <code>sudo systemctl disable &lt;service&gt;</code></p> </li> <li> <p><code>cnode.service</code> (main <code>cardano-node</code> launcher)</p> </li> <li><code>cnode-cncli-sync.service</code></li> <li><code>cnode-cncli-leaderlog.service</code></li> <li><code>cnode-cncli-validate.service</code></li> <li><code>cnode-cncli-ptsendtip.service</code></li> <li><code>cnode-cncli-ptsendslots.service</code></li> <li><code>cnode-logmonitor.service</code> (see Log Monitor)</li> </ol>"},{"location":"Scripts/cncli/#configuration","title":"Configuration","text":"<p>You can override the values in the script at the User Variables section shown below. POOL_ID, POOL_VRF_SKEY and POOL_VRF_VKEY should automatically be detected if <code>POOL_NAME</code> is set in the common <code>env</code> file and can be left commented. PT_API_KEY and POOL_TICKER need to be set in the script if PoolTool <code>sendtip</code>/<code>sendslots</code> are to be used before starting the services. For the rest of the commented values, if the defaults do not provide the right values, uncomment and make adjustments.</p> <pre><code>#POOL_ID=\"\"                               # Automatically detected if POOL_NAME is set in env. Required for leaderlog calculation &amp; pooltool sendtip, lower-case hex pool id\n#POOL_VRF_SKEY=\"\"                         # Automatically detected if POOL_NAME is set in env. Required for leaderlog calculation, path to pool's vrf.skey file\n#POOL_VRF_VKEY=\"\"                         # Automatically detected if POOL_NAME is set in env. Required for block validation, path to pool's vrf.vkey file\n#PT_API_KEY=\"\"                            # POOLTOOL sendtip: set API key, e.g \"a47811d3-0008-4ecd-9f3e-9c22bdb7c82d\"\n#POOL_TICKER=\"\"                           # POOLTOOL sendtip: set the pools ticker, e.g. \"TCKR\"\n#PT_HOST=\"127.0.0.1\"                      # POOLTOOL sendtip: connect to a remote node, preferably block producer (default localhost)\n#PT_PORT=\"${CNODE_PORT}\"                  # POOLTOOL sendtip: port of node to connect to (default is CNODE_PORT from the env file)\n#CNCLI_DIR=\"${CNODE_HOME}/guild-db/cncli\" # path to the directory for cncli sqlite db\n#SLEEP_RATE=60                            # CNCLI leaderlog/validate: time to wait until next check (in seconds)\n#CONFIRM_SLOT_CNT=600                     # CNCLI validate: require at least these many slots to have passed before validating\n#CONFIRM_BLOCK_CNT=15                     # CNCLI validate: require at least these many blocks on top of minted before validating\n#TIMEOUT_LEDGER_STATE=300                 # CNCLI leaderlog: timeout in seconds for ledger-state query\n#BATCH_AUTO_UPDATE=N                      # Set to Y to automatically update the script if a new version is available without user interaction\n</code></pre>"},{"location":"Scripts/cncli/#run","title":"Run","text":"<p>Services are controlled by <code>sudo systemctl &lt;status|start|stop|restart&gt; &lt;service name&gt;</code> All services are configured as child services to <code>cnode.service</code> and as such, when an action is taken against this service it's replicated to all child services. E.g running <code>sudo systemctl start cnode.service</code> will also start all child services. </p> <p>Log output is handled by journald. <code>journalctl -f -u &lt;service&gt;</code> can be used to check service output (follow mode). Other logging configurations are not covered here. </p> <p>Recommended workflow to get started with CNCLI blocklog.</p> <ol> <li>Install and deploy services according to Installation section.</li> <li>Set required user variables according to Configuration section.</li> <li>(optional) If a previous blocklog db exist created by cntoolsBlockCollector, run this command to migrate json storage to new SQLite DB:</li> <li><code>$CNODE_HOME/scripts/cncli.sh migrate &lt;path&gt;</code> where  is the location to the directory containing all blocks_.json files. <li>Start deployed services with:</li> <li><code>sudo systemctl start cnode-cncli-sync.service</code> (starts <code>leaderlog</code>, <code>validate</code> &amp; <code>ptsendslots</code> automatically)</li> <li><code>sudo systemctl start cnode-logmonitor.service</code></li> <li><code>sudo systemctl start cnode-cncli-ptsendtip.service</code> (optional but recommended)</li> <li>alternatively restart the main service that will trigger a start of all services with:</li> <li><code>sudo systemctl restart cnode.service</code></li> <li>Run init command to fill the db with all blocks made by your pool known to the blockchain</li> <li><code>$CNODE_HOME/scripts/cncli.sh init</code></li> <li>Enjoy full blocklog automation and visit View Blocklog section for instructions on how to show blocks from the blocklog DB.</li> <pre><code>Usage: cncli.sh [operation &lt;sub arg&gt;]\nScript to run CNCLI, best launched through systemd deployed by 'deploy-as-systemd.sh'\n\nsync        Start CNCLI chainsync process that connects to cardano-node to sync blocks stored in SQLite DB (deployed as service)\nleaderlog   One-time leader schedule calculation for current epoch, then continuously monitors and calculates schedule for coming epochs, 1.5 days before epoch boundary on the mainnet (deployed as service)\n  force     Manually force leaderlog calculation and overwrite even if already done, exits after leaderlog is calculated\nvalidate    Continuously monitor and confirm that the blocks made actually was accepted and adopted by chain (deployed as service)\n  all       One-time re-validation of all blocks in blocklog db\n  epoch     One-time re-validation of blocks in blocklog db for the specified epoch \nptsendtip   Send node tip to PoolTool for network analysis and to show that your node is alive and well with a green badge (deployed as service)\nptsendslots Securely sends PoolTool the number of slots you have assigned for an epoch and validates the correctness of your past epochs (deployed as service)\ninit        One-time initialization adding all minted and confirmed blocks to blocklog\nmigrate     One-time migration from old blocklog (cntoolsBlockCollector) to new format (post cncli)\n  path      Path to the old cntoolsBlockCollector blocklog folder holding json files with blocks created\n</code></pre>"},{"location":"Scripts/cncli/#view-blocklog","title":"View Blocklog","text":"<p>Best and easiest viewed in CNTools and <code>gLiveView</code> but the blocklog database is a SQLite DB so if you are comfortable with SQL, the <code>sqlite3</code> command can be used to query the DB. </p> <p>Block status</p> <pre><code>- Leader    : Scheduled to make block at this slot\n- Ideal     : Expected/Ideal number of blocks assigned based on active stake (sigma)\n- Luck      : Leader slots assigned vs ideal slots for this epoch\n- Adopted   : Block created successfully\n- Confirmed : Block created validated to be on-chain with the certainty set in `cncli.sh` for `CONFIRM_BLOCK_CNT`\n- Missed    : Scheduled at slot but no record of it in CNCLI DB and no other pool has made a block for this slot\n- Ghosted   : Block created but marked as orphaned and no other pool has made a valid block for this slot -&gt; height battle or block propagation issue\n- Stolen    : Another pool has a valid block registered on-chain for the same slot\n- Invalid   : Pool failed to create block, base64 encoded error message can be decoded with `echo &lt;base64 hash&gt; | base64 -d | jq -r`\n</code></pre> CNTools <p>Open CNTools and select <code>[b] Blocks</code> to open the block viewer. Either select <code>Epoch</code> and enter the epoch you want to see a detailed view for or choose <code>Summary</code> to display blocks for last x epochs.</p> <p>If the node was elected to create blocks in the selected epoch it could look something like this:</p> Summary <pre><code> &gt;&gt; BLOCKS\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\nCurrent epoch: 96\n\n+--------+---------------------------+----------------------+--------------------------------------+\n| Epoch  | Leader | Ideal | Luck     | Adopted | Confirmed  | Missed | Ghosted | Stolen | Invalid  |\n+--------+---------------------------+----------------------+--------------------------------------+\n| 96     | 34     | 31.66 | 107.39%  | 18      | 18         | 0      | 0       | 0      | 0        |\n| 95     | 32     | 30.57 | 104.68%  | 32      | 32         | 0      | 0       | 0      | 0        |\n+--------+---------------------------+----------------------+--------------------------------------+\n\n[h] Home | [b] Block View | [i] Info | [*] Refresh\n</code></pre> Epoch <pre><code> &gt;&gt; BLOCKS\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\nCurrent epoch: 96\n\n+---------------------------+----------------------+--------------------------------------+\n| Leader | Ideal | Luck     | Adopted | Confirmed  | Missed | Ghosted | Stolen | Invalid  |\n+---------------------------+----------------------+--------------------------------------+\n| 34     | 31.66 | 107.39%  | 18      | 18         | 0      | 0       | 0      | 0        |\n+---------------------------+----------------------+--------------------------------------+\n\n+-----+------------+----------+---------------------+--------------------------+-------+-------------------------------------------------------------------+\n| #   | Status     | Block    | Slot | SlotInEpoch  | Scheduled At             | Size  | Hash                                                              |\n+-----+------------+----------+---------------------+--------------------------+-------+-------------------------------------------------------------------+\n| 1   | confirmed  | 2043444  | 11142827 | 40427    | 2020-11-16 08:34:03 CET  | 3     | ec216d3fb01e4a3cc3e85305145a31875d9561fa3bbcc6d0ee8297236dbb4115  |\n| 2   | confirmed  | 2044321  | 11165082 | 62682    | 2020-11-16 14:44:58 CET  | 3     | b75c33a5bbe49a74e4b4cc5df4474398bfb10ed39531fc65ec2acc51f89ddce5  |\n| 3   | confirmed  | 2044397  | 11166970 | 64570    | 2020-11-16 15:16:26 CET  | 3     | c1ea37fd72543779b6dab46e3e29e0e422784b5fd6188f828ace9eabcc87088f  |\n| 4   | confirmed  | 2044879  | 11178909 | 76509    | 2020-11-16 18:35:25 CET  | 3     | 35a116cec80c5dc295415e4fc8e6435c562b14a5d6833027006c988706c60307  |\n| 5   | confirmed  | 2046965  | 11232557 | 130157   | 2020-11-17 09:29:33 CET  | 3     | d566e5a1f6a3d78811acab4ae3bdcee6aa42717364f9afecd6cac5093559f466  |\n| 6   | confirmed  | 2047101  | 11235675 | 133275   | 2020-11-17 10:21:31 CET  | 3     | 3a638e01f70ea1c4a660fe4e6333272e6c61b11cf84dc8a5a107b414d1e057eb  |\n| 7   | confirmed  | 2047221  | 11238453 | 136053   | 2020-11-17 11:07:49 CET  | 3     | 843336f132961b94276603707751cdb9a1c2528b97100819ce47bc317af0a2d6  |\n| 8   | confirmed  | 2048692  | 11273507 | 171107   | 2020-11-17 20:52:03 CET  | 3     | 9b3eb79fe07e8ebae163870c21ba30460e689b23768d2e5f8e7118c572c4df36  |\n| 9   | confirmed  | 2049058  | 11282619 | 180219   | 2020-11-17 23:23:55 CET  | 3     | 643396ea9a1a2b6c66bb83bdc589fa19c8ae728d1f1181aab82e8dfe508d430a  |\n| 10  | confirmed  | 2049321  | 11289237 | 186837   | 2020-11-18 01:14:13 CET  | 3     | d93d305a955f40b2298247d44e4bc27fe9e3d1486ef3ef3e73b235b25247ccd7  |\n| 11  | confirmed  | 2049747  | 11299205 | 196805   | 2020-11-18 04:00:21 CET  | 3     | 19a43deb5014b14760c3e564b41027c5ee50e0a252abddbfcac90c8f56dc0245  |\n| 12  | confirmed  | 2050415  | 11316075 | 213675   | 2020-11-18 08:41:31 CET  | 3     | dd2cb47653f3bfb3ccc8ffe76906e07d96f1384bafd57a872ddbab3b352403e3  |\n| 13  | confirmed  | 2050505  | 11318274 | 215874   | 2020-11-18 09:18:10 CET  | 3     | deb834bc42360f8d39eefc5856bb6d7cabb6b04170c842dcbe7e9efdf9dbd2e1  |\n| 14  | confirmed  | 2050613  | 11320754 | 218354   | 2020-11-18 09:59:30 CET  | 3     | bf094f6fde8e8c29f568a253201e4b92b078e9a1cad60706285e236a91ec95ff  |\n| 15  | confirmed  | 2050807  | 11325239 | 222839   | 2020-11-18 11:14:15 CET  | 3     | 21f904346ba0fd2bb41afaae7d35977cb929d1d9727887f541782576fc6a62c9  |\n| 16  | confirmed  | 2050997  | 11330062 | 227662   | 2020-11-18 12:34:38 CET  | 3     | 109799d686fe3cad13b156a2d446a544fde2bf5d0e8f157f688f1dc30f35e912  |\n| 17  | confirmed  | 2051286  | 11336791 | 234391   | 2020-11-18 14:26:47 CET  | 3     | bb1beca7a1d849059110e3d7dc49ecf07b47970af2294fe73555ddfefb9561a8  |\n| 18  | confirmed  | 2051734  | 11348498 | 246098   | 2020-11-18 17:41:54 CET  | 3     | 87940b53c2342999c1ba4e185038cda3d8382891a16878a865f5114f540683de  |\n| 19  | leader     | -        | 11382001 | 279601   | 2020-11-19 03:00:17 CET  | -     | -                                                                 |\n| 20  | leader     | -        | 11419959 | 317559   | 2020-11-19 13:32:55 CET  | -     | -                                                                 |\n| 21  | leader     | -        | 11433174 | 330774   | 2020-11-19 17:13:10 CET  | -     | -                                                                 |\n| 22  | leader     | -        | 11434241 | 331841   | 2020-11-19 17:30:57 CET  | -     | -                                                                 |\n| 23  | leader     | -        | 11435289 | 332889   | 2020-11-19 17:48:25 CET  | -     | -                                                                 |\n| 24  | leader     | -        | 11440314 | 337914   | 2020-11-19 19:12:10 CET  | -     | -                                                                 |\n| 25  | leader     | -        | 11442361 | 339961   | 2020-11-19 19:46:17 CET  | -     | -                                                                 |\n| 26  | leader     | -        | 11443861 | 341461   | 2020-11-19 20:11:17 CET  | -     | -                                                                 |\n| 27  | leader     | -        | 11446997 | 344597   | 2020-11-19 21:03:33 CET  | -     | -                                                                 |\n| 28  | leader     | -        | 11453110 | 350710   | 2020-11-19 22:45:26 CET  | -     | -                                                                 |\n| 29  | leader     | -        | 11455323 | 352923   | 2020-11-19 23:22:19 CET  | -     | -                                                                 |\n| 30  | leader     | -        | 11505987 | 403587   | 2020-11-20 13:26:43 CET  | -     | -                                                                 |\n| 31  | leader     | -        | 11514983 | 412583   | 2020-11-20 15:56:39 CET  | -     | -                                                                 |\n| 32  | leader     | -        | 11516010 | 413610   | 2020-11-20 16:13:46 CET  | -     | -                                                                 |\n| 33  | leader     | -        | 11518958 | 416558   | 2020-11-20 17:02:54 CET  | -     | -                                                                 |\n| 34  | leader     | -        | 11533254 | 430854   | 2020-11-20 21:01:10 CET  | -     | -                                                                 |\n+-----+------------+----------+---------------------+--------------------------+-------+-------------------------------------------------------------------+\n</code></pre> gLiveView <p>Currently shows a block summary for current epoch. For full block details use CNTools for now. Invalid, missing, ghosted and stolen blocks only shown in case of a non-zero value.</p> <pre><code>\u2502--------------------------------------------------------------\u2502\n\u2502 BLOCKS   Leader  | Ideal  | Luck    | Adopted | Confirmed    \u2502\n\u2502          24        27.42    87.53%    1         1            \u2502\n\u2502          08:07:57 until leader XXXXXXXXX.....................\u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n</code></pre>"},{"location":"Scripts/cntools-changelog/","title":"Changelog","text":"<p>All notable changes to this tool will be documented in this file.</p> <p>Whenever you're updating between versions where format/hash of keys have changed , or you're changing networks - it is recommended to Backup your Wallet and Pool folders before you proceed with launching cntools on a fresh network.</p> <p>The format is based on Keep a Changelog, and this adheres to Semantic Versioning.</p>"},{"location":"Scripts/cntools-changelog/#1310-2024-08-01","title":"[13.1.0] - 2024-08-01","text":""},{"location":"Scripts/cntools-changelog/#added","title":"Added","text":"<ul> <li>New Vote menu option to hold everything related to voting on Cardano.</li> <li>Governance features according to CIP-1694 with Conway era.</li> <li>Catalyst vote registration.</li> <li>MultiSig wallet support through advanced menu item.</li> <li>Creation of new mnemonic(seed phrase) generated wallet.</li> <li>Custom mnemonic derivation path on mnemonic/hardware import.</li> <li>For a more in-depth list, see https://github.com/cardano-community/guild-operators/pull/1783</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed","title":"Changed","text":"<ul> <li>Removed CIP-0095 SPO poll</li> </ul>"},{"location":"Scripts/cntools-changelog/#1302-2024-06-07","title":"[13.0.2] - 2024-06-07","text":""},{"location":"Scripts/cntools-changelog/#fixed","title":"Fixed","text":"<ul> <li>Mnemonic import.</li> </ul>"},{"location":"Scripts/cntools-changelog/#1300-2024-06-07","title":"[13.0.0] - 2024-06-07","text":""},{"location":"Scripts/cntools-changelog/#added_1","title":"Added","text":"<ul> <li>New light mode using Koios API instead of local node for blockchain queries. Mode can be selected at runtime using -l (light), -n (node, default) or -o (offline).</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_1","title":"Changed","text":"<ul> <li>Improved submission handling</li> <li>Removed complicated file descriptor re-direction for logging. This simplified the code and improves screen printing speed as special hacks around FDs could be removed.</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_1","title":"Fixed","text":"<ul> <li>Update handling and reload of script.</li> </ul>"},{"location":"Scripts/cntools-changelog/#1210-2024-01-19","title":"[12.1.0] - 2024-01-19","text":""},{"location":"Scripts/cntools-changelog/#changed_2","title":"Changed","text":"<ul> <li>ADA price discovery through CoinGecko in set currency incl 24hr change. Shown in main UI, wallet list &amp; show as well as pool show. Disabled by default.</li> <li>Convert calculate-min-fee command references to function</li> </ul>"},{"location":"Scripts/cntools-changelog/#1202-2024-02-11","title":"[12.0.2] - 2024-02-11","text":""},{"location":"Scripts/cntools-changelog/#fixed_2","title":"Fixed","text":"<ul> <li>Retiring pools should be modifiable in case the user wants to cancel the pending retirement.</li> </ul>"},{"location":"Scripts/cntools-changelog/#1201-2024-01-26","title":"[12.0.1] - 2024-01-26","text":""},{"location":"Scripts/cntools-changelog/#fixed_3","title":"Fixed","text":"<ul> <li>Funds &gt; Send had a regression introduced in 12.0.0 thats now fixed (thanks Homer :)</li> </ul>"},{"location":"Scripts/cntools-changelog/#1200-2024-01-19","title":"[12.0.0] - 2024-01-19","text":""},{"location":"Scripts/cntools-changelog/#changed_3","title":"Changed","text":"<ul> <li>Minimum version bumps</li> <li>node 8.7.3</li> <li>cardano-hw-cli 1.14.0</li> <li>Ledger 5.0.1</li> <li>Trezor 2.6.4</li> <li>Send &gt; Funds flow updated to ask for Ada amount after tokens and destination to be able to calculate minimum amount valid to send correctly.</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_4","title":"Fixed","text":"<ul> <li>Minimum utxo calculation changed to use cardano-cli calculation tool (was broken)</li> </ul>"},{"location":"Scripts/cntools-changelog/#1102-2023-10-30","title":"[11.0.2] - 2023-10-30","text":""},{"location":"Scripts/cntools-changelog/#fixed_5","title":"Fixed","text":"<ul> <li>Fix additional Ada printing. Now omits trailing zeros from fraction part of Ada output.</li> </ul>"},{"location":"Scripts/cntools-changelog/#1101-2023-10-25","title":"[11.0.1] - 2023-10-25","text":""},{"location":"Scripts/cntools-changelog/#fixed_6","title":"Fixed","text":"<ul> <li>Fix display for Pool Cost and Pledge to accept integer as well as decimal format of ADA</li> </ul>"},{"location":"Scripts/cntools-changelog/#1100-2023-07-05","title":"[11.0.0] - 2023-07-05","text":""},{"location":"Scripts/cntools-changelog/#changed_4","title":"Changed","text":"<ul> <li>CNTools now part of Koios brand</li> </ul>"},{"location":"Scripts/cntools-changelog/#1040-2023-06-19","title":"[10.4.0] - 2023-06-19","text":""},{"location":"Scripts/cntools-changelog/#added_2","title":"Added","text":"<ul> <li>Support for SRV records</li> <li>Support for cardano-node 8.1.1</li> </ul>"},{"location":"Scripts/cntools-changelog/#1031-2023-06-03","title":"[10.3.1] - 2023-06-03","text":""},{"location":"Scripts/cntools-changelog/#fixed_7","title":"Fixed","text":"<ul> <li>Backup didn't properly exclude private keys</li> </ul>"},{"location":"Scripts/cntools-changelog/#1030-2023-05-18","title":"[10.3.0] - 2023-05-18","text":""},{"location":"Scripts/cntools-changelog/#added_3","title":"Added","text":"<ul> <li>Support for voting as per CIP-0094</li> </ul>"},{"location":"Scripts/cntools-changelog/#1023-2023-04-28","title":"[10.2.3] - 2023-04-28","text":""},{"location":"Scripts/cntools-changelog/#fixed_8","title":"Fixed","text":"<ul> <li>Additional HW signing fixes</li> </ul>"},{"location":"Scripts/cntools-changelog/#1022-2023-04-24","title":"[10.2.2] - 2023-04-24","text":""},{"location":"Scripts/cntools-changelog/#fixed_9","title":"Fixed","text":"<ul> <li>Add special case handling for hardware wallets to use stake keys as witness for registering stake address</li> </ul>"},{"location":"Scripts/cntools-changelog/#1021-2023-04-04","title":"[10.2.1] - 2023-04-04","text":""},{"location":"Scripts/cntools-changelog/#fixed_10","title":"Fixed","text":"<ul> <li>Moved <code>test_koios</code> call from cntools.library to cntools.sh</li> </ul>"},{"location":"Scripts/cntools-changelog/#1020-2023-03-13","title":"[10.2.0] - 2023-03-13","text":""},{"location":"Scripts/cntools-changelog/#fixed_11","title":"Fixed","text":"<ul> <li>HW signing fix due to deprecated cardano-hw-cli sign call.</li> <li>The check whether to use Koios API or not (env config) wasn't properly handled.</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_5","title":"Changed","text":"<ul> <li>Disabled Koios for balance lookup to prefer local node check. In most circumstances this will be faster due to low latency. If needed, set WALLET_SELECTION_FILTER_LIMIT in cntools.sh to a lower limit to skip balance lookup on wallet selection if you have many wallets and delay is too long.</li> </ul>"},{"location":"Scripts/cntools-changelog/#1011-2023-02-07","title":"[10.1.1] - 2023-02-07","text":""},{"location":"Scripts/cntools-changelog/#fixed_12","title":"Fixed","text":"<ul> <li>Disable <code>dialog</code> by default, it is an optional component - and no longer installed by default.</li> </ul>"},{"location":"Scripts/cntools-changelog/#1010-2023-01-17","title":"[10.1.0] - 2023-01-17","text":""},{"location":"Scripts/cntools-changelog/#added_4","title":"Added","text":"<ul> <li>Hardware Wallets: Allow signing using cold keys for a pool, use it for rotating KES keys.</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_6","title":"Changed","text":"<ul> <li>Keep deployment consistent with guild-deploy.sh</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_13","title":"Fixed","text":"<ul> <li>Fix parsing space in the name of assets</li> </ul>"},{"location":"Scripts/cntools-changelog/#1005-2022-11-07","title":"[10.0.5] - 2022-11-07","text":""},{"location":"Scripts/cntools-changelog/#changed_7","title":"Changed","text":"<ul> <li>Updated testnet token registry to be reused for each non-mainnet network</li> <li>Remove stale code for remote chain analysis</li> </ul>"},{"location":"Scripts/cntools-changelog/#1004-2022-08-26","title":"[10.0.4] - 2022-08-26","text":""},{"location":"Scripts/cntools-changelog/#changed_8","title":"Changed","text":"<ul> <li>Allow pool cost to use fraction of ADA</li> <li>Starts using koios-1.0.7 endpoints to fetch information</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_14","title":"Fixed","text":"<ul> <li>Fixes an issue with reusage of variable name and updated param name for cardano-cli.</li> <li>Fix token minting and burn assets</li> </ul>"},{"location":"Scripts/cntools-changelog/#1003-2022-08-16","title":"[10.0.3] - 2022-08-16","text":""},{"location":"Scripts/cntools-changelog/#fixed_15","title":"Fixed","text":"<ul> <li>env file was sourced after calling cntools.library, overriding test_koios result</li> </ul>"},{"location":"Scripts/cntools-changelog/#1002-2022-08-13","title":"[10.0.2] - 2022-08-13","text":""},{"location":"Scripts/cntools-changelog/#fixed_16","title":"Fixed","text":"<ul> <li>Bump min cardano-hw-cli version to 1.10.0</li> <li>Requires cardano-hw-cli to be present on online node for pool registration/modification to be able to transform tx if needed</li> <li>Transform tx if needed before any witnessing/signing is done.</li> <li>Wrong arguments in call to cardano-hw-cli for cddl-formatted tx</li> </ul>"},{"location":"Scripts/cntools-changelog/#1001-2022-07-14","title":"[10.0.1] - 2022-07-14","text":""},{"location":"Scripts/cntools-changelog/#changed_9","title":"Changed","text":"<ul> <li>Transactions now built using cddl-format to ensure that the formatting of transaction adheres the ledger specs.</li> <li>Default to mary era transaction building format for now.</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_17","title":"Fixed","text":"<ul> <li>Cold signing fix for pool registration / update. Last key was added twice when assemling witnesses.</li> </ul>"},{"location":"Scripts/cntools-changelog/#1000-2022-06-28","title":"[10.0.0] - 2022-06-28","text":""},{"location":"Scripts/cntools-changelog/#added_5","title":"Added","text":"<ul> <li>Support for Vasil Fork</li> <li>Preliminary support for Post HF updates (a short release will follow post fork in coming days)</li> <li>Minimum version for Node bumped to 1.35.0</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_10","title":"Changed","text":"<ul> <li>Pool &gt; Rotate code now uses kes-periodinfo CLI query to get counter from node (fallback for Koios)</li> <li>Pool &gt; Show Info updated to include current KES counter</li> <li>Update getEraIdentifier to include Babbage era</li> </ul>"},{"location":"Scripts/cntools-changelog/#910-2022-05-11","title":"[9.1.0] - 2022-05-11","text":""},{"location":"Scripts/cntools-changelog/#changed_11","title":"Changed","text":"<ul> <li>Harmonize flow for reusing old wallet configuration on pool modification vs setting new wallets.</li> <li>Remove the requirement for reward stake signing key in wallet registration/modification</li> <li>Reward wallet no longer auto-delegated on pool registration just like for multi-owners. </li> </ul>"},{"location":"Scripts/cntools-changelog/#9010-2022-05-03","title":"[9.0.10] - 2022-05-03","text":""},{"location":"Scripts/cntools-changelog/#fixed_18","title":"Fixed","text":"<ul> <li>Detect if cardano-hw-cli has execution permission</li> </ul>"},{"location":"Scripts/cntools-changelog/#909-2022-03-14","title":"[9.0.9] - 2022-03-14","text":""},{"location":"Scripts/cntools-changelog/#changed_12","title":"Changed","text":"<ul> <li>Add version (-v) argument to cntools script to print current version</li> </ul>"},{"location":"Scripts/cntools-changelog/#908-2022-03-07","title":"[9.0.8] - 2022-03-07","text":""},{"location":"Scripts/cntools-changelog/#changed_13","title":"Changed","text":"<ul> <li>Remove HASH_IDENTIFIER variable references (Ddz issue which required this seperation was resolved a while ago)</li> <li>Replace NETWORKID check with NWMAGIC variable</li> </ul>"},{"location":"Scripts/cntools-changelog/#907-2022-03-02","title":"[9.0.7] - 2022-03-02","text":""},{"location":"Scripts/cntools-changelog/#fixed_19","title":"Fixed","text":"<ul> <li>Call Test Koios function at start of CNTools, instead of calling by default every time env is sourced</li> </ul>"},{"location":"Scripts/cntools-changelog/#906-2022-02-20","title":"[9.0.6] - 2022-02-20","text":""},{"location":"Scripts/cntools-changelog/#fixed_20","title":"Fixed","text":"<ul> <li>Fix for update check if not executed from default scripts folder.</li> </ul>"},{"location":"Scripts/cntools-changelog/#905-2022-02-16","title":"[9.0.5] - 2022-02-16","text":""},{"location":"Scripts/cntools-changelog/#fixed_21","title":"Fixed","text":"<ul> <li>Script update code fixed to better handle in-app update. Would sometimes update but not source library correctly.</li> </ul>"},{"location":"Scripts/cntools-changelog/#904-2022-02-14","title":"[9.0.4] - 2022-02-14","text":""},{"location":"Scripts/cntools-changelog/#fixed_22","title":"Fixed","text":"<ul> <li>Update request for pool_info endpoint from Koios</li> </ul>"},{"location":"Scripts/cntools-changelog/#903-2022-02-01","title":"[9.0.3] - 2022-02-01","text":""},{"location":"Scripts/cntools-changelog/#added_6","title":"Added","text":"<ul> <li>Add a config variable TX_TTL to allow transaction to be valid (by default for 3600 slots) from the point of creation - previous default of 10 minutes on mainnet could be hit-and-miss with the state of network.</li> </ul>"},{"location":"Scripts/cntools-changelog/#902-2022-01-22","title":"[9.0.2] - 2022-01-22","text":""},{"location":"Scripts/cntools-changelog/#changed_14","title":"Changed","text":"<ul> <li>Add decimal param to token metadata creator and increase ticker max length to 9 chars according to spec changes.</li> </ul>"},{"location":"Scripts/cntools-changelog/#901-2022-01-17","title":"[9.0.1] - 2022-01-17","text":""},{"location":"Scripts/cntools-changelog/#changed_15","title":"Changed","text":"<ul> <li>Removing tool credits in offline metadata registry due to \"out of protocol\".</li> </ul>"},{"location":"Scripts/cntools-changelog/#900-2022-01-10","title":"[9.0.0] - 2022-01-10","text":""},{"location":"Scripts/cntools-changelog/#changed_16","title":"Changed","text":"<ul> <li>Due to changes in cardano-node 1.33.x -&gt; for utxo ledger lookup and previous heavy pool-params query, Koios API is now the default option for these lookups.</li> <li>You can update KOIOS_API env variable to connect to a local instance of koios (open source and incentivises all to build a decentralised query layer) if you'd not like to connect to remote instance.</li> <li>Visit the https://www.koios.rest/ for more information about Koios or check out the API documentation at https://api.koios.rest.</li> <li>If you'd like to revert to old behaviour (use CLI which could be slow to retrieve UTxOs), you can set ENABLE_KOIOS environment variable to N.</li> </ul>"},{"location":"Scripts/cntools-changelog/#882-2021-12-28","title":"[8.8.2] - 2021-12-28","text":""},{"location":"Scripts/cntools-changelog/#fixed_23","title":"Fixed","text":"<ul> <li>Transform txBody using canonical order before signing/witnessing in case of HW wallet.</li> <li>Bump minimum HW wallet versions:</li> <li>Ledger &gt;= 3.0.0</li> <li>Trezor &gt;= 2.4.3</li> <li>cardano-hw-cli &gt;= 1.9.0</li> </ul>"},{"location":"Scripts/cntools-changelog/#881-2021-12-18","title":"[8.8.1] - 2021-12-18","text":""},{"location":"Scripts/cntools-changelog/#fixed_24","title":"Fixed","text":"<ul> <li>Fallback to Mary era in build commands to keep ledger compatibility</li> </ul>"},{"location":"Scripts/cntools-changelog/#880-2021-12-15","title":"[8.8.0] - 2021-12-15","text":""},{"location":"Scripts/cntools-changelog/#fixed_25","title":"Fixed","text":"<ul> <li>Asset handling after cardano-node 1.32.1 version bump. ascii -&gt; hex change in cardano-cli.</li> </ul>"},{"location":"Scripts/cntools-changelog/#873-2021-11-30","title":"[8.7.3] - 2021-11-30","text":""},{"location":"Scripts/cntools-changelog/#fixed_26","title":"Fixed","text":"<ul> <li>Remove stale cntools.config comments</li> </ul>"},{"location":"Scripts/cntools-changelog/#872-2021-11-08","title":"[8.7.2] - 2021-11-08","text":""},{"location":"Scripts/cntools-changelog/#changed_17","title":"Changed","text":"<ul> <li>Remove check if pool reward wallet is a hw wallet, enforce that its also a multi-owner to the pool</li> </ul>"},{"location":"Scripts/cntools-changelog/#871-2021-11-04","title":"[8.7.1] - 2021-11-04","text":""},{"location":"Scripts/cntools-changelog/#fixed_27","title":"Fixed","text":"<ul> <li>Balance check of wrong wallet in certain circumstances for pool modify</li> </ul>"},{"location":"Scripts/cntools-changelog/#870-2021-10-05","title":"[8.7.0] - 2021-10-05","text":""},{"location":"Scripts/cntools-changelog/#changed_18","title":"Changed","text":"<ul> <li>CNTools configuration moved from cntools.config to cntools.sh</li> </ul>"},{"location":"Scripts/cntools-changelog/#866-2021-09-26","title":"[8.6.6] - 2021-09-26","text":""},{"location":"Scripts/cntools-changelog/#fixed_28","title":"Fixed","text":"<ul> <li>Pool rotation date calculation fix, 8.6.4 didn't properly fix it</li> </ul>"},{"location":"Scripts/cntools-changelog/#865-2021-09-15","title":"[8.6.5] - 2021-09-15","text":""},{"location":"Scripts/cntools-changelog/#fixed_29","title":"Fixed","text":"<ul> <li>Minimum utxo output calculation post Alonzo</li> </ul>"},{"location":"Scripts/cntools-changelog/#864-2021-09-14","title":"[8.6.4] - 2021-09-14","text":""},{"location":"Scripts/cntools-changelog/#fixed_30","title":"Fixed","text":"<ul> <li>Pool rotation date calculation fix (display only)</li> </ul>"},{"location":"Scripts/cntools-changelog/#863-2021-08-31","title":"[8.6.3] - 2021-08-31","text":""},{"location":"Scripts/cntools-changelog/#fixed_31","title":"Fixed","text":"<ul> <li>Pool retire fix</li> </ul>"},{"location":"Scripts/cntools-changelog/#862-2021-08-30","title":"[8.6.2] - 2021-08-30","text":""},{"location":"Scripts/cntools-changelog/#fixed_32","title":"Fixed","text":"<ul> <li>Revert <code>--whole-utxo</code> flag, as it returns all address and will not accept <code>--address</code></li> </ul>"},{"location":"Scripts/cntools-changelog/#861-2021-08-27","title":"[8.6.1] - 2021-08-27","text":""},{"location":"Scripts/cntools-changelog/#changed_19","title":"Changed","text":"<ul> <li>Alonzo related changes for era and minimum utxo.</li> </ul>"},{"location":"Scripts/cntools-changelog/#860-2021-08-27","title":"[8.6.0] - 2021-08-27","text":""},{"location":"Scripts/cntools-changelog/#changed_20","title":"Changed","text":"<ul> <li>Add <code>--whole-utxo</code> flag when query UTxO, as required by cardano-cli 1.28, to keep behaviour same as before.</li> <li>Baseline compatibility with 1.29</li> </ul>"},{"location":"Scripts/cntools-changelog/#8415-2021-07-15","title":"[8.4.15] - 2021-07-15","text":""},{"location":"Scripts/cntools-changelog/#changed_21","title":"Changed","text":"<ul> <li>Switch default to 'No' adding a message when sending funds</li> </ul>"},{"location":"Scripts/cntools-changelog/#8414-2021-07-14","title":"[8.4.14] - 2021-07-14","text":""},{"location":"Scripts/cntools-changelog/#fixed_33","title":"Fixed","text":"<ul> <li>Fix for upcoming unreleased dbsync rest endpoint </li> </ul>"},{"location":"Scripts/cntools-changelog/#8413-2021-07-08","title":"[8.4.13] - 2021-07-08","text":""},{"location":"Scripts/cntools-changelog/#changed_22","title":"Changed","text":"<ul> <li>Documentation references updated to new site layout</li> </ul>"},{"location":"Scripts/cntools-changelog/#8412-2021-06-28","title":"[8.4.12] - 2021-06-28","text":""},{"location":"Scripts/cntools-changelog/#fixed_34","title":"Fixed","text":"<ul> <li>Pre-source env in offline/online mode for checkUpdate depending on argument provided to cntools.sh</li> </ul>"},{"location":"Scripts/cntools-changelog/#8411-2021-06-25","title":"[8.4.11] - 2021-06-25","text":""},{"location":"Scripts/cntools-changelog/#changed_23","title":"Changed","text":"<ul> <li>KES calculation moved from CNTools &amp; gLiveView into a common function in env file. For online mode node metrics is used for KES expiration instead of static pool KES start period.</li> <li>General message metadata support added to 'funds &gt;&gt; send' according to CIP-0020.</li> </ul>"},{"location":"Scripts/cntools-changelog/#8410-2021-06-15","title":"[8.4.10] - 2021-06-15","text":""},{"location":"Scripts/cntools-changelog/#fixed_35","title":"Fixed","text":"<ul> <li>Fix display issue for CLI that were upgraded to Alonzo-Blue networks</li> </ul>"},{"location":"Scripts/cntools-changelog/#849-2021-06-15","title":"[8.4.9] - 2021-06-15","text":""},{"location":"Scripts/cntools-changelog/#changed_24","title":"Changed","text":"<ul> <li>Handle Various updates to grest queries [disabled] to make them independent of instances. Note: Version incremented thrice on PR branch itself</li> </ul>"},{"location":"Scripts/cntools-changelog/#846-2021-06-04","title":"[8.4.6] - 2021-06-04","text":""},{"location":"Scripts/cntools-changelog/#fixed_36","title":"Fixed","text":"<ul> <li>Add balance check for main pool owner, that there is at least one utxo available</li> <li>Allow utxo without lovelace (for future when we might have tokens on utxo without Ada, like on Alonzo TestNet)</li> <li>pctToFraction helper function didn't properly handle 0 value</li> </ul>"},{"location":"Scripts/cntools-changelog/#845-2021-05-31","title":"[8.4.5] - 2021-05-31","text":""},{"location":"Scripts/cntools-changelog/#fixed_37","title":"Fixed","text":"<ul> <li>Reset IFS at main loop, fixes invalid tip difference on home screen after going to Block &gt; Summary</li> </ul>"},{"location":"Scripts/cntools-changelog/#844-2021-05-19","title":"[8.4.4] - 2021-05-19","text":""},{"location":"Scripts/cntools-changelog/#fixed_38","title":"Fixed","text":"<ul> <li>Typo in Ledger ledger version requirement error and make it clearer that its the app version, not fw version.</li> </ul>"},{"location":"Scripts/cntools-changelog/#843-2021-05-17","title":"[8.4.3] - 2021-05-17","text":""},{"location":"Scripts/cntools-changelog/#fixed_39","title":"Fixed","text":"<ul> <li>Token Mint/Burn script file signing not completely removed in all places (1.27.0 change)</li> </ul>"},{"location":"Scripts/cntools-changelog/#842-2021-05-16","title":"[8.4.2] - 2021-05-16","text":""},{"location":"Scripts/cntools-changelog/#fixed_40","title":"Fixed","text":"<ul> <li>cardano-hw-cli version limited to 1.2.0 for current Trezor fw v2.3.6. Please manually downgrade version, available at https://github.com/vacuumlabs/cardano-hw-cli/releases , placing files in $HOME/bin/cardano-hw-cli</li> </ul>"},{"location":"Scripts/cntools-changelog/#841-2021-05-16","title":"[8.4.1] - 2021-05-16","text":""},{"location":"Scripts/cntools-changelog/#changed_25","title":"Changed","text":"<ul> <li>Wallet &gt;&gt; Show no longer require payment.vkey to be present, as long as either payment or base .addr file(s) exist</li> </ul>"},{"location":"Scripts/cntools-changelog/#840-2021-05-16","title":"[8.4.0] - 2021-05-16","text":""},{"location":"Scripts/cntools-changelog/#added_7","title":"Added","text":"<ul> <li>Compatibility with cardano-address 3.4.0 (while retaining support for 2.1.0)</li> </ul>"},{"location":"Scripts/cntools-changelog/#830-2021-05-15","title":"[8.3.0] - 2021-05-15","text":""},{"location":"Scripts/cntools-changelog/#added_8","title":"Added","text":"<ul> <li>New env variable called PGREST_API and if set and reachable, used instead of local node queries and for advanced modes</li> <li>New library function isPoolRegistered() for verifying if a pool is registered or not using either simple reg cert file detection (if REST API not set/reachable) or proper dbsync lookup using REST API. Used by Pool &gt;&gt; Show|List|Register|Modify</li> <li>Option to mint/burn assets in hybrid mode</li> <li>Transaction &gt;&gt; Sign now automatically tries to find the correct signing keys instead of having the user manually select the correct files</li> <li>** ADVANCED FEATURE ** - Chain Queries</li> <li>Menu is dynamically built based on queries(JSON files) in DBSYNC_QUERY_FOLDER (env variable, default files/dbsync/queries) three levels deep.</li> <li>A download option lets the user download the latest uploaded queries on Guild Operators GitHub site.</li> <li>Query files<ul> <li>Contains menu path, description, variables, and queries(multiple)</li> <li>Executes a predefined DBSync RPC/function through PostgREST API</li> <li>Variables used in RPC call can either be user input, CNTools variables like EKG metrics, or an item in the result from a previous query(in the same query file)</li> <li>Result from RPC call can either be printed or silent(only to be used for later query)</li> <li>Output is printed as JSON</li> </ul> </li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_26","title":"Changed","text":"<ul> <li>Minimum node version bumped to 1.27.0</li> <li>Menu has been re-designed with both back &amp; home options. Instead of returning to home menu after the completed operation user is returned to the last menu.</li> <li>Pool &gt;&gt; Show now use PostgREST API(if set), or the new pool-params cli query as fallback method.</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_41","title":"Fixed","text":"<ul> <li>1.27.0 introduced a few changes in CLI commands for assets minting/burning</li> </ul>"},{"location":"Scripts/cntools-changelog/#822-2021-05-02","title":"[8.2.2] - 2021-05-02","text":""},{"location":"Scripts/cntools-changelog/#fixed_42","title":"Fixed","text":"<ul> <li>KES expiration date fix</li> </ul>"},{"location":"Scripts/cntools-changelog/#821-2021-04-26","title":"[8.2.1] - 2021-04-26","text":""},{"location":"Scripts/cntools-changelog/#changed_27","title":"Changed","text":"<ul> <li>Make use of UPDATE_CHECK environment variable to skip any checks to internet by default</li> </ul>"},{"location":"Scripts/cntools-changelog/#820-2021-04-18","title":"[8.2.0] - 2021-04-18","text":""},{"location":"Scripts/cntools-changelog/#added_9","title":"Added","text":"<ul> <li>Ability to create &amp; update a Cardano Token Registry submission JSON file</li> <li>Requires 'token-metadata-creator' tool, instructions to download/build this tool added to Guild Operators documentation:</li> <li>https://cardano-community.github.io/guild-operators/Build/offchainMetadataTools</li> <li>Token Registry lookup in Wallet &gt;&gt; Show</li> <li>Token asset fingerprint generation according to https://github.com/cardano-foundation/CIPs/pull/64</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_28","title":"Changed","text":"<ul> <li>Redesigned input handling to be more flexible and improve output</li> </ul>"},{"location":"Scripts/cntools-changelog/#816-2021-04-14","title":"[8.1.6] - 2021-04-14","text":""},{"location":"Scripts/cntools-changelog/#changed_29","title":"Changed","text":"<ul> <li>Metadata creation now offer the choice to add a metadata JSON scaffold to see the required structure</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_43","title":"Fixed","text":"<ul> <li>Fixed metadata creation entering JSON metadata through text editor</li> </ul>"},{"location":"Scripts/cntools-changelog/#815-2021-04-09","title":"[8.1.5] - 2021-04-09","text":""},{"location":"Scripts/cntools-changelog/#fixed_44","title":"Fixed","text":"<ul> <li>Offline mode fix to ignore error when sourcing env</li> </ul>"},{"location":"Scripts/cntools-changelog/#814-2021-04-05","title":"[8.1.4] - 2021-04-05","text":""},{"location":"Scripts/cntools-changelog/#changed_30","title":"Changed","text":"<ul> <li>Enhanced minimum utxo calculation (credits to Martin providing this)</li> <li>based on calculations from https://github.com/intersectmbo/cardano-ledger/blob/master/doc/explanations/min-utxo-mary.rst</li> <li>Validation of wallet address balance on transactions improved</li> </ul>"},{"location":"Scripts/cntools-changelog/#813-2021-04-01","title":"[8.1.3] - 2021-04-01","text":""},{"location":"Scripts/cntools-changelog/#fixed_45","title":"Fixed","text":"<ul> <li>Alignment fix in blocks table</li> </ul>"},{"location":"Scripts/cntools-changelog/#812-2021-03-31","title":"[8.1.2] - 2021-03-31","text":""},{"location":"Scripts/cntools-changelog/#changed_31","title":"Changed","text":"<ul> <li>Manual CNTools update replaced with automatic by asking to update on startup like the rest of the scripts in the guild repository. </li> <li>Changelog truncated up to v6.0.0. Minor and patch version changelog entries merged with next major release changelog.</li> </ul>"},{"location":"Scripts/cntools-changelog/#811-2021-03-30","title":"[8.1.1] - 2021-03-30","text":""},{"location":"Scripts/cntools-changelog/#fixed_46","title":"Fixed","text":"<ul> <li>Relay registration condition</li> <li>Version number</li> </ul>"},{"location":"Scripts/cntools-changelog/#810-2021-03-26","title":"[8.1.0] - 2021-03-26","text":""},{"location":"Scripts/cntools-changelog/#added_10","title":"Added","text":"<ul> <li>IPv6 support in pool registration/modification</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_32","title":"Changed","text":"<ul> <li>Wallet delegation now lets you specify Pool ID in addition to local CNTools pool instead of previous cold.vkey cbor string</li> <li>A couple of functions regarding number validation moved to common env file</li> <li>Code adapted for changes in ledger-state dump used by 'Pool &gt;&gt; Show'</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_47","title":"Fixed","text":"<ul> <li>Backup &amp; restore now exclude gpg encrypted keys from online backup and suppression of false alarms</li> </ul>"},{"location":"Scripts/cntools-changelog/#802-2021-03-15","title":"[8.0.2] - 2021-03-15","text":""},{"location":"Scripts/cntools-changelog/#fixed_48","title":"Fixed","text":"<ul> <li>Bump cardano-hw-cli minimum version to 1.2.0</li> <li>Add Ledger Cardano app version check with minimum enforced version of 2.2.0</li> <li>Add Trezor firmware check with minimum enforced version of 2.3.6</li> </ul>"},{"location":"Scripts/cntools-changelog/#801-2021-03-05","title":"[8.0.1] - 2021-03-05","text":""},{"location":"Scripts/cntools-changelog/#fixed_49","title":"Fixed","text":"<ul> <li>Add BASH version check, version 4.4 or newer required</li> </ul>"},{"location":"Scripts/cntools-changelog/#800-2021-02-28","title":"[8.0.0] - 2021-02-28","text":""},{"location":"Scripts/cntools-changelog/#added_11","title":"Added","text":"<ul> <li>Multi Asset Token compatibility added throughout all CNTools operations. </li> <li>Sending Ada and custom tokens is done through the normal 'Funds &gt;&gt; Send' operation</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_33","title":"Changed","text":"<ul> <li>Metadata moved to a new Advanced section used for devs/advanced operations not normally used by SPOs.</li> <li>Accessed by enabling developer/advanced mode in cntools.config or by providing runtime flag '-a'</li> <li>Redesign of backup and restore.</li> <li>Deletion of private keys moved from backup to new section under <code>Advanced</code></li> <li>Backup now only contain content of priv folder (files &amp; scripts folders dropped)</li> <li>Restore operation now restore directly to priv folder instead of a random user selected folder to make restore easier and better. Before restore, a new full backup of priv folder is made and stored encrypted in priv/archive</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_50","title":"Fixed","text":"<ul> <li>JQ limitation workaround for large numbers</li> <li>Dialog compatibility improvement by preventing dialog launching a subshell on some systems causing dialog not to run</li> </ul>"},{"location":"Scripts/cntools-changelog/#716-2021-02-10","title":"[7.1.6] - 2021-02-10","text":"<ul> <li>Update curl commands when file isnt downloaded correctly (to give correct return code)</li> </ul>"},{"location":"Scripts/cntools-changelog/#715-2021-02-03","title":"[7.1.5] - 2021-02-03","text":""},{"location":"Scripts/cntools-changelog/#changed_34","title":"Changed","text":"<ul> <li>Guild Announcement/Support Telegram channel added to CNTools GUI</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_51","title":"Fixed","text":"<ul> <li>Fix for a special case using an incomplete wallet (missing stake keys) </li> </ul>"},{"location":"Scripts/cntools-changelog/#714-2021-02-01","title":"[7.1.4] - 2021-02-01","text":""},{"location":"Scripts/cntools-changelog/#fixed_52","title":"Fixed","text":"<ul> <li>Typo in function name after harmonization between scripts</li> </ul>"},{"location":"Scripts/cntools-changelog/#713-2021-01-30","title":"[7.1.3] - 2021-01-30","text":""},{"location":"Scripts/cntools-changelog/#fixed_53","title":"Fixed","text":"<ul> <li>Vacuumlabs cardano-hw-cli 1.1.3 support, now the minimum supported version</li> <li>Improved error handling</li> </ul>"},{"location":"Scripts/cntools-changelog/#711-2021-01-29","title":"[7.1.1] - 2021-01-29","text":""},{"location":"Scripts/cntools-changelog/#changed_35","title":"Changed","text":"<ul> <li>Minor change to future update notification for common env file</li> </ul>"},{"location":"Scripts/cntools-changelog/#710-2021-01-29","title":"[7.1.0] - 2021-01-29","text":""},{"location":"Scripts/cntools-changelog/#changed_36","title":"Changed","text":"<ul> <li>Remove ChainDB metrics references to align with cardano-node 1.25.1</li> <li>Moved some functions to env for reusability between tools</li> </ul>"},{"location":"Scripts/cntools-changelog/#702-2021-01-17","title":"[7.0.2] - 2021-01-17","text":""},{"location":"Scripts/cntools-changelog/#changed_37","title":"Changed","text":"<ul> <li>Re-add the option in offline workflow to use wallet folder that only contains stake keys for multi-owner pools</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_54","title":"Fixed","text":"<ul> <li>Verification of signing key in offline mode for extended signing keys (mnemonics imported wallets)</li> </ul>"},{"location":"Scripts/cntools-changelog/#701-2021-01-13","title":"[7.0.1] - 2021-01-13","text":""},{"location":"Scripts/cntools-changelog/#changed_38","title":"Changed","text":"<ul> <li>Add prompt before updating common env file, instead of auto-update</li> </ul>"},{"location":"Scripts/cntools-changelog/#700-2021-01-11","title":"[7.0.0] - 2021-01-11","text":"<p>Though mostly unchanged in the user interface, this is a major update with most of the code re-written/touched in the back-end. Only the most noticeable changes added to changelog. </p>"},{"location":"Scripts/cntools-changelog/#added_12","title":"Added","text":"<ul> <li>HW Wallet support through Vacuumlabs cardano-hw-cli (Ledger Nano X/S &amp; Trezor T)</li> <li>Vacuumlabs cardano-hw-cli added as build option to prereqs.sh, option '-w' incl Ledger udev rules. Software from Vacuumlabs and Ledger app still early in development and may contain limitations that require workarounds. Users are recommended to familiarise their usage using test wallets first.</li> <li>Because of HW wallet support, transaction signing has been re-designed. For CLI and HW wallet pool reg, raw tx is first witnessed by all signing keys separately and then assembled and signed instead of signing directly with all signing keys. But for all other HW wallet transactions, signing is done directly without first witnessing.</li> <li>Requires updated Cardano app in Ledger/Trezor set to be released in January 2021 to use in pool registration/modification.</li> <li>Option added to disable Dialog for file/dir input in cntools.config</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_39","title":"Changed","text":"<ul> <li>Logging completely re-designed from the ground. Previous selective logging wasn't very useful. All output(almost) now outputted to both the screen and to a timestamped log file. One log file is created per CNTools session. Old log file archived in logs/archive subfolder and last 10 log files kept, rest is pruned on CNTools startup.</li> <li>DEBUG    : Verbose output, most output printed on screen is logged as debug messages except explicitly disabled, like menu printing.</li> <li>INFO     : Informational and the most important output.</li> <li>ACTION   : e.g cardano-cli executions etc</li> <li>ERROR    : error messages and stderr output</li> <li>Verbosity setting in cntools.config removed.</li> <li>Offline workflow now use a single JSON transaction file holding all data needed. This allows us to bake in additional data in the JSON file in addition to the tx body to make it much more clear what the offline transaction do. Like signing key verification, transaction data like fee, source wallet, pool name etc. It also lets the user know on offline computer what signing keys is needed to sign the transaction.</li> <li>Sign Tx moved to Transaction &gt;&gt; Sign</li> <li>Submit Tx moved to Transaction &gt;&gt; Submit</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_55","title":"Fixed","text":"<ul> <li>Remove intermediate prompt for showing changelog, so that it's directly visible.</li> </ul>"},{"location":"Scripts/cntools-changelog/#631-2020-12-14","title":"[6.3.1] - 2020-12-14","text":""},{"location":"Scripts/cntools-changelog/#fixed_56","title":"Fixed","text":"<ul> <li>Array expansion not correctly handled for multi-owner signing keys</li> <li>KES rotation output fix in OFFLINE mode, op.cert should be copied, not cold.counter</li> <li>Output and file explorer workflow redesigned a bit for a better flow</li> <li>formatLovelace() thousand separator fix after forcing locale to C.UTF-8 in env</li> <li>formatAda() function added to pretty print pledge and cost w/o Lovelace</li> </ul>"},{"location":"Scripts/cntools-changelog/#630-2020-12-03","title":"[6.3.0] - 2020-12-03","text":""},{"location":"Scripts/cntools-changelog/#changed_40","title":"Changed","text":"<ul> <li>printTable function replaced with bash printf due to compatibility issues</li> <li>Improved workflow in pool registration/modification for relays and multi-owner.</li> <li>Standardized names for wallet and pool files/folders moved to env file from cntools.config</li> <li>Compatibility with 1.24.2 node (accomodate ledger schema and CLI changes), use 1.24.2 as baseline</li> <li>Move version check to env</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_57","title":"Fixed","text":"<ul> <li>Error output for prerequisite checks</li> </ul>"},{"location":"Scripts/cntools-changelog/#621-2020-11-28","title":"[6.2.1] - 2020-11-28","text":""},{"location":"Scripts/cntools-changelog/#changed_41","title":"Changed","text":"<ul> <li>Compatibility changes for cardano-node 1.23.0, now minimum version to run CNTools 6.2.1</li> <li>Cleanup of old code</li> </ul>"},{"location":"Scripts/cntools-changelog/#620-alpha-branch","title":"[6.2.0] - (alpha branch)","text":""},{"location":"Scripts/cntools-changelog/#added_13","title":"Added","text":"<ul> <li>Ability to post metadata on-chain, e.g. (but not limited to) Adams https://vote.crypto2099.io/</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_42","title":"Changed","text":"<ul> <li>Blocks view updated to adapt to the added CNCLI integration and changes made to block collector(logMonitor)</li> <li>CNCLI</li> <li>Log Monitor</li> <li>chattr file locking now optional to use, a new setting in cntools.config added for it.</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_58","title":"Fixed","text":"<ul> <li>unnecessary bech32 conversion in wallet import (non-breaking) </li> </ul>"},{"location":"Scripts/cntools-changelog/#610-2020-10-22","title":"[6.1.0] - 2020-10-22","text":""},{"location":"Scripts/cntools-changelog/#added_14","title":"Added","text":"<ul> <li>Wallet de-registration with key deposit refund (new cntools.config parameter, WALLET_STAKE_DEREG_FILENAME)</li> <li>Default values loaded for all config variables if omitted/missing in cntools.config</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_43","title":"Changed","text":"<ul> <li>Prometheus node metrics replaced with EKG</li> <li>Allow and handle missing pool.config in pool &gt;&gt; modify and show</li> <li>Cancel and return added in several helper functions if cardano-cli execution fails</li> <li>Various tweaks to the output</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_59","title":"Fixed","text":"<ul> <li>Script execution permissions after internal update</li> <li>Handle redirect in curl metadata fetch</li> </ul>"},{"location":"Scripts/cntools-changelog/#603-2020-10-16","title":"[6.0.3] - 2020-10-16","text":""},{"location":"Scripts/cntools-changelog/#fixed_60","title":"Fixed","text":"<ul> <li>Shelley epoch transition calculation used the wrong byron metric in the calculation</li> </ul>"},{"location":"Scripts/cntools-changelog/#602-2020-10-16","title":"[6.0.2] - 2020-10-16","text":""},{"location":"Scripts/cntools-changelog/#fixed_61","title":"Fixed","text":"<ul> <li>Internal update had the wrong path to env file breaking automatic update, please use prereqs.sh to update</li> <li>Fix in 6.0.1 broke pool id retrieval, now compatible with both pre and post cardano-node 1.21.2 syntax.</li> </ul>"},{"location":"Scripts/cntools-changelog/#601-2020-10-16","title":"[6.0.1] - 2020-10-16","text":""},{"location":"Scripts/cntools-changelog/#fixed_62","title":"Fixed","text":"<ul> <li>As per change to cardano-cli syntax, pool ID requires <code>--cold-verification-key-file</code> instead of <code>--verification-key-file</code></li> </ul>"},{"location":"Scripts/cntools-changelog/#600-2020-10-15","title":"[6.0.0] - 2020-10-15","text":"<p>This is a major release with a lot of changes. It is highly recommended that you familiarise yourself with the usage for Hybrid or Online v/s Offline mode on a testnet environment before doing it on production. Please visit https://cardano-community.github.io/guild-operators/upgrade for details.</p>"},{"location":"Scripts/cntools-changelog/#added_15","title":"Added","text":"<ul> <li>Allow CNTools to operate in offline mode. Offline features include:</li> <li>Simplified Walet Show/List menu</li> <li>Wallet delete without balance check option</li> <li>Pool KES Rotation</li> <li>Sign a staging transaction.</li> <li>Allow creation of staging tx files using ttl as input in an online/offline-hybrid mode, that can be sent to offline server to sign.</li> <li>To Transfer Funds</li> <li>Withdraw Rewards</li> <li>Delegate</li> <li>Register/Modify/Retire pool</li> <li>Allow import of a signed transaction to submit in online mode</li> <li>Allow import of 15/24 words based wallets. Note that you'd need <code>cardano-address</code> and <code>bech32</code> in yout $PATH to use this feature (available if you rebuild <code>cardano-node</code> using updated <code>cabal-build-all.sh</code>), reusing guide from @ilap.</li> <li>Backup now offer the ability to create an online backup without wallet payment/stake and pool cold sign keys</li> <li>Regular(offline) backup now display a warning if wallet payment/stake and pool cold sign keys are missing due to being deleted manually or by previous backup</li> <li>Retire notification in pool &gt;&gt; show</li> <li>Sanity check before launching a second instance of cnode.sh</li> <li>Doc update to run cnode.sh as a systemd service</li> <li>Use secure remove (<code>srm</code>) when available when deleting files.</li> <li>Balance check notification added before wallet selection menus are shown to know that work is done in the background</li> <li>Ability to select a different pool owner and reward wallet</li> <li>Multi-owner support using stake vkey/skey files</li> <li>Added TIMEOUT_LEDGER_STATE(default 300s) in cntools.config to be used instead of static 60 seconds for querying shelley ledger-state.</li> <li>Option to delete private keys after successful backup</li> <li>itnRewards.sh script to claim ITN rewards incl docs update</li> <li>More explicit error messages at startup</li> <li>Basic sanity checks for socket file</li> <li>Backup &amp; Restore of wallets, pools and configuration files</li> <li>External KES rotation script using CNTools library</li> <li>Add few flags to control prereqs to allow skipping overwriting files, deploying OS packages, etc</li> <li>cntools.sh: Drop an error if log not found, indicating config with no JSON being used</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_44","title":"Changed","text":"<ul> <li>Improved trap/exit handling</li> <li>Allow thousand separator(<code>,</code>) in user input for sending ADA and pledge/cost at pool registration to make it easier to count the zeros</li> <li>User input for files and directories now open a dialog gui to make it easier to find the correct path</li> <li>CNTools now uses and works with <code>cardano-node 1.19.0</code>, please upgrade if you're not using this version.</li> <li>Use manual calculation based on slot tip to get KES period</li> <li>Removed ledger dump dependency from Pool Register, Modify, Retire and List.</li> <li>The cost of the ledger dump is too high, replaced with a simple check if pools registration certificate exist in pool folder</li> <li>Pool &gt;&gt; Show|Delegators are now the only options dumping the ledger-state</li> <li>Wallet vkeys no longer encrypted as skeys are the only ones in need of protection</li> <li>Update command change (change applied after this release is active):</li> <li>Minor/Patch release: it will warn, backup and replace CNTools script files including cntools.config</li> <li>Major release: No change, prompt user to backup and run prereqs.sh according to instructions.</li> <li>Troubleshooting improvements:</li> <li>Split 'config in json format' and 'hasPrometheus' checks</li> <li>Output node sync stats if Shelley transition epoch is to be calculated</li> <li>Protocol parameters output check to give an improved error message</li> <li>Pool &gt;&gt; Show view updated to show modified pool values if Pool &gt;&gt; Modify has been used to update pool parameters</li> <li>The section has also been updated to make it a little bit easier to read</li> <li>Pool &gt;&gt; Delegators view also use updated pledge value if a pool modification has been registered to check if pledge is met</li> <li>Use mainnet as default, since other testnets are either retired or not being maintained :(</li> <li>Backup original files when doing upgrades, so that users do not lose their changes.</li> <li>Major update description updated</li> <li>env file update removed from minor update</li> <li>Prometheus metrics used for various functions and now required to run CNTools, enabled by default</li> <li>Changed references to ptn0 to generalize the usage</li> <li>Change CNTools changelog heading format - +1 sublevels for headings (used by newer documentation)</li> <li>Delegators previously displayed in <code>Pool &gt;&gt; Show</code> now moved to its own menu option   This is to de-clutter and because it takes time to parse this data from ledger-state</li> <li>stake.cert no longer encrypted in wallet</li> <li>Meta description now has a limit of 255 chars to match smash server limit</li> <li>ledger-state timeout increased to 60s</li> <li>Update ptn0 config to align with hydra config as much as possible, while keeping trace options on</li> <li>moved update check to be one of the first things CNTools does after start to be able to show critical changes before anything else runs.</li> <li>Parse node logs to check the transition from Byron to shelley era, and save the epoch for transition in db folder. This is required for calculating KES keys.</li> <li>Please make sure to use config files created by the prereqs.sh, or enable JSON loggers for your config.</li> <li>Update cnode.sh.templ to archive logs every time node is restart, this ensures that we're not searching for previous log history when network was changed. Network being changed would automatically deduce db folder was deleted.</li> <li>Update default network to MC3</li> </ul>"},{"location":"Scripts/cntools-changelog/#removed","title":"Removed","text":"<ul> <li><code>Pool &gt;&gt; Delegators</code> removed.</li> <li>If/when a better option than dumping and parsing ledger-state dump arise re-adding it will be considered. </li> <li>Utilize the community explorers listed at https://cardano-community.github.io/support-faq/explorers </li> <li>POOL_PLEDGECERT_FILENAME removed from config, WALLET_DELEGCERT_FILENAME is used instead for delegation cert to pool, no need to keep a separate cert in pool folder for this, its the wallet that is delegated.</li> <li>Redundant sections in guide</li> <li>Stale delegate.counter</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_63","title":"Fixed","text":"<ul> <li>Check <code>pool &gt;&gt; show</code> stake distribution showing up as always 0.</li> <li>KES expiration calculation</li> <li>Slot interval calculation</li> <li>Custom vname replacement(when using <code>prereqs.sh -t</code>) fix for internal update</li> <li>Pool registration and de-registration certificates removed in case of retire/re-registration</li> <li>KES Expiry to use KES Period instead of Epoch duration</li> <li>Block Collector script adapted for cardano-node 1.19.0.</li> <li>Block hash is now truncated in log, issue https://github.com/intersectmbo/cardano-node/issues/1738</li> <li>High cpu usage reported in a few cases when running Block Collector</li> <li>Depending on log level, parsing and byte64 enc each entry with jq could potentially put high load on weaker systems. Replaced with grep to only parse entries containing specific traces.</li> <li>Docs for creating systemd block collector service file updated to include user env in run command</li> <li>cardano-node 1.19.0 introduced an issue that required us to use KES as current - 1 while rotating.</li> <li>A new getPoolID helper function added to extract both hex and bech32 pool ID</li> <li>Added <code>--output-format hex</code> when extracting pool ID in hex format</li> <li>A new pool.id-bech32 file gets created if cold.vkey is available and decrypted</li> <li>Added error check to see if cardano-cli is in $PATH before continuing.</li> <li>Backup &amp; Restore paths were failing on machines due to alnum class availability on certain interpreters.</li> <li>Rewards were not counted in stake and pledge</li> <li>Removed +i file locking on .addr files when using <code>Wallet &gt;&gt; Encrypt</code> as these are re-generated from keys and need to be writable</li> <li>Balance check added to <code>Funds &gt;&gt; Withdraw</code> for base address as this is used to pay the withdraw transaction fee</li> <li>Resolve issue with Multi Owner causing an error with new pool registration (error was due to quotes)</li> <li>Mainnet uses dedicated condition for slot checks</li> <li>Timeout moved to a variable in cntools.library</li> <li>KES Calculation for current KES period and KES expiration date   Please re-check expiration date using Pool &gt;&gt; Show</li> <li>calc_slots to be network independent</li> <li>prom_host should be calculated from config file, instead of having to update a config</li> <li>Minor typo in menu</li> <li>Parse Config for virtual forks, which adds supports for MC4</li> <li>CNTools block collector fix</li> <li>column application added as a prereq, bsdmainutils/util-linux</li> <li>cntoolsBlockCollector.sh get epoch using function</li> <li>KES count was not showing up in Katip</li> <li>Funds -&gt; Delegation was broken as per recent changes in 1.17, corrected key type for delegation certificate</li> <li><code>Pool &gt;&gt; Show</code> delegator rewards parsing from ledger-state</li> <li>Slot sync format improvement</li> <li>kesExpiration function to use 17 fixed byron transition epochs </li> </ul>"},{"location":"Scripts/cntools-changelog/#500-2020-07-20","title":"[5.0.0] - 2020-07-20","text":""},{"location":"Scripts/cntools-changelog/#added_16","title":"Added","text":"<ul> <li>HASH_IDENTIFIER where applicable to differentiate between network modes for commands used, required due to legacy Byron considerations</li> <li>add ptn0-praos.json and ptn0-combinator.json to reduce confusion between formats, make prereqs default to combinator, and accept p argument to indicate praos mode.</li> <li>cardano-node 1.16.0 refers to txhash using quotes, sed them out</li> <li>show what's new at startup after update</li> <li>file size check for pool metadata file</li> <li>Add nonce in pool metadata JSON to keep registration attempts unique, avoiding one hash pointing to multiple URLs</li> <li>Change default network to <code>mainnet_candidate</code>, and add second argument (g) to run prereqs against guild network</li> <li>allow the use of pre-existing metadata from URL when registering or modifying pool</li> <li>minimum pool cost check against protocol</li> <li>Refresh option to home screen</li> <li>Ability to register multiple relay DNS A records as well as a mix of DNS A and IPv4</li> <li>Valid for pool registration and modification</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_45","title":"Changed","text":"<ul> <li>Default config switched to combinator instead of testnet</li> <li>Start maintaining seperate versions of praos and combinator config files.</li> <li>Add 10s timeout to wget commmands in case of issue</li> <li>timestamp added to pool metadata file to make every creation unique</li> <li>Cancel shortcut changed from <code>[c]</code> to <code>[Esc]</code></li> <li>Default pool cost from 256 -&gt; 400</li> <li>slotinterval calculation to include decentralisation parameter</li> <li>mainnet candidate compatible slot calculation, 17 fixed byron transition epochs (needs to be fixed for mainnet)</li> <li>Pool metadata information to copy file as-is as well as wait for keypress to make sure file is copied before proceeding with registration.</li> <li>Now use internal table builder to display previous relays</li> <li>Instead of giving relays from previous registration as default values it will now ask if you want to re-register relays exactly as before to minimize steps and complexity</li> </ul>"},{"location":"Scripts/cntools-changelog/#removed_1","title":"Removed","text":"<ul> <li>Delete cntools-updater script</li> <li>NODE_SOCKET_PATH config parameter(replaced by CARDANO_NODE_SOCKET_PATH)</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_64","title":"Fixed","text":"<ul> <li>Slots reference was mixing up for shelley testnet in absence of a combinator network</li> <li>numfmt dependency removed in favor of printf formatting</li> <li>Vkey delegation fix due to json format switch</li> <li>ADA not displayed in a couple of the wallet selection menus</li> <li>KES calculation support for both MC and Shelley Testnet</li> <li>Slot tip reference calculation for shelley testnet</li> </ul>"},{"location":"Scripts/cntools-changelog/#400-2020-07-13","title":"[4.0.0] - 2020-07-13","text":""},{"location":"Scripts/cntools-changelog/#added_17","title":"Added","text":"<ul> <li>Add PROTOCOL_IDENTIFIER and NETWORK_IDENTIFIER instead of harcoded entries for combinator v/s TPraos &amp; testnet v/s magic differentiators respectively.</li> <li>Keep both ptn0.yaml and ptn0-combinator.yaml to keep validity with mainnet-combinator</li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_46","title":"Changed","text":"<ul> <li>Revert back default for Public network to Shelley_Testnet as per https://t.me/CardanoStakePoolWorkgroup/282606</li> </ul>"},{"location":"Scripts/cntools-changelog/#300-2020-07-12","title":"[3.0.0] - 2020-07-12","text":""},{"location":"Scripts/cntools-changelog/#added_18","title":"Added","text":"<ul> <li>Basic health check data on main menu</li> <li>Epoch, time until next epoch, node tip vs calculated reference tip and a warning if node is lagging behind.</li> <li>Address era and encoding to <code>Wallet &gt;&gt; Show</code></li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_47","title":"Changed","text":"<ul> <li>Release <code>2.1.1</code> included a change to env file and thus require a major version bump.</li> <li>Modified output on Update screen slightly.</li> <li>KES calculation, now take into account the byron era and the transition period until shelley start</li> <li>Credit to Martin @ ATADA for inspiration on how to calculate this</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_65","title":"Fixed","text":"<ul> <li>Version fix to include patch version</li> </ul>"},{"location":"Scripts/cntools-changelog/#200-2020-07-12","title":"[2.0.0] - 2020-07-12","text":""},{"location":"Scripts/cntools-changelog/#added_19","title":"Added","text":"<ul> <li>Support for cardano-node 1.15.x  </li> <li>calculate-min-fee update to reflect change in 1.15.     change was required to support byron witnesses.</li> <li>gettip update as output is now json formatted</li> <li>bech32 addressing in 1.15 required changes to delegator lookup in <code>Pool &gt;&gt; Show</code></li> <li>add --cardano-mode to query parameters</li> <li>--mainnet flag for address generation</li> <li>Output verbosity   A new config parameter added for output verbosity using say function.   0 = Minimal - Show relevant information (default)   1 = Normal  - More information about whats going on behind the scene   2 = Maximal - Debug level for troubleshooting</li> <li>Improve delegators list in <code>Pool &gt;&gt; Show</code></li> <li>Identify owners delegations</li> <li>Display owner stake in red if <code>(stake + reward)</code> is below pledge (single-owner only for now)</li> <li>Display all lovelace values in floating point ADA with 6 decimals (lovelaces) using locales</li> <li>Block Collector summary view</li> <li>KES rotation notification/warning on startup and in pool list/show views</li> <li>Live stake and delegators in <code>Pool &gt;&gt; Show</code></li> <li>Changelog </li> </ul>"},{"location":"Scripts/cntools-changelog/#changed_48","title":"Changed","text":"<ul> <li>op-cert creation moved from <code>Pool &gt;&gt; New</code> to <code>Pool &gt;&gt; Register</code>.</li> <li>Output changed in various places throughout.</li> <li>Include reward in delegators stake.</li> <li>Release now include patch version in addition to major and minor version.   In-app update modified to reflect this change.</li> <li>Block Collector table view</li> <li>Various minor code improvements</li> </ul>"},{"location":"Scripts/cntools-changelog/#removed_2","title":"Removed","text":"<ul> <li>Enterprise wallet upgrade option in <code>Wallet &gt;&gt; List</code> </li> <li><code>Not a registered wallet on chain</code> information from Wallet listing</li> <li>en_US.UTF-8 locale dependency</li> </ul>"},{"location":"Scripts/cntools-changelog/#fixed_66","title":"Fixed","text":"<ul> <li>meta_json_url check</li> <li>Invalid tx_in when registering stake wallet</li> <li>Delegators rewards in <code>Pool &gt;&gt; Show</code></li> <li>Work-around awk versions that only support 32-bit integers</li> <li>Sometimes cardano-node log contain duplicate traces for the same slot at log file rollover, now filtered</li> <li>Correct nwmagic - was hardcoded to 42</li> <li>Set script locale to fix format issue</li> </ul>"},{"location":"Scripts/cntools-changelog/#100-2020-07-07","title":"[1.0.0] - 2020-07-07","text":"<ul> <li>First official major release</li> </ul>"},{"location":"Scripts/cntools-common/","title":"Common Tasks","text":"<p>Important</p> <p>Familiarize yourself with the Online workflow of creating wallets and pools on the Preview/Preprod/Guild network first. You can then move on to test the Offline Workflow. The Offline workflow means that the private keys never touch the Online node. When comfortable with both the online and offline CNTools workflow, it's time to deploy what you learnt on the mainnet.</p> <p>This chapter describes some common use-cases for wallet and pool creation when running CNTools in Online mode. CNTools contains much more functionality not described here.</p> Create Wallet <p>A wallet is needed for pledge and to pay for pool registration fee.</p> <ol> <li>Choose <code>[w] Wallet</code> and you will be presented with the following menu:   <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; WALLET\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n Wallet Management\n\n ) New         - create a new wallet\n ) Import      - import a Daedalus/Yoroi 24/25 mnemonic or Ledger/Trezor HW wallet\n ) Register    - register a wallet on chain\n ) De-Register - De-Register (retire) a registered wallet\n ) List        - list all available wallets in a compact view\n ) Show        - show detailed view of a specific wallet\n ) Remove      - remove a wallet\n ) Decrypt     - remove write protection and decrypt wallet\n ) Encrypt     - encrypt wallet keys and make all files immutable\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n Select Wallet Operation\n\n  [n] New\n  [i] Import\n  [r] Register\n  [z] De-Register\n  [l] List\n  [s] Show\n  [x] Remove\n  [d] Decrypt\n  [e] Encrypt\n  [h] Home\n</code></pre></li> <li>Choose <code>[n] New</code> to create a new wallet. <code>[i] Import</code> can also be used to import a Daedalus/Yoroi based 15 or 24 word wallet seed  </li> <li>Give the wallet a name  </li> <li>CNTools will give you the wallet address.  For example:   <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; WALLET &gt;&gt; NEW\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nName of new wallet: Test\n\nNew Wallet         : Test\nAddress            : addr_test1qpq5qjr774cyc6kxcwp060k4t4hwp42q43v35lmcg3gcycu5uwdwld5yr8m8fgn7su955zf5qahtrgljqfjfa4nr8jfsj4alxk\nEnterprise Address : addr_test1vpq5qjr774cyc6kxcwp060k4t4hwp42q43v35lmcg3gcyccuxhdka\n\nYou can now send and receive Ada using the above addresses.\nNote that Enterprise Address will not take part in staking.\nWallet will be automatically registered on chain if you\nchoose to delegate or pledge wallet when registering a stake pool.\n</code></pre></li> <li>Send some money to this wallet. Either through the faucet or have a friend send you some.</li> </ol> <ul> <li>The wallet must have funds in it before you can proceed.</li> <li>The Wallet created from here is not derived from mnemonics, please use next tab if you'd like to use a wallet that can also be accessed from Daedalus/Yoroi</li> </ul> Import Daedalus/Yoroi/HW Wallet <p>The <code>Import</code> feature of CNTools is originally based on this guide from Ilap.</p> <p>If you would like to use <code>Import</code> function to import a Daedalus/Yoroi based 15 or 24 word wallet seed, please ensure that <code>cardano-address</code> and <code>bech32</code> bineries are available in your <code>$PATH</code> environment variable:   <pre><code>bech32 --version\n1.1.0\n\ncardano-address --version\n3.5.0\n</code></pre></p> <p>If the version is not as per above, please run the latest <code>guild-deploy.sh</code> from here and rebuild <code>cardano-node</code> as instructed here.</p> <p>To import a Daedalus/Yoroi wallet to CNTools, open CNTools and select the <code>[w] Wallet</code> option, and then select the <code>[i] Import</code>, the following menu will appear:   <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; WALLET &gt;&gt; IMPORT\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n Wallet Import\n\n ) Mnemonic  - Daedalus/Yoroi 24 or 25 word mnemonic\n ) HW Wallet - Ledger/Trezor hardware wallet\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n Select Wallet operation\n\n  [m] Mnemonic\n  [w] HW Wallet\n  [h] Home\n</code></pre></p> <p>Note</p> <p>You can import Hardware wallet using <code>[w] HW Wallet</code> above, but please note that before you are able to use hardware wallet in CNTools, you need to ensure you can detect your hardware device at OS level using <code>cardano-hw-cli</code></p> <p>Select the wallet you want to import, for Daedalus / Yoroi wallets select <code>[m] Mnemonic</code>:   <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; WALLET &gt;&gt; IMPORT &gt;&gt; MNEMONIC\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nName of imported wallet: TEST\n\n24 or 15 word mnemonic(space separated):\n</code></pre> Give your wallet a name (in this case 'TEST'), and enter your mnemonic phrase. Please ensure that you **READ* through the complete notes presented by CNTools before proceeding.</p> Create Pool <p>Create the necessary pool keys.</p> <ol> <li>From the main menu select <code>[p] Pool</code> <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; POOL\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n Pool Management\n\n ) New      - create a new pool\n ) Register - register created pool on chain using a stake wallet (pledge wallet)\n ) Modify   - change pool parameters and register updated pool values on chain\n ) Retire   - de-register stake pool from chain in specified epoch\n ) List     - a compact list view of available local pools\n ) Show     - detailed view of specified pool\n ) Rotate   - rotate pool KES keys\n ) Decrypt  - remove write protection and decrypt pool\n ) Encrypt  - encrypt pool cold keys and make all files immutable\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n Select Pool Operation\n\n  [n] New\n  [r] Register\n  [m] Modify\n  [x] Retire\n  [l] List\n  [s] Show\n  [o] Rotate\n  [d] Decrypt\n  [e] Encrypt\n  [h] Home\n</code></pre></li> <li>Select <code>[n] New</code> to create a new pool  </li> <li>Give the pool a name. In our case, we call it TEST. The result should look something like this:   <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; POOL &gt;&gt; NEW\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nPool Name: TEST\n\nPool: TEST\nID (hex)    : 8d5a3510f18ce241115da38a1b2419ed82d308599c16e98caea1b4c0\nID (bech32) : pool134dr2y833n3yzy2a5w9pkfqeakpdxzzenstwnr9w5x6vqtnclue\n</code></pre></li> </ol> Register Pool <p>Register the pool on-chain.</p> <ol> <li>From the main menu select <code>[p] Pool</code> </li> <li>Select <code>[r] Register</code> </li> <li>Select the pool you just created  </li> <li>CNTools will give you prompts to set pledge, margin, cost, metadata, and relays. Enter values that are useful to you.  </li> </ol> <p>Make sure you set your pledge low enough to insure your funds in your wallet will cover pledge plus pool registration fees.  </p> <ol> <li>Select wallet to use as pledge wallet, <code>Test</code> in our case. As this is a newly created wallet, you will be prompted to continue with wallet registration. When complete and if successful, both wallet and pool will be registered on-chain.</li> </ol> <p>It will look something like this:   <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; POOL &gt;&gt; REGISTER\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nOnline mode  -  The default mode to use if all keys are available\n\nHybrid mode  -  1) Go through the steps to build a transaction file\n                2) Copy the built tx file to an offline node\n                3) Sign it using 'Sign Tx' with keys on offline node\n                   (CNTools started in offline mode '-o' without node connection)\n                4) Copy the signed tx file back to the online node and submit using 'Submit Tx'\n\nSelected value: [o] Online\n\n# Select pool\nSelected pool: TEST\n\n# Pool Parameters\npress enter to use default value\n\nPledge (in Ada, default: 50,000):\nMargin (in %, default: 7.5):\nCost (in Ada, minimum: 340, default: 340):\n\n# Pool Metadata\n\nEnter Pool's JSON URL to host metadata file - URL length should be less than 64 chars (default: https://foo.bat/poolmeta.json):\n\nEnter Pool's Name (default: TEST):\nEnter Pool's Ticker , should be between 3-5 characters (default: TEST):\nEnter Pool's Description (default: No Description):\nEnter Pool's Homepage (default: https://foo.com):\n\nOptionally set an extended metadata URL?\nSelected value: [n] No\n{\n  \"name\": \"TEST\",\n  \"ticker\": \"TEST\",\n  \"description\": \"No Description\",\n  \"homepage\": \"https://foo.com\",\n  \"nonce\": \"1613146429\"\n}\n\nPlease host file /opt/cardano/guild/priv/pool/TEST/poolmeta.json as-is at https://foo.bat/poolmeta.json\n\n# Pool Relay Registration\nSelected value: [d] A or AAAA DNS record (single)\nEnter relays's DNS record, only A or AAAA DNS records: relay.foo.com\nEnter relays's port: 6000\nAdd more relay entries?\nSelected value: [n] No\n\n# Select main owner/pledge wallet (normal CLI wallet)\nSelected wallet: Test (100,000.000000 Ada)\nWallet Test3 not registered on chain\n\nWaiting for new block to be created (timeout = 600 slots, 600s)\nINFO: press any key to cancel and return (won't stop transaction)\n\nOwner #1 : Test added!\n\nRegister a multi-owner pool (you need to have stake.vkey of any additional owner in a seperate wallet folder under $CNODE_HOME/priv/wallet)?\nSelected value: [n] No\n\nUse a separate rewards wallet from main owner?\nSelected value: [n] No\n\nWaiting for new block to be created (timeout = 600 slots, 600s)\nINFO: press any key to cancel and return (won't stop transaction)\n\nPool TEST successfully registered!\nOwner #1      : Test\nReward Wallet : Test\nPledge        : 50,000 Ada\nMargin        : 7.5 %\nCost          : 340 Ada\n\nUncomment and set value for POOL_NAME in ./env with 'TEST'\n\nINFO: Total balance in 1 owner/pledge wallet(s) are: 99,497.996518 Ada\n</code></pre></p> <ol> <li>As mentioned in the above output: Uncomment and set value for <code>POOL_NAME</code> in <code>./env</code> with 'TEST' (in our case, the <code>POOL_NAME</code> is <code>TEST</code>). The <code>cnode.sh</code> script will automatically detect whether the files required to run as a block producing node are present in the <code>$CNODE_HOME/priv/pool/&lt;POOL_NAME&gt;</code> directory.</li> </ol> Rotate KES Keys <p>The node runs with an operational certificate, generated using the KES hot key. For security reasons, the protocol asks to re-generate (or rotate) your KES key once reaching expiry. On mainnet, this expiry is in 62 cycles of 18 hours (thus, to ask for rotation quarterly), after which your node will not be able to forge valid blocks unless rotated. To be able to rotate KES keys, your cold keys files (<code>cold.skey</code>, <code>cold.vkey</code> and <code>cold.counter</code>) need to be present on the machine where you run CNTools to rotate your KES key.</p> <ol> <li> <p>To Rotate KES keys and generate the operational certificate - <code>op.cert</code>.</p> </li> <li> <p>From the main menu select <code>[p] Pool</code> </p> </li> <li>Select <code>[o] Rotate</code> </li> <li>Select the pool you just created  </li> </ol> <p>The output should look like:</p> <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; POOL &gt;&gt; ROTATE KES\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nSelect pool to rotate KES keys on\nSelected pool: TEST\n\nPool KES keys successfully updated\nNew KES start period  : 240\nKES keys will expire  : 302 - 2021-09-04 11:24:31 UTC\n\nRestart your pool node for changes to take effect\n\npress any key to return to home menu\n</code></pre> <ol> <li>Start or restart your <code>cardano-node</code>. If deployed as a <code>systemd</code> service as shown here, you can run <code>sudo systemctl restart cnode</code>.</li> <li>Ensure the node is running as a block producing (core) node.</li> </ol> <p>You can use gLiveView - the output at the top should say <code>&gt; Cardano Node - (Core - Guild)</code>.</p> <p>Alternatively, you can check the node logs in <code>$CNODE_HOME/logs/</code> to see whether the node is performing leadership checks (<code>TraceStartLeadershipCheck</code>, <code>TraceNodeIsNotLeader</code>, etc.) </p>"},{"location":"Scripts/cntools/","title":"Overview","text":"<p>Important</p> <ul> <li>Ensure the Pre-Requisites are in place before you proceed.</li> <li>The active testers for this script use Fedora/CentOS/RHEL/Ubuntu operating systems, other OS may require customisations.</li> <li>The tool uses the folder structure defined here. Everyone is free to customise, but while doing so beware that you may introduce changes that may not be tested during updates.</li> <li>Always use Preview/Preprod/Guild network first to familiarise, read the warning/messages in full, maintain your keys/backups with passwords (no one other than yourself can retrieve the funds if you make an accident), before performing actions on mainnet.</li> </ul> <p>Koios CNTools is like a swiss army knife for pool operators to simplify typical operations regarding their wallet keys and pool management. Please note that this tool only aims to simplify usual tasks for its users, but it should NOT act as an excuse to skip understanding how to manually work through things or basics of Linux operations. The skills highlighted on the home page are paramount for a stake pool operator, and so is the understanding of configuration files and network. Please ensure you've read and understood the disclaimers before proceeding.</p> <p>Visit the Changelog section to see progress and current release.</p>"},{"location":"Scripts/cntools/#overview","title":"Overview","text":"<p>The tool consist of three files.  </p> <ul> <li><code>cntools.sh</code> - the main script to launch cntools.</li> <li><code>cntools.library</code> - internal script with helper functions.</li> </ul> <p>In addition to the above files, there is also a dependency on the common <code>env</code> file. CNTools connects to your node through the configuration in the <code>env</code> file located in the same directory as the script. Customize <code>env</code> and <code>cntools.sh</code> files to your needs.</p> <p>Additionally, CNTools can integrate and enable optional functionalities based on external components:</p> <ul> <li><code>cncli.sh</code> is a companion script with optional functionalities to run on the core node (block producer) such as monitoring created blocks, calculating leader schedules and block validation.</li> <li><code>logMonitor.sh</code> is another companion script meant to be run together with the <code>cncli.sh</code> script to give a more complete picture.</li> </ul> <p>See CNCLI and Log Monitor sections for more details.</p> <p>Koios CNTools can operate in following modes:</p> <ul> <li>Online - The default mode using either a local node or Koios API to query the blockchain.</li> <li>Local <code>-n</code> - The local node is used to query the blockchain for needed data. This is the default mode when you start CNTools without parameters.</li> <li>Light <code>-l</code> - Koios query layer is used and removes the need for a local node deployment. This mode is both quicker and lighter on resources but comes with a third party dependency.</li> <li>Hybrid - When running in online mode, this option can be used in menus to create offline transaction files that can be passed to Offline CNTools to sign.</li> <li>Offline <code>-o</code> - Launches CNTools with a limited set of features. This mode does not require access to cardano-node or access to an internet connection. It is mainly used to create Wallet/Pool and access <code>Transaction &gt;&gt; Sign</code> to sign an offline transaction file created in Hybrid mode.</li> <li>Advanced <code>-a</code> - Exposes a new <code>Advanced</code> menu, which allows users to manage (create/mint/burn) new assets.</li> </ul> <p>In addition to above mentioned runtime arguments to launch CNTools in different modes, it can also be persisted by editing User Variables section within <code>cntools.sh</code> script.</p>"},{"location":"Scripts/cntools/#download-and-update","title":"Download and Update","text":"<p>The update functionality is provided from within CNTools. In case of breaking changes, please follow the prompts post-upgrade. If stuck, it's always best to re-run the latest <code>guild-deploy.sh</code> before proceeding.</p> <p>If you have not updated in a while, it is possible that you might come from a release with breaking changes. If so, please be sure to check out the upgrade instructions.</p>"},{"location":"Scripts/cntools/#navigation","title":"Navigation","text":"<p>The scripts menu supports both arrow key navigation and shortcut key selection. The character within the square brackets is the shortcut to press for quick navigation. For other selections like wallet and pool menu that don't contain shortcuts, there is a third way to navigate. Key pressed is compared to the first character of the menu option and if there is a match the selection jumps to this location. A handy way to quickly navigate a large menu. </p>"},{"location":"Scripts/cntools/#hardware-wallet","title":"Hardware Wallet","text":"<p>CNTools includes hardware wallet support since version <code>7.0.0</code> through Vacuumlabs <code>cardano-hw-cli</code> application. Initialize and update firmware/app on the device to the latest version before usage following the manufacturer instructions.</p> <p>To enable hardware support run <code>guild-deploy.sh -s w</code>. This downloads and installs Vacuumlabs <code>cardano-hw-cli</code> including <code>udev</code> configuration. When a new version of Vacuumlabs <code>cardano-hw-cli</code> is released, run <code>guild-deploy.sh -s w</code> again to update. For additional runtime options, run <code>guild-deploy.sh -h</code>.</p> Ledger <ul> <li>Supported devices: Nano S / Nano X  </li> <li>Make sure the latest cardano app is installed on the device.</li> </ul> Trezor <ul> <li>Supported devices: Model T  </li> <li>Make sure the latest firmware is installed on the device. In addition to this, install <code>Trezor Bridge</code> for your system before trying to use your Trezor device in CNTools. You can find the latest version of the bridge at https://wallet.trezor.io/#/bridge</li> </ul>"},{"location":"Scripts/cntools/#offline-workflow","title":"Offline Workflow","text":"<p>CNTools can be run in online and offline mode. At a very high level, for working with offline devices, remember that you need to use CNTools in an online node to generate a staging transaction for the desired type of transaction, and then move the staging transaction to an offline node to sign (authorize) using the signing keys on your offline node - and then bring back the signed transaction to the online node for submission to the chain. </p> <p>For the offline workflow, all the wallet and pool keys should be kept on the offline node. The backup function in CNTools has an option to create a backup without private keys (sensitive signing keys) to be transferred to online node. All other files are included in the backup to be transferred to the online node. </p> <p>Keys excluded from backup when created without private keys: Wallet - <code>payment.skey</code>, <code>stake.skey</code> Pool   - <code>cold.skey</code></p> <p>Note that setting up an offline server requires good SysOps background (you need to be aware of how to set up your server with offline mirror repository, how to transfer files across and be fairly familiar with the disk layout presented in the documentation). The <code>guild-deploy.sh</code> in its current state is not expected to run on an offline machine. Essentially, you simply need the <code>cardano-cli</code>, <code>bech32</code>, <code>cardano-address</code> binaries in your <code>$PATH</code>, OS level dependency packages [<code>jq</code>, <code>coreutils</code>, <code>pkgconfig</code>, <code>gcc-c++</code> and <code>bc</code> ], and perhaps a copy from your online <code>cnode</code> directory (to ensure you have the right <code>genesis</code>/<code>config</code> files on your offline server). We strongly recommend you to familiarise yourself with the workflow on the preview / preprod / guild networks first, before attempting on mainnet.</p> <p>Example workflow for creating a wallet and pool:</p>  sequenceDiagram     Note over Offline: Create/Import a wallet     Note over Offline: Create a new pool     Note over Offline: Rotate KES keys to generate op.cert     Note over Offline: Create a backup w/o private keys     Offline-&gt;&gt;Online: Transfer backup to online node     Note over Online: Fund the wallet base address with enough Ada     Note over Online: Register wallet using ' Wallet \u00bb Register ' in hybrid mode     Online-&gt;&gt;Offline: Transfer built tx file back to offline node     Note over Offline: Use ' Transaction &gt;&gt; Sign ' with payment.skey from wallet to sign transaction     Offline-&gt;&gt;Online: Transfer signed tx back to online node     Note over Online: Use ' Transaction &gt;&gt; Submit ' to send signed transaction to blockchain     Note over Online: Register pool in hybrid mode     loop         Offline--&gt;Online: Repeat steps to sign and submit built pool registration transaction     end     Note over Online: Verify that pool was successfully registered with ' Pool \u00bb Show '   Online mode <p>To start CNTools in Online (advanced) Mode, execute the script from the <code>$CNODE_HOME/scripts/</code> directory: <pre><code>cd $CNODE_HOME/scripts\n./cntools.sh -a\n</code></pre></p> <p>You should get a screen that looks something like this:</p> <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; Koios CNTools vX.X.X - Guild - CONNECTED &lt;&lt;\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n Main Menu    Telegram Announcement / Support channel: t.me/CardanoKoios/9759\n\n ) Wallet      - create, show, remove and protect wallets\n ) Funds       - send, withdraw and delegate\n ) Pool        - pool creation and management\n ) Transaction - Sign and Submit a cold transaction (hybrid/offline mode)\n ) Blocks      - show core node leader schedule &amp; block production statistics\n ) Backup      - backup &amp; restore of wallet/pool/config\n ) Advanced    - Developer and advanced features: metadata, multi-assets, ...\n ) Refresh     - reload home screen content\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n                                                  Epoch 276 - 3d 19:08:27 until next\n What would you like to do?                                         Node Sync: 12 :)\n\n  [w] Wallet\n  [f] Funds\n  [p] Pool\n  [t] Transaction\n  [b] Blocks\n  [u] Update\n  [z] Backup &amp; Restore\n  [a] Advanced\n  [r] Refresh\n  [q] Quit\n</code></pre> Offline mode <p>To start CNTools in Offline Mode, execute the script from the <code>$CNODE_HOME/scripts/</code> directory using the <code>-o</code> flag: <pre><code>cd $CNODE_HOME/scripts\n./cntools.sh -o\n</code></pre></p> <p>The main menu header should let you know that node is started in offline mode: <pre><code>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n &gt;&gt; Koios CNTools vX.X.X - Guild - OFFLINE &lt;&lt;\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n Main Menu    Telegram Announcement / Support channel: t.me/CardanoKoios/9759\n\n ) Wallet      - create, show, remove and protect wallets\n ) Funds       - send, withdraw and delegate\n ) Pool        - pool creation and management\n ) Transaction - Sign and Submit a cold transaction (hybrid/offline mode)\n\n ) Backup      - backup &amp; restore of wallet/pool/config\n\n ) Refresh     - reload home screen content\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n                                                  Epoch 276 - 3d 19:03:46 until next\n What would you like to do?\n\n  [w] Wallet\n  [f] Funds\n  [p] Pool\n  [t] Transaction\n  [z] Backup &amp; Restore\n  [r] Refresh\n  [q] Quit\n</code></pre></p>"},{"location":"Scripts/env/","title":"Common env","text":"<p>A common environment file called <code>env</code> is sourced by most scripts in the Guild Operators repository. This file holds common variables and functions needed by other scripts. There are several benefits to this, not having to specify duplicate settings and being able to reuse functions decreasing the risk of misconfiguration and inconsistency.</p>"},{"location":"Scripts/env/#installation","title":"Installation","text":"<p><code>env</code> file is downloaded together with the rest of the scripts when Pre-Requisites if followed and located in the <code>$CNODE_HOME/scripts/</code> directory. The file is also automatically downloaded/updated by some of the individual scripts if missing, like <code>cntools.sh</code>, <code>gLiveView.sh</code> and <code>topologyUpdater.sh</code>. All custom changes in User Variables section are untouched on updates unless a forced overwrite is selected when running <code>guild-deploy.sh</code>.</p>"},{"location":"Scripts/env/#configuration","title":"Configuration","text":"<p>Most variables can be left commented to use the automatically detected or default value. But there are some that need to be set as explained below.</p> <ul> <li><code>CNODE_PORT</code> - This is the most important variable and needs to be set. Used when launching the node through <code>cnode.sh</code> and to identify the correct process of the node.</li> <li><code>CNODE_HOME</code> - The root directory of the Cardano node holding all the files needed. Can be left commented if <code>guild-deploy.sh</code> has been run as this variable is then exported and added as a system environment variable.</li> <li><code>POOL_NAME</code> - If the node is to be started as a block producer by <code>cnode.sh</code> this variable needs to be uncommented and set. This is the name given to the pool in CNTools (not ticker), i.e. the pool directory name under <code>$CNODE_HOME/priv/pool/&lt;POOL_NAME&gt;</code></li> </ul> <p>Take your time and look through the different variables and their explanations and decide if you need/want to change the default setting. For a default deployment using <code>guild-deploy.sh</code>, the <code>CNODE_PORT</code> (all installs) and <code>POOL_NAME</code> (only block producer) should be the only variables needed to be set.</p> <pre><code>######################################\n# User Variables - Change as desired #\n# Leave as is if unsure              #\n######################################\n\n#CCLI=\"${HOME}/.local/bin/cardano-cli\"                  # Override automatic detection of path to cardano-cli executable\n#CNCLI=\"${HOME}/.local/bin/cncli\"                       # Override automatic detection of path to cncli executable (https://github.com/AndrewWestberg/cncli)\n#CNODE_HOME=\"/opt/cardano/cnode\"                        # Override default CNODE_HOME path (defaults to /opt/cardano/cnode)\nCNODE_PORT=6000                                         # Set node port\n#CONFIG=\"${CNODE_HOME}/files/config.json\"               # Override automatic detection of node config path\n#SOCKET=\"${CNODE_HOME}/sockets/node.socket\"             # Override automatic detection of path to socket\n#TOPOLOGY=\"${CNODE_HOME}/files/topology.json\"           # Override default topology.json path\n#LOG_DIR=\"${CNODE_HOME}/logs\"                           # Folder where your logs will be sent to (must pre-exist)\n#DB_DIR=\"${CNODE_HOME}/db\"                              # Folder to store the cardano-node blockchain db\n#UPDATE_CHECK=\"Y\"                                       # Check for updates to scripts, it will still be prompted before proceeding (Y|N).\n#TMP_DIR=\"/tmp/cnode\"                                   # Folder to hold temporary files in the various scripts, each script might create additional subfolders\n#EKG_HOST=127.0.0.1                                     # Set node EKG host IP\n#EKG_PORT=12788                                         # Override automatic detection of node EKG port\n#PROM_HOST=127.0.0.1                                    # Set node Prometheus host IP\n#PROM_PORT=12798                                        # Override automatic detection of node Prometheus port\n#EKG_TIMEOUT=3                                          # Maximum time in seconds that you allow EKG request to take before aborting (node metrics)\n#CURL_TIMEOUT=10                                        # Maximum time in seconds that you allow curl file download to take before aborting (GitHub update process)\n#BLOCKLOG_DIR=\"${CNODE_HOME}/guild-db/blocklog\"         # Override default directory used to store block data for core node\n#BLOCKLOG_TZ=\"UTC\"                                      # TimeZone to use when displaying blocklog - https://en.wikipedia.org/wiki/List_of_tz_database_time_zones\n#SHELLEY_TRANS_EPOCH=208                                # Override automatic detection of shelley epoch start, e.g 208 for mainnet\n#TG_BOT_TOKEN=\"\"                                        # Uncomment and set to enable telegramSend function. To create your own BOT-token and Chat-Id follow guide at:\n#TG_CHAT_ID=\"\"                                          # https://cardano-community.github.io/guild-operators/Scripts/sendalerts\n#USE_EKG=\"N\"                                            # Use EKG metrics from the node instead of Promethus. Promethus metrics(default) should yield slightly better performance\n#TIMEOUT_LEDGER_STATE=300                               # Timeout in seconds for querying and dumping ledger-state\n#IP_VERSION=4                                           # The IP version to use for push and fetch, valid options: 4 | 6 | mix (Default: 4)\n\n#WALLET_FOLDER=\"${CNODE_HOME}/priv/wallet\"              # Root folder for Wallets\n#POOL_FOLDER=\"${CNODE_HOME}/priv/pool\"                  # Root folder for Pools\n# Each wallet and pool has a friendly name and subfolder containing all related keys, certificates, ...\n#POOL_NAME=\"\"                                           # Set the pool's name to run node as a core node (the name, NOT the ticker, ie folder name)\n\n#WALLET_PAY_VK_FILENAME=\"payment.vkey\"                  # Standardized names for all wallet related files\n#WALLET_PAY_SK_FILENAME=\"payment.skey\"\n#WALLET_HW_PAY_SK_FILENAME=\"payment.hwsfile\"\n#WALLET_PAY_ADDR_FILENAME=\"payment.addr\"\n#WALLET_BASE_ADDR_FILENAME=\"base.addr\"\n#WALLET_STAKE_VK_FILENAME=\"stake.vkey\"\n#WALLET_STAKE_SK_FILENAME=\"stake.skey\"\n#WALLET_HW_STAKE_SK_FILENAME=\"stake.hwsfile\"\n#WALLET_STAKE_ADDR_FILENAME=\"reward.addr\"\n#WALLET_STAKE_CERT_FILENAME=\"stake.cert\"\n#WALLET_STAKE_DEREG_FILENAME=\"stake.dereg\"\n#WALLET_DELEGCERT_FILENAME=\"delegation.cert\"\n\n#POOL_ID_FILENAME=\"pool.id\"                             # Standardized names for all pool related files\n#POOL_HOTKEY_VK_FILENAME=\"hot.vkey\"\n#POOL_HOTKEY_SK_FILENAME=\"hot.skey\"\n#POOL_COLDKEY_VK_FILENAME=\"cold.vkey\"\n#POOL_COLDKEY_SK_FILENAME=\"cold.skey\"\n#POOL_OPCERT_COUNTER_FILENAME=\"cold.counter\"\n#POOL_OPCERT_FILENAME=\"op.cert\"\n#POOL_VRF_VK_FILENAME=\"vrf.vkey\"\n#POOL_VRF_SK_FILENAME=\"vrf.skey\"\n#POOL_CONFIG_FILENAME=\"pool.config\"\n#POOL_REGCERT_FILENAME=\"pool.cert\"\n#POOL_CURRENT_KES_START=\"kes.start\"\n#POOL_DEREGCERT_FILENAME=\"pool.dereg\"\n\n#ASSET_FOLDER=\"${CNODE_HOME}/priv/asset\"                # Root folder for Multi-Assets containing minted assets and subfolders for Policy IDs\n#ASSET_POLICY_VK_FILENAME=\"policy.vkey\"                 # Standardized names for all multi-asset related files\n#ASSET_POLICY_SK_FILENAME=\"policy.skey\"\n#ASSET_POLICY_SCRIPT_FILENAME=\"policy.script\"           # File extension '.script' mandatory\n#ASSET_POLICY_ID_FILENAME=\"policy.id\"\n</code></pre>"},{"location":"Scripts/gliveview/","title":"gLiveView","text":"<p>Reminder !!</p> <p>Ensure the Pre-Requisites are in place before you proceed.</p> <p>Koios gLiveView is a local monitoring tool to use in addition to remote monitoring tools like Prometheus/Grafana, Zabbix or IOG's RTView. This is especially useful when moving to a systemd deployment - if you haven't done so already - as it offers an intuitive UI to monitor the node status.</p>"},{"location":"Scripts/gliveview/#configuration-startup","title":"Configuration &amp; Startup","text":"<p>For most setups, it's enough to set <code>CNODE_PORT</code> in the <code>env</code> file. The rest of the variables should automatically be detected. If required, modify User Variables in <code>env</code> and <code>gLiveView.sh</code> to suit your environment (if the environment is customised). This should lead you to a stage where you can now start running <code>./gLiveView.sh</code> in the folder you downloaded the script (the default location would be <code>$CNODE_HOME/scripts</code>). Note that the script is smart enough to automatically detect when you're running as a Core or Relay and will show fields accordingly.</p> <p>The tool can be run in legacy mode with only standard ASCII characters for terminals with trouble displaying the box-drawing characters. Run <code>./gLiveView.sh -h</code> to show available command-line parameters or permanently set it directly in script.</p> <p>Note !!</p> <p>Keeping gLiveView to it's intent of being a dashboard and not a full-fledged monitoring tool, we intend to keep most relevant information for a node operator in a minimalistic dashboard, accordingly - gLiveView runs by default in compact mode. One can enable verbose mode by pressing 'v' to unhide additional fields.</p> <p>A sample output from both core and relay together with peer analysis:</p> Core <p></p> Relay <p></p> Peer Analysis <p></p>"},{"location":"Scripts/gliveview/#upper-main-section","title":"Upper main section","text":"<p>Displays live metrics from cardano-node gathered through the nodes EKG/Prometheus(env setting) endpoint.</p> <ul> <li>Epoch Progress - Epoch number and progress is live from the node while date calculation until epoch boundary is based on offline genesis parameters.  </li> <li>Block - The nodes current block height since genesis start.  </li> <li>Slot - The nodes current slot height since current epoch start.  </li> <li>Density - With the current chain parameters(MainNet), a block is created roughly every 20 seconds(<code>activeSlotsCoeff</code>). A slot on MainNet happens every 1 second(<code>slotLength</code>), thus the max chain density can be calculated as <code>slotLength/activeSlotsCoeff = 5%</code>. Normally, the value should fluctuate around this value.  </li> <li>Total Tx - The total number of transactions processed since node start.  </li> <li>Pending Tx - The number of transactions and the bytes(total, in kb) currently in mempool to be included in upcoming blocks.  </li> <li>Tip (ref) - Reference tip is an offline calculation based on genesis values for current slot height since genesis start.  </li> <li>Tip (diff) / Status - Will either show node status as <code>starting|sync xx.x%</code> or if close to reference tip, the tip difference <code>Tip (ref) - Tip (node)</code> to see how far of the tip (diff value) the node is. With current parameters a slot diff up to 40 from reference tip is considered good but it should usually stay below 30. It's perfectly normal to see big differences in slots between blocks. It's the built in randomness at play. To see if a node is really healthy and staying on tip you would need to compare the tip between multiple nodes.  </li> <li>Forks - The number of forks since node start. Each fork means the blockchain evolved in a different direction, thereby discarding blocks. A high number of forks means there is a higher chance of orphaned blocks.  </li> <li>Peers In / Out - Shows how many connections the node has established in and out. See Peer analysis section for how to get more details of incoming and outgoing connections.  </li> <li>P2P Mode </li> <li><code>Cold</code> peers indicate the number of inactive but known peers to the node.</li> <li><code>Warm</code> peers tell how many established connections the node has.</li> <li><code>Hot</code> peers how many established connections are actually active.</li> <li><code>Bi-Dir</code>(bidirectional) and <code>Uni-Dir</code>(unidirectional) indicate how the handshake protocol negotiated the connection. The connection between p2p nodes will always be bidirectional, but it will be unidirectional between p2p nodes and non-p2p nodes. </li> <li><code>Duplex</code> shows the connections that are actually used in both directions, only bidirectional connections have this potential.</li> <li>Mem (RSS) - RSS is the Resident Set Size and shows how much memory is allocated to cardano-node and that is in RAM. It does not include memory that is swapped out. It does include memory from shared libraries as long as the pages from those libraries are actually in memory. It does include all stack and heap memory.  </li> <li>Mem (Live) / (Heap) - GC (Garbage Collector) values that show how much memory is used for live/heap data. A large difference between them (or the heap approaching the physical memory limit) means the node is struggling with the garbage collector and/or may begin swapping.  </li> <li>GC Minor / Major - Collecting garbage from \"Young space\" is called a Minor GC. Major (Full) GC is done more rarily and is a more expensive operation. Explaining garbage collection is a topic outside the scope of this documentation and google is your friend for this.  </li> <li>Block propagation - Last Block measures the duration between when the last block was scheduled to be produced and when the node learned about it. Late blocks are blocks whose delay is larger than 5s. If the node is not synching, the number of late blocks needs to stay low. Within \u2153/5s estimates the chance of observing a delay of \u2153/5s (based on the delays observed for previous blocks). A healthy node needs to stay above 95% of blocks within 3s. Finally, served blocks counts how many blocks were fetched by \"in\" peers. If this does not increase for a long time, it means the \"in\" peers are learning about new blocks from somewhere else (and therefore this node is not contributing towards accelerating the propagation). Overall, these metrics are helpful in tweaking the topology and/or performance of the network links.  </li> </ul>"},{"location":"Scripts/gliveview/#core-section","title":"Core section","text":"<p>If the node is run as a core, identified by the 'forge-about-to-lead' parameter, a second core section is displayed.</p> <ul> <li>KES period / expiration - This section contain the current and remaining KES periods as well as a calculated date for the expiration. When getting close to expire date the values will change color.  </li> <li> <p>Missed slot checks - A value that show if the node have missed slots for attempting leadership checks (as absolute value and percentage since node startup).   !!! info \"Missed Slot Leadership Check\"  </p> <p>Note that while this counter should ideally be close to zero, you would often see a higher value if the node is busy (e.g. paused for garbage collection or busy with reward calculations). A consistently high percentage of missed slots would need further investigation (assistance for troubleshooting can be seeked here ), as in extremely remote cases - it can overlap with a slot that your node could be a leader for.</p> </li> <li> <p>Blocks - If CNCLI is activated to store blocks created in a blocklog DB, data from this blocklog is displayed. See linked CNCLI documentation for details regarding the different block metrics. If CNCLI is not deployed, block metrics displayed are taken from node metrics and show blocks created by the node since node start.</p> </li> </ul>"},{"location":"Scripts/gliveview/#peer-analysis","title":"Peer analysis","text":"<p>A manual peer analysis can be triggered by key press <code>p</code>. A latency test will be done on incoming and outgoing connections to the node.</p> <p>Note</p> <p>Note that with P2P enabled, an incoming/outgoing connection can be reused for bi-directional traffic. There isnt a way to distinctly identify the P2P peer's direction yet for a given IP.</p> <p>Outgoing connections(peers in topology file), ping type used is done in this order: 1. cncli - If available, this gives the most accurate measure as it checks the entire handshake process against the remote peer. 2. ss - Sends a TCP SYN package to ping the remote peer on the <code>cardano-node</code> port. Should give ~100% success rate. 2. tcptraceroute - Same as ss. 3. ping - fallback method using ICMP ping against IP. Will only work if firewall of remote peer accept ICMP traffic.  </p> <p>For incoming connections, only ICMP ping is used as remote peer port is unknown. It's not uncommon to see many undetermined peers for incoming connections as it's a good security practice to disable ICMP in firewall.</p> <p>Once the analysis is finished, it will display the RTTs (return-trip times) for the peers and group them in ranges 0-50, 50-100, 100-200, 200&lt;. The analysis is NOT live. Press <code>[h] Home</code> to go back to default view or <code>[i] Info</code> to show in-script help text. <code>Up</code> and <code>Down</code> arrow keys is used to select incoming or outgoing detailed list of IPs and their RTT value. <code>Left (&lt;)</code> and <code>Right (&gt;)</code> arrow keys can be used to navigate the pages in the selected list.</p>"},{"location":"Scripts/gliveview/#troubleshootingcustomisations","title":"Troubleshooting/Customisations","text":"<p>In case you run into trouble while running the script, you might want to edit <code>env</code> &amp; <code>gLiveView.sh</code> and look at User Variables section. You can override the values if the automatic detection do not provide the right information, but we would appreciate if you could also notify us by raising an issue against the GitHub repository:</p> <p>gLiveView.sh <pre><code>######################################\n# User Variables - Change as desired #\n######################################\n\nNODE_NAME=\"Cardano Node\"                  # Change your node's name prefix here, keep at or below 19 characters!\nREFRESH_RATE=2                            # How often (in seconds) to refresh the view (additional time for processing and output may slow it down)\nLEGACY_MODE=false                         # (true|false) If enabled unicode box-drawing characters will be replaced by standard ASCII characters\nRETRIES=3                                 # How many attempts to connect to running Cardano node before erroring out and quitting\nPEER_LIST_CNT=6                           # Number of peers to show on each in/out page in peer analysis view\nTHEME=\"dark\"                              # dark  = suited for terminals with a dark background\n# light = suited for terminals with a bright background\nENABLE_IP_GEOLOCATION=\"Y\"                 # Enable IP geolocation on outgoing and incoming connections using ip-api.com\n</code></pre></p>"},{"location":"Scripts/itnrewards/","title":"Itnrewards","text":""},{"location":"Scripts/itnrewards/#concept","title":"Concept","text":"<p>To claim rewards earned during the Incentivized TestNet the private and public keys from ITN must be converted to Shelley stake keys. A script called <code>itnRewards.sh</code> has been created to guide you through the process of converting the keys and to create a CNTools compatible wallet from were the rewards can be withdrawn. </p> graph TB     A([\"itnRewards.sh\"])     A --x B([\"ITN Owner skey (ed25519[e]_sk)..\"]) --x D([\"cardano-cli shelley key convert-itn-key ..\"])     A --x C([\"ITN Owner vkey (ed25519_pk)..\"]) --x D     D --x E([\"Stake skey/vkey\"]) --x L     A --x F([\"cardano-cli shelley ..\"])     F --x G([\"Payment skey/vkey/addr\"]) --x L     F --x H([\"Reward addr\"]) --x L     F --x I([\"Base addr\"]) --x L     L[CNTools Wallet]     ;"},{"location":"Scripts/itnrewards/#steps","title":"Steps","text":"<ul> <li>If the secret key used for <code>jcli</code> account in ITN was ed25519_sk (not extended), you can run the <code>itnRewards.sh</code> script providing the name for the CNTools wallet and ITN owner public/secret keys that were used to register your pool as below.   <pre><code>cd $CNODE_HOME/scripts\n./itnRewards.sh MyITNWallet ~/jormu/account/priv/owner.sk ~/jormu/account/priv/owner.pk\n</code></pre></li> <li>Start CNTools and verify that the correct balance is shown in the wallet reward address</li> <li>Fund base address of the wallet with enough funds to pay the withdraw tx fee</li> <li>Use <code>FUNDS &gt;&gt; WITHDRAW</code> to move rewards to the base address of wallet</li> <li>You can now spend/move funds as you see fit</li> </ul>"},{"location":"Scripts/itnwitness/","title":"Itnwitness","text":"<p>Disclaimer</p> <p>Currently this is to protect the existing pools from the ITN who already have a delegator base against spoofing - to avoid scammers building on results of ITN from known pools. There would be a solution in the future for Mainnet nodes too - but it doesn't apply to those in its current form.</p>"},{"location":"Scripts/itnwitness/#concept","title":"Concept","text":"<p>Due to the expected ticker spoofing attack for pools that were famous during ITN, some of the community members have proposed an interim solution to verify the legitimacy of a pool for delegators. You can check the high-level workflow below:</p> graph TB     A(\"ITN Owner skey (ed25519/ed25519e) ..\") --x C([\"jcli key sign ..\"])     B(\"Haskell Pool ID (pool.id) ..\") --x C     C --x D(\"Signature key, (pool.sig) ..\")     E(\"ITN Owner vkey (ed25519_pk) ..\") --x F(\"Extended Metadata JSON (poolmeta_extended.json) ..\")     D --x F     F --x G(\"Pool Meta JSON (poolmeta.json) ..\")     ;"},{"location":"Scripts/itnwitness/#steps","title":"Steps","text":"<p>The actual implementation is pretty straightforward, we will keep it brisk - as we assume ones participating are fairly familiar with <code>jcli</code> usage.</p> <ul> <li>You need to use your owner keys that were used to register your pool, and it should match the owner public key you presented on official cardano-foundation github while registering metadata.</li> <li>Store your pool ID in a file (eg: <code>mainnet_pool.id</code>)</li> <li>Sign the file using your owner secret key from ITN (eg: <code>owner_skey</code>) as per below:   <pre><code>jcli key sign --secret-key ~/jormu/account/priv/owner.sk $CNODE_HOME/priv/pool/TEST/pool.id --output mainnet_pool.sig\ncat mainnet_pool.sig\n# ed25519_sig1sn32v3z...d72rg7rc6gs\n</code></pre></li> <li>Add this signature and owner public key to the extended pool JSON , so that it looks like below:   <pre><code>{\n\"itn\": {\n\"owner\": \"ed25519_pk1...\",\n\"witness\": \"ed25519_sig1...\"\n}\n}\n</code></pre></li> <li>Host this signature file online at a URL with raw contents easily accessible on the internet (eg: https://my.pool.com/extended-metadata.json)</li> <li>When you register/modify a pool using CNTools, use the above mentioned URL to add to your pool metadata.</li> </ul> <p>If the process is approved to appear for wallets, we may consider providing easier alternatives. If any queries about the process, or any additions please create a git issue/PR against guild repository - to capture common queries and update instructions/help text where appropriate.</p>"},{"location":"Scripts/itnwitness/#sample-output-of-json-files-generated","title":"Sample output of JSON files generated","text":"<ul> <li> <p>Metadata JSON used for registering pool (one that will be hosted URL used to define pool, eg: https://hosting.site/poolmeta.json)   <pre><code>{\n\"name\":\"Test\",\n\"ticker\":\"TEST\",\n\"description\":\"For demo purposes only\",\n\"homepage\":\"https://hosting.site\",\n\"nonce\":\"1595816423\",\n\"extended\":\"https://hosting.site/poolmeta_extended.json\"\n}\n</code></pre></p> </li> <li> <p>Extended Metadata JSON used for hosting additional metadata  (hosted at URL referred in <code>extended</code> field above, thus - eg : https://hosting.site/poolmeta_extended.json)</p> </li> </ul> <pre><code>{\n\"itn\": {\n\"owner\": \"ed25519_pk1...\",\n\"witness\": \"ed25519_sig1...\"\n}\n}\n</code></pre>"},{"location":"Scripts/logmonitor/","title":"Log Monitor","text":"<p>Reminder !!</p> <p>Ensure the Pre-Requisites are in place before you proceed.</p> <p><code>logMonitor.sh</code> is a general purpose JSON log monitoring script for traces created by <code>cardano-node</code>. Currently, it looks for traces related to leader slots and block creation but other uses could be added in the future. </p>"},{"location":"Scripts/logmonitor/#block-traces","title":"Block traces","text":"<p>For the core node (block producer) the <code>logMonitor.sh</code> script can be run to monitor the JSON log file created by <code>cardano-node</code> for traces related to leader slots and block creation.   </p> <p>For optimal coverage, it's best run together with CNCLI scripts as they provide different functionalities. Together, they create a complete picture of blocks assigned, created, validated or invalidated due to node issues. </p>"},{"location":"Scripts/logmonitor/#installation","title":"Installation","text":"<p>The script is best run as a background process. This can be accomplished in many ways but the preferred method is to run it as a systemd service. A terminal multiplexer like tmux or screen could also be used but not covered here.</p> <p>Use the <code>deploy-as-systemd.sh</code> script to create a systemd unit file (deployed together with CNCLI). Log output is handled by journald. <code>journalctl -f -u cnode-logmonitor.service</code> can be used to check service output (follow mode). Other logging configurations are not covered here.  </p>"},{"location":"Scripts/logmonitor/#view-blocklog","title":"View Blocklog","text":"<p>Best viewed in CNTools or gLiveView. See CNCLI for example output.</p>"},{"location":"Scripts/mithril-client/","title":"Client","text":"<p><code>mithril-client.sh</code> is a script to manage the Mithril client, a tool used to set up the Mithril client environment and manage downloading Mithril snapshots and stake distributions. The main features include:</p> <ul> <li>environment - Creates a new <code>mithril.env</code> file with all the necessary environment variables for the Mithril client.</li> <li>cardano-db - Download, list all or show a specific available Mithril snapshot.</li> <li>stake-distribution - Download or list available Mithril stake distributions.</li> <li>-u - Skip script update check.</li> </ul>"},{"location":"Scripts/mithril-client/#usage","title":"Usage","text":"<pre><code>Usage: bash [-u] &lt;command&gt; &lt;subcommand&gt; [&lt;sub arg&gt;]\nA script to run Cardano Mithril Client\n\n-u          Skip script update check overriding UPDATE_CHECK value in env (must be first argument to script)\n\nCommands:\nenvironment           Manage mithril environment file\n  setup               Setup mithril environment file\n  override            Override default variable in the mithril environment file\n  update              Update mithril environment file\ncardano-db            Interact with Cardano DB\n  download            Download Cardano DB from Mithril snapshot\n  snapshot            Interact with Mithril snapshots\n    list              List available Mithril snapshots\n      json            List availble Mithril snapshots in JSON format\n    show              Show details of a Mithril snapshot\n      json            Show details of a Mithril snapshot in JSON format\nstake-distribution    Interact with Mithril stake distributions\n  download            Download latest stake distribution\n  list                List available stake distributions\n    json              Output latest Mithril snapshot in JSON format\n</code></pre>"},{"location":"Scripts/mithril-client/#preparing-a-relay-or-block-producer-node","title":"Preparing a Relay or Block Producer Node","text":"<p>To prepare a relay or block producer node, you should follow these steps:</p> <ol> <li>Create the Mithril environment file: Run the script with the <code>environment setup</code> command. This will create a new <code>mithril.env</code> file with all the necessary environment variables for the Mithril client.</li> </ol> <pre><code>./mithril-client.sh environment setup\n</code></pre> <ol> <li>Download the latest Mithril snapshot: Once the environment file is set up, you can download the latest Mithril snapshot by running the script with the <code>snapshot download</code> command. This snapshot contains the latest state of the Cardano blockchain db from a Mithril Aggregator.</li> </ol> <pre><code>./mithril-client.sh cardano-db download\n</code></pre>"},{"location":"Scripts/mithril-client/#investigating-available-snapshots","title":"Investigating Available Snapshots","text":"<p>You can investigate the available snapshots by using the <code>snapshot list</code> and <code>snapshot show</code> commands:</p> <ul> <li>List all available Mithril snapshots: You can list all available Mithril snapshots by running the script with the <code>snapshot list</code> command. Add <code>json</code> at the end to get the output in JSON format.</li> </ul> <pre><code>./mithril-client.sh cardano-dbsnapshot list\n./mithril-client.sh cardano-dbsnapshot list json\n</code></pre> <ul> <li>Show details of a specific Mithril snapshot: You can show details of a specific Mithril snapshot by running the script with the <code>snapshot show &lt;DIGEST&gt;</code> command, where <code>&lt;DIGEST&gt;</code> is the digest of the snapshot. Add <code>json</code> at the end to get the output in JSON format.</li> </ul> <pre><code>./mithril-client.sh cardano-dbsnapshot show &lt;DIGEST&gt;\n./mithril-client.sh cardano-dbsnapshot show &lt;DIGEST&gt; json\n./mithril-client.sh cardano-dbsnapshot show json &lt;DIGEST&gt;\n</code></pre>"},{"location":"Scripts/mithril-client/#managing-stake-distributions","title":"Managing Stake Distributions","text":"<p>You can manage stake distributions by using the <code>stake-distribution download</code> and <code>stake-distribution list</code> commands:</p> <ul> <li>Download the latest Mithril stake distribution: You can download the latest Mithril stake distribution by running the script with the <code>stake-distribution download</code> command.</li> </ul> <pre><code>./mithril-client.sh stake-distribution download\n</code></pre> <ul> <li>List all available Mithril stake distributions: You can list all available Mithril stake distributions by running the script with the <code>stake-distribution list</code> command. Add <code>json</code> at the end to get the output in JSON format.</li> </ul> <pre><code>./mithril-client.sh stake-distribution list\n./mithril-client.sh stake-distribution list json\n</code></pre>"},{"location":"Scripts/mithril-relay/","title":"Relay","text":"<p><code>mithril-relay.sh</code> is a bash script for deployment of Squid Mithril Relays and a Nginx loadbalancer. It provides functionalities such as:</p> <ul> <li>Installing and configuring Squid as a relay for a Cardano Block Producer.</li> <li>Installing and configuring Nginx as a load balancer for multiple Mithril Relays.</li> </ul>"},{"location":"Scripts/mithril-relay/#usage","title":"Usage","text":"<pre><code>bash [-d] [-l] [-u] [-h]\nA script to setup Cardano Mithril relays\n\n-d  Install squid and configure as a relay\n-l  Install nginx and configure as a load balancer\n-u  Skip update check\n-h  Show this help text\n</code></pre>"},{"location":"Scripts/mithril-relay/#description","title":"Description","text":"<p>The <code>mithril-relay.sh</code> script is a bash script for managing the Mithril Relay Server. It provides functionalities such as installing and configuring Squid as a relay, installing and configuring Nginx as a load balancer.</p>"},{"location":"Scripts/mithril-relay/#environment-variables","title":"Environment Variables","text":"<p>The script uses the following environment variable:</p> <ul> <li><code>RELAY_LISTENING_PORT</code>: The port on which the relay server listens.</li> </ul>"},{"location":"Scripts/mithril-relay/#execution","title":"Execution","text":"<p>The script parses command line options and performs the corresponding actions based on the options provided. If the <code>-d</code> option is provided, it installs Squid and configures it as a relay. If the <code>-l</code> option is provided, it installs Nginx and configures it as a load balancer. If no options are provided, it displays the usage message.</p>"},{"location":"Scripts/mithril-signer/","title":"Signer","text":"<p><code>mithril-signer.sh</code> is a bash script for managing the Mithril Signer Server. It provides functionalities such as deploying the server as a systemd service and updating the environment file to contain variables specific to the Mithril Signer.</p>"},{"location":"Scripts/mithril-signer/#usage","title":"Usage","text":"<pre><code>Usage: bash [-d] [-D] [-e] [-k] [-r] [-s] [-u] [-h]\nA script to setup, run and verify Cardano Mithril Signer\n\n-d    Deploy mithril-signer as a systemd service\n-D    Run mithril-signer as a daemon\n-e    Update mithril environment file\n-k    Stop signer using SIGINT\n-r    Verify signer registration\n-s    Verify signer signature\n-u    Skip update check\n-h    Show this help text\n</code></pre>"},{"location":"Scripts/mithril-signer/#description","title":"Description","text":"<p>This script is a bash script for managing the Mithril Signer Server. It provides functionalities such as deploying the server as a systemd service, updating the environment file, and running the server.</p>"},{"location":"Scripts/mithril-signer/#environment-variables","title":"Environment Variables","text":"<p>The script uses several environment variables, some of which are:</p> <ul> <li><code>MITHRILBIN</code>: Path for mithril-signer binary, if not in <code>$PATH</code>.</li> <li><code>HOSTADDR</code>: Default Listen IP/Hostname for Mithril Signer Server.</li> <li><code>POOL_NAME</code>: The name of the pool.</li> <li><code>NETWORK_NAME</code>: The name of the network.</li> </ul>"},{"location":"Scripts/mithril-signer/#execution","title":"Execution","text":"<p>The script parses command line options, sources the environment file, sets default values, and performs basic sanity checks. It then checks if the <code>-d</code> or <code>-u</code> options were specified and performs the corresponding actions. If no options were specified, it runs the Mithril Signer Server.</p>"},{"location":"Scripts/sendalerts/","title":"Sendalerts","text":"<p>!&gt; Ensure the Pre-Requisites are in place before you proceed.</p> <p>This section describes the ways in which CNTools can send important messages to the operator.</p>"},{"location":"Scripts/sendalerts/#telegram-alerts","title":"Telegram alerts","text":"<p>If known but unwanted errors occur on your node, or if characteristic values indicate an unusual status , CNTools can send you Telegram alert messages. </p> <p>To do this, you first have to activate your own bot and link it to your own Telegram user. Here is an explanation of how this works:</p> <ol> <li> <p>Open Telegram and search for \"botfather\".</p> </li> <li> <p>Write him your wish: <code>/newbot</code>.</p> </li> <li> <p>Define a name for your bot, such as <code>cntools_[POOLNAME]_alerts</code>.</p> </li> <li> <p>Botfather will confirm the creation of your bot by giving you the unique bot access token. Keep it safe and private.</p> </li> <li> <p>Now send at least one direct message to your new bot.</p> </li> <li> <p>Open this URL in your browser by using your own, just created bot access token:</p> </li> </ol> <pre><code>https://api.telegram.org/bot&lt;your-access-token&gt;/getUpdates\n</code></pre> <ol> <li>the result is a JSON. Look for the value of <code>result.message.chat.id</code>.     This chat id should be a large integer number.</li> </ol> <p>This is all you need to enable your Telegram alerts in the <code>scripts/env</code> file - uncomment and add the chat ID to the <code>TG_CHAT_ID</code> user variable in the <code>env</code> file: <pre><code>...\nTG_CHAT_ID=\"&lt;YOUR_TG_CHAT_ID&gt;\"\n...  \n</code></pre></p>"},{"location":"Scripts/topologyupdater/","title":"Topology Updater","text":"<p>Reminder !!</p> <ul> <li>Starting cardano-node release 8.9.2, it is recommended to run node with P2P config enabled. If running node with P2P, one does not require to run topology updater anymore.</li> <li>Since the network had to get along without the P2P network module for a while, it needed static topology files. This \"TopologyUpdater\" service, which was far from being perfect due to its centralization factor, was intended to be a temporary solution to allow everyone to activate their relay nodes without having to postpone and wait for manual topology completion requests.</li> <li>You should NOT set up topologyUpdater for your block producing nodes.</li> </ul> <p>The topologyUpdater shell script must be executed on the relay node as a cronjob exactly every 60 minutes. After 4 consecutive requests (3 hours) the node is considered a new relay node in listed in the topology file. If the node is turned off, it's automatically delisted after 3 hours.</p>"},{"location":"Scripts/topologyupdater/#download","title":"Download and Configure","text":"<p>If you have run guild-deploy.sh, this should already be available in your scripts folder and make this step unnecessary.</p> <p>Before the updater can make a valid request to the central topology service, it must query the current tip/blockNo from the well-synced local node. It connects to your node through the configuration in the script as well as the common <code>env</code> configuration file. Customize these files for your needs.</p> <p>To download <code>topologyUpdater.sh</code> manually, you can execute the commands below and test executing Topology Updater once (it's OK if first execution gives back an error): <pre><code>cd $CNODE_HOME/scripts\ncurl -s -o topologyUpdater.sh https://raw.githubusercontent.com/cardano-community/guild-operators/master/scripts/cnode-helper-scripts/topologyUpdater.sh\ncurl -s -o env https://raw.githubusercontent.com/cardano-community/guild-operators/master/scripts/cnode-helper-scripts/env\nchmod 750 topologyUpdater.sh\n./topologyUpdater.sh\n</code></pre></p>"},{"location":"Scripts/topologyupdater/#modify","title":"Examine and modify the variables within topologyUpdater.sh script","text":"<p>Out of the box, the scripts might come with some assumptions, that may or may not be valid for your environment. One of the common changes as an SPO would be to the complete CUSTOM_PEERS section as below to include your local relays/BP nodes (described in the How do I add my own nodes section), and any additional peers you'd like to be always available at minimum. Please do take time to update the variables in User Variables section in  <code>env</code> &amp; <code>topologyUpdater.sh</code>:</p> <pre><code>### topologyUpdater.sh\n\n######################################\n# User Variables - Change as desired #\n######################################\n\nCNODE_HOSTNAME=\"CHANGE ME\"                                # (Optional) Must resolve to the IP you are requesting from\nCNODE_VALENCY=1                                           # (Optional) for multi-IP hostnames\nMAX_PEERS=15                                              # Maximum number of peers to return on successful fetch\n#CUSTOM_PEERS=\"None\"                                      # Additional custom peers to (IP,port[,valency]) to add to your target topology.json\n# eg: \"10.0.0.1,3001|10.0.0.2,3002|relays.mydomain.com,3003,3\"\n#BATCH_AUTO_UPDATE=N                                      # Set to Y to automatically update the script if a new version is available without user interaction\n</code></pre> <p>Any customisations you add above, will be saved across future <code>guild-deploy.sh</code> executions, unless you specify the <code>-f</code> flag to overwrite completely.</p>"},{"location":"Scripts/topologyupdater/#deploy","title":"Deploy the script","text":"<p>systemd service The script can be deployed as a background service in different ways but the recommended and easiest way if guild-deploy.sh was used, is to utilize the <code>deploy-as-systemd.sh</code> script to setup and schedule the execution. This will deploy both push &amp; fetch service files as well as timers for a scheduled 60 min node alive message and cnode restart at the user set interval (default: 24 hours) when running the deploy script.</p> <ul> <li><code>cnode-tu-push.service</code>    : pushes a node alive message to Topology Updater API</li> <li><code>cnode-tu-push.timer</code>      : schedules the push service to execute once every hour</li> <li><code>cnode-tu-fetch.service</code>   : fetches a fresh topology file before the <code>cnode.service</code> file is started/restarted</li> <li><code>cnode-tu-restart.service</code> : handles the restart of <code>cardano-node</code> (<code>cnode.sh</code>)</li> <li><code>cnode-tu-restart.timer</code>   : schedules the <code>cardano-node</code> restart service, default every 24h</li> </ul> <p><code>systemctl list-timers</code> can be used to to check the push and restart service schedule.</p> <p>crontab job Another way to deploy the <code>topologyUpdater.sh</code> script is as a <code>crontab</code> job. Add the script to be executed once per hour at a minute of your choice (eg xx:25 o'clock in the example below). The example below will handle both the fetch and push in a single call to the script once an hour. In addition to the below crontab job for topologyUpdater, it's expected that you also add a scheduled restart of the relay node to pick up a fresh topology file fetched by topologyUpdater script with relays that are alive and well.</p> <pre><code>25 * * * * /opt/cardano/cnode/scripts/topologyUpdater.sh\n</code></pre>"},{"location":"Scripts/topologyupdater/#logs","title":"Logs","text":"<p>You can check the last result of push message in <code>logs/topologyUpdater_lastresult.json</code>. If deployed as systemd service, use <code>sudo journalctl -u &lt;service&gt;</code> to check output from service.</p> <p>If one of the parameters is outside the allowed ranges, invalid or missing the returned JSON will tell you what needs to be fixed.</p> <p>Don't try to execute the script more often than once per hour. It's completely useless and may lead to a temporary blacklisting.</p>"},{"location":"Scripts/topologyupdater/#why-does-my-topology-file-only-contain-iog-peers","title":"Why does my topology file only contain IOG peers?","text":"<p>Each subscribed node (4 consecutive requests) is allowed to fetch a subset of other nodes to prove loyalty/stability of the relay. Until reaching this point, your fetch calls will only return IOG peers combined with any custom peers added in USER VARIABLES section of <code>topologyUpdater.sh</code> script</p> <p>The engineers of <code>cardano-node</code> network stack suggested to use around 20 peers. More peers create unnecessary and unwanted system load and delays.</p> <p>In its default setting, topologyUpdater returns a list of 15 remote peers. </p> <p>Note that the change in topology is only effective upon restart of your node. Make sure you account for some scheduled restarts on your relays, to help onboard newer relays onto the network (as described in the systemd section).</p>"},{"location":"Scripts/topologyupdater/#how-do-i-add-my-own-relaysstatic-nodes-in-addition-to-dynamic-list-generated-by-topologyupdater","title":"How do I add my own relays/static nodes in addition to dynamic list generated by topologyUpdater?","text":"<p>Most of the Stake Pool Operators may have few preferences (own relays, close friends, etc) that they would like to add to their topology by default. This is where the <code>CUSTOM_PEERS</code> variable in <code>topologyUpdater.sh</code> comes in. You can add a list of peers in the format of: <code>hostname/IP:port[:valency]</code> here and the output <code>topology.json</code> formed will already include the custom peers that you supplied. Every custom peer is defined in the form <code>[address]:[port]</code> and optional <code>:[valency]</code> (if not specified, the valency defaults to <code>1</code>). Multiple custom peers are separated by <code>|</code>. An example of a valid <code>CUSTOM_PEERS</code> variable would be:</p> <p><pre><code>CUSTOM_PEERS=\"foo.bar.io,3001,2|198.175.21.197,6001|36.233.3.89,6000\n</code></pre> The list above would add three custom peers with the specified addresses and ports, with the first one additionally specifying the optional valency parameter (in this case <code>2</code>).</p>"},{"location":"Scripts/topologyupdater/#how-are-the-peers-for-my-topology-file-selected","title":"How are the peers for my topology file selected?","text":"<p>We calculate the distance on the Earth's surface from your node's IP to all subscribed peers. We then order the peers by distance (closest first) and start by selecting one peer. We then skip some, pick the next, skip, pick, skip, pick ... until we reach the end of the list (furthest away). The number of skipped records is calculated in a way to have the desired number of peers at the end.</p> <p>Every requesting node has its personal distance to all other nodes.</p> <p>We assume this should result in a well-distributed and interconnected peering network.</p>"},{"location":"docker/build/","title":"Build","text":""},{"location":"docker/build/#intro","title":"Intro","text":"<p>\ud83d\udca1 Docker containers are the fastest way to run a Cardano node in both \"Relay\" and \"Block-Producing\" (Pool) mode.</p>"},{"location":"docker/build/#how-to-build","title":"How to build","text":"<pre><code>docker build -t cardanocommunity/cardano-node:latest - &lt; dockerfile_bin\n</code></pre>"},{"location":"docker/build/#for-windows-users","title":"For Windows Users","text":"<p>With Powershell on Windows, you can run docker by typing the following command:</p> <pre><code>Get-Content dockerfile_bin  | docker build -t guild-operators/cardano-node:latest -\n</code></pre>"},{"location":"docker/build/#see-also","title":"See also","text":"<p>Docker Tips</p> <p>Docker Official Docs</p>"},{"location":"docker/docker/","title":"Overview","text":"<p>Running your own Cardano node has never been so fast and easy.</p> <p>But first, a kind reminder to the security aspects of running docker containers.</p>"},{"location":"docker/docker/#external-resources","title":"External resources","text":"<ul> <li>DockerHub Guild's images</li> <li>YouTube Guild's Videos</li> </ul>"},{"location":"docker/docker/#built-in-cardano-software","title":"\ud83d\udd14 Built-in Cardano software","text":"<ul> <li>cardano-address</li> <li>cardano-cli</li> <li>cardano-hw-cli</li> <li>cardano-node</li> <li>cardano-submit-api</li> <li>mithril-client</li> <li>mithril-signer</li> </ul>"},{"location":"docker/docker/#mithril","title":"Mithril","text":""},{"location":"docker/docker/#built-in-tools","title":"\ud83d\udd14 Built-in tools","text":"<ul> <li>CNTools</li> <li>gLiveView</li> <li>CNCLI</li> <li>Ogmios</li> <li>Cardano Hardware CLI</li> <li>Cardano Signer</li> <li>Monitoring ready (with EKG and Prometheus)</li> </ul>"},{"location":"docker/docker/#docker-splash-screen","title":"Docker Splash screen","text":""},{"location":"docker/docker/#cntools","title":"Cntools","text":""},{"location":"docker/docker/#gliveview","title":"gLiveView","text":""},{"location":"docker/docker/#gliveview-peers-analyzer","title":"gLiveView Peers analyzer","text":""},{"location":"docker/docker/#cncli","title":"CNCLI","text":""},{"location":"docker/docker/#strategy","title":"Guild Operators Docker strategy ( mainnet/ preview / preprod / guild)","text":"<p>Modular docker images based on Debian.</p> <p>Based on the Guild's work the Cardano Node image is built in a single stage: -&gt; dockerfile_bin</p> <ul> <li>Uses <code>guild-deploy.sh</code> to:</li> <li>Install the os prerequisites</li> <li>Add the cardano software from release binaries</li> <li>Add the guild's SPO tools and the node's configuration files.</li> </ul>"},{"location":"docker/docker/#additional-docs","title":"Additional docs","text":"<p>If you prefer to build the images your own than you can check:</p> <ul> <li>Docker Build Documentation</li> <li>Docker Tips</li> </ul>"},{"location":"docker/docker/#port-mapping","title":"Port mapping","text":"<p>The dockerfiles are located in ./files/docker/</p> Node Ports Wallet Ports Flavor Node  (6000) Wallet (8090) Debian Prometheus (12798) Prometheus (12798) EKG (12781)"},{"location":"docker/run/","title":"Run","text":""},{"location":"docker/run/#os-requirements","title":"OS Requirements","text":"<ul> <li><code>docker-ce</code> installed - Get Docker.</li> </ul> Private mode Public mode <p>Note</p> <p>1) <code>--entrypoint=bash</code> # This option won't start the node's container but only the OS running (the node software wont actually start, you'll need to manually execute entrypoint.sh ), ready to get in (trough the command <code>docker exec -it &lt; container name or hash &gt;  /bin/bash</code>) and play/explore around with it in command line mode. 2) all guild tools env variable can be used to start a new container using custom values by using the \"-e\" option. 3) CPU and RAM and Shared Memory allocation option for the container can be used when you start the container (i.e. --shm-size or --memory or --cpus official docker resource docs) 4) <code>--env MITHRIL_DOWNLOAD=Y</code> # This option will allow Mithril client to download the latest Mithril snapshot of the blockchain when the container starts and does not have a copy of the blockchain yet. This is useful when you want to start a new node from scratch and don't want to wait for the node to sync from the network. This option is not currently available for the guild network. 5) <code>--env ENTRYPOINT_PROCESS=mithril-signer.sh</code> # This option will allow the container to start the Mithril signer process instead of the node process. This is useful when you want to run a Mithril signer node and have the container setup the configuration files based on the NETWORK environment varaible.</p>"},{"location":"docker/run/#use-cases","title":"Use Cases","text":"<ul> <li>Pool Management</li> <li>Wallet Management</li> <li>Node testing</li> </ul> <pre><code>docker run --init -dit\n--name &lt;YourCName&gt;\n--security-opt=no-new-privileges\n-e NETWORK=mainnet\n-v &lt;your_custom_path&gt;:/opt/cardano/cnode/priv\n-v &lt;your_custom_db_path&gt;:/opt/cardano/cnode/db\ncardanocommunity/cardano-node\n</code></pre>"},{"location":"docker/run/#use-cases_1","title":"Use Cases:","text":"<ul> <li>Node Relay</li> </ul> <pre><code>docker run --init -dit\n--name &lt;YourCName&gt;\n--security-opt=no-new-privileges\n-e NETWORK=mainnet\n-p 6000:6000\n-v &lt;your_custom_path&gt;:/opt/cardano/cnode/priv\n-v &lt;your_custom_db_path&gt;:/opt/cardano/cnode/db\ncardanocommunity/cardano-node\n</code></pre> <ul> <li>Node Relay with custom permanent cfg by passing the env variable CONFIG    (Mapping your configuration folder as below will allow you to retain configurations if you update or delete your container)</li> </ul> <pre><code>docker run --init -dit\n--name &lt;YourCName&gt;\n--security-opt=no-new-privileges\n-e NETWORK=mainnet\n-e CONFIG=/opt/cardano/cnode/priv/&lt;your own configuration files&gt;.yml\n-p 6000:6000\n-v &lt;your_custom_path&gt;:/opt/cardano/cnode/priv\n-v &lt;your_custom_db_path&gt;:/opt/cardano/cnode/db\ncardanocommunity/cardano-node\n</code></pre>"},{"location":"docker/security/","title":"Security","text":""},{"location":"docker/security/#docker-security-best-practices","title":"Docker Security best practices","text":""},{"location":"docker/security/#intro","title":"Intro","text":"<p>On the security front, Docker developers are faced with different types of security attacks such as:</p> <ul> <li>Kernel exploits: Since the host\u2019s kernel is shared in the container, a compromised container can attack the entire host.</li> <li>Container breakouts: Caused when the user is able to escape the container namespace and interact with other processes on the host.</li> <li>Denial-of-service attacks: Occur when some containers take up enough resources to hamper the functioning of other applications.</li> <li>Poisoned images: Caused when an untrusted image is being run and a hacker is able to access application data and, potentially, the host itself.</li> </ul> <p>Docker containers are now being exploited to covertly mine for cryptocurrency, marking a shift from ransomware to cryptocurrency malware.  As with all things in security, also Docker security is a moving target \u2014 so it\u2019s helpful to have access to up-to-date information, including experience-based best practices, for securing your containerized environments.</p>"},{"location":"docker/security/#here-below-some-key-concepts","title":"Here below some key concepts:","text":"<ol> <li> <p>Use a Third-Party Security Tool Docker allows you to use containers from untrusted public repositories, which increases the need to scrutinize whether the container was created securely and whether it is free of any corrupt or malicious files. For this, use a multi-purpose security tool that gives extensive dev-to-production security controls.(keep reading below)</p> </li> <li> <p>Manage Vulnerability It is best to have a sound vulnerability management program that has multiple checks throughout the container lifecycle. Vulnerability management should incorporate quality gates to detect access issues and weaknesses for a potential exploit from dev-to-production environments.</p> </li> <li> <p>Monitor and Audit Container Activity It is vital to monitor the container ecosystem and detect suspicious activity. Container monitoring activities provide real-time reports that can help you react promptly to a security breach.</p> </li> <li> <p>Enable Docker Content Trust Docker Content Trustis a new feature incorporated into Docker 1.8. It is disabled by default, but once enabled, allows you to verify the integrity, authenticity, and publication date of all Docker images from the Docker Hub Registry.</p> </li> <li> <p>Use Docker Bench for Security You should consider Docker Bench for Security as your must-use script. Once the script is run, you will notice a lot of information regarding configuration best practices for deploying Docker containers that can be used to further secure your Docker server and containers.</p> </li> <li> <p>Resource Utilization To reduce performance impacts and denial-of-service attacks, it is a good practice to implement limits on the system resources that the containers can consume. If, for example, a web server is compromised, it helps to limit the impact to the other processes that are running on a host.</p> </li> <li> <p>RBAC RBAC is role-based access control. If you have multiple users accessing you enviroment, this is a must-have. It can be quite expensive to implement but portainer makes it super easy.</p> </li> </ol>"},{"location":"docker/security/#security-docker-best-practices","title":"Security Docker best practices:","text":""},{"location":"docker/security/#the-guild-docker-images-are-not-using-all-the-following-tips-due-to-functional-purpose","title":"The Guild Docker images are not using all the following tips due to functional purpose","text":"<p>Guild tips:</p> <ul> <li> <p><code>NEVER NEVER NEVER expose Docker API publicly!!!</code> (disabled by default)</p> </li> <li> <p>Keep Docker Host Up-to-date</p> </li> <li>Reverse uptime: containers that are frequently shut down and replaced by new container are more difficult for hackers to attack.</li> <li>Use a Firewall or Expose only the ports you need to be public.</li> <li>Use a *<code>Reverse Proxy</code></li> <li>Do not Change **<code>Docker Socket Ownership</code></li> <li>Do not <code>Run Docker Containers as Root</code></li> <li><code>Use Trusted Docker Images</code></li> <li><code>Use Privileged Mode Carefully</code> (This is usually done by adding --privileged you can use <code>--security-opt=no-new-privileges</code> instead)</li> </ul> <p>Some more general tips:</p> <ul> <li>Restrict container capabilities: <code>\"--cap-drop ALL\"</code></li> <li>Use Docker Secrets</li> <li>Change DOCKER_OPTS to ***Respect IP Table Firewall </li> <li>Control Docker Resource Usage</li> <li>Rate Limit: is quite common to mitigate brute force or denial of service attacks.</li> <li>Fail2ban: Fail2ban scans your log files and bans IP address that shows malicious intent</li> <li>Container Vulnerability Scanner</li> </ul>"},{"location":"docker/security/#notes","title":"Notes:","text":"<ul> <li>*Nginx is a very good choice as load balancer and/or reverse proxy.</li> <li>**By default the socket is owned by root user and docker group.</li> <li>*** On Ubuntu/Debian based systems, edit /etc/default/docker and add the following line: <code>DOCKER_OPTS= \"--iptables=false\"</code></li> </ul>"},{"location":"docker/tips/","title":"Tips","text":""},{"location":"docker/tips/#how-to-run-a-cardano-node-with-docker","title":"How to run a Cardano Node with Docker","text":"<p>With this quick guide you will be able to run a cardano node in seconds and also have the powerfull Koios SPO scripts built-in.</p>"},{"location":"docker/tips/#how-to-operate-interactively-within-the-container","title":"How to operate interactively within the container","text":"<p>Once executed the container as a deamon with attached tty you are then able to enter the container by using the flag <code>-dit</code> .</p> <p>While if you have a hook within the container console, use the following command (change <code>CN</code> with your container name):</p> <pre><code>docker exec -it CN bash </code></pre> <p>This command will bring you within the container bash env ready to use the Koios tools.</p>"},{"location":"docker/tips/#docker-flags-explained","title":"Docker flags explained","text":"<pre><code>\"docker build\" options explained:\n -t : option is to \"tag\" the image you can name the image as you prefer as long as you maintain the references between dockerfiles.\n\n\"docker run\" options explained:\n -d : for detach the container\n -i : interactive enabled -t : terminal session enabled\n -e : set an Env Variable\n -p : set exposed ports (by default if not specified the ports will be reachable only internally)\n--hostname : Container's hostname\n --name : Container's name\n</code></pre>"},{"location":"docker/tips/#custom-container-with-your-own-cfg","title":"Custom container with your own cfg","text":"<pre><code>docker run --init -itd  \n-name Relay                                   # Optional (recommended for quick access): set a name for your newly created container.\n-p 9000:6000                                  # Optional: to expose the internal container's port (6000) to the host &lt;IP&gt; port 9000\n-e NETWORK=mainnet                            # Mandatory: mainnet / preprod / guild-mainnet / guild\n--security-opt=no-new-privileges              # Option to prevent privilege escalations\n-v &lt;YourNetPath&gt;:/opt/cardano/cnode/sockets   # Optional: useful to share the node socket with other containers\n-v &lt;YourCfgPath&gt;:/opt/cardano/cnode/priv      # Optional: if used has to contain all the sensitive keys needed to run a node as core\n-v &lt;YourDBbk&gt;:/opt/cardano/cnode/db           # Optional: if not set a fresh DB will be downloaded from scratch\ncardanocommunity/cardano-node:latest          # Mandatory: image to run\n</code></pre> <p>Note</p> <p>To be able to use the CNTools encryption key feature you need to manually change in \"cntools.config\" ENABLE_CHATTR to \"true\" and not use the <code>--security-opt=no-new-privileges</code> docker run option.</p>"},{"location":"docker/tips/#docker-cli-managment","title":"Docker CLI managment","text":""},{"location":"docker/tips/#official","title":"Official","text":"<ul> <li>docker inspect</li> <li>docker ps</li> <li>docker ls</li> <li>docker stop</li> </ul>"},{"location":"docker/tips/#un-official-docker-managment-cli-tool","title":"Un-Official Docker managment cli tool","text":"<ul> <li>Lazydocker</li> </ul>"},{"location":"docker/tips/#docker-backups-and-restores","title":"Docker backups and restores","text":"<p>The docker container has an optional backup and restore functionality that can be used to backup the <code>/opt/cardano/cnode/db</code> directory. To have the  backup persist longer than the countainer, the backup directory should be mounted as a volume.</p> <p>[!NOTE] The backup and restore functionality is disabled by default.</p> <p>[!WARNING] Make sure adequate space exists on the host as the backup will double the space consumed by the database. </p>"},{"location":"docker/tips/#creating-a-backup","title":"Creating a Backup","text":"<p>When the container is started with the ENABLE_BACKUP environment variable set to Y the container will automatically create a backup in the <code>/opt/cardano/cnode/backup/$NETWORK-db</code> directory. The backup will be created when the container is started and if the backup directory is smaller than the db directory.</p>"},{"location":"docker/tips/#restoring-from-a-backup","title":"Restoring from a Backup","text":"<p>When the container is started with the ENABLE_RESTORE environment variable set to Y the container will automatically restore the latest backup from the <code>/opt/cardano/cnode/backup/$NETWORK-db</code> directory. The database will be restored when the container is started and if the backup directory is larger than the db directory.</p>"},{"location":"docker/tips/#configuration-update-check-functionality","title":"Configuration Update Check Functionality","text":"<p>The container now includes a static copy of each network's configuration files (Mainnet, Preprod, Preview, Sanchonet, and Guild networks). The <code>NETWORK</code> environment variable passed into the container determines which configuration files are copied into <code>$CNODE_HOME/files</code>.</p> <p>The <code>UPDATE_CHECK</code> environment variable controls whether the container updates these configuration files from GitHub before starting. By default, the container has the environment variable set to <code>UPDATE_CHECK=N</code>, meaning the container uses the configuration files it was built with. This can be overriden either persistently or dynamically.</p>"},{"location":"docker/tips/#persistently-updating-configuration-files","title":"Persistently updating configuration files","text":"<p>To always update the configuration files from GitHub, set the <code>UPDATE_CHECK</code> environment variable when creating the container by using the <code>--env</code> option, for example <code>--env UPDATE_CHECK=Y</code>.</p> <p>To always update the configuration files from a specific GitHub account, set the <code>G_ACCOUNT</code> environment variable when creating the container by using the <code>--env</code> option, for example <code>--env G_ACCOUNT=gh-fork-user</code>.</p> <p>[!NOTE] There is no way to change the environment variable of an already running container. To rollback the configuration files and scripts stop and remove the container and start it without setting the environment variable.</p>"},{"location":"docker/tips/#dynamically-updating-configuration-files","title":"Dynamically updating configuration files","text":"<p>Set an environment file during create/run using <code>--env-file=file</code>, for example <code>--env-file=/opt/cardano/cnode/.env</code>.</p> <ul> <li>When <code>UPDATE_CHECK</code> is not defined in the environment file, the container will use the built-in configs.</li> <li>When <code>UPDATE_CHECK=Y</code> is defined in the environment file the container will update configs and scripts from the   <code>cardano-community</code> GitHub repository.</li> <li>When <code>G_ACCOUNT</code> is defined in the environment file, the container will update configs and scripts from the GitHub   repository of the specified account.</li> </ul> <p>To rollback the configuration files to the built-in versions, remove the <code>UPDATE_CHECK=Y</code> or set it to <code>UPDATE_CHECK=N</code> in the environment file. The static configuration files in the container will be used, however the scripts will remain updated. If you want both the configuration files and scripts to be rolled back, you will need to stop and remove the container and create a new one.</p>"},{"location":"docker/tips/#building-images-from-forked-repositories","title":"Building Images from Forked Repositories","text":"<p>Run the Docker Image GitHub Action to build and push images to the <code>ghcr.io</code> registry.</p> <ul> <li>The <code>G_ACCOUNT</code> will be inherited from the <code>GITHUB_REPOSITORY_OWNER</code>.</li> <li>It will be all lowercase so it matches container image name requirements.</li> <li>All images not from master branch or when Testing workflow is checked will be pushed to <code>ghcr.io</code>.</li> <li>Images from the master branch will also be pushed to the <code>ghcr.io</code> registry as long as the Testing workflow remains checked.</li> </ul>"}]}
\ No newline at end of file
diff --git a/sitemap.xml b/sitemap.xml
index 441016425..fb616035b 100644
--- a/sitemap.xml
+++ b/sitemap.xml
@@ -2,192 +2,192 @@
 <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
     <url>
          <loc>https://cardano-community.github.io/guild-operators/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/basics/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/build/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/catalystf11/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/contributors/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/grest-meets/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/sidebar/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/upgrade/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Appendix/RecoverByronWallet/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Appendix/monitoring/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Appendix/postgres/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Build/dbsync/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Build/grest-changelog/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Build/grest/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Build/node-cli/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Build/offchain-metadata-tools/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Build/wallet/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Mithril/mithril-overview/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Scripts/blockperf/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Scripts/cncli/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Scripts/cntools-changelog/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Scripts/cntools-common/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Scripts/cntools/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Scripts/env/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Scripts/gliveview/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Scripts/itnrewards/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Scripts/itnwitness/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Scripts/logmonitor/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Scripts/mithril-client/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Scripts/mithril-relay/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Scripts/mithril-signer/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Scripts/sendalerts/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/Scripts/topologyupdater/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/docker/build/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/docker/docker/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/docker/run/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/docker/security/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
     <url>
          <loc>https://cardano-community.github.io/guild-operators/docker/tips/</loc>
-         <lastmod>2024-08-06</lastmod>
+         <lastmod>2024-08-15</lastmod>
          <changefreq>daily</changefreq>
     </url>
 </urlset>
\ No newline at end of file
diff --git a/sitemap.xml.gz b/sitemap.xml.gz
index 64fb01d1ad0e2caabd2d9a87c7da3412741a1979..f63dd93970116a64b36afea8db8063d849173a96 100644
GIT binary patch
delta 36
scmbQhGJ%C%zMF&Nd*t4U>~c&S87HbQXMDHuyd@*YqRM$+yJQ#`0N~3E#Q*>R

delta 36
scmbQhGJ%C%zMF&N{OU~;+2xoFc_ykaXZ*JDyd@*Y?DvLUT`~*|0NHX34gdfE