Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install rust globally #331

Merged
merged 36 commits into from
Oct 2, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
a5b0c84
install rust globally
popenta Sep 13, 2023
7250ffa
try fix env
popenta Sep 13, 2023
69fba5e
code cleanup
popenta Sep 13, 2023
0bb5667
update version
popenta Sep 15, 2023
5295378
remove code related to old rust env
popenta Sep 15, 2023
2845d07
install rust for tests
popenta Sep 15, 2023
6906afa
remove explicit rust installation for workflow
popenta Sep 18, 2023
cad2cbd
Merge branch 'main' into globally-install-rust
popenta Sep 18, 2023
47c897e
add pytest flag for showing logs
popenta Sep 18, 2023
7ef887f
set flag for pytest logs
popenta Sep 18, 2023
6c22bac
remove pytest for testing purposes
popenta Sep 21, 2023
e9542fe
debugging
popenta Sep 21, 2023
67defe7
Merge branch 'main' into globally-install-rust
popenta Sep 21, 2023
45e9698
remove python 3.8 from GH action
popenta Sep 21, 2023
362cb9b
install sc-meta on GH action
popenta Sep 21, 2023
edb1c59
set rust toolchain to nightly
popenta Sep 21, 2023
0c9aaa9
remove default sc-meta installation when installing rust
popenta Sep 21, 2023
d7681dd
fix typo
popenta Sep 21, 2023
87df9d4
add new pytest marker
popenta Sep 22, 2023
84ce195
install sc-meta in GH workflow
popenta Sep 22, 2023
b3c6a0c
remove unnecessary prints
popenta Sep 22, 2023
3cbff95
bump version
popenta Sep 22, 2023
c597e34
started refactoring
popenta Sep 22, 2023
485bfaf
Install sc-meta when installing rust
popenta Sep 25, 2023
d5999e9
skip some tests on windows
popenta Sep 25, 2023
837b794
skip test on windows
popenta Sep 25, 2023
39f1b77
install wasm-opt and twiggy using cargo
popenta Sep 26, 2023
177d263
fix resolution
popenta Sep 27, 2023
648cb4c
Merge pull request #340 from multiversx/cargo-install-wasm-opt
popenta Sep 27, 2023
340e5e1
remove clang installation
popenta Sep 27, 2023
fc8609a
fix localnet and unit tests
popenta Sep 27, 2023
8eea743
fix workflow
popenta Sep 27, 2023
ad585ea
run GH actions using newer python version & wasm-opt fixes
popenta Sep 28, 2023
851bd30
fix test assert
popenta Sep 28, 2023
d2af8f5
fixes after review
popenta Sep 28, 2023
d8bb7a4
add link to mxpy cookbook
popenta Sep 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 39 additions & 39 deletions .github/workflows/build-windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,54 +5,54 @@ name: build

on:
pull_request:
branches: [ main, feat/* ]
branches: [main, feat/*]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some unrelated whitespace changes in this file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll blame it on the formatter.

workflow_dispatch:

jobs:
build:
name: Build and Test mxpy for ${{ matrix.os }}, python ${{ matrix.python-version }}

runs-on: ${{ matrix.os }}

strategy:
matrix:
os: [windows-2019]
python-version: [3.8]
python-version: [3.11]

steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
shell: bash
run: |
python3 -m pip install --upgrade pip
pip3 install -r requirements.txt
pip3 install pytest
- name: Set github_api_token
shell: bash
run: |
mkdir ~/multiversx-sdk
export PYTHONPATH=.
python3 -m multiversx_sdk_cli.cli config new test
python3 -m multiversx_sdk_cli.cli config set github_api_token ${{ secrets.GITHUB_TOKEN }}
- name: Setup test dependencies
shell: bash
run: |
python3 -m multiversx_sdk_cli.cli deps install testwallets
- name: Run unit tests
shell: bash
run: |
export PYTHONPATH=.
pytest -m "not skip_on_windows" .
- name: Run CLI tests
shell: bash
run: |
export PROXY=https://testnet-gateway.multiversx.com
export CHAIN_ID=T
cd ./multiversx_sdk_cli/tests
source ./test_cli_tx.sh && testAll || return 1
source ./test_cli_dns.sh && testOffline || return 1
source ./test_cli_validators.sh && testAll || return 1
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
shell: bash
run: |
python3 -m pip install --upgrade pip
pip3 install -r requirements.txt
pip3 install pytest
- name: Set github_api_token
shell: bash
run: |
mkdir ~/multiversx-sdk
export PYTHONPATH=.
python3 -m multiversx_sdk_cli.cli config new test
python3 -m multiversx_sdk_cli.cli config set github_api_token ${{ secrets.GITHUB_TOKEN }}
- name: Setup test dependencies
shell: bash
run: |
python3 -m multiversx_sdk_cli.cli deps install testwallets
- name: Run unit tests
shell: bash
run: |
export PYTHONPATH=.
pytest -m "not skip_on_windows" .
- name: Run CLI tests
shell: bash
run: |
export PROXY=https://testnet-gateway.multiversx.com
export CHAIN_ID=T
cd ./multiversx_sdk_cli/tests
source ./test_cli_tx.sh && testAll || return 1
source ./test_cli_dns.sh && testOffline || return 1
source ./test_cli_validators.sh && testAll || return 1
75 changes: 37 additions & 38 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,53 +5,52 @@ name: build

on:
pull_request:
branches: [ main, feat/* ]
branches: [main, feat/*]
workflow_dispatch:

jobs:
build:
name: Build and Test mxpy for ${{ matrix.os }}, python ${{ matrix.python-version }}

runs-on: ${{ matrix.os }}

strategy:
matrix:
os: [ubuntu-latest, macos-latest]
python-version: [3.8, 3.11]
python-version: [3.11]

steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
pip3 install -r requirements.txt
pip3 install pytest
- name: Install libtinfo5
if: ${{ matrix.os != 'macos-latest' }}
run: |
sudo apt update
sudo apt install -y libtinfo5
- name: Set github_api_token
run: |
mkdir ~/multiversx-sdk
export PYTHONPATH=.
python3 -m multiversx_sdk_cli.cli config new test
python3 -m multiversx_sdk_cli.cli config set github_api_token ${{ secrets.GITHUB_TOKEN }}
- name: Setup test dependencies
run: |
python3 -m multiversx_sdk_cli.cli deps install testwallets
python3 -m multiversx_sdk_cli.cli deps install wasm-opt
- name: Run unit tests
run: |
export PYTHONPATH=.
pytest .
- name: Run CLI tests
run: |
python3 -m multiversx_sdk_cli.cli config set dependencies.vmtools.tag v1.4.60
cd ./multiversx_sdk_cli/tests
source ./test_cli_contracts.sh && testAll || return 1
source ./test_cli_dns.sh && testOffline || return 1
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
pip3 install -r requirements.txt
pip3 install pytest
- name: Install libtinfo5
if: ${{ matrix.os != 'macos-latest' }}
run: |
sudo apt update
sudo apt install -y libtinfo5
- name: Set github_api_token
run: |
mkdir ~/multiversx-sdk
export PYTHONPATH=.
python3 -m multiversx_sdk_cli.cli config new test
python3 -m multiversx_sdk_cli.cli config set github_api_token ${{ secrets.GITHUB_TOKEN }}
- name: Setup test dependencies
run: |
python3 -m multiversx_sdk_cli.cli deps install testwallets
- name: Run unit tests
run: |
export PYTHONPATH=.
pytest .
- name: Run CLI tests
run: |
python3 -m multiversx_sdk_cli.cli config set dependencies.vmtools.tag v1.4.60
cd ./multiversx_sdk_cli/tests
source ./test_cli_contracts.sh && testAll || return 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How come that we don't need to install clang / llvm, as well? No C smart contracts in our tests?

If so, we should add some tests for C contracts in the next PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we only use rust contracts in our tests

source ./test_cli_dns.sh && testOffline || return 1
4 changes: 2 additions & 2 deletions .github/workflows/install-macos.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@ on:
workflow_dispatch:

env:
BRANCH_NAME: ${{ github.head_ref || github.ref_name }}
BRANCH_NAME: ${{ github.head_ref || github.ref_name }}

jobs:
install:
runs-on: macos-latest

strategy:
matrix:
python-version: [3.8]
python-version: [3.11]

steps:
- name: Set up Python ${{ matrix.python-version }}
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/install-ubuntu.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@ on:
workflow_dispatch:

env:
BRANCH_NAME: ${{ github.head_ref || github.ref_name }}
BRANCH_NAME: ${{ github.head_ref || github.ref_name }}

jobs:
install:
runs-on: ubuntu-latest

strategy:
matrix:
python-version: [3.8]
python-version: [3.11]

steps:
- name: Set up Python ${{ matrix.python-version }}
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/install-windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@ on:
workflow_dispatch:

env:
BRANCH_NAME: ${{ github.head_ref || github.ref_name }}
BRANCH_NAME: ${{ github.head_ref || github.ref_name }}

jobs:
install:
runs-on: windows-2019

strategy:
matrix:
python-version: [3.8]
python-version: [3.11]

steps:
- name: Set up Python ${{ matrix.python-version }}
Expand Down
5 changes: 2 additions & 3 deletions multiversx_sdk_cli/cli_deps.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ def setup_parser(subparsers: Any) -> Any:
sub = cli_shared.add_command_subparser(subparsers, "deps", "install", "Install dependencies or multiversx-sdk modules.")
sub.add_argument("name", choices=choices, help="the dependency to install")
sub.add_argument("--overwrite", action="store_true", default=False, help="whether to overwrite an existing installation")
sub.add_argument("--tag", help="the tag or version to install")
bogdan-rosianu marked this conversation as resolved.
Show resolved Hide resolved
sub.set_defaults(func=install)

sub = cli_shared.add_command_subparser(subparsers, "deps", "check", "Check whether a dependency is installed.")
Expand All @@ -30,9 +29,8 @@ def setup_parser(subparsers: Any) -> Any:

def install(args: Any):
name: str = args.name
tag: str = args.tag
overwrite: bool = args.overwrite
dependencies.install_module(name, tag, overwrite)
dependencies.install_module(name, overwrite)


def check(args: Any):
Expand All @@ -42,6 +40,7 @@ def check(args: Any):
default_tag: str = config.get_dependency_tag(module.key)
tag_to_check = tag or default_tag
resolution: str = config.get_dependency_resolution(module.key)
resolution = resolution if resolution else "HOST"

logger.info(f"Checking dependency: module = {module.key}, tag = {tag_to_check}; default tag = {default_tag}, resolution = {resolution}")

Expand Down
13 changes: 2 additions & 11 deletions multiversx_sdk_cli/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -150,28 +150,19 @@ def get_defaults() -> Dict[str, Any]:
"dependencies.vmtools.urlTemplate.linux": "https://github.com/multiversx/mx-chain-vm-go/archive/{TAG}.tar.gz",
"dependencies.vmtools.urlTemplate.osx": "https://github.com/multiversx/mx-chain-vm-go/archive/{TAG}.tar.gz",
"dependencies.vmtools.urlTemplate.windows": "https://github.com/multiversx/mx-chain-vm-go/archive/{TAG}.tar.gz",
"dependencies.llvm.tag": "v9-19feb",
# ide.elrond.com will be removed, TBD if clang will still be downloaded
"dependencies.llvm.urlTemplate.linux": "https://ide.elrond.com/vendor-llvm/{TAG}/linux-amd64.tar.gz?t=19feb",
"dependencies.llvm.urlTemplate.osx": "https://ide.elrond.com/vendor-llvm/{TAG}/darwin-amd64.tar.gz?t=19feb",
"dependencies.rust.resolution": "SDK",
"dependencies.rust.tag": "nightly-2023-05-26",
"dependencies.golang.resolution": "SDK",
"dependencies.golang.tag": "go1.20.7",
"dependencies.golang.urlTemplate.linux": "https://golang.org/dl/{TAG}.linux-amd64.tar.gz",
"dependencies.golang.urlTemplate.osx": "https://golang.org/dl/{TAG}.darwin-amd64.tar.gz",
"dependencies.golang.urlTemplate.windows": "https://golang.org/dl/{TAG}.windows-amd64.zip",
"dependencies.wasm-opt.tag": "latest",
"dependencies.twiggy.tag": "latest",
"dependencies.sc-meta.tag": "latest",
"dependencies.testwallets.tag": "latest",
"dependencies.testwallets.urlTemplate.linux": "https://github.com/multiversx/mx-sdk-testwallets/archive/{TAG}.tar.gz",
"dependencies.testwallets.urlTemplate.osx": "https://github.com/multiversx/mx-sdk-testwallets/archive/{TAG}.tar.gz",
"dependencies.testwallets.urlTemplate.windows": "https://github.com/multiversx/mx-sdk-testwallets/archive/{TAG}.tar.gz",
"dependencies.wasm-opt.resolution": "SDK",
"dependencies.wasm-opt.tag": "version_112",
"dependencies.wasm-opt.urlTemplate.linux": "https://github.com/WebAssembly/binaryen/releases/download/{TAG}/binaryen-{TAG}-x86_64-linux.tar.gz",
"dependencies.wasm-opt.urlTemplate.osx": "https://github.com/WebAssembly/binaryen/releases/download/{TAG}/binaryen-{TAG}-x86_64-macos.tar.gz",
"dependencies.wasm-opt.urlTemplate.windows": "https://github.com/WebAssembly/binaryen/releases/download/{TAG}/binaryen-{TAG}-x86_64-windows.tar.gz",
"dependencies.wasm-opt.tag": "0.112.0",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

"github_api_token": "",
}

Expand Down
30 changes: 8 additions & 22 deletions multiversx_sdk_cli/dependencies/install.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,22 @@
from typing import Dict, List

from multiversx_sdk_cli import config, errors
from multiversx_sdk_cli.dependencies.modules import (CargoModule, TestWalletsModule, VMToolsModule,
DependencyModule, GolangModule, Rust, StandaloneModule, WasmOptModule)
from multiversx_sdk_cli.dependencies.modules import (DependencyModule,
GolangModule, Rust,
TestWalletsModule,
VMToolsModule)

logger = logging.getLogger("install")


def install_module(key: str, tag: str = "", overwrite: bool = False):
def install_module(key: str, overwrite: bool = False):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, we are getting rid of --tag from mxpy deps. Tags are stored within the config file.

if key == 'all':
modules = _get_implicitly_installable_deps()
modules = _get_all_deps()
else:
modules = [get_module_by_key(key)]

for module in modules:
module.install(tag, overwrite)
module.install(overwrite)


def get_module_directory(key: str) -> Path:
Expand Down Expand Up @@ -46,26 +48,10 @@ def get_deps_dict() -> Dict[str, DependencyModule]:


def _get_all_deps() -> List[DependencyModule]:
return _get_explicitly_installable_deps() + _get_implicitly_installable_deps()


def _get_explicitly_installable_deps() -> List[DependencyModule]:
return [
StandaloneModule(key="llvm", aliases=["clang", "cpp"]),
Rust(key="rust"),
GolangModule(key="golang")
]


def _get_implicitly_installable_deps() -> List[DependencyModule]:
# See: https://github.com/multiversx/mx-sdk-py-cli/pull/55

return [
GolangModule(key="golang"),
VMToolsModule(key="vmtools"),
StandaloneModule(key="mx_chain_go", repo_name="mx-chain-go", organisation="multiversx"),
StandaloneModule(key="mx_chain_proxy_go", repo_name="mx-chain-proxy-go", organisation="multiversx"),
WasmOptModule(key="wasm-opt"),
CargoModule(key="twiggy"),
TestWalletsModule(key="testwallets")
]

Expand Down
Loading
Loading