Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update README.md + fix bug introduced into ci #91

Merged
merged 1 commit into from
Jan 23, 2024
Merged

Update README.md + fix bug introduced into ci #91

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 5 additions & 9 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -467,14 +467,8 @@ jobs:
cmake --build . --target clang clang-repl --parallel ${{ env.ncpus }}
fi
cd ../../
#FIXME: Shouldn't be necessary to copy osx and Ubuntu differently
OS=$(uname -s)
if [[ "$OS" == "Darwin" ]]; then
mkdir ./cppyy-backend/llvm-project/
cp -r ./llvm-project/ ./cppyy-backend/llvm-project/
else
cp ./llvm-project/ ./cppyy-backend/
fi
mkdir ./cppyy-backend/llvm-project/
cp -r ./llvm-project/ ./cppyy-backend/llvm-project/
echo "Copy llvm-project folder to cppyy-backend for caching"

- name: Build LLVM/Cling on Windows systems if the cache is invalid
Expand Down Expand Up @@ -549,7 +543,9 @@ jobs:
cmake --build . --config Release --target clang clang-repl --parallel ${{ env.ncpus }}
}
cd ../../
cp -r .\llvm-project\ .\cppyy-backend\
mkdir .\cppyy-backend\llvm-project\
cp -r .\llvm-project\ .\cppyy-backend\llvm-project\
echo "Copy llvm-project folder to cppyy-backend for caching"

- name: Save Cache LLVM/Clang runtime build directory
uses: actions/cache/save@v3
Expand Down
259 changes: 207 additions & 52 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,96 @@
# Setup cppyy for development

## Installing the packages
### Build Instructions (Includes instructions both Unix systems and Windows)
Build instructions for CppInterOp and its dependencies are as follows. CppInterOP can be built with either Cling and Clang-REPL, so instructions will differ slightly depending on which option you would like to build, but should be clear from the section title which instructions to follow.

### Install cling
#### Clone CppInterOp and cppyy-backend
First clone the CppInterOp repository, as this contains patches that need to be applied to the subsequently cloned llvm-project repo (these patches are only applied if building CppInterOp with Clang-REPL)
```
git clone --depth=1 https://github.com/compiler-research/CppInterOp.git
```
and clone cppyy-backend repository where we will be installing the CppInterOp library
```
git clone --depth=1 https://github.com/compiler-research/cppyy-backend.git
```

Build cling with LLVM and clang:
#### Setup Clang-REPL
Clone the 17.x release of the LLVM project repository.
```
git clone --depth=1 --branch release/17.x https://github.com/llvm/llvm-project.git
cd llvm-project
```
Get the following patches required for development work. To apply these patches on Linux and MacOS execute the following command
```
git apply -v ../CppInterOp/patches/llvm/clang17-*.patch
```
and
```
cp -r ..\CppInterOp\patches\llvm\clang17* .
git apply -v clang17-1-NewOperator.patch
```
on Windows.

##### Build Clang-REPL
Clang-REPL is an interpreter that CppInterOp works alongside. Build Clang (and
Clang-REPL along with it). On Linux and MaxOS you do this by executing the following
command
```
mkdir build
cd build
cmake -DLLVM_ENABLE_PROJECTS=clang \
-DLLVM_TARGETS_TO_BUILD="host;NVPTX" \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DLLVM_USE_LINKER=lld \
-DCLANG_ENABLE_STATIC_ANALYZER=OFF \
-DCLANG_ENABLE_ARCMT=OFF \
-DCLANG_ENABLE_FORMAT=OFF \
-DCLANG_ENABLE_BOOTSTRAP=OFF \
../llvm
cmake --build . --target clang clang-repl --parallel $(nproc --all)
```
On Windows you would do this by executing the following
```
git clone --depth=1 https://github.com/root-project/cling.git
$env:ncpus = %NUMBER_OF_PROCESSORS%
mkdir build
cd build
cmake -DLLVM_ENABLE_PROJECTS=clang `
-DLLVM_TARGETS_TO_BUILD="host;NVPTX" `
-DCMAKE_BUILD_TYPE=Release `
-DLLVM_ENABLE_ASSERTIONS=ON `
-DCLANG_ENABLE_STATIC_ANALYZER=OFF `
-DCLANG_ENABLE_ARCMT=OFF `
-DCLANG_ENABLE_FORMAT=OFF `
-DCLANG_ENABLE_BOOTSTRAP=OFF `
..\llvm
cmake --build . --target clang clang-repl --parallel $env:ncpus
```
Note the 'llvm-project' directory location. On linux and MacOS you execute the following
```
cd ../
export LLVM_DIR=$PWD
cd ../
```
On Windows you execute the following
```
cd ..\
$env:LLVM_DIR= $PWD.Path
cd ..\
```

#### Build Cling and related dependencies
Besides the Clang-REPL interpreter, CppInterOp also works alongside the Cling
interpreter. Cling depends on its own customised version of `llvm-project`,
hosted under the `root-project` (see the git path below).
Use the following build instructions to build on Linux and MacOS
```
git clone https://github.com/root-project/cling.git
cd ./cling/
git checkout tags/v1.0
cd ..
git clone --depth=1 -b cling-llvm13 https://github.com/root-project/llvm-project.git
cd llvm-project
mkdir build
cd build
mkdir llvm-project/build
cd llvm-project/build
cmake -DLLVM_ENABLE_PROJECTS=clang \
-DLLVM_EXTERNAL_PROJECTS=cling \
-DLLVM_EXTERNAL_CLING_SOURCE_DIR=../../cling \
Expand All @@ -28,57 +107,144 @@ cmake --build . --target clang --parallel $(nproc --all)
cmake --build . --target cling --parallel $(nproc --all)
cmake --build . --target gtest_main --parallel $(nproc --all)
```

Note down the `llvm-project` directory location as we will need it later:
Use the following build instructions to build on Windows
```
git clone https://github.com/root-project/cling.git
cd .\cling\
git checkout tags/v1.0
cd ..
git clone --depth=1 -b cling-llvm13 https://github.com/root-project/llvm-project.git
$env:ncpus = %NUMBER_OF_PROCESSORS%
$env:PWD_DIR= $PWD.Path
$env:CLING_DIR="$env:PWD_DIR\cling"
mkdir llvm-project\build
cd llvm-project\build
cmake -DLLVM_ENABLE_PROJECTS=clang `
-DLLVM_EXTERNAL_PROJECTS=cling `
-DLLVM_EXTERNAL_CLING_SOURCE_DIR="$env:CLING_DIR" `
-DLLVM_TARGETS_TO_BUILD="host;NVPTX" `
-DCMAKE_BUILD_TYPE=Release `
-DLLVM_ENABLE_ASSERTIONS=ON `
-DCLANG_ENABLE_STATIC_ANALYZER=OFF `
-DCLANG_ENABLE_ARCMT=OFF `
-DCLANG_ENABLE_FORMAT=OFF `
-DCLANG_ENABLE_BOOTSTRAP=OFF `
../llvm
cmake --build . --target clang --parallel $env:ncpus
cmake --build . --target cling --parallel $env:ncpus
cmake --build . --target gtest_main --parallel $env:ncpus
```
Note the 'llvm-project' directory location. On linux and MacOS you execute the following
```
cd ../
export LLVM_DIR=$PWD
cd ../
```
On Windows you execute the following
```
cd ..\
$env:LLVM_DIR= $PWD.Path
cd ..\
```

### Install CppInterOp

Clone the CppInterOp repo. Build it using cling and install. Note down the path
to CppInterOp install directory. This will be referred to as `CPPINTEROP_DIR`:
#### Environment variables
Regardless of whether you are building CppInterOP with Cling or Clang-REPL you will need to define the following Envirnoment variables (as they clear for a new session, it is recommended that you also add these to your .bashrc in linux, .bash_profile if on MacOS, or profile.ps1 on Windows). On Linux and MacOS you define as follows
```
export CB_PYTHON_DIR="$PWD/cppyy-backend/python"
export CPPINTEROP_DIR="$CB_PYTHON_DIR/cppyy_backend"
export CPLUS_INCLUDE_PATH="${CPLUS_INCLUDE_PATH}:${LLVM_DIR}/llvm/include:${LLVM_DIR}/clang/include:${LLVM_DIR}/build/include:${LLVM_DIR}/build/tools/clang/include"
export PYTHONPATH=$PYTHONPATH:$CPYCPPYY_DIR:$CB_PYTHON_DIR
```
If on MacOS you will also need the following envirnoment variable defined
```
export SDKROOT=`xcrun --show-sdk-path`
```
On Windows you define as follows (assumes you have defined $env:PWD_DIR= $PWD.Path )
```
$env:CB_PYTHON_DIR="$env:PWD_DIR\cppyy-backend\python"
$env:CPPINTEROP_DIR="$env:CB_PYTHON_DIR\cppyy_backend"
$env:CPLUS_INCLUDE_PATH="$env:CPLUS_INCLUDE_PATH;$env:LLVM_DIR\llvm\include;$env:LLVM_DIR\clang\include;$env:LLVM_DIR\build\include;$env:LLVM_DIR\build\tools\clang\include"
$env:PYTHONPATH="$env:PYTHONPATH;$env:CPYCPPYY_DIR;$env:CB_PYTHON_DIR"
```

#### Build CppInterOp
Now CppInterOp can be installed. On Linux and MacOS execute
```
mkdir CppInterOp/build/
cd CppInterOp/build/
```
On Windows execute
```
export CPPINTEROP_DIR=$PWD/cppyy-backend/python/cppyy_backend
mkdir CppInterOp\build\
cd CppInterOp\build\
```

Now if you want to build CppInterOp with Clang-REPL then execute the following commands on Linux and MacOS
```
git clone https://github.com/compiler-research/CppInterOp.git
cd CppInterOp
mkdir build && cd build
CPPINTEROP_BUILD_DIR=$(PWD)
cmake -DBUILD_SHARED_LIBS=ON -DUSE_CLING=ON -DUSE_REPL=Off -DCling_DIR=$LLVM_DIR/build -DCMAKE_INSTALL_PREFIX=$CPPINTEROP_DIR ..
cmake --build . --target install
cmake -DBUILD_SHARED_LIBS=ON -DUSE_CLING=Off -DUSE_REPL=ON -DLLVM_DIR=$LLVM_DIR/build/lib/cmake/llvm -DClang_DIR=$LLVM_DIR/build/lib/cmake/clang -DCMAKE_INSTALL_PREFIX=$CPPINTEROP_DIR ..
cmake --build . --target install --parallel $(nproc --all)
```
and
```
cmake -DUSE_CLING=Off -DUSE_REPL=ON -DLLVM_DIR=$LLVM_DIR\build\lib\cmake\llvm -DClang_DIR=$LLVM_DIR\build\lib\cmake\clang -DCMAKE_INSTALL_PREFIX=$env:CPPINTEROP_DIR ..
cmake --build . --target install --parallel $env:ncpus
```
on Windows. If alternatively you would like to install CppInterOp with Cling then execute the following commands on Linux and MacOS
```
cmake -DBUILD_SHARED_LIBS=ON -DUSE_CLING=ON -DUSE_REPL=Off -DCling_DIR=$LLVM_DIR/build/tools/cling -DLLVM_DIR=$LLVM_DIR/build/lib/cmake/llvm -DClang_DIR=$LLVM_DIR/build/lib/cmake/clang -DCMAKE_INSTALL_PREFIX=$CPPINTEROP_DIR ..
cmake --build . --target install --parallel $(nproc --all)
```
and
```
cmake -DUSE_CLING=ON -DUSE_REPL=Off -DCling_DIR=$LLVM_DIR\build\tools\cling -DLLVM_DIR=$LLVM_DIR\build\lib\cmake\llvm -DClang_DIR=$LLVM_DIR\build\lib\cmake\clang -DCMAKE_INSTALL_PREFIX=$env:CPPINTEROP_DIR ..
cmake --build . --target install --parallel $env:ncpus
```

#### Testing CppInterOp
To test the built CppInterOp execute the following command in the CppInterOP build folder on Linux and MacOS
```
cmake --build . --target check-cppinterop --parallel $(nproc --all)
```
and
```
cmake --build . --target check-cppinterop --parallel $env:ncpus
```
on Windows. Now go back to the top level directory in which your building CppInterOP. On Linux and MacOS you do this by executing
```
cd ../..
```
and
```
cd ..\..
```
on Windows. Now you are in a position to install cppyy following the instructions below.

### Install cppyy-backend
#### Building and Install cppyy-backend

Clone the repo, build it and copy library files into `python/cppyy-backend` directory:

```
git clone https://github.com/compiler-research/cppyy-backend.git
cd cppyy-backend
mkdir -p python/cppyy_backend/lib build && cd build
# Install CppInterOp first to appear in python/cppyy_backend/
(cd $CPPINTEROP_BUILD_DIR && cmake --build . --target install)

mkdir -p python/cppyy_backend/lib build
cd build
cmake -DCppInterOp_DIR=$CPPINTEROP_DIR ..
cmake --build .
```
If on a linux system now execute the following command
```
cp libcppyy-backend.so ../python/cppyy_backend/lib/
```
and if on MacOS execute the following command
```
cp libcppyy-backend.dylib ../python/cppyy_backend/lib/
```

Note down the path to `cppyy-backend/python` directory as "CB_PYTHON_DIR":
Note go back to the top level build directory
```
cd ../python
export CB_PYTHON_DIR=$PWD
cd ../..
```

### Install CPyCppyy
#### Install CPyCppyy

Create virtual environment and activate it:
```
Expand All @@ -87,9 +253,9 @@ source .venv/bin/activate
```

```
git clone https://github.com/compiler-research/CPyCppyy.git
cd CPyCppyy
mkdir build && cd build
git clone --depth=1 https://github.com/compiler-research/CPyCppyy.git
mkdir CPyCppyy/build
cd CPyCppyy/build
cmake ..
cmake --build .
```
Expand All @@ -100,40 +266,29 @@ export CPYCPPYY_DIR=$PWD
cd ../..
```

### Install cppyy
#### Install cppyy

```
git clone https://github.com/compiler-research/cppyy.git
git clone --depth=1 https://github.com/compiler-research/cppyy.git
cd cppyy
python -m pip install --upgrade . --no-deps
cd ..
```

## Run cppyy
#### Run cppyy

Each time you want to run cppyy you need to:
1. Activate the virtual environment
```
source .venv/bin/activate
```
2. Add `CPYCPPYY_DIR` and `CB_PYTHON_DIR` to `PYTHONPATH`:
```
export PYTHONPATH=$PYTHONPATH:$CPYCPPYY_DIR:$CB_PYTHON_DIR
```
The `CPYCPPYY_DIR` and `CB_PYTHON_DIR` will not be set if you start a new
terminal instance so you can replace them with the actual path that they
refer to.
3. Add paths to `CPLUS_INCLUDE_PATH`
```
export CPLUS_INCLUDE_PATH="${CPLUS_INCLUDE_PATH}:${LLVM_DIR}/llvm/include:${LLVM_DIR}/clang/include:${LLVM_DIR}/build/include:${LLVM_DIR}/build/tools/clang/include"
```
Activate the virtual environment
```
source .venv/bin/activate
```

Now you can `import cppyy` in `python`
```
python -c "import cppyy"
```

## Run the tests
#### Run cppyy tests

**Follow the steps in Run cppyy.** Change to the test directory, make the library files and run pytest:
```
Expand Down
Loading