Skip to content

Commit

Permalink
Update README.md + fix bug introduced into ci (#91)
Browse files Browse the repository at this point in the history
  • Loading branch information
@mcbarton
mcbarton authored Jan 23, 2024
1 parent f2f5ff6 commit c5fea06
Show file tree
Hide file tree
Showing 2 changed files with 212 additions and 61 deletions.
14 changes: 5 additions & 9 deletions .github/workflows/ci.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -467,14 +467,8 @@ jobs:
cmake --build . --target clang clang-repl --parallel ${{ env.ncpus }}
fi
cd ../../
#FIXME: Shouldn't be necessary to copy osx and Ubuntu differently
OS=$(uname -s)
if [[ "$OS" == "Darwin" ]]; then
mkdir ./cppyy-backend/llvm-project/
cp -r ./llvm-project/ ./cppyy-backend/llvm-project/
else
cp ./llvm-project/ ./cppyy-backend/
fi
mkdir ./cppyy-backend/llvm-project/
cp -r ./llvm-project/ ./cppyy-backend/llvm-project/
echo "Copy llvm-project folder to cppyy-backend for caching"
- name: Build LLVM/Cling on Windows systems if the cache is invalid
Expand Down Expand Up @@ -549,7 +543,9 @@ jobs:
cmake --build . --config Release --target clang clang-repl --parallel ${{ env.ncpus }}
}
cd ../../
cp -r .\llvm-project\ .\cppyy-backend\
mkdir .\cppyy-backend\llvm-project\
cp -r .\llvm-project\ .\cppyy-backend\llvm-project\
echo "Copy llvm-project folder to cppyy-backend for caching"
- name: Save Cache LLVM/Clang runtime build directory
uses: actions/cache/save@v3
Expand Down
259 changes: 207 additions & 52 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,17 +1,96 @@
# Setup cppyy for development

## Installing the packages
### Build Instructions (Includes instructions both Unix systems and Windows)
Build instructions for CppInterOp and its dependencies are as follows. CppInterOP can be built with either Cling and Clang-REPL, so instructions will differ slightly depending on which option you would like to build, but should be clear from the section title which instructions to follow.

### Install cling
#### Clone CppInterOp and cppyy-backend
First clone the CppInterOp repository, as this contains patches that need to be applied to the subsequently cloned llvm-project repo (these patches are only applied if building CppInterOp with Clang-REPL)
```
git clone --depth=1 https://github.com/compiler-research/CppInterOp.git
```
and clone cppyy-backend repository where we will be installing the CppInterOp library
```
git clone --depth=1 https://github.com/compiler-research/cppyy-backend.git
```

Build cling with LLVM and clang:
#### Setup Clang-REPL
Clone the 17.x release of the LLVM project repository.
```
git clone --depth=1 --branch release/17.x https://github.com/llvm/llvm-project.git
cd llvm-project
```
Get the following patches required for development work. To apply these patches on Linux and MacOS execute the following command
```
git apply -v ../CppInterOp/patches/llvm/clang17-*.patch
```
and
```
cp -r ..\CppInterOp\patches\llvm\clang17* .
git apply -v clang17-1-NewOperator.patch
```
on Windows.

##### Build Clang-REPL
Clang-REPL is an interpreter that CppInterOp works alongside. Build Clang (and
Clang-REPL along with it). On Linux and MaxOS you do this by executing the following
command
```
mkdir build
cd build
cmake -DLLVM_ENABLE_PROJECTS=clang \
-DLLVM_TARGETS_TO_BUILD="host;NVPTX" \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DLLVM_USE_LINKER=lld \
-DCLANG_ENABLE_STATIC_ANALYZER=OFF \
-DCLANG_ENABLE_ARCMT=OFF \
-DCLANG_ENABLE_FORMAT=OFF \
-DCLANG_ENABLE_BOOTSTRAP=OFF \
../llvm
cmake --build . --target clang clang-repl --parallel $(nproc --all)
```
On Windows you would do this by executing the following
```
git clone --depth=1 https://github.com/root-project/cling.git
$env:ncpus = %NUMBER_OF_PROCESSORS%
mkdir build
cd build
cmake -DLLVM_ENABLE_PROJECTS=clang `
-DLLVM_TARGETS_TO_BUILD="host;NVPTX" `
-DCMAKE_BUILD_TYPE=Release `
-DLLVM_ENABLE_ASSERTIONS=ON `
-DCLANG_ENABLE_STATIC_ANALYZER=OFF `
-DCLANG_ENABLE_ARCMT=OFF `
-DCLANG_ENABLE_FORMAT=OFF `
-DCLANG_ENABLE_BOOTSTRAP=OFF `
..\llvm
cmake --build . --target clang clang-repl --parallel $env:ncpus
```
Note the 'llvm-project' directory location. On linux and MacOS you execute the following
```
cd ../
export LLVM_DIR=$PWD
cd ../
```
On Windows you execute the following
```
cd ..\
$env:LLVM_DIR= $PWD.Path
cd ..\
```

#### Build Cling and related dependencies
Besides the Clang-REPL interpreter, CppInterOp also works alongside the Cling
interpreter. Cling depends on its own customised version of `llvm-project`,
hosted under the `root-project` (see the git path below).
Use the following build instructions to build on Linux and MacOS
```
git clone https://github.com/root-project/cling.git
cd ./cling/
git checkout tags/v1.0
cd ..
git clone --depth=1 -b cling-llvm13 https://github.com/root-project/llvm-project.git
cd llvm-project
mkdir build
cd build
mkdir llvm-project/build
cd llvm-project/build
cmake -DLLVM_ENABLE_PROJECTS=clang \
-DLLVM_EXTERNAL_PROJECTS=cling \
-DLLVM_EXTERNAL_CLING_SOURCE_DIR=../../cling \
Expand All @@ -28,57 +107,144 @@ cmake --build . --target clang --parallel $(nproc --all)
cmake --build . --target cling --parallel $(nproc --all)
cmake --build . --target gtest_main --parallel $(nproc --all)
```

Note down the `llvm-project` directory location as we will need it later:
Use the following build instructions to build on Windows
```
git clone https://github.com/root-project/cling.git
cd .\cling\
git checkout tags/v1.0
cd ..
git clone --depth=1 -b cling-llvm13 https://github.com/root-project/llvm-project.git
$env:ncpus = %NUMBER_OF_PROCESSORS%
$env:PWD_DIR= $PWD.Path
$env:CLING_DIR="$env:PWD_DIR\cling"
mkdir llvm-project\build
cd llvm-project\build
cmake -DLLVM_ENABLE_PROJECTS=clang `
-DLLVM_EXTERNAL_PROJECTS=cling `
-DLLVM_EXTERNAL_CLING_SOURCE_DIR="$env:CLING_DIR" `
-DLLVM_TARGETS_TO_BUILD="host;NVPTX" `
-DCMAKE_BUILD_TYPE=Release `
-DLLVM_ENABLE_ASSERTIONS=ON `
-DCLANG_ENABLE_STATIC_ANALYZER=OFF `
-DCLANG_ENABLE_ARCMT=OFF `
-DCLANG_ENABLE_FORMAT=OFF `
-DCLANG_ENABLE_BOOTSTRAP=OFF `
../llvm
cmake --build . --target clang --parallel $env:ncpus
cmake --build . --target cling --parallel $env:ncpus
cmake --build . --target gtest_main --parallel $env:ncpus
```
Note the 'llvm-project' directory location. On linux and MacOS you execute the following
```
cd ../
export LLVM_DIR=$PWD
cd ../
```
On Windows you execute the following
```
cd ..\
$env:LLVM_DIR= $PWD.Path
cd ..\
```

### Install CppInterOp

Clone the CppInterOp repo. Build it using cling and install. Note down the path
to CppInterOp install directory. This will be referred to as `CPPINTEROP_DIR`:
#### Environment variables
Regardless of whether you are building CppInterOP with Cling or Clang-REPL you will need to define the following Envirnoment variables (as they clear for a new session, it is recommended that you also add these to your .bashrc in linux, .bash_profile if on MacOS, or profile.ps1 on Windows). On Linux and MacOS you define as follows
```
export CB_PYTHON_DIR="$PWD/cppyy-backend/python"
export CPPINTEROP_DIR="$CB_PYTHON_DIR/cppyy_backend"
export CPLUS_INCLUDE_PATH="${CPLUS_INCLUDE_PATH}:${LLVM_DIR}/llvm/include:${LLVM_DIR}/clang/include:${LLVM_DIR}/build/include:${LLVM_DIR}/build/tools/clang/include"
export PYTHONPATH=$PYTHONPATH:$CPYCPPYY_DIR:$CB_PYTHON_DIR
```
If on MacOS you will also need the following envirnoment variable defined
```
export SDKROOT=`xcrun --show-sdk-path`
```
On Windows you define as follows (assumes you have defined $env:PWD_DIR= $PWD.Path )
```
$env:CB_PYTHON_DIR="$env:PWD_DIR\cppyy-backend\python"
$env:CPPINTEROP_DIR="$env:CB_PYTHON_DIR\cppyy_backend"
$env:CPLUS_INCLUDE_PATH="$env:CPLUS_INCLUDE_PATH;$env:LLVM_DIR\llvm\include;$env:LLVM_DIR\clang\include;$env:LLVM_DIR\build\include;$env:LLVM_DIR\build\tools\clang\include"
$env:PYTHONPATH="$env:PYTHONPATH;$env:CPYCPPYY_DIR;$env:CB_PYTHON_DIR"
```

#### Build CppInterOp
Now CppInterOp can be installed. On Linux and MacOS execute
```
mkdir CppInterOp/build/
cd CppInterOp/build/
```
On Windows execute
```
export CPPINTEROP_DIR=$PWD/cppyy-backend/python/cppyy_backend
mkdir CppInterOp\build\
cd CppInterOp\build\
```

Now if you want to build CppInterOp with Clang-REPL then execute the following commands on Linux and MacOS
```
git clone https://github.com/compiler-research/CppInterOp.git
cd CppInterOp
mkdir build && cd build
CPPINTEROP_BUILD_DIR=$(PWD)
cmake -DBUILD_SHARED_LIBS=ON -DUSE_CLING=ON -DUSE_REPL=Off -DCling_DIR=$LLVM_DIR/build -DCMAKE_INSTALL_PREFIX=$CPPINTEROP_DIR ..
cmake --build . --target install
cmake -DBUILD_SHARED_LIBS=ON -DUSE_CLING=Off -DUSE_REPL=ON -DLLVM_DIR=$LLVM_DIR/build/lib/cmake/llvm -DClang_DIR=$LLVM_DIR/build/lib/cmake/clang -DCMAKE_INSTALL_PREFIX=$CPPINTEROP_DIR ..
cmake --build . --target install --parallel $(nproc --all)
```
and
```
cmake -DUSE_CLING=Off -DUSE_REPL=ON -DLLVM_DIR=$LLVM_DIR\build\lib\cmake\llvm -DClang_DIR=$LLVM_DIR\build\lib\cmake\clang -DCMAKE_INSTALL_PREFIX=$env:CPPINTEROP_DIR ..
cmake --build . --target install --parallel $env:ncpus
```
on Windows. If alternatively you would like to install CppInterOp with Cling then execute the following commands on Linux and MacOS
```
cmake -DBUILD_SHARED_LIBS=ON -DUSE_CLING=ON -DUSE_REPL=Off -DCling_DIR=$LLVM_DIR/build/tools/cling -DLLVM_DIR=$LLVM_DIR/build/lib/cmake/llvm -DClang_DIR=$LLVM_DIR/build/lib/cmake/clang -DCMAKE_INSTALL_PREFIX=$CPPINTEROP_DIR ..
cmake --build . --target install --parallel $(nproc --all)
```
and
```
cmake -DUSE_CLING=ON -DUSE_REPL=Off -DCling_DIR=$LLVM_DIR\build\tools\cling -DLLVM_DIR=$LLVM_DIR\build\lib\cmake\llvm -DClang_DIR=$LLVM_DIR\build\lib\cmake\clang -DCMAKE_INSTALL_PREFIX=$env:CPPINTEROP_DIR ..
cmake --build . --target install --parallel $env:ncpus
```

#### Testing CppInterOp
To test the built CppInterOp execute the following command in the CppInterOP build folder on Linux and MacOS
```
cmake --build . --target check-cppinterop --parallel $(nproc --all)
```
and
```
cmake --build . --target check-cppinterop --parallel $env:ncpus
```
on Windows. Now go back to the top level directory in which your building CppInterOP. On Linux and MacOS you do this by executing
```
cd ../..
```
and
```
cd ..\..
```
on Windows. Now you are in a position to install cppyy following the instructions below.

### Install cppyy-backend
#### Building and Install cppyy-backend

Clone the repo, build it and copy library files into `python/cppyy-backend` directory:

```
git clone https://github.com/compiler-research/cppyy-backend.git
cd cppyy-backend
mkdir -p python/cppyy_backend/lib build && cd build
# Install CppInterOp first to appear in python/cppyy_backend/
(cd $CPPINTEROP_BUILD_DIR && cmake --build . --target install)
mkdir -p python/cppyy_backend/lib build
cd build
cmake -DCppInterOp_DIR=$CPPINTEROP_DIR ..
cmake --build .
```
If on a linux system now execute the following command
```
cp libcppyy-backend.so ../python/cppyy_backend/lib/
```
and if on MacOS execute the following command
```
cp libcppyy-backend.dylib ../python/cppyy_backend/lib/
```

Note down the path to `cppyy-backend/python` directory as "CB_PYTHON_DIR":
Note go back to the top level build directory
```
cd ../python
export CB_PYTHON_DIR=$PWD
cd ../..
```

### Install CPyCppyy
#### Install CPyCppyy

Create virtual environment and activate it:
```
Expand All @@ -87,9 +253,9 @@ source .venv/bin/activate
```

```
git clone https://github.com/compiler-research/CPyCppyy.git
cd CPyCppyy
mkdir build && cd build
git clone --depth=1 https://github.com/compiler-research/CPyCppyy.git
mkdir CPyCppyy/build
cd CPyCppyy/build
cmake ..
cmake --build .
```
Expand All @@ -100,40 +266,29 @@ export CPYCPPYY_DIR=$PWD
cd ../..
```

### Install cppyy
#### Install cppyy

```
git clone https://github.com/compiler-research/cppyy.git
git clone --depth=1 https://github.com/compiler-research/cppyy.git
cd cppyy
python -m pip install --upgrade . --no-deps
cd ..
```

## Run cppyy
#### Run cppyy

Each time you want to run cppyy you need to:
1. Activate the virtual environment
```
source .venv/bin/activate
```
2. Add `CPYCPPYY_DIR` and `CB_PYTHON_DIR` to `PYTHONPATH`:
```
export PYTHONPATH=$PYTHONPATH:$CPYCPPYY_DIR:$CB_PYTHON_DIR
```
The `CPYCPPYY_DIR` and `CB_PYTHON_DIR` will not be set if you start a new
terminal instance so you can replace them with the actual path that they
refer to.
3. Add paths to `CPLUS_INCLUDE_PATH`
```
export CPLUS_INCLUDE_PATH="${CPLUS_INCLUDE_PATH}:${LLVM_DIR}/llvm/include:${LLVM_DIR}/clang/include:${LLVM_DIR}/build/include:${LLVM_DIR}/build/tools/clang/include"
```
Activate the virtual environment
```
source .venv/bin/activate
```

Now you can `import cppyy` in `python`
```
python -c "import cppyy"
```

## Run the tests
#### Run cppyy tests

**Follow the steps in Run cppyy.** Change to the test directory, make the library files and run pytest:
```
Expand Down

0 comments on commit c5fea06

Please sign in to comment.