Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[other]: Require python-scipy package #105

Open
srianvesh opened this issue Oct 4, 2023 · 7 comments
Open

[other]: Require python-scipy package #105

srianvesh opened this issue Oct 4, 2023 · 7 comments
Assignees
@wycwyhwyq

Comments

@srianvesh
Copy link

Other

I have been trying to build python-scipy package. However, the build fails due to toolchain not supporting FORTRAN. python-scipy package depends on lapack. LAPACK requires FORTRAN supported by toolchain. I am unable to proceed because of this.
buildroot_error_scipy_lapack_fortran.txt
@MrThanlon
Copy link
Member

Trying to install a cross-compiler for RISC-V Fortran, if you are using debian/ubuntu, run apt-get install gfortran-riscv64-linux-gnu

@srianvesh
Copy link
Author

Please elaborate. I am using ubuntu. i ran apt-get install gfortran-riscv64-linux-gnu. but still getting the error.
-- The Fortran compiler identification is unknown
-- The C compiler identification is GNU 7.3.0
CMake Error at CMakeLists.txt:3 (project):
No CMAKE_Fortran_COMPILER could be found.

`[CAN k510_buildroot]$ make
if [ ! -f toolchain/nds64le-linux-glibc-v5d.txz ] || [ "c87c87089fe2f0445a94054e8ae7b947" != "c87c87089fe2f0445a94054e8ae7b947" ]; then
rm -rf toolchain/nds64le-linux-glibc-v5d* ;
wget -c https://github.com/andestech/Andes-Development-Kit/releases/download/ast-v3_2_1-release-linux/nds64le-linux-glibc-v5d.txz -O toolchain/nds64le-linux-glibc-v5d.txz;
tar --no-same-owner -xf /opt/k510_buildroot/toolchain/nds64le-linux-glibc-v5d.txz -C /opt/k510_buildroot/toolchain;
fi;
if [ ! -f toolchain/nds64le-elf-mculib-v5d.txz ] || [ "d2dcf2729c2e6b654c4be197c2c75987" != "d2dcf2729c2e6b654c4be197c2c75987" ]; then
rm -rf toolchain/nds64le-elf-mculib-v5d* ;
wget -c https://github.com/andestech/Andes-Development-Kit/releases/download/ast-v3_2_3-release-linux/nds64le-elf-mculib-v5d.txz -O toolchain/nds64le-elf-mculib-v5d.txz;
tar --no-same-owner -xf /opt/k510_buildroot/toolchain/nds64le-elf-mculib-v5d.txz -C /opt/k510_buildroot/toolchain;
fi;
make -C /opt/k510_buildroot/buildroot-2020.02.11 "k510_crb_lp3_v1_2_defconfig" O=/opt/k510_buildroot/"k510_crb_lp3_v1_2_defconfig" BR2_EXTERNAL=/opt/k510_buildroot
make[1]: Entering directory '/opt/k510_buildroot/buildroot-2020.02.11'
GEN /opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/Makefile

configuration written to /opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/.config

make[1]: Leaving directory '/opt/k510_buildroot/buildroot-2020.02.11'
make -C /opt/k510_buildroot/"k510_crb_lp3_v1_2_defconfig"
make[1]: Entering directory '/opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig'
/usr/bin/make -j1 O=/opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig HOSTCC="/usr/bin/gcc" HOSTCXX="/usr/bin/g++" syncconfig
GEN /opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/Makefile

lapack 3.9.0 Downloading
Will not apply HSTS. The HSTS database must be a regular and non-world-writable file.
ERROR: could not open HSTS store at '/root/.wget-hsts'. HSTS will be disabled.
--2023-10-08 21:22:49-- https://github.com/Reference-LAPACK/lapack/archive/v3.9.0/lapack-3.9.0.tar.gz
Resolving github.com (github.com)... 20.207.73.82
Connecting to github.com (github.com)|20.207.73.82|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/Reference-LAPACK/lapack/tar.gz/refs/tags/v3.9.0 [following]
--2023-10-08 21:22:49-- https://codeload.github.com/Reference-LAPACK/lapack/tar.gz/refs/tags/v3.9.0
Resolving codeload.github.com (codeload.github.com)... 20.207.73.88
Connecting to codeload.github.com (codeload.github.com)|20.207.73.88|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: '/opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/build/.lapack-3.9.0.tar.gz.0KQQhr/output'

/opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig [ <=> ] 7.18M 2.02MB/s in 3.6s

2023-10-08 21:22:55 (2.02 MB/s) - '/opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/build/.lapack-3.9.0.tar.gz.0KQQhr/output' saved [7534567]

lapack-3.9.0.tar.gz: OK (sha256: 106087f1bb5f46afdfba7f569d0cbe23dacb9a07cd24733765a0e89dbe1ad573)

lapack 3.9.0 Extracting
gzip -d -c /opt/k510_buildroot/dl/lapack/lapack-3.9.0.tar.gz | /opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/host/bin/tar --strip-components=1 -C /opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/build/lapack-3.9.0 -xf -
lapack 3.9.0 Patching
lapack 3.9.0 Configuring
(mkdir -p /opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/build/lapack-3.9.0//buildroot-build && cd /opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/build/lapack-3.9.0//buildroot-build && rm -f CMakeCache.txt && PATH="/opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/host/bin:/opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/host/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" /opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/host/bin/cmake /opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/build/lapack-3.9.0/ -DCMAKE_TOOLCHAIN_FILE="/opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/host/share/buildroot/toolchainfile.cmake" -DCMAKE_INSTALL_PREFIX="/usr" -DCMAKE_COLOR_MAKEFILE=OFF -DBUILD_DOC=OFF -DBUILD_DOCS=OFF -DBUILD_EXAMPLE=OFF -DBUILD_EXAMPLES=OFF -DBUILD_TEST=OFF -DBUILD_TESTS=OFF -DBUILD_TESTING=OFF -DBUILD_SHARED_LIBS=ON -DLAPACKE=ON -DCBLAS=ON -DBUILD_COMPLEX=ON -DBUILD_COMPLEX16=ON )
-- The Fortran compiler identification is unknown
-- The C compiler identification is GNU 7.3.0
CMake Error at CMakeLists.txt:3 (project):
No CMAKE_Fortran_COMPILER could be found.

Tell CMake where to find the compiler by setting either the environment
variable "FC" or the CMake cache entry CMAKE_Fortran_COMPILER to the full
path to the compiler, or to the compiler name if it is in the PATH.

-- Check for working C compiler: /opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/host/bin/riscv64-linux-gcc
-- Check for working C compiler: /opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/host/bin/riscv64-linux-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Configuring incomplete, errors occurred!
See also "/opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/build/lapack-3.9.0/buildroot-build/CMakeFiles/CMakeOutput.log".
See also "/opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/build/lapack-3.9.0/buildroot-build/CMakeFiles/CMakeError.log".
make[2]: *** [package/pkg-generic.mk:259: /opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig/build/lapack-3.9.0/.stamp_configured] Error 1
make[1]: *** [Makefile:23: _all] Error 2
make[1]: Leaving directory '/opt/k510_buildroot/k510_crb_lp3_v1_2_defconfig'
make: *** [Makefile:80: all] Error 2`

@MrThanlon
Copy link
Member

It seems that CMake is using the toolchain specified by Buildroot, but the Andes toolchain does not provide a Fortran compiler. The Andes toolchain is open source, and you can download the source code from https://github.com/andestech/Andes-Development-Kit/releases/download/ast-v3_2_1-release-linux/nds64le-linux-glibc-v5d.txz. You may need to compile the Andes toolchain yourself to add gfortran, or you can choose not to use the Buildroot and instead compile LAPACK & python separately. The programs generated by the generic riscv compiler can be used for the K510 Linux, but the Linux kernel in Buildroot needs to be compiled using the Andes compiler.

@srianvesh
Copy link
Author

As suggested, i have tried to cross-compile scipy in the following environment.

  1. In Ubuntu i installed PYTHON with all depencies.
  2. Installed gcc-riscv64-linux-gnu, gfortran-8-riscv64-linux-gnu on ubuntu.
  3. compiled LAPACK, BLAS libraries using gcc-riscv64-linux-gnu
  4. downloaded scipy-1.8.1 from github and then set environment variables and lapack libraries for scipy
    export FC=riscv64-linux-gnu-gfortran
    export CC=riscv64-linux-gnu-gcc
    and ran the following command
    [scipy]$ python3 setup.py build --fcompiler=riscv64-linux-gnu-gfortran --compiler=riscv64-linux-gnu-gcc

output: (getting error)
error: don't know how to compile C/C++ code on platform 'posix' with 'riscv64-linux-gnu-gcc' compiler

scipy_crosscompile.txt

@MrThanlon
Copy link
Member

I haven't compiled scipy before, so I'm not sure about the cause of this issue. However, I checked the scipy code repository, and it seems to have a significant amount of C++ code. This means that besides specifying a C compiler, you may also need to specify a C++ compiler. Additionally, I noticed that the latest version of scipy has switched to using meson for compilation, while version 1.8.1 still uses setup.py. I haven't compiled Python native packages before, so I'm not familiar with how compilers are typically specified.

@chenhb3
Copy link

chenhb3 commented Oct 23, 2023

@srianvesh If there is no feedback on the results or situation for more than 7 days, the issue will be closed.. If you have any questions later, please create it again, thank you!

@srianvesh
Copy link
Author

@srianvesh If there is no feedback on the results or situation for more than 7 days, the issue will be closed.. If you have any questions later, please create it again, thank you!

I'm trying the options mentioned by @MrThanlon. out of the two options mentioned, i have tried cross-compiling scipy using cross-compiler riscv toolchain. However i am struck during scipy compilation that it searches for riscv-target python.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wycwyhwyq wycwyhwyq

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@srianvesh @MrThanlon @wycwyhwyq @chenhb3