Plant phenotyping analysis is a crucial task in agricultural research, aiming to measure plant morphological data using tools such as calipers and radars. However, traditional methods generally struggle to accurately capture the surface textures and structural features of plants. In this paper, we propose a unified architecture to reconstruct high fidelity 3D surface models and render realistic plant views,which enhances geometric accuracy during Gaussian densification and mesh extraction from 2D images. First, we apply semantic segmentation with large language models to extract plant objects from 2D RGB images, generating sparse mappings and camera poses. These images and point clouds are then processed to produce Gaussian ellipsoids and 3D textured models, with our method detecting smooth regions during densification. To ensure precise alignment of Gaussians with object surfaces, our robust 3D Gaussian splatting method includes an outlier removal algorithm, produces more accurate models with less noise than traditional techniques. Experimental results demonstrate that our method outperforms existing plant modeling approaches, surpassing the NeRF-based method instantNGP and the 3DGS-based surface modeling method SuGaR in terms of PSNR, LPIPS, and SSIM metrics.
GRAPE (Ours)'s mesh results on some classes of the GRAPE dataset.
# https
git clone https://github.com/DYH200009/GRAPE
# or ssh
git clone [email protected]:DYH200009/GRAPE.git
If you have an environment used for 3dgs, use it.
If not, please refer to the environment configuration of 3DGS environment
The environment.yml file for 3DGS has been provided in our repo.
Additionally, you also need to install the
open3d scipy matplotlib pillow
libraries.
# If you have already installed the 3dgs environment,
# please activate the environment and execute the following command :
conda activate gaussian_splatting
pip install open3d scipy matplotlib pillow
cd submodules/diff-gaussian-rasterization/
pip install -e .
cd ../simple-knn/
pip install -e .
cd ../../
In addition, you need to install SuGaR's running environment separately If you have an environment used for SuGaR, use it. If not, please refer to the environment configuration of SuGaR environment
- Create a new
datafolder - Download the file (GRAPE-Dataset).
- Unzip it to
datafolder.
In order to run our code,
# python train.py -s [path_to_dataset] -m [path_to_output] --eval
# cd MeshExtractor
# python train.py -s [path_to_dataset] -r <"density" or "sdf"> -c [path_to_output]
Run demo:
python train.py -s data/GRAPE/grape -m outputCKT/grape
cd MeshExtractor
python train.py -s data/GRAPE/grape -r density -c outputCKT/grape
This project is based on GeoGussian and SuGaR Thanks for the contribution of the open source community.