Skip to content

Create New Presets

Jump to bottom Edit New page
Samuel Audet edited this page Dec 6, 2020 · 16 revisions

Introduction

So you have a C/C++ library you would like to use from inside Java, but it is not yet available as part of the JavaCPP Presets? You have come to the right place!

Since all JavaCPP Presets are currently defined using a set of cppbuild.sh Bash scripts and pom.xml Maven build files, the recommended way to get started while obtaining the latest versions of the parent script and build files is by first cloning the Git repositories and installing JavaCPP itself:

$ git clone https://github.com/bytedeco/javacpp.git
$ git clone https://github.com/bytedeco/javacpp-presets.git
$ cd javacpp
$ mvn clean install

Please make sure to install all the required software as indicated in their README.md files and on the Build Environments page. Then, to add new presets for a library inside the javacpp-presets directory:

  1. Create a new subdirectory whose name corresponds to the desired final name of the JAR file and artifact, which should be all lowercase by convention.
  2. Inside this new subdirectory, create new project files as described in detail in the following sections: the cppbuild.sh file, the pom.xml file, and the Java configuration files in the org.bytedeco.<moduleName>.presets package, as well as at least one platform/pom.xml file.
  3. After confirming that everything is in a working order by running mvn clean install inside the javacpp-presets directory, add the necessary files to the .github/workflows directory for the CI servers, send a pull request to have your code added to the main repository, and check the status of your builds to make sure everything passes.

To make the explanations clearer, the sample content below was actually designed to build and wrap a small, simple, but popular library: zlib. (In fact, it comes bundled with the JDK as part of the java.util.zip package.) To get a quick impression of the whole procedure, please feel free to copy/paste into the appropriate files and try it out that way. There is also a helloworld sample project in the helloworld branch ready to be built:

For convenience, the zlib sample presets below is also available on this branch ready to be built:

The cppbuild.sh file

The Bash script is the agent that takes care of building the native library itself. (To use Bash under Windows, we can install environments such as MSYS2.) Ideally, it should do these three things:

  1. Acquire the source code somehow,
  2. Build the binary files someway, and
  3. Install the header and library files in the cppbuild/<javacpp.platform> subdirectory.

Note that if the library can be installed in a portable fashion in some other way, we can use that method instead of hacking a script file together. Unfortunately, this is rarely the case, but Bash scripts have proven to be quite an effective solution that happens to have become an industry standard. Bazel or Gradle may eventually become a better alternative as a good portable candidate that could not only obviate the need for Bash, but that could also take over the whole build process, including Java files currently assembled by Maven.

Before this becomes a reality though, we are going to make do with cppbuild.sh files. For our small demo zlib library, it could look like this:

#!/bin/bash
# This file is meant to be included by the parent cppbuild.sh script
if [[ -z "$PLATFORM" ]]; then
    pushd ..
    bash cppbuild.sh "$@" zlib
    popd
    exit
fi

ZLIB_VERSION=1.2.11
download http://zlib.net/zlib-$ZLIB_VERSION.tar.gz zlib-$ZLIB_VERSION.tar.gz
mkdir -p $PLATFORM
cd $PLATFORM
tar -xzvf ../zlib-$ZLIB_VERSION.tar.gz
cd zlib-$ZLIB_VERSION

case $PLATFORM in
    android-arm)
        CC="$ANDROID_CC" CFLAGS="$ANDROID_FLAGS" ./configure --prefix=.. --static
        make -j $MAKEJ
        make install
        ;;
    android-arm64)
        CC="$ANDROID_CC" CFLAGS="$ANDROID_FLAGS" ./configure --prefix=.. --static
        make -j $MAKEJ
        make install
        ;;
    android-x86)
        CC="$ANDROID_CC" CFLAGS="$ANDROID_FLAGS" ./configure --prefix=.. --static
        make -j $MAKEJ
        make install
        ;;
    android-x86_64)
        CC="$ANDROID_CC" CFLAGS="$ANDROID_FLAGS" ./configure --prefix=.. --static
        make -j $MAKEJ
        make install
        ;;
    linux-x86)
        CC="gcc -m32 -fPIC" ./configure --prefix=.. --static
        make -j $MAKEJ
        make install
        ;;
    linux-x86_64)
        CC="gcc -m64 -fPIC" ./configure --prefix=.. --static
        make -j $MAKEJ
        make install
        ;;
    macosx-x86_64)
        ./configure --prefix=.. --static
        make -j $MAKEJ
        make install
        ;;
    windows-x86)
        nmake -f win32/Makefile.msc zlib.lib
        mkdir -p ../include ../lib
        cp zconf.h zlib.h ../include/
        cp zlib.lib ../lib/
        ;;
    windows-x86_64)
        nmake -f win32/Makefile.msc zlib.lib
        mkdir -p ../include ../lib
        cp zconf.h zlib.h ../include/
        cp zlib.lib ../lib/
        ;;
    *)
        echo "Error: Platform \"$PLATFORM\" is not supported"
        ;;
esac

cd ../..

After calling bash cppbuild.sh install zlib from inside the parent directory of javacpp-presets it should successfully download, build, and install the library in the cppbuild subdirectory, as desired.

The pom.xml file

Most of the pom.xml file is boilerplate that we cannot abstract away with Maven via the parent pom.xml file. Simply adjusting the content below to the appropriate names and versions of the libraries should suffice:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <parent>
    <groupId>org.bytedeco</groupId>
    <artifactId>javacpp-presets</artifactId>
    <version>1.5.5-SNAPSHOT</version>
  </parent>

  <groupId>org.bytedeco</groupId>
  <artifactId>zlib</artifactId>
  <version>1.2.11-${project.parent.version}</version>
  <name>JavaCPP Presets for zlib</name>

  <dependencies>
    <dependency>
      <groupId>org.bytedeco</groupId>
      <artifactId>javacpp</artifactId>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <artifactId>maven-resources-plugin</artifactId>
      </plugin>
      <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
      </plugin>
      <plugin>
        <groupId>org.bytedeco</groupId>
        <artifactId>javacpp</artifactId>
      </plugin>
      <plugin>
        <artifactId>maven-jar-plugin</artifactId>
      </plugin>
      <plugin>
        <groupId>org.moditect</groupId>
        <artifactId>moditect-maven-plugin</artifactId>
      </plugin>
      <plugin>
        <artifactId>maven-dependency-plugin</artifactId>
      </plugin>
      <plugin>
        <artifactId>maven-source-plugin</artifactId>
      </plugin>
      <plugin>
        <artifactId>maven-javadoc-plugin</artifactId>
      </plugin>
    </plugins>
  </build>

</project>

Although not crucial, the calls to moditect-maven-plugin create for us module-info.class files that are required by the JPMS, in which case we need to create an additional src/main/java9/module-info.java file such as the following:

module org.bytedeco.zlib {
  requires transitive org.bytedeco.javacpp;
  exports org.bytedeco.zlib;
  exports org.bytedeco.zlib.global;
  exports org.bytedeco.zlib.presets;
}

The Java configuration files

Next, we need to specify such things as the desired names of the target Java interface files and packages, as well as the ones of C/C++ header files and native library files previously built and installed by the cppbuild.sh scripts. We place that information in Java source code files inside the org.bytedeco.<moduleName>.presets package, by convention, under the src/main/java/org/bytedeco/<moduleName>/presets subdirectory. For consistency, we should create one configuration file per native library file. In the case of the zlib library, which comes with only one library file, we could be satisfied with the following alone:

package org.bytedeco.zlib.presets;

import org.bytedeco.javacpp.*;
import org.bytedeco.javacpp.annotation.*;
import org.bytedeco.javacpp.tools.*;

@Properties(
    value = {
        @Platform(include = "<zlib.h>", link = "[email protected]#"),
        @Platform(value = "windows", link = "zlib#")
    },
    target = "org.bytedeco.zlib",
    global = "org.bytedeco.zlib.global.zlib"
)
public class zlib implements InfoMapper {
    static { Loader.checkVersion("org.bytedeco", "zlib"); }

    public void map(InfoMap infoMap) {
        infoMap.put(new Info("ZEXTERN", "ZEXPORT", "z_const", "zlib_version").cppTypes().annotations())
               .put(new Info("FAR").cppText("#define FAR"))
               .put(new Info("OF").cppText("#define OF(args) args"))
               .put(new Info("Z_ARG").cppText("#define Z_ARG(args) args"))
               .put(new Info("Byte", "Bytef", "charf").cast().valueTypes("byte").pointerTypes("BytePointer"))
               .put(new Info("uInt", "uIntf").cast().valueTypes("int").pointerTypes("IntPointer"))
               .put(new Info("uLong", "uLongf", "z_crc_t", "z_off_t", "z_size_t").cast().valueTypes("long").pointerTypes("CLongPointer"))
               .put(new Info("z_off64_t").cast().valueTypes("long").pointerTypes("LongPointer"))
               .put(new Info("voidp", "voidpc", "voidpf").valueTypes("Pointer"))
               .put(new Info("gzFile_s").pointerTypes("gzFile"))
               .put(new Info("gzFile").valueTypes("gzFile"))
               .put(new Info("Z_LARGE64", "!defined(ZLIB_INTERNAL) && defined(Z_WANT64)").define(false))
               .put(new Info("inflateGetDictionary", "gzopen_w", "gzvprintf").skip());
    }
}

That file also contains information useful to the Parser specified via the Info configuration objects added to an InfoMap container through the InfoMapper interface. To understand how to use the annotations and configuration objects, please refer to the Mapping Recipes for C/C++ Libraries, while the API documentation of JavaCPP should also come in handy during this phase.

The platform/pom.xml files

Finally, we should create at least one pom.xml as below that depends on the artifacts for all platforms, but that users can also configure transitively at runtime as described on the Reducing the Number of Dependencies page. We can install those platform pom.xml files with mvn clean install -Djavacpp.platform.host, which prevents Maven from trying to download dependencies that might not have been deployed from all platforms yet.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <parent>
    <groupId>org.bytedeco</groupId>
    <artifactId>javacpp-presets</artifactId>
    <version>1.5.5-SNAPSHOT</version>
    <relativePath>../../</relativePath>
  </parent>

  <groupId>org.bytedeco</groupId>
  <artifactId>zlib-platform</artifactId>
  <version>1.2.11-${project.parent.version}</version>
  <name>JavaCPP Presets Platform for zlib</name>

  <properties>
    <javacpp.moduleId>zlib</javacpp.moduleId>
  </properties>

  <dependencies>
    <dependency>
      <groupId>${project.groupId}</groupId>
      <artifactId>${javacpp.moduleId}</artifactId>
      <version>${project.version}</version>
    </dependency>
    <dependency>
      <groupId>${project.groupId}</groupId>
      <artifactId>${javacpp.moduleId}</artifactId>
      <version>${project.version}</version>
      <classifier>${javacpp.platform.android-arm}</classifier>
    </dependency>
    <dependency>
      <groupId>${project.groupId}</groupId>
      <artifactId>${javacpp.moduleId}</artifactId>
      <version>${project.version}</version>
      <classifier>${javacpp.platform.android-arm64}</classifier>
    </dependency>
    <dependency>
      <groupId>${project.groupId}</groupId>
      <artifactId>${javacpp.moduleId}</artifactId>
      <version>${project.version}</version>
      <classifier>${javacpp.platform.android-x86}</classifier>
    </dependency>
    <dependency>
      <groupId>${project.groupId}</groupId>
      <artifactId>${javacpp.moduleId}</artifactId>
      <version>${project.version}</version>
      <classifier>${javacpp.platform.android-x86_64}</classifier>
    </dependency>
    <dependency>
      <groupId>${project.groupId}</groupId>
      <artifactId>${javacpp.moduleId}</artifactId>
      <version>${project.version}</version>
      <classifier>${javacpp.platform.linux-x86}</classifier>
    </dependency>
    <dependency>
      <groupId>${project.groupId}</groupId>
      <artifactId>${javacpp.moduleId}</artifactId>
      <version>${project.version}</version>
      <classifier>${javacpp.platform.linux-x86_64}</classifier>
    </dependency>
    <dependency>
      <groupId>${project.groupId}</groupId>
      <artifactId>${javacpp.moduleId}</artifactId>
      <version>${project.version}</version>
      <classifier>${javacpp.platform.macosx-x86_64}</classifier>
    </dependency>
    <dependency>
      <groupId>${project.groupId}</groupId>
      <artifactId>${javacpp.moduleId}</artifactId>
      <version>${project.version}</version>
      <classifier>${javacpp.platform.windows-x86}</classifier>
    </dependency>
    <dependency>
      <groupId>${project.groupId}</groupId>
      <artifactId>${javacpp.moduleId}</artifactId>
      <version>${project.version}</version>
      <classifier>${javacpp.platform.windows-x86_64}</classifier>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <artifactId>maven-jar-plugin</artifactId>
        <executions>
          <execution>
            <id>default-jar</id>
            <configuration>
              <archive>
                <manifestEntries>
                  <Class-Path>${javacpp.moduleId}.jar ${javacpp.moduleId}-linux-x86.jar ${javacpp.moduleId}-linux-x86_64.jar ${javacpp.moduleId}-macosx-x86_64.jar ${javacpp.moduleId}-windows-x86.jar ${javacpp.moduleId}-windows-x86_64.jar</Class-Path>
                </manifestEntries>
              </archive>
            </configuration>
          </execution>
          <execution>
            <id>empty-javadoc-jar</id>
            <goals>
              <goal>jar</goal>
            </goals>
            <configuration>
              <classifier>javadoc</classifier>
            </configuration>
          </execution>
          <execution>
            <id>empty-sources-jar</id>
            <goals>
              <goal>jar</goal>
            </goals>
            <configuration>
              <classifier>sources</classifier>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.moditect</groupId>
        <artifactId>moditect-maven-plugin</artifactId>
        <executions>
          <execution>
            <id>add-module-infos</id>
            <phase>none</phase>
          </execution>
          <execution>
            <id>add-platform-module-info</id>
            <phase>package</phase>
            <goals>
              <goal>add-module-info</goal>
            </goals>
            <configuration>
              <modules>
                <module>
                  <file>${project.build.directory}/${project.artifactId}.jar</file>
                  <moduleInfoSource>
                    module org.bytedeco.${javacpp.moduleId}.platform {
                      requires static org.bytedeco.${javacpp.moduleId}.android.arm;
                      requires static org.bytedeco.${javacpp.moduleId}.android.arm64;
                      requires static org.bytedeco.${javacpp.moduleId}.android.x86;
                      requires static org.bytedeco.${javacpp.moduleId}.android.x86_64;
                      requires static org.bytedeco.${javacpp.moduleId}.linux.x86;
                      requires static org.bytedeco.${javacpp.moduleId}.linux.x86_64;
                      requires static org.bytedeco.${javacpp.moduleId}.macosx.x86_64;
                      requires static org.bytedeco.${javacpp.moduleId}.windows.x86;
                      requires static org.bytedeco.${javacpp.moduleId}.windows.x86_64;
                    }
                  </moduleInfoSource>
                </module>
              </modules>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Now, inside the parent directory of javacpp-presets, after adding the module name zlib to the module lists in the parent pom.xml file, and the corresponding path in the parent platform/pom.xml one, we can try to build the project by calling mvn clean install --projects .,zlib followed by mvn clean install -f platform --projects ../zlib/platform -Djavacpp.platform.host. If all goes well, we should see lines among the output of Maven that confirm that the target class and the native wrapping library get created. If you are having trouble getting JavaCPP to parse the header files or to generate proper interfaces for your libraries, please open a new issue about that so we can fix it.

When satisfied with the overall result, please send a pull request with your changes! Moreover, since portability is one of Java's main benefits, we should offer binaries for as many platforms as possible, so please consider offering builds for platforms other than your own, which we can easily test and debug remotely with GitHub Actions. Nevertheless, other contributors will hopefully do the same for your library presets on platforms that interest them.

If you are looking for a more "turnkey solution", please refer to these commercial services offered on xs:code:

Thank you very much for your interest in this project, and please feel free to post your questions on the mailing list and open new issues to communicate your suggestions.

Add a custom footer
Add a custom sidebar
Clone this wiki locally