X. Engine Management III (InstalledEngines)
When JDLL is first installed it is "empty", to be able to run models, the corresponding Deep Learning (DL) frameworks (engines) are need to be downloaded and setup (explained here).
Each model might need a different DL framework and version, thus for several models the user might end up with many new engines installed. JDLL offers tools to keep track of what is available and what has been installed to minimize conflicts and improve user experience.
These tools are separted in two classes, one helps understanding which engines have already been installed: io.bioimage.modelrunner.versionmanagement.InstalledEngines and the other one provides information about the engines that can be installed: io.bioimage.modelrunner.versionmanagement.AvailableEngines.
In this page the class io.bioimage.modelrunner.versionmanagement.InstalledEngines to help users understand how they can retrieve information about the engines they have installed.
Class that contains a collection of methods to get information about the already installed engines.
All the information about all the engines that can be installed and used by JDLL can be found in the engines JSON file. This file enumerates all the engines available in JDLL together with the name of their DL framework, the version number in Python, the version number in Java, the operating system that supports it, the minimum Java version required to use it, whether it supports GPU or not, whether it supports CPU or not, whether it supports Rosetta or not and all the JAR files that are needed to run the engine.
The information of each of the engines can be instantiated as a DeepLearningVersion object to easily access each of its fields. This is better described in the JDLL Wiki page Engine Management I (DeepLearningVersion).
Example of locally installed engines
Method that returns a list of all the DL engines installed on the wanted directory. The output is a List of instances of DeepLearningVersion from which the wanted information about the engine can be obtained.
The method returns all the engines installed, even if they are not supported by the operating system or the Java version. To return only the engines installed completely compatible use InstalledEngines.getDownloadedForOS(String enginesPath).
- enginesPath: directory where the engines are installed.
Below there is an example using the method. The engine installation for the example is the one depicted in the image above:
String enginesPath = "C:\\Users\\carlos\icy\\engines";
List<DeepLearningVersions> installedVersions = InstalledEngines.getAll(enginesPath);
System.out.println(installedVersions.get(0).getFramework());
System.out.println(installedVersions.get(0).getVersion());
Output:
pytorch
1.4.0
Method that returns a list of all the DL engines installed on the wanted directory for the OS and Java version used. In addition, in the case the computer is using Rosetta the engines that do not support it are filtered too. The output is a List of instances of DeepLearningVersion from which the wanted information about the engine can be obtained.
- enginesPath: directory where the engines are installed.
Below there is an example using the method. The engine installation for the example is the one depicted in the image above:
String enginesPath = "C:\\Users\\carlos\icy\\engines";
List<DeepLearningVersions> installedVersions = InstalledEngines.getAll(enginesPath);
System.out.println(installedVersions.get(0).getFramework());
System.out.println(installedVersions.get(0).getVersion());
Output:
pytorch
1.4.0
List<DeepLearningVersion> InstalledEngines.getDownloadedForFramework(String enginesPath, String framework)
Method that returns a list of all the DL engines installed on the wanted directory for a specific DL framework for the OS and Java version used. In addition, in the case the computer is using Rosetta the engines that do not support it are filtered too. The output is a List of instances of DeepLearningVersion from which the wanted information about the engine can be obtained.
- enginesPath: directory where the engines are installed.
- framework: DL framework for which the engines want to be retrieved. The list of possible frameworks can be found here.
Below there is an example using the method. The engine installation for the example is the one depicted in the image above:
String enginesPath = "C:\\Users\\carlos\icy\\engines";
String framework = "tensorflow";
List<DeepLearningVersions> installedVersions = InstalledEngines.getDownloadedForFramework(enginesPath, framework);
System.out.println(installedVersions.get(0).getFramework());
System.out.println(installedVersions.get(0).getVersion());
Output:
tensorflow
1.12.0
List<String> InstalledEngines.getDownloadedPythonVersionsForFramework(String enginesPath, String framework)
Method that returns a list of the versions installed for the engine of teh corresponding framework. The output is a List of String.
- enginesPath: directory where the engines are installed.
- framework: DL framework for which the engines want to be retrieved
Below there is an example using the method. The engine installation for the example is the one depicted in the image above:
String enginesPath = "C:\\Users\\carlos\icy\\engines";
String framework = "tensorflow";
List<String> installedVersions = InstalledEngines.getDownloadedPythonVersionsForFramework(enginesPath, framework);
System.out.println(installedVersions);
Output:
[1.12.0, 1.15.0]
boolean InstalledEngines.checkFrameworkVersionInstalled(String framework, String version, String enginesDir)
Method that searches if a DL engine from the wanted DL framework and version has been installed or not and returns true if it has or false otherwise. note that this method only looks for engines compatible with the OS and Java version used. In addition, in the case the computer is using Rosetta the engines that do not support it are filtered too.
- framework: DL framework of interest. The list of possible frameworks can be found here.
- version: version of the DL framework of interest
- enginesDir: directory where the engines are installed.
Below there is an example using the method. The engine installation for the example is the one depicted in the image above:
String enginesPath = "C:\\Users\\carlos\icy\\engines";
String framework = "tensorflow";
String version = "1.15.0";
boolean installed = InstalledEngines.checkFrameworkVersionInstalled(framework, version, enginesPath);
String notInstalledVersion = "2.0.0";
boolean notInstalled = InstalledEngines.checkFrameworkVersionInstalled(framework, notInstalledVersion, enginesPath);
System.out.println(installed);
System.out.println(notInstalled);
Output:
true
false
List<DeepLearningVersion> InstalledEngines.checkEngineWithArgsInstalled(String framework, String version, Boolean cpu, Boolean gpu, Boolean rosetta, Integer minJavaVersion, String enginesDir)
Method to find and engine among the ones installed by setting the wanted parameters. The engines filtered are only the ones from the system OS (either Windows, Linux or MacOS).
The output is a List of instances of DeepLearningVersion from which the wanted information about the engine can be obtained.
- framework: name of the DL framework, if
nullit will look for any DL framework. The list of possible frameworks can be found here. - version: version of the DL framework of interest, if
nullit will look for any version - cpu: whether it supports CPU or not, if
nullit will look for both - gpu: whether it supports GPU or not, if
nullit will look for both - rosetta: special parameter for ARM64 chips. Java only has full support for ARM64 chips after from version 11 and onwards. If a computer with this chip tries to launch a Java program using Java 8 for example, a x86_64 chip will be simulated synthetically to avoid crashes. This translation process is called Rosetta (for more info click here). For JDLL, some engines work with Rosetta and others do not. This method allows filtering engines that support Rosetta by setting
rosetta=true. If therosetta=nullthe paramter will be ignored and not used for filtering engines. However, not that this parameter at the moment is only useful for MacOS systems (M1 and M2 Macs) and a few Linux computers. Whether an engine is compatible with Rosetta or not can be seen in the engines json file. - minJavaVersion: the minimum Java version an engine requires to run on. If a software is running on Java 17, it will be able to run every engine that has a minimum Java version of 17 or older. However, if the program uses Java 8, it will not be able to use engines that require a minimum Java version of 11 for example. If this argument is set to
null, it will be ignored for filtering. The minimum required Java version for an engine is describe with the fieldjavain the engines json file. - enginesDir: directory where the engines are installed
Below there are several examples on how the method works using the image above as the installation directory.
In this example, all the engines that support both CPU and GPU computation are retrieved. Note how Boolean and Integer are used instead of boolean and int, this happens because the first ones allow setting the values to null.
String enginesDir = "C:\\Users\\carlos\\icy\engines";
Boolean cpu = true;
Boolean gpu = true;
// The rest of parameters are set to null because there is not interest in them
String framework = null;
String version = null;
Boolean rosetta = null;
Integer minJavaVersion = null;
List<DeepLearningVersion> engines = InstalledEngines.checkEngineWithArgsInstalled(framework, version, cpu, gpu, rosetta, minJavaVersion, enginesDir);
System.out.println(engines.size());
System.out.println(engines.get(0).getFramework());
Output (looking at the example image above, only 8 engines out of 10 support CPU and GPU, and all of them are for pytorch):
8
pytorch
If all the parameters are set to null, all the installed engines are retrieved:
String enginesDir = "C:\\Users\\carlos\\icy\engines";
Boolean cpu = null;
Boolean gpu = null;
String framework = null;
String version = null;
Boolean rosetta = null;
Integer minJavaVersion = null;
List<DeepLearningVersion> engines = InstalledEngines.checkEngineWithArgsInstalled(framework, version, cpu, gpu, rosetta, minJavaVersion, enginesDir);
System.out.println(engines.size());
Output (looking at the example image above, 10 engines are installed):
10
Now look for Tensorflow engines, no more constraints:
String enginesDir = "C:\\Users\\carlos\\icy\engines";
// Note that the tag 'tensorflow_saved_model_bundled' would be equivalent
String framework = "tensorflow";
// Everything else is set to null
Boolean cpu = null;
Boolean gpu = null;
String version = null;
Boolean rosetta = null;
Integer minJavaVersion = null;
List<DeepLearningVersion> engines = InstalledEngines.checkEngineWithArgsInstalled(framework, version, cpu, gpu, rosetta, minJavaVersion, enginesDir);
System.out.println(engines.size());
System.out.println(engines.get(0).getFramework());
Output:
2
tensorflow
Filter for Tensorflow version 1.15.0, no more constraints:
String enginesDir = "C:\\Users\\carlos\\icy\engines";
// Note that the tag 'tensorflow_saved_model_bundled' would be equivalent
String framework = "tensorflow";
String version = 1.15.0;
// Everything else is set to null
Boolean cpu = null;
Boolean gpu = null;
Boolean rosetta = null;
Integer minJavaVersion = null;
List<DeepLearningVersion> engines = InstalledEngines.checkEngineWithArgsInstalled(framework, version, cpu, gpu, rosetta, minJavaVersion, enginesDir);
System.out.println(engines.size());
System.out.println(engines.get(0).getVersion());
Output:
1
1.15.0
This example looks for a tensorfow 1.15.0 engine that supports GPU computation:
String enginesDir = "C:\\Users\\carlos\\icy\engines";
// Note that the tag 'tensorflow_saved_model_bundled' would be equivalent
String framework = "tensorflow";
String version = 1.15.0;
Boolean gpu = true;
// Everything else is set to null
Boolean cpu = null;
Boolean rosetta = null;
Integer minJavaVersion = null;
List<DeepLearningVersion> engines = InstalledEngines.checkEngineWithArgsInstalled(framework, version, cpu, gpu, rosetta, minJavaVersion, enginesDir);
System.out.println(engines.size());
Output (no engine satistisfies the criteria):
0
The next example shows teh behaviour of the minimum Java version argument. First try with a new Java version (17) when all the engines installed require a Java version of at least 8:
String enginesDir = "C:\\Users\\carlos\\icy\engines";
Integer minJavaVersion = 17;
// Everything else is set to null
Boolean cpu = null;
Boolean rosetta = null;
String framework = null;
String version = null;
Boolean gpu = null;
List<DeepLearningVersion> engines = InstalledEngines.checkEngineWithArgsInstalled(framework, version, cpu, gpu, rosetta, minJavaVersion, enginesDir);
System.out.println(engines.size());
Output (all the engines are retrieved because 17 is higher than 8, all the engines will be able to run on a Java 17 system because they require at least 8):
10
However using an older Java version, 5:
String enginesDir = "C:\\Users\\carlos\\icy\engines";
Integer minJavaVersion = 5;
// Everything else is set to null
Boolean cpu = null;
Boolean rosetta = null;
String framework = null;
String version = null;
Boolean gpu = null;
List<DeepLearningVersion> engines = InstalledEngines.checkEngineWithArgsInstalled(framework, version, cpu, gpu, rosetta, minJavaVersion, enginesDir);
System.out.println(engines.size());
Output (no engine is retrieved because all the engines require at least Java 8, thus Java 5 is insufficient):
0
List<DeepLearningVersion> InstalledEngines.checkEngineWithArgsInstalledForOS(String framework, String version, Boolean cpu, Boolean gpu, String enginesDir)
Method similar to InstalledEngines.checkEngineWithArgsInstalled(String framework, String version, Boolean cpu, Boolean gpu, Boolean rosetta, Integer minJavaVersion, String enginesDir). The difference is that this method has the arguments defining the system (rosetta and minJavaVersion) set automatically. They cannot be set to null or any other value, their value will be the value of the system that is running the software. Thus this method will be equivalent to the other when the values for the arguments rosetta and minJavaVersion align with the reality of the process. For example, in a process where Rosetta is being used and the Java version of the JVM is 8, InstalledEngines.checkEngineWithArgsInstalledForOS(null, null, null, null, enginesDir) and InstalledEngines.checkEngineWithArgsInstalled(null, null, null, null, true, 8, enginesDir) are equivalent.
The output is a List of instances of DeepLearningVersion from which the wanted information about the engine can be obtained.
- framework: name of the DL framework, if
nullit will look for any DL framework. The list of possible frameworks can be found here. - version: version of the DL framework of interest, if
nullit will look for any version - cpu: whether it supports CPU or not, if
nullit will look for both - gpu: whether it supports GPU or not, if
nullit will look for both - enginesDir: directory where the engines are installed
Below there are several examples on how the method works using the image above as the installation directory.
In this example, all the engines that support both CPU and GPU computation are retrieved. Note how Boolean and Integer are used instead of boolean and int, this happens because the first ones allow setting the values to null.
String enginesDir = "C:\\Users\\carlos\\icy\engines";
Boolean cpu = true;
Boolean gpu = true;
// The rest of parameters are set to null because there is not interest in them
String framework = null;
String version = null;
List<DeepLearningVersion> engines = InstalledEngines.checkEngineWithArgsInstalledForOS(framework, version, cpu, gpu, enginesDir);
System.out.println(engines.size());
System.out.println(engines.get(0).getFramework());
Output (looking at the example image above, only 8 engines out of 10 support CPU and GPU, and all of them are for pytorch):
8
pytorch
If all the parameters are set to null, all the installed engines are retrieved:
String enginesDir = "C:\\Users\\carlos\\icy\engines";
Boolean cpu = null;
Boolean gpu = null;
String framework = null;
String version = null;
List<DeepLearningVersion> engines = InstalledEngines.checkEngineWithArgsInstalledForOS(framework, version, cpu, gpu, enginesDir);
System.out.println(engines.size());
Output (looking at the example image above, 10 engines are installed):
10
Now look for Tensorflow engines, no more constraints:
String enginesDir = "C:\\Users\\carlos\\icy\engines";
// Note that the tag 'tensorflow_saved_model_bundled' would be equivalent
String framework = "tensorflow";
// Everything else is set to null
Boolean cpu = null;
Boolean gpu = null;
String version = null;
List<DeepLearningVersion> engines = InstalledEngines.checkEngineWithArgsInstalledForOS(framework, version, cpu, gpu, enginesDir);
System.out.println(engines.size());
System.out.println(engines.get(0).getFramework());
Output:
2
tensorflow
Filter for Tensorflow version 1.15.0, no more constraints:
String enginesDir = "C:\\Users\\carlos\\icy\engines";
// Note that the tag 'tensorflow_saved_model_bundled' would be equivalent
String framework = "tensorflow";
String version = 1.15.0;
// Everything else is set to null
Boolean cpu = null;
Boolean gpu = null;
List<DeepLearningVersion> engines = InstalledEngines.checkEngineWithArgsInstalledForOS(framework, version, cpu, gpu, enginesDir);
System.out.println(engines.size());
System.out.println(engines.get(0).getVersion());
Output:
1
1.15.0
This example looks for a tensorfow 1.15.0 engine that supports GPU computation:
String enginesDir = "C:\\Users\\carlos\\icy\engines";
// Note that the tag 'tensorflow_saved_model_bundled' would be equivalent
String framework = "tensorflow";
String version = 1.15.0;
Boolean gpu = true;
// Everything else is set to null
Boolean cpu = null;
List<DeepLearningVersion> engines = InstalledEngines.checkEngineWithArgsInstalledForOS(framework, version, cpu, gpu, enginesDir);
System.out.println(engines.size());
Output (no engine satistisfies the criteria):
0