Skip to content

Latest commit

 

History

History
259 lines (180 loc) · 12.8 KB

programming-language-interoperability.md

File metadata and controls

259 lines (180 loc) · 12.8 KB
Raw

Ouinet programming language interoperability

  • Document version: 0.1.0
  • Last update: February, 2024

0. Overview

This document aims to clarify the current status of the library in terms of cross-language integration and possible improvements. The challenges and goals resulting from the analysis are organized in three different sections:

  1. Using Ouinet libraries with other programming languages
  2. Using libraries from other programming languages in Ouinet
  3. Ouinet refactoring strategy

The sources used as a reference for this plan are stored in gitlab.com/equalitie/ouinet.git and consists of around 22 thousand lines of C++ code with the following distribution:

Lines of code analysis

Language files blank comment code
C++ 94 5200 1770 21981
C/C++ Header 146 3332 1585 11822
CMake 22 199 123 1241
Kotlin 7 86 20 801
Java 3 96 99 782
Python 7 251 259 706
Bourne Shell 9 125 123 629
Go 1 85 45 283

C++ standard support

The base code of Ouinet's core supports C++ 20 but some optional dependencies like i2pd still require an upgrade.

module standard
ouinet-base c++20
boost c++20
i2pd c++11

1. Using Ouinet libraries with other programming languages

Current JNI implementation

There's already an integration with Android that uses JNI to expose a Config builder and a Client class to Java and Kotlin applications.

Based on this implementation we've identified the following methods that are enough to interact with the Ouinet service; Start, and Stop to control the Client, GetStatus to monitor it, and an optional method to retrieve the TLS certificates root defined in Ouinet's conf file GetCARootCert.

  • Client control:
StartClient
StopClient
  • Client monitoring:
GetClientState
  • Utilities:
GetCARootCert

Some attributes related to the Client wrapper are also exposed and can be classified in 5 groups; General, Logging, Injector, BEP5 Cache and Request mechanism

  • General:
String frontEndEp
String listenOnTcp
String localDomain
String maxCachedAge
String originDohBase
String ouinetDirectory
  • Logging:
boolean enableLogFile
LogLevel logLevel
  • Injector:
String injectorCredentials
String injectorTlsCert
String injectorTlsCertPath
String caRootCertPath
String tlsCaCertStorePath
  • BEP5 Cache:
boolean cachePrivate
String cacheHttpPubKey
String cacheStaticContentPath
String cacheStaticPath
String cacheType
Set<String> btBootstrapExtras
  • Request mechanism:
boolean disableBridgeAnnouncement
boolean disableInjectorAccess
boolean disableOriginAccess
boolean disableProxyAccess

C API

Considering that most of the FFI mechanisms available are based on the linking of C-like methods and variables, it'd be useful to expose first a Ouinet C API that contains the main control methods and the attributes needed to use the service. This API should be flexible enough to be reused by the Rust and the Golang bindings. It's not mandatory but desirable that this API should be also compatible with the binding generator tools (e.g. SWIG) for other high level languages like Python or Ruby.

An useful experiment to define the structure of this API would be to create a small C++ application and integrate Ouinet libraries, this exercise will clarify the usability of the current public libraries before exposing them to other languages.

Python bindings

Ouinet makes use of Boost libraries so using Boost.Python could help us to have a simple and consistent exposure of the new bindings.

One of the main problems to solve if we follow this approach is that Ouinet builds Boost statically which is incompatible with the resulting shared objects generated by the binding targets. Removing this blocker should be the first task before starting with the exposure of the wrapper objects and attributes.

Tasks:

  • Solve problems related to build Boost.Python with static Boost libraries or use an alternative like SWIG.
  • Modify CMake scripts to build the targets.
  • Package bindings, create a setup tools script and simplify its install.
  • Test the bindings with threads and co-routines.
  • Perform an example integration with a Python application or service.

Rust via FFI

Rust has support for integrating C libraries into Rust programs via extern. A similar integration with C++ could be achieved using the same FFI capabilities and using extern on the C++ side to generate C header files that can be used with Rust.

The most challenging part of this integration would be to deal with memory management and throwing errors and exceptions on the C++, these particular aspects be carefully covered by the automated tests.

Tasks:

  • Extern the objects and attributes needed to C header libraries.
  • Modify CMake files in Ouinet to automatically build the C .h files.
  • Import the C headers in Rust.
  • Implement the error handling mechanism on the Rust side.
  • Check the feasibility of packaging it as crate.
  • Perform an example integration with a Rust application or service.

Golang via FFI

cgo is one of the most used packages in Golang to interface with C/C++ libraries. Similarly to the Rust workflow we will rely on the C API to implement the required handlers to pass values between both languages via FFI.

Tasks:

  • Extern the objects and attributes needed to C header libraries.
  • Implement cgo handlers to use the C++ library.
  • Check the feasibility of packaging it.
  • Perform an example integration with a Golang application or service.

2. Using libraries from other programming languages in Ouinet

The main purpose of reusing the code generated by similar applications is to make our Ouinet base code more stable and easier to maintain. One of the main integration targets is btdht package, written in Rust for the Ouisync project.

Rust

cbindgen is a good candidate to automatically generate C headers that can be used on the Ouinet C++ base code. We will use Ouisync's btdht as an integration proof of concept.

Tasks:

  • Define the public API that will be externalized.
  • Add cbindgen to the Cargo targets.
  • Modify CMake scripts on Ouinet's side to get the btdht crate.
  • Build a C++ test program to use the DHT.

Golang

We have already an integration example in Ouinet. The OuiService interface has an implementation of LampShade and expose its bindings to C++ via cgo.

Tasks:

  • Choose a Golang package that could be interesting for Ouinet.
  • Define the public API that will be externalized.
  • Use cgo to expose the API to C++.
  • Modify Ouinet CMake scripts to build the package.
  • Build a C++ test program to use the package.

3. Ouinet refactoring strategy

One of the main benefits of reusing libraries from Rust and Golang is the possibility of gradually refactoring Ouinet, starting by the components that add more complexity or those that are most difficult to maintain.

This section offers an overview of the main Ouinet components and also defines possible targets to start the refactoring. It's important to mention that some of the current components could be splitted or removed during the refactoring process as the redesing could alse benefit from changes to the code structure as long as the specs defined in the whitepaper are met.

Main components

The following list points to the main Ouinet components classified and organized from a functionality perspective. The purpose of this section is to have a catalog of components that would be necesary to port (or implement from scratch) in order to be compatible with Ouinet specs defined in the white paper.

Candidates for refactoring

  1. Considering that the functionality of the BEP5 DHT is critical in the current implementation of Ouinet this component appears in the top of the list of components to refactor. Processes using the DHT are complex and difficult to optimize and maintain, reusing btdht could improve the performance and reduce the amount of time to find and fix bugs.

  2. DNS over HTTPS is implemented but not optimized, as the feature is not actively used becomes a good candidate to implement with a well maintained library like doh-server.

  3. The HTTP Proxy which could be implemented with a custom crate that relies on the native Rust library http and a crate for dealing with the TLS encryption.

Prerequisites of the refactoring

  • Estimate the time needed to peform a refactoring when most of the tasks from sections 1 and 2 of the current plan are complete.
  • Define team capacity, goals and schedule the milestones.
  • Make sure that the integration and acceptance test suites are complete and stable before starting any refactoring.
  • Establish quality and performance metrics that should be fulfilled by the ported components.
  • Plan the releases containing refactoring.

References