Serval - an API supporting Computer Aided Translation for all the languages of the world!

Serval is a REST API for natural language processing services for all languages. For the REST documentation use the Swagger site here.

Serval Architecture

Serval is designed using a microservice architecture with the following deployments:

• Serval.APIServer
  • This is the top level API layer. All REST calls are made through this layer.
  • It primarily handles the definition of files and corpora, per-client permissions, validation of requests and assembling NLP requests from files and database entries, such as inserting pretranslations into the USFM structure.
  • This includes all components under ./src/Serval/src except for the auto-generated Client and gRPC projects, which are used to handle communication with the API layer.
• Serval.Machine.EngineServer
  • Exposes multiple engine types to the APIServer
  • Can be directly invoked by the APIServer
  • Handles short-lived NLP requests, including loading SMT models for on-demand word graphs.
  • Queues up jobs for the JobServer
  • Primary functionality is defined in Serval.Machine.Shared, with the deployment configuration is defined in Serval.Machine.EngineServer
  • Reports back status to the APILayer over gRPC
• Serval.Machine.JobServer
  • Executes Hangfire NLP jobs
  • Queues up ClearML NLP jobs, using the ClearML API and the S3 bucket
  • Preprocesses training and inferencing data
  • Postprocesses inferencing results and models from the ClearML job
  • Primary functionality is defined in Serval.Machine.Shared, with the deployment configuration is defined in Serval.Machine.JobServer
  • Reports back status to the APILayer over gRPC
• Echo.EchoTranslationEngine
  • The echo engine is for testing both the API layer and the deployment in general. Other components of the microservice are:
• SIL.DataAccess
  • Abstracts all MongoDB operations
  • Enables in-memory database for testing purposes
  • Replicates the functionality of EF Core
• ServiceToolkit
  • Defines common models, configurations and services both used by the Serval.APIServal deployment and the Serval.Machine deployments.

Development

Setting up Your Environment

• Use VS Code with all the recommended extensions
• Development is supported in Ubuntu and Windows WSL2
  • For Ubunutu, use microsoft's distribution of .net
  • Ubuntu 22.04 and 24.04 are currently supported
• Install the repositories:
  • To develop Serval, you will also likely need to make changes to the Machine repo as well - they are intricately tied.
  • To enable Serval to use your current edits in Machine (rather than the nuget package) you need to install Machine in an adjacent folder to Serval
    • i.e., if your serval repo is cloned into /home/username/repos/serval, machine should be in /home/username/repos/machine
  • Make sure that you build Machine using before you build Serval

Option 1: Docker Compose local testing deployment

These instructions are for developing/testing using docker-compose (rather than locally/bare metal) With both the serval and machine repos installed, in the serval root folder, run ./docker_deploy.sh To debug in VSCode, launch "DockerComb" after to containers come up (about 5 -10 seconds). This will allow you to debug all 4 serval containers.

Option 2: Bare metal local testing deployment

Alternatively, you can develop without containerizing Serval.

Install MongoDB 6.0 as a replica set run it on localhost:27017. (You can run docker compose -f docker-compose.mongo.yml up from the root of the serval repo to do so).

Make sure that the environment variable ASPNETCORE_ENVIRONMENT is set to "Development" by running export ASPNETCORE_ENVIRONMENT=Development or adding it to your .bashrc.

Open "Serval.sln" and debug the ApiServer.

Coding guidelines

Coding guidelines are documented on the wiki

(Optional) Get your machine.py images setup

When jobs are run, they are queued up on ClearML. If you want to have your own agents for integration testing (and you have a GPU with 24GB RAM), you can do the following:

• clone the machine.py repo
• Build the docker image with docker build . -t local.mpy for a GPU inamge or docker build . -f dockerfile.cpu_only -t local.mpy.cpu_only for a CPU only image.
• Register your machine as a ClearML agent (see dev team for details)
  • Make sure you do NOT "always pull the image"! The images you are building are stored locally.
• Set the following environment variables:

export MACHINE_PY_IMAGE=local.mpy
export MACHINE_PY_CPU_IMAGE=local.mpy.cpu_only

Debugging

To access Serval API

• Internal QA:
  • Use the VPN
  • In C:\Windows\System32\drivers\etc\hosts, enter in a line for 10.3.0.119 serval-api.org
  • go to https://machine-api.org/swagger and accept the security warning
• External QA:
  • go to https://qa.serval-api.org/swagger and accept the security warning

To view pod logs:

• Run: kubectl get pods to get the currently running pods
• Run: kubectl logs <pod name>
• Run: kubectl describe pod <pod name> to check a stalled pod stuck in containercreating

Running the API E2E Tests

In order to run the E2E tests, you will need to have the appropriate credentials

• Get Client ID and Client Secret from auth0.com
  • Login, go to Applications-> Applications -> "Machine API (Test Application)" or similar
  • Copy Client ID into Environment variable SERVAL_CLIENT_ID
  • Copy Client Secret into Environment variable SERVAL_CLIENT_SECRET
  • Copy the auth0 url into Environment variable SERVAL_AUTH_URL (e.g. SERVAL_AUTH_URL=https://sil-appbuilder.auth0.com)
  • Set SERVAL_HOST_URL to the api's URL (e.g. SERVAL_HOST_URL=http://localhost) Now, when you run the tests from Serval.E2ETests, the token will automatically be retrieved from Auth0.

Debugging the S3 bucket

To view files stored in the bucket, run

aws s3 ls s3://silnlp/<deployment environment>

Mongo debugging

• First, get the mongo pod name: kubectl get pods -n serval
• Then forward to a local port, such as 28015: kubectl port-forward <pod name> 28015:27017 -n serval
• Download MongoDB Compass.
• Then, open MongoDB Compass and connect to mongodb://localhost:28015/?directConnection=true

Deployment

ASPNETCORE Environments:

The deployment environment is set by the environment variable ASPNETCORE_ENVIRONMENT.

• ASPNETCORE_ENVIRONMENT=Production
  • Full deployment, full NMT and SMT builds.
  • Used in all kubernetes deployments
• ASPNETCORE_ENVIRONMENT=Staging:
  • Full Deploymnet, dummy NMT building (10 steps, small model).
    • These default build configurations can be overwritten.
  • Used in Docker-compose local staging
• ASPNETCORE_ENVIRONMENT=Development:
  • Non-docker use only

Kubernetes deployments:

There are 3 different environments that Serval is deployed to:

• Internal QA for testing out the deployment
• External QA for clients to test new updates before production
• Production

To deploy the cluster

• Add the dallas-rke KubeConfig to your kubectl configs
• Run kubectl config use-context dallas-rke
• First, startup the storage (using internal qa for example)
• helm install serval-pvc deploy/serval-pvc -n nlp -f deploy/qa-int-values.yaml
• Now you can turn on Serval
• helm install serval deploy/serval -n nlp -f deploy/qa-int-values.yaml

To update the cluster

• To upgrade Serval:
  • For QA internal Run:
    • kubectl config use-context dallas-stage
    • helm upgrade serval deploy/serval -n nlp -f deploy/qa-int-values.yaml
  • For QA external Run:
    • kubectl config use-context dallas-stage
    • helm upgrade serval deploy/serval -n serval -f deploy/qa-ext-values.yaml
  • For Production Run:
    • kubectl config use-context aws-rke
    • helm upgrade serval deploy/serval -n serval -f deploy/values.yaml

Special thanks to

BugSnag for error reporting: