Distributed Tracing

Distributed Tracing

Tracing support is provided via opentracing and the openzipkin implementation.

Any of the gRPC service implementations or delegate classes called by the services can be instrumented with @Traced annotation to indicate that it needs to be traced - i.e. participate in an existing trace or start an entirely new trace.

See GreeterService and HelloServiceBean as examples for tracing at gRPC service and delegate classes.

Follow these steps to add Tracing to GJEX methods:

• Add @Traced annotation to public methods of any class that is either directly instantiated via the service's Guice bundle - e.g. HelloWorldModule or instantiated via injection in referenced classes.
  • @Traced may be provided additional attribute like withSamplingRate=0.5f to indicate sampling at 50% of requests at granularity of 100 request blocks.
  • @Traced may be provided additional attribute like withTracingSampler=AllWhitelistTracingSampler.class to indicate global Tracing on-off overrides. Note that this is currently interpreted only for gRPC Service classes i.e. sub-types of BindableService
  • Additional information may be added as tags to the active Span (method with @Traced annotation) by calling addToTrace(key,value) as seen below:

  @Traced
  public void sayHelloInBean(@NotNull @Valid HelloBean helloBean) {
      ...........
      addToTrace("Request", helloBean.toString());
  }

• Run OpenZipkin or Run Jaeger with Zipkin compatible endpoint
• Edit GJEX Configuration if required to update host and port of Zipkin collector you are running.
• Run your service via GJEX. You will be able to see traces like below in Zipkin:

And in Jaeger as below (see Span with "Request" Tag):

Modifying Tracing configuration at runtime

Tracing sampling can be modified on a running GJEX instance using the admin Http/1 API. For e.g. an API request to http://<hostname?:9998/tracingconfig like so:

{
   "apiName" : "helloworld.greeter/sayhello",	
   "componentName":"greeterservice.sayhello",
   "samplingRate":"1.0"
}

turns on tracing for the sayHello() gRPC method of the sample Greeter service. Setting the "samplingRate" to 0.0f turns off tracing for this component.

Note that configurations modified this way are not persisted and therefore do not survive GJEX instance restarts.