Sample http4s REST client/server with client macro derivation #552

L-Lavigne · 2019-01-29T01:24:18Z

Adds sample http4s REST client/server implementations, which we will eventually derive from the same @service-annotated classes as we do for RPC. The roadmap for this is described in #182.

Also implements HTTP client-side derivation using @http annotations, based on #368. Those are tested with unary clients, but support for streaming exists in our http4s samples and can be auto-derived as well. The server-side remains to be done.

Note that this client derivation is implemented inside the serviceImpl class that processes the @service annotation, due to difficulties in separating the macro processing between this class and another one for the http module. It still needs to be moved there.

Warning: do not merge in current state as this does not pass all unit tests, because auto-derived HTTP servers are not yet implemented.

This is basically a conversion to/from the internal fs2.Stream.

Note: Update of Monix/FS2 conversions is in progress with some outdated implementations removed, which fails several tests.

codecov · 2019-02-18T19:42:36Z

Codecov Report

Merging #552 into master will increase coverage by 0.09%.
The diff coverage is 81.81%.

@@            Coverage Diff             @@
##           master     #552      +/-   ##
==========================================
+ Coverage   84.61%   84.71%   +0.09%     
==========================================
  Files          69       71       +2     
  Lines        1066     1099      +33     
  Branches       15       16       +1     
==========================================
+ Hits          902      931      +29     
- Misses        164      168       +4

Impacted Files	Coverage Δ
...c/main/scala/higherkindness/mu/http/protocol.scala	`100% <100%> (ø)`
.../main/scala/higherkindness/mu/http/implicits.scala	`81.25% <81.25%> (ø)`
...herkindness/mu/rpc/channel/cache/ClientCache.scala	`85.71% <0%> (+3.57%)`	⬆️
...c/main/scala/higherkindness/mu/MonixAdapters.scala	`92.3% <0%> (+3.84%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bc51c6f...bf6e853. Read the comment docs.

rafaparadela · 2019-03-04T01:51:12Z

This PR is ready for review. Below a brief summary about how it works:

HTTP Protocol

Given this example protocol below:

import monix.reactive.Observable
@service(Avro) trait Greeter[F[_]] {

  @http def sayHellos(requests: Observable[HelloRequest]): F[HelloResponse]

  @http def sayHelloAll(request: HelloRequest): Observable[HelloResponse]

  @http def sayHellosAll(requests: Observable[HelloRequest]): Observable[HelloResponse]

}

With the annotation @http we would be letting the macro know that an HTTP Server should be built, which must be able to serve responses to call from the paths: sayHellos, sayHelloAll, and sayHellosAll.

Run the HTTP Server

Assuming that implicits ExecutionContext and ContextShift are already in the scope:

implicit val ec                   = monix.execution.Scheduler.Implicits.global
implicit val cs: ContextShift[IO] = IO.contextShift(ec)

We need to provide an implicit instance of the interpreter of the service, that is, the Greeter[F] with the actual implementation of every operation.

implicit val greeterHndlerIO = new GreeterHandler[IO]

Then we need a server builder by:

val server = HttpServer.bind(80, "localhost", Greeter.route[IO])

And finally, we can run the server:

server.resource.use(_ => IO.never).start.unsafeRunSync()

Run the HTTP Client

Firstly, we should set the Uri to reach the server:

val serviceUri: Uri = Uri.unsafeFromString(s"http://localhost:80")

Then, we can instantiate the derived client:

val client = Greeter.httpClient[IO](serviceUri)

And now we can perform rest calls. For instance, by:

val requests = Observable(HelloRequest("hey"), HelloRequest("there"))
val response = BlazeClientBuilder[IO](ec).resource.use(client.sayHellos(requests)(_))

where response is IO[HelloResponse] as expected.

fedefernandez · 2019-03-04T07:48:29Z

@rafaparadela this looks pretty cool. I didn't take a look at the code yet, but I have some questions/doubts:

Is it possible now or planned for the future the ability to modify the paths? Use the method name seems a bit strict for me.
Do we need a monix Scheduler or an ExecutionContext?
What's the type returned by HttpServer.bind

project/ProjectPlugin.scala

modules/http/src/main/scala/higherkindness/mu/http/Utils.scala

modules/http/src/test/scala/higherkindness/mu/rpc/http/GreeterRestTests.scala

...c/test/scala/higherkindness/mu/rpc/prometheus/client/BaseMonitorClientInterceptorTests.scala

project/ProjectPlugin.scala

build.sbt

pepegar

This looks great! thank you @rafaparadela !

I've added a couple of minor comments

modules/http/src/main/scala/higherkindness/mu/http/Utils.scala

rafaparadela · 2019-03-06T07:28:55Z

Addressed all the comments by @juanpedromoreno and @pepegar

modules/internal/src/main/scala/higherkindness/mu/rpc/internal/serviceImpl.scala

juanpedromoreno · 2019-03-06T12:47:07Z

modules/internal/src/main/scala/higherkindness/mu/rpc/internal/serviceImpl.scala

+
+      val scheduler: List[Tree] = operations
+        .find(_.operation.isMonixObservable)
+        .map(_ => q"import _root_.monix.execution.Scheduler.Implicits.global")


The execution context needs to be asked as implicit parameter. We cannot add the global here.

Moved at fb91483

modules/internal/src/main/scala/higherkindness/mu/rpc/internal/serviceImpl.scala

project/ProjectPlugin.scala

…otocols' into feature/182-http-support-from-protocols # Conflicts: # project/ProjectPlugin.scala

juanpedromoreno

This is great, thanks so much @L-Lavigne and @rafaparadela !

fedefernandez

Left some comments. Thanks @rafaparadela @L-Lavigne

fedefernandez · 2019-03-08T08:27:02Z

modules/http/src/main/scala/higherkindness/mu/http/implicits.scala

+      l or r
+    }
+
+    implicit private val throwableDecoder: Decoder[Throwable] =


Please, help me to understand this. This is only used for streams and observable, right? The server generates an Either that is parsed in the client. Am I right?

The implicits.scala was enterally created by @L-Lavigne, so I'll let him answer all your questions.

fedefernandez · 2019-03-08T08:28:38Z

modules/http/src/main/scala/higherkindness/mu/http/implicits.scala

+            .getConstructor(classOf[String])
+            .newInstance(msg)
+            .asInstanceOf[Throwable]
+      }


Since we're losing the stacktrace here, I'd use our custom case class:

final case class UnexpectedError(status: Status, msg: Option[String] = None) extends RuntimeException(status + msg.fold("")(": " + _)) with NoStackTrace //... implicit val unexpectedErrorDecoder: Decoder[UnexpectedError] = deriveDecoder

IMO we need to avoid custom logic in the encoders/decoders

fedefernandez · 2019-03-08T08:31:14Z

modules/http/src/main/scala/higherkindness/mu/http/implicits.scala

+        final def apply(a: Either[A, B]): Json = a match {
+          case Left(a)  => a.asJson
+          case Right(b) => b.asJson
+        }


new Encoder[Either[A, B]] { final def apply(a: Either[A, B]): Json = a.fold(_.asJson, _.asJson) }

fedefernandez · 2019-03-08T08:32:19Z

modules/http/src/main/scala/higherkindness/mu/http/implicits.scala

+
+    implicit val throwableEncoder: Encoder[Throwable] = new Encoder[Throwable] {
+      def apply(ex: Throwable): Json = (ex.getClass.getName, ex.getMessage).asJson
+    }


Based on the above:

implicit val unexpectedErrorEncoder: Encoder[UnexpectedError] = deriveEncoder

fedefernandez · 2019-03-08T08:33:53Z

modules/http/src/main/scala/higherkindness/mu/http/implicits.scala

+      def apply(ex: Throwable): Json = (ex.getClass.getName, ex.getMessage).asJson
+    }
+
+    def asJsonEither(implicit encoder: Encoder[A]): Stream[F, Json] = stream.attempt.map(_.asJson)


def asJsonEither(implicit encoder: Encoder[A]): Stream[F, Json] = stream.attempt.bimap(e => UnexpectedError(e.getClass.getName, Option(e.getMessage)), _.asJson)

I'm sorry @fedefernandez, I don't really understand how you expect to instantiate the UnexpectedError without having the Status.

Maybe you're proposing

case class UnexpectedError(status: String, msg: Option[String] = None)

instead of

case class UnexpectedError(status: Status, msg: Option[String] = None)

rafaparadela · 2019-03-08T20:23:59Z

@fedefernandez Please, have a look at implicits.scala again and let me know if now makes more sense to you. Thanks

fedefernandez

LGTM!

L-Lavigne

Good work! A few comments.

L-Lavigne · 2019-03-08T21:34:00Z

modules/http/src/test/scala/higherkindness/mu/rpc/http/GreeterRestTests.scala

@@ -254,15 +250,5 @@ class GreeterRestTests
        .getOrElse(sys.error("Stuck!")) shouldBe Nil
    }

-    "serve ScalaCheck-generated POST requests with bidirectional Observable streaming" in {


Just curious, were there any issues with the ScalaCheck tests?

(resolved offline)

L-Lavigne · 2019-03-08T22:23:37Z

modules/http/src/main/scala/higherkindness/mu/http/implicits.scala

 }

+final case class UnexpectedError(className: String, msg: Option[String])


The name is a bit generic - most errors are unexpected. The way we're using those, they're closer to "RequestException"s, or 4XX-status ResponseExceptions. Can we combine those error types or rename this one?

(resolved offline)

L-Lavigne · 2019-03-08T23:54:45Z

As discussed offline, I approve PR (can't set it to that status as I'm the owner), and I'll make a separate PR to adjust the naming of the exceptions.
(@rafaparadela please let me know if you don't have merge permissions and then I'll do it)

juanpedromoreno and others added 11 commits July 18, 2018 13:47

Initial steps for http integration (#203)

ed9cf04

Implemented sample client and server REST handlers to be generated

93f8ace

Added monix.Observable implementation

b03637c

This is basically a conversion to/from the internal fs2.Stream.

Implemented error handling for unary and streaming REST services (#258)

2a1cc42

Tentative fix for the hanging Monix-Observable tests

a74604e

Merge branch 'master' into feature/182-http-support-from-protocols

6424dab

Undo Single Abstract Method syntax to restore 2.11 compatibility

8caee87

Merge master into branch

65d705d

Add auto-derived HTTP client implementation, move packages

98df299

Note: Update of Monix/FS2 conversions is in progress with some outdated implementations removed, which fails several tests.

Fix Monix/FS2 conversions using updated dependency

a216c63

fixes Scala 2.11 compilation error

051519d

rafaparadela added 16 commits February 20, 2019 14:43

fixes unit tests to prove http client derivation

8eb36b4

removes some println

c172539

adds more tests

20aae64

removes the macro params that can be inferred

15abe94

builds the client according to the typology of the request

5d5b726

fixes macro

683147c

advances with client derivation

54990d8

adds more unit test to prove the derived http client

fe34e47

restores the derivation of the rpc server, without the refactoring

1132fc2

re-applies part of the refactoring little by little

97d1c73

applies the refactoring again with the fix

6db7ab3

derived the simplest http route that only serves GET calls

2a9cd3d

derived the stream reaquests http server

06f2b83

fixes the binding pattern in POST routes

3888be7

adds tests to cover all the possible types of endpoints

965fdd4

removes unused imports

d6ce882

Merge branch 'master' into feature/182-http-support-from-protocols

795eb12

juanpedromoreno reviewed Mar 5, 2019

View reviewed changes

build.sbt Outdated Show resolved Hide resolved

pepegar reviewed Mar 5, 2019

View reviewed changes

solves all the comments in code review

c922e0f

juanpedromoreno reviewed Mar 6, 2019

View reviewed changes

modules/internal/src/main/scala/higherkindness/mu/rpc/internal/serviceImpl.scala Outdated Show resolved Hide resolved

juanpedromoreno reviewed Mar 6, 2019

View reviewed changes

modules/internal/src/main/scala/higherkindness/mu/rpc/internal/serviceImpl.scala Outdated Show resolved Hide resolved

modules/internal/src/main/scala/higherkindness/mu/rpc/internal/serviceImpl.scala Outdated Show resolved Hide resolved

juanpedromoreno reviewed Mar 6, 2019

View reviewed changes

fedefernandez reviewed Mar 6, 2019

View reviewed changes

modules/internal/src/main/scala/higherkindness/mu/rpc/internal/serviceImpl.scala Outdated Show resolved Hide resolved

rafaparadela added 4 commits March 6, 2019 17:15

expressed type as FQN and propagated encoder/decoders constraints

9b0a8f3

removes the import of monix.Scheduler in the macro

fb91483

replaces executionContext by Schedule at some points

854b0d6

adds _root_ to ExecutionContext

f6eab68

juanpedromoreno reviewed Mar 7, 2019

View reviewed changes

project/ProjectPlugin.scala Outdated Show resolved Hide resolved

project/ProjectPlugin.scala Outdated Show resolved Hide resolved

juanpedromoreno and others added 3 commits March 7, 2019 23:39

Apply suggestions from code review

ad90c2d

removes circe-generic

f1f60ff

Merge remote-tracking branch 'origin/feature/182-http-support-from-pr…

94dba66

…otocols' into feature/182-http-support-from-protocols # Conflicts: # project/ProjectPlugin.scala

juanpedromoreno approved these changes Mar 8, 2019

View reviewed changes

fedefernandez reviewed Mar 8, 2019

View reviewed changes

replaces Throwable by UnexpectedError and its encoder/decoder

bf6e853

fedefernandez approved these changes Mar 8, 2019

View reviewed changes

L-Lavigne commented Mar 8, 2019

View reviewed changes

rafaparadela merged commit cdf7e99 into master Mar 8, 2019

rafaparadela mentioned this pull request Mar 11, 2019

Monix.Observable <> Fs2.Stream #574

Closed

L-Lavigne mentioned this pull request Mar 18, 2019

WIP: derive http client from service #368

Closed

4 tasks

fedefernandez mentioned this pull request Apr 10, 2019

Make the HTTP feature more flexible to foster easy migrations #583

Open

diesalbla deleted the feature/182-http-support-from-protocols branch May 7, 2019 15:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sample http4s REST client/server with client macro derivation #552

Sample http4s REST client/server with client macro derivation #552

L-Lavigne commented Jan 29, 2019

codecov bot commented Feb 18, 2019 •

edited

Loading

rafaparadela commented Mar 4, 2019

fedefernandez commented Mar 4, 2019

pepegar left a comment

rafaparadela commented Mar 6, 2019

juanpedromoreno Mar 6, 2019

rafaparadela Mar 7, 2019

juanpedromoreno left a comment

fedefernandez left a comment

fedefernandez Mar 8, 2019

rafaparadela Mar 8, 2019

fedefernandez Mar 8, 2019

fedefernandez Mar 8, 2019

fedefernandez Mar 8, 2019

fedefernandez Mar 8, 2019

fedefernandez Mar 8, 2019

rafaparadela Mar 8, 2019 •

edited

Loading

rafaparadela commented Mar 8, 2019

fedefernandez left a comment

L-Lavigne left a comment

L-Lavigne Mar 8, 2019

L-Lavigne Mar 8, 2019

L-Lavigne Mar 8, 2019

L-Lavigne Mar 8, 2019

L-Lavigne commented Mar 8, 2019

		}

		final case class UnexpectedError(className: String, msg: Option[String])

Sample http4s REST client/server with client macro derivation #552

Sample http4s REST client/server with client macro derivation #552

Conversation

L-Lavigne commented Jan 29, 2019

codecov bot commented Feb 18, 2019 • edited Loading

Codecov Report

rafaparadela commented Mar 4, 2019

HTTP Protocol

Run the HTTP Server

Run the HTTP Client

fedefernandez commented Mar 4, 2019

pepegar left a comment

Choose a reason for hiding this comment

rafaparadela commented Mar 6, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

juanpedromoreno left a comment

Choose a reason for hiding this comment

fedefernandez left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rafaparadela Mar 8, 2019 • edited Loading

Choose a reason for hiding this comment

rafaparadela commented Mar 8, 2019

fedefernandez left a comment

Choose a reason for hiding this comment

L-Lavigne left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

L-Lavigne commented Mar 8, 2019

codecov bot commented Feb 18, 2019 •

edited

Loading

rafaparadela Mar 8, 2019 •

edited

Loading