Skip to content

Commit

Permalink
Improve assertion docs (#318)
Browse files Browse the repository at this point in the history
  • Loading branch information
keynmol authored Jun 23, 2021
1 parent b019de7 commit c182dce
Show file tree
Hide file tree
Showing 7 changed files with 237 additions and 17 deletions.
186 changes: 171 additions & 15 deletions docs/expectations.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,30 +7,180 @@ Expectations are pure, composable values. This forces developers to separate the

The easiest way to construct expectactions is to call the `expect` macro, which is built using the [expecty](https://github.com/eed3si9n/expecty/) library.

## TL;DR

- Assert on boolean values using `expect`:

```scala mdoc:compile-only
expect(myVar == 25 && list.size == 4)
```

- Compose expectations using `and`/`or`

```scala mdoc:compile-only
(expect(1 == 1) and expect(2 > 1)) or expect(5 == 5)
```

- Use varargs short form for asserting on all boolean values

```scala mdoc:compile-only
expect.all(1 == 1, 2 == 2, 3 > 2)
```

- Use `forEach` to test every element of a collection (or anything that
implements `Foldable`)

```scala mdoc:compile-only
forEach(List(1, 2, 3))(i => expect(i < 5))
```

- Use `exists` to assert that at least one element of collection matches
expectations:

```scala mdoc:compile-only
exists(Option(5))(n => expect(n > 3)
```

- Use `expect.eql` for strict equality comparison (types that implement `Eq`
typeclass) and string representation diffing (using `Show` typeclass, fall
back to `toString` if no instance found) in
case of failure

```scala mdoc:compile-only
expect.eql(List(1, 2, 3), (1 to 3).toList)
```

See below how the output looks in case of failure

- Use `expect.same` for relaxed equality comparison (if no `Eq` instance is
found, fall back to universal equality) and relaxed string diffing (fall
back to `toString` implementation)

```scala mdoc:compile-only
expect.same(List(1, 2, 3), (1 to 3).toList)
```

- Use `success` or `failure` to create succeeding/failing expectations without
conditions

```scala mdoc:compile-only
val result = if(5 == 5) success else failure("oh no")
```

- Use `.failFast` to evaluate the expectation eagerly and raise the assertion error in your effect type

```scala mdoc:compile-only
for {
x <- IO("hello")
_ <- expect(x.length == 4).failFast
y = x + "bla"
_ <- expect(y.size > x.size).failFast
} yield expect(y.contains(x))
```

## Example suite

```scala mdoc
import weaver._
import cats.effect.IO

object MySuite2 extends SimpleIOSuite {
object ExpectationsSuite extends SimpleIOSuite {

object A {
object B {
object C {
def test(a: Int) = a + 5
}
}
}

pureTest("And/Or composition") {
pureTest("Simple expectations (success)") {
val z = 15

expect(A.B.C.test(z) == z + 5)
}

pureTest("Simple expectations (failure)") {
val z = 15

expect(A.B.C.test(z) % 7 == 0)
}


pureTest("And/Or composition (success)") {
expect(1 != 2) and expect(2 != 1) or expect(2 != 3)
}

pureTest("Varargs composition") {
pureTest("And/Or composition (failure") {
(expect(1 != 2) and expect(2 == 1)) or expect(2 == 3)
}

pureTest("Varargs composition (success)") {
// expect(1 + 1 == 2) && expect (2 + 2 == 4) && expect(4 * 2 == 8)
expect.all(1 + 1 == 2, 2 + 2 == 4, 4 * 2 == 8)
}

pureTest("Pretty string diffs") {
expect.same("foo", "bar")
pureTest("Varargs composition (failure)") {
// expect(1 + 1 == 2) && expect (2 + 2 == 4) && expect(4 * 2 == 8)
expect.all(1 + 1 == 2, 2 + 2 == 5, 4 * 2 == 8)
}

pureTest("Working with collections (success)") {
forEach(List(1, 2, 3))(i => expect(i < 5)) and
forEach(Option("hello"))(msg => expect.same(msg, "hello")) and
exists(List("a", "b", "c"))(i => expect(i == "c")) and
exists(Vector(true, true, false))(i => expect(i == false))
}

pureTest("Working with collections (failure 1)") {
forEach(Vector("hello", "world"))(msg => expect.same(msg, "hello"))
}

pureTest("Working with collections (failure 2)") {
exists(Option(39))(i => expect(i > 50))
}

pureTest("Foldable operations") {
val list = List(1,2,3)
import cats.instances.list._
forEach(list)(i => expect(i > 0)) and
exists(list)(i => expect(i == 3))
import cats.Eq
case class Test(d: Double)

implicit val eqTest: Eq[Test] = Eq.by[Test, Double](_.d)

pureTest("Strict equality (success)") {
expect.eql("hello", "hello") and
expect.eql(List(1, 2, 3), List(1, 2, 3)) and
expect.eql(Test(25.0), Test(25.0))
}

pureTest("Strict equality (failure 1)") {
expect.eql("hello", "world")
}

pureTest("Strict equality (failure 2)") {
expect.eql(List(1, 2, 3), List(1, 19, 3))
}

pureTest("Strict equality (failure 3)") {
expect.eql(Test(25.0), Test(50.0))
}

// Note that we don't have an instance of Eq[Hello]
// anywhere in scope
class Hello(val d: Double) {
override def toString = s"Hello to $d"

override def equals(other: Any) =
if(other != null && other.isInstanceOf[Hello])
other.asInstanceOf[Hello].d == this.d
else
false
}

pureTest("Relaxed equality comparison (success)") {
expect.same(new Hello(25.0), new Hello(25.0))
}

pureTest("Relaxed equality comparison (failure)") {
expect.same(new Hello(25.0), new Hello(50.0))
}

pureTest("Non macro-based expectations") {
Expand All @@ -41,23 +191,26 @@ object MySuite2 extends SimpleIOSuite {
test("Failing fast expectations") {
for {
h <- IO.pure("hello")
_ <- expect(h.nonEmpty).failFast
_ <- expect(h.isEmpty).failFast
} yield success
}

}
```

```scala mdoc:passthrough
println(weaver.docs.Output.runSuites(ExpectationsSuite).unsafeRunSync())
```

## Tracing locations of failed expectations

As of 0.5.0, failed expectations carry a `NonEmptyList[SourceLocation]`, which can be used to manually trace the callsites that lead to a failure.

By default, the very location where the expectation is created is captured, but the `traced` method can be use to add additional locations to the expectation.

```scala mdoc
object MySuite3 extends SimpleIOSuite {
object TracingSuite extends SimpleIOSuite {

pureTest("And/Or composition") {
pureTest("Tracing example") {
foo
}

Expand All @@ -66,6 +219,9 @@ object MySuite3 extends SimpleIOSuite {
def bar() = baz().traced(here)

def baz() = expect(1 != 1)

}
```

```scala mdoc:passthrough
println(weaver.docs.Output.runSuites(TracingSuite).unsafeRunSync())
```
5 changes: 4 additions & 1 deletion docs/logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ object LoggedTests extends IOSuite {
}


// We can oviously have tests receive loggers AND shared resources
// We can obviously have tests receive loggers AND shared resources
override type Res = String
override def sharedResource : Resource[IO, Res] =
Resource.pure[IO, Res]("hello")
Expand All @@ -34,3 +34,6 @@ object LoggedTests extends IOSuite {
}
```

```scala mdoc:passthrough
println(weaver.docs.Output.runSuites(LoggedTests).unsafeRunSync())
```
41 changes: 41 additions & 0 deletions docs/resources.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,44 @@ object HttpSuite extends IOSuite {
}
```

### Resources lifecycle

We can demonstrate the resource lifecycle with this example:

```scala mdoc
import java.util.concurrent.ConcurrentLinkedQueue

// We will store the messages in this queue
val order = new ConcurrentLinkedQueue[String]()

object ResourceDemo extends IOSuite {

def record(msg: String) = IO(order.add(msg)).void

override type Res = Int
override def sharedResource = {
val acquire = record("Acquiring resource") *> IO.pure(42)
val release = (i: Int) => record(s"Releasing resource $i")
Resource.make(acquire)(release)
}

test("Test 1") { res =>
record(s"Test 1 is using resource $res").as(success)
}

test("Test 2") { res =>
record(s"Test 2 is using resource $res").as(expect(res == 45))
}
}
```

```scala mdoc:passthrough
println(weaver.docs.Output.runSuites(ResourceDemo).unsafeRunSync())

println("Contents of `order` are:\n")
println("```")
order.toArray.foreach { el =>
println(s"// * $el")
}
println("```")
```
7 changes: 7 additions & 0 deletions docs/scalacheck.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,13 @@ object ForallExamples extends SimpleIOSuite with Checkers {
expect(a1 * a2 * a3 == a3 * a2 * a1)
}
}

test("Failure example") {
// There are 6 overloads, to pass 1-6 parameters
forall { (a1: Int, a2: Int) =>
expect(a1 + a2 % 2 == 0)
}
}

}
```
Expand Down
8 changes: 8 additions & 0 deletions docs/specs2.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,14 @@ object MatchersSpec extends SimpleIOSuite with IOMatchers {
pureTest("pureTest { 1 === 1 }") {
1 === 1
}

pureTest("failure example") {
1 must beEqualTo(2)
}
}

```

```scala mdoc:passthrough
println(weaver.docs.Output.runSuites(MatchersSpec).unsafeRunSync())
```
6 changes: 5 additions & 1 deletion docs/tagging.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import weaver._
import cats.effect.IO
import cats.syntax.all._

object MySuite extends SimpleIOSuite {
object TaggingSuite extends SimpleIOSuite {

test("Only on CI") {
for {
Expand All @@ -32,3 +32,7 @@ object MySuite extends SimpleIOSuite {

}
```

```scala mdoc:passthrough
println(weaver.docs.Output.runSuites(TaggingSuite).unsafeRunSync())
```
1 change: 1 addition & 0 deletions website/static/css/custom.css
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,5 @@ div.terminal pre {
background-color: black;
color: white;
padding: 20px;
overflow: auto
}

0 comments on commit c182dce

Please sign in to comment.