Skip to content

Commit

Permalink
update introduction section.
Browse files Browse the repository at this point in the history
  • Loading branch information
khajavi committed Aug 2, 2023
1 parent 47af326 commit 7bb60b1
Showing 1 changed file with 46 additions and 34 deletions.
80 changes: 46 additions & 34 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,20 +10,29 @@ sidebar_label: "Introduction"

## Introduction

Schema is a structure of a data type. ZIO Schema reifies the concept of structure for data types. It makes a high-level description of any data type and makes them as first-class values.
ZIO Schema helps us to solve some of the most common problems in distributed computing, such as serialization, deserialization, and data migration.

It turns a compiled-time construct (the type of a data structure) into a runtime construct (a value that can be read, manipulated, and composed at runtime). A schema is a structure of a data type. ZIO Schema reifies the concept of structure for data types. It makes a high-level description of any data type and makes them first-class values.

Creating a schema for a data type helps us to write codecs for that data type. So this library can be a host of functionalities useful for writing codecs and protocols like JSON, Protobuf, CSV, and so forth.

With schema descriptions that can be automatically derived for case classes and sealed traits, _ZIO Schema_ will be going to provide powerful features for free (Note that the project is in the development stage and all these features are not supported yet):
## What Problems Does ZIO Schema Solve?

With schema descriptions that can be automatically derived for case classes and sealed traits, _ZIO Schema_ will be going to provide powerful features for free:

- Codecs for any supported protocol (JSON, protobuf, etc.), so data structures can be serialized and deserialized in a principled way
- Diffing, patching, merging, and other generic-data-based operations
- Migration of data structures from one schema to another compatible schema
- Derivation of arbitrary type classes (`Eq`, `Show`, `Ord`, etc.) from the structure of the data
1. Metaprogramming without macros, reflection, or complicated implicit derivations.
1. Creating serialization and deserialization codecs for any supported protocol (JSON, Protobuf, etc.)
2. Deriving standard type classes (`Eq`, `Show`, `Ordering`, etc.) from the structure of the data
3. Default values for data types
2. Automate ETL (Extract, Transform, Load) pipelines
1. Diffing: diffing between two values of the same type
2. Patching: applying a diff to a value to update it
3. Migration: migrating values from one type to another
3. Computations as data: Not only we can turn types into values, but we can also turn computations into values. This opens up a whole new world of possibilities concerning distributed computing.

When our data structures need to be serialized, deserialized, persisted, or transported across the wire, then _ZIO Schema_ lets us focus on data modeling and automatically tackle all the low-level, messy details for us.

_ZIO Schema_ is used by a growing number of ZIO libraries, including _ZIO Flow_, _ZIO Redis_, _ZIO Web_, _ZIO SQL_ and _ZIO DynamoDB_.
_ZIO Schema_ is used by a growing number of ZIO libraries, including [ZIO Flow](https://zio.dev/zio-flow), [ZIO Redis](https://zio-redis), [ZIO SQL](https://zio.dev/zio-sql) and [ZIO DynamoDB](https://zio.dev/zio-dynamodb).

## Installation

Expand All @@ -37,51 +46,54 @@ libraryDependencies += "dev.zio" %% "zio-schema-json" % "@VERSION@"
libraryDependencies += "dev.zio" %% "zio-schema-msg-pack" % "@VERSION@"
libraryDependencies += "dev.zio" %% "zio-schema-protobuf" % "@VERSION@"
libraryDependencies += "dev.zio" %% "zio-schema-thrift" % "@VERSION@"
libraryDependencies += "dev.zio" %% "zio-schema-zio-test" % "@VERSION@"

// Required for automatic generic derivation of schemas
// Required for the automatic generic derivation of schemas
libraryDependencies += "dev.zio" %% "zio-schema-derivation" % "@VERSION@"
libraryDependencies += "org.scala-lang" % "scala-reflect" % scalaVersion.value % "provided"
```

## Example

In this simple example first, we create a schema for `Person` and then run the _diff_ operation on two instances of the `Person` data type, and finally we encode a Person instance using _Protobuf_ protocol:
In this simple example first, we create a schema for `Person` and then run the _diff_ operation on two instances of the `Person` data type, and finally, we encode a Person instance using _Protobuf_ protocol:

```scala
import zio.console.putStrLn
import zio.schema.codec.ProtobufCodec._
```scala mdoc:compile-only
import zio._
import zio.stream._
import zio.schema.codec.{BinaryCodec, ProtobufCodec}
import zio.schema.{DeriveSchema, Schema}
import zio.stream.ZStream
import zio.{Chunk, ExitCode, URIO}

final case class Person(name: String, age: Int, id: String)
final case class Person(name: String, age: Int)

object Person {
implicit val schema: Schema[Person] = DeriveSchema.gen[Person]
implicit val schema: Schema[Person] = DeriveSchema.gen
val protobufCodec: BinaryCodec[Person] = ProtobufCodec.protobufCodec
}

Person.schema

import zio.schema.syntax._

Person("Alex", 31, "0123").diff(Person("Alex", 31, "124"))
object Main extends ZIOAppDefault {
def run =
ZStream
.succeed(Person("John", 43))
.via(Person.protobufCodec.streamEncoder)
.runCollect
.flatMap(x =>
Console.printLine(s"Encoded data with protobuf codec: ${toHex(x)}")
)

def toHex(chunk: Chunk[Byte]): String =
chunk.map("%02X".format(_)).mkString
}
```

def toHex(chunk: Chunk[Byte]): String =
chunk.toArray.map("%02X".format(_)).mkString
Here is the output of running the above program:

zio.Runtime.default.unsafe.run(
ZStream
.succeed(Person("Thomas", 23, "2354"))
.transduce(
encoder(Person.schema)
)
.runCollect
.flatMap(x => putStrLn(s"Encoded data with protobuf codec: ${toHex(x)}"))
).getOrThrowFiberFailure()
```scala
Encoded data with protobuf codec: 0A044A6F686E102B
```

## Resources

- [Zymposium - ZIO Schema](https://www.youtube.com/watch?v=GfNiDaL5aIM) by John A. De Goes, Adam Fraser and Kit Langton (May 2021)
- [Zymposium - ZIO Schema](https://www.youtube.com/watch?v=GfNiDaL5aIM) by John A. De Goes, Adam Fraser, and Kit Langton (May 2021)
- [ZIO SCHEMA: A Toolkit For Functional Distributed Computing](https://www.youtube.com/watch?v=lJziseYKvHo&t=481s) by Dan Harris (Functional Scala 2021)
- [Creating Declarative Query Plans With ZIO Schema](https://www.youtube.com/watch?v=ClePN4P9_pg) by Dan Harris (ZIO World 2022)
- [Describing Data...with free applicative functors (and more)](https://www.youtube.com/watch?v=oRLkb6mqvVM) by Kris Nuttycombe (Scala World) on the idea behind the [xenomorph](https://github.com/nuttycom/xenomorph) library
- [Describing Data...with free applicative functors (and more)](https://www.youtube.com/watch?v=oRLkb6mqvVM) by Kris Nuttycombe (Scala World) on the idea behind the [xenomorph](https://github.com/nuttycom/xenomorph) library

0 comments on commit 7bb60b1

Please sign in to comment.