design-principles.md

@title Polyphony's Design

Polyphony's Design

Polyphony is a new gem that aims to enable developing high-performance concurrent applications in Ruby using a fluent, compact syntax and API. Polyphony enables fine-grained concurrency - the splitting up of operations into a large number of concurrent tasks, each concerned with small part of the whole and advancing at its own pace. Polyphony aims to solve some of the problems associated with concurrent Ruby programs using a novel design that sets it apart from other approaches currently being used in Ruby.

Origins

The Ruby core language (at least in its MRI implementation) currently provides two main constructs for performing concurrent work: threads and fibers. While Ruby threads are basically wrappers for OS threads, fibers are essentially continuations, allowing pausing and resuming distinct computations. Fibers have been traditionally used mostly for implementing enumerators and generators.

In addition to the core Ruby concurrency primitives, some Ruby gems have been offering an alternative solution to writing concurrent Ruby apps, most notably EventMachine, which implements an event reactor and offers an asynchronous callback-based API for writing concurrent code.

In the last couple of years, however, fibers have been receiving more attention as a possible constructs for writing concurrent programs. In particular, the Async framework, created by Samuel Williams, offering a comprehensive set of libraries, employs fibers in conjunction with an event reactor provided by the nio4r gem, which wraps the C library libev.

In addition, recently some effort was undertaken to provide a way to automatically switch between fibers whenever a blocking operation is performed, or to integrate a fiber scheduler into the core Ruby code.

Nevertheless, while work is being done to harness fibers for providing a better way to do concurrency in Ruby, fibers remain a mistery for most Ruby programmers, a perplexing unfamiliar corner right at the heart of Ruby.

The History of Polyphony

Polyphony started as an experiment, but over about three years of slow, jerky evolution turned into something I'm really excited to share with the Ruby community. Polyphony's design is both similar to and different than the projects mentioned above.

Polyphony today looks nothing like the way it began. A careful examination of the CHANGELOG would show how Polyphony explored not only different event reactor designs, but also different API designs incorporating various concurrent paradigms such as promises, async/await, fibers, and finally structured concurrency.

Throughout the development process, it was my intention to create a programming interface that would make it easy to author highly-concurrent Ruby programs.

Design Principles

While Polyphony, like nio4r or EventMachine, uses an event reactor to turn blocking operations into non-blocking ones, it completely embraces fibers and in fact does not provide any callback-based APIs.

Furthermore, Polyphony provides fullblown fiber-aware implementations of blocking operations, such as read/write, sleep or waitpid, instead of just event watching primitives.

Polyphony's design is based on the following principles:

• The concurrency model should feel "baked-in". The API should allow concurrency with minimal effort. Polyphony should facilitate writing both large apps and small scripts with as little boilerplate code as possible. There should be no calls to initialize the event reactor, or other ceremonial code:

  require 'polyphony'
  
  # start 10 fibers, each sleeping for 3 seconds
  10.times { spin { sleep 3 } }
  
  puts 'going to sleep now'
  # wait for other fibers to terminate
  suspend

• Blocking operations should yield to other concurrent tasks without any decoration or wrapper APIs. This means no async/await notation, and no async callback-style APIs.

  # in Polyphony, I/O ops might block the current fiber, but implicitly yield to
  # other concurrent fibers:
  clients.each do |client|
    spin { client.puts 'Elvis has left the chatroom' }
  end

• Concurrency primitives should be accessible using idiomatic Ruby techniques (blocks, method chaining...) and should feel as much as possible "part of the language". The resulting API is fundamentally based on methods rather than classes, for example spin or move_on_after, leading to a coding style that is both more compact and more legible:

  fiber = spin {
    move_on_after(3) {
      do_something_slow
    }
  }

• Polyphony should embrace Ruby's standard raise/rescue/ensure exception handling mechanism. Exception handling in a highly concurrent environment should be robust and foolproof:

  cancel_after(0.5) do
    puts 'going to sleep'
    sleep 1
    # this will not be printed
    puts 'wokeup'
  ensure
    # this will be printed
    puts 'done sleeping'
  end

• Concurrency primitives should allow creating higher-order concurrent constructs through composition.

• The entire design should embrace fibers. There should be no callback-based asynchronous APIs. The library and its ecosystem will foster the development of techniques and tools for converting callback-based APIs to fiber-based ones.

• Use of extensive monkey patching of Ruby core modules and classes such as Kernel, Fiber, IO and Timeout. This allows porting over non-Polyphony code, as well as using a larger part of stdlib in a concurrent manner, without having to use custom non-standard network classes or other glue code.

  require 'polyphony'
  
  # use TCPServer from Ruby's stdlib
  server = TCPServer.open('127.0.0.1', 1234)
  while (client = server.accept)
    spin {
      while (data = client.gets)
        client.write("you said: #{ data.chomp }\n")
      end
    }
  end

• Enhance Ruby's I/O capabilities by providing additional APIs for splicing (on Linux) and moving data between file descriptors. Polyphony provides APIs for compressing / uncompressing data on the fly between file descriptors. This in turn enables the creation of arbitrarily-complex data manipulation pipelines that maximize performance and provide automatic backpressure.

Emergent Patterns for Ruby Apps Using Polyphony

During the development of Polyphony and its usage in a few small- and medium-size custom Ruby apps, a few patterns have emerged. We belive embracing these patterns will lead to better-written concurrent programs that take advantage of all the benefits provided by Polyphony. Here are some of them:

• Infinite loops make sense for fibers: normally, developers are taught to avoid writing infinite loops, and to make sure any loop will be ended at one point. With Polyphony, however, a fiber can run an infinite loop, performing work such as responding to messages received on its mailbox, without the programmer having to worry about it blocking the entire program:

  server = spin do
    loop do
      client, data = receive
      result = do_something_with_data(data)
      client << result
    end
  end

  In the above example, a server of some sorts runs an infinite loop, taking messages off its mailbox and handling them, sending the result back to the corresponding client fiber. The programmer does not need to worry about signalling the server fiber when it's time to finish its work. A simple call to server.stop will stop it, as of course will its parent fiber stopping.

• Message passing between fibers as a means to synchronize and pass data between different parts of the application. The message passing ability integrated into Polyphony allows writing programs where each fiber is responsible for a single task, and receives its work by popping messages off its mailbox. If we reconsider the above example, here's how a client might talk to the server fiber:

  results = incoming_data.map do |data|
    server << [Fiber.current, data]
    receive
  end

  Look at all the things we don't need to do: we don't need to worry about synchronizing access to shared variables between the different parts of the app, and we also don't need to worry about how to handle backpressure - the work will progress as fast as the slowest part in the app, without any requests accumulating unnecessarily.