Typesafe Activator

BigPipe using Play, RxJava, and Hystrix

BigPipe using Play, RxJava, and Hystrix

knutwalker
Source
May 20, 2015
playframework scala hystrix rxjava bigpipe scaladays2014

Reactive Frontend - an implementation of BigPipe using Play, RxJava, and Hystrix.

How to get "BigPipe using Play, RxJava, and Hystrix" on your computer

There are several ways to get this template.

Option 1: Choose play-bigpipe-with-rxjava-hystrix in the Typesafe Activator UI.

Already have Typesafe Activator (get it here)? Launch the UI then search for play-bigpipe-with-rxjava-hystrix in the list of templates.

Option 2: Download the play-bigpipe-with-rxjava-hystrix project as a zip archive

If you haven't installed Activator, you can get the code by downloading the template bundle for play-bigpipe-with-rxjava-hystrix.

  1. Download the Template Bundle for "BigPipe using Play, RxJava, and Hystrix"
  2. Extract the downloaded zip file to your system
  3. The bundle includes a small bootstrap script that can start Activator. To start Typesafe Activator's UI:

    In your File Explorer, navigate into the directory that the template was extracted to, right-click on the file named "activator.bat", then select "Open", and if prompted with a warning, click to continue:

    Or from a command line:

     C:\Users\typesafe\play-bigpipe-with-rxjava-hystrix> activator ui 
    This will start Typesafe Activator and open this template in your browser.

Option 3: Create a play-bigpipe-with-rxjava-hystrix project from the command line

If you have Typesafe Activator, use its command line mode to create a new project from this template. Type activator new PROJECTNAME play-bigpipe-with-rxjava-hystrix on the command line.

Option 4: View the template source

The creator of this template maintains it at https://github.com/knutwalker/play-bigpipe-with-rxjava-hystrix#master.

Option 5: Preview the tutorial below

We've included the text of this template's tutorial below, but it may work better if you view it inside Activator on your computer. Activator tutorials are often designed to be interactive.

Preview the tutorial

BigPipe, Play, and Hystrix

This is an implementation of the BigPipe pattern using Play, backed by Hystrix.

You can navigate this tutorial to see some of the new features using the arrows above, or you can jump to any particular feature at any time by clicking the title of the current part of the tutorial, which will open a drop down.

BigPipe / Motivation

BigPipe is a technique described by Facebook for speeding up webpage performance, in particular the perceived latency.

BigPipe was developed to overcome the problems of the existing web page serving systems.

In the traditional model, the browser sends a request to the web server, which parses and processes it. Only when the complete response is available, it will be sent back to the browser. Now, the browser can process the response, render the DOM and load additional resources like CSS and JavaScript files. All operations are performed sequentially, which means that the browser has to wait until the server has finished processing everything.

This is not what we want in a reactive web application. People expect web applications to react immediately. The server shouldn't have to wait until all HTML snippets are available and the browser shouldn't have to wait for the server.

With BigPipe, the server's generation time overlaps with the browser's rendering time. The browser receives almost immediately the first bytes and can start rendering the page and download the assets. Whenever the server pushes some new page content to the browser, it can react accordingly to it.

To make excellent use of this technique, the web page should consist of several modules that can operate independently of each other. In the traditional model, pages have to wait till all modules are loaded completly. With BigPipe, the modules can operate asynchronously and push new content to the browser whenever they have it available.

This is achieved by using Chunked Transfer Encoding (Play documentation). This is very simple and is already widely supported. Possible alternatives are found under the umbrella term Comet.

This tutorial represents a fancy new social media site, that heavily features kittens to satisfy the need of the internet.

Kittens Haven

Run this application and open your browser at http://localhost:9000/blocking. You will see a web page with some social media items. You may also notice, that the web page took some time to load.

There are some modules on this page, two in the sidebar and one in the news stream. The numbers and values of these modules are randomly generated and each module simulates network latency and failures, thus taking a variable amount of time to generate their number, or they might be failing.

The idea is, that the modules, or rather their individual commands, are provided by some kind of micro-service architecture. Each one could be fed by a different data source, profile view counts could be supplied by Redis, while comments could be delivered by Cassandra. It doesn't really matter where the data comes from, only that the application is not a monolithic block but a distributed network of services.

Hystrix

The modules and their commands are implemented using Hystrix by Netflix. Hystrix is a fault tolerance library, that provides useful mechanics for isolating each module. It allows you to write simple code in a traditional synchronous manner while providing latency and fault tolerance, concurrent execution, and realtime monitoring for you.

Browse to commands.scala. These are all the commands, one for each value of the five parts of all modules. The commands are fairly simple, most of their code actually comes from BogusHelper.scala. This is a trait, that Thread.sleep()s and throws RuntimeExceptions in order to simulate an unreliable network.

HystrixCommands can have a fallback to allow for graceful degradation. Whether or not to use a fallback and what it should return is driven by the business logic. Every command in Kittens Haven implements a fallback, which is mostly some static default value. The fallback is triggered for all types af failure, such as run() failure, timeouts, thread pool rejection, or circuit-breaker short-circuiting. The default timeout for any HystrixCommand in one seconds, which is raised for most commands in Kittens Haven, purely for demonstration purpose.

HystrixCommands can be executed in a synchronous, asynchronous, or reactive manner. Kittens Haven uses the reactive execution, which returns an rx.Observable[T] from RxJava. Browse to PageletFactory.scala at line 30 to see where it is used. The HystrixCommands' observe() method returns an Observable. The view of this single command is mapped over the observable and then rendered into a Pagelet. You can read more about Pagelets later.

RxJava and its Observables fit nicely into the model of composing single modules reactively. Multiple Observables can be merged together, creating a new Observable that will push the modules' content as soon as it is available. Browse to Pagelet.scala at line 59 to see it in action. A caveat of this merging is, that the resulting observable pushes its pagelets out of web page order, so they have to be reordered later via the means of JavaScript.

Hystrix is not a requirement for BigPipe, neither is RxJava (or Play, really, for that matter). You can also use Akka or any other toolkit, that enables fault and latency tolerance and reactive execution of simple commands.

Pagelets

A Pagelet is the single unit of processing for the BigPipe technique. It represents a snippet of HTML, together with an ID of the element into which the HTML snippet should be injected.

As said before, the individual Pagelets are streamed out of order — they appear in the order they're available in, not in the order they're needed in. To put them in the right place, a little Coffescript snippet takes the HTML and puts it into place.

Pagelets itself consist of two script tags. One contains the HTML and has a type of text/x-html-stream. If a browser encounters a script tag, they should download it and insert its content in the DOM. Despite that, only tags with the type set to text/javascript should be evaluated. This makes script tags a convenient way to inject HTML or other texts into the DOM, so that it is never rendered or visible to the user but available for programmatic use. Several JavaScript based template engines use this technique as well. The other script tag just calls the injection snippet, so that the content is injected in the DOM. Browse to pagelet.scala.html for the template view.

Pagelet Modules

On the server side, a Pagelet is powered by an Observable[Html] and a regular HTML template. sayHello.scala.html for example, is the template for the NewestKittenCommand.

Several Pagelets can make up a Pageletmodule. Such a module consists of a controller and a layout template. This template is not a regular HTML template but a streamable template. You can read more about these later on; for now, assume they're regular templates.

For example, the NewKittens module consists only of the NewKittens/sayHello Pagelet. Its layout template is at body.scala.pagelet. There is a placeholder, the div#newest-kitten. This is where the content of the Pagelet will be injected.

Browse to NewKittens.scala for the module controller. Most of the implementation is provided by the CommandStreamController trait, so that modules are really just a definition of the above — a list of Pagelets, each consisting of:

  • A command (NewestKittenCommand())
  • A view (sayHello.apply)
  • An ID of the DOM element ("newest-kitten")

The CommandStreamController trait defines some helper methods to create Pagelet factories out of commands or required resources. It also implements the Play action by merging the modules' Observables together and transform them into an Enumerator.

Plays Enumerators ans RxJavas Observables are very similar in what they're achieving to solve. Both provide a push based alternative to enumerate several values asynchronously, instead of the synchronous, pull-based iteration with Iterators. Here, Observables were used, because that's what was already provided by Hystrix and they offer all composing functionality that is needed. Alternatively, one could use Enumerators from the beginning and have them composed (e.g. Enumerator.interleave instead of Observable.merge and Enumerator.andThen instead of Observable.concat).

A PageletFactory is just a function that produces a Pagelet with an additional method to do so in a traditional, blocking manner. This blocking behaviour is only used for demonstration purpose, it is not needed for BigPipe and this abstraction can be removed in production code.

Controllers define one additional thing, the layout. This it the streaming template that is used, when the module is viewed as a standalone page. By providing this, you can easily text and develop the modules independent from each other, without having to keep the big picture around all the time.

Streamable Templates

In order to stream the Pagelets to the browser, the templates have to be made asynchronous as well. The default HTML templates are rendered into an Html, that is finally sent to the browser. What is needed are templates, that would instead render into an Observable[Html], so that we can still use the view separation with BigPipe.

Browse to build.sbt. You'll see a configuration option called TwirlKeys.templateFormats.

TwirlKeys.templateFormats += ("pagelet" -> "pagelet.PageletObservableFormat")

This defines a custom template format and registers the file ending .pagelet for it. You have already encountered such a template, it doesn't look different from a regular template.

Browse to PageletObservableFormat. This is the Format definition for the streamable templates. It mirrors the HtmlFormat but works with parts of PageletObservables instead of Html.

Browse to PageletObservable. This is the Appendable definition for the streamable templates. Instead of appending text to some StringBuffer, it concatenates Observables into one single Observable. The usage of ++ (or concat) is important here, since templates are order sensitive — you don't want to have your HTML layout scattered all over the place. The ++ operator ensures, that each part of the template appears in the order it is defined. Note, that this is different from the Pagelets itself, which are encouraged to appear out of order.

Browse to streamed.scala.pagelet for an example of a module page. Here, body are the modules' Pagelets. Everything that appears before @body will be sent to the browser immediately. The body itself will be streamed as the Pagelets arrive and finally, the transfer is finished with closing the body and html tags. Therefore, @body should always appear at the very last position, right before the closing </body>, so that as much as possible is sent to the browser immediately. In other words — @body is the part that would normally block, but doesn't, thanks to BigPipe.

Put it together

Now, all pieces are laid out to implement BigPipe.

  1. The data is provided asynchronously by a HystrixCommand (or, could be a web/micro service) => Observable[T]
  2. A Pagelets view template is mapped over this Observable[T] => Observable[Html]
  3. This result is wrapped in the Pagelet template => Observable[Pagelet]
  4. Each modules Pagelets are merged together and turned into a => PageletObservable
  5. The layout of the Pagelet module is rendered with the previous PageletObservable as its body, providing the outline of the module => PageletObservable
  6. The resulting PageletObservable is transformed into an => Enumerator[Html]
  7. This Enumerator[Html] is streamed to the browser, using Plays Ok.chunked() => Action

The main page is implemented by the Aggregator.scala, which is really simple. It just concatenates all Pageletes from all modules and defines the global template.

Open the browser at http://localhost:9000/. and compare it with the traditional blocking version of http://localhost:9000/blocking. You will notice, that the layout of the page appears almost immediately and you can see how some numbers suddenly pop into place, that would have otherwise staled the HTML generation on the server side. Additionally, you can see the result in the network section of your browsers developer tools, e.g.:

blocking

vs.

streaming

With the traditional model, the browser had to wait several seconds before it could do anything. With BigPipe, the first bytes were received after only 6 milliseconds, and the browser could download additional resources while the server was still busy, generating the Pagelets.

Also, if you refresh the page several times, you can see the fallback strategy being applied. Every time, there is a 0 count of something, the underlying service failed. But this failure did not affect the overall page in any way, only its own segment. This outage might have been due to a tripped circuit, a timeout, an exception or other things, but Hystrix allows the page to survive this fairly easily and the error does not propagate the complete system.

Monitoring Dashboard

Another aspect of Hystrix is the Dashboard. The default distribution of the Dashboard and its companion – the Metrics Event Stream – is via Servlets/JEE. This is not suitable for Play, so this tutorial has them both integrated into this Play app.

HystrixMetricsStream.scala is a Play implementation of the hystrix-metrics-event-stream module. It is fully compliant with the default implementation, so you can use this stream to push Hystrix metrics to your own dashboard or Turbine.

http://localhost:9000/dashboard is an integrated/hosted version of the dashboard, providing useful insights into your commands.

dashboard

Next steps

Here are some suggestions or improvements that could be made.

  • Since Hystrix 1.4, there exists an HystrixObservableCommand, so that one can implement the commands completely asynchronous and non-blocking as well.
  • Also, Hystrix does offer a great palette of interesting features like bulked or cached commands
  • Alternatively, replace Hystrix and RxJava with e.g. Akka
  • Sessions with Cookies were ignored for now. Cookies are already provided by the request, but one has to export custom module headers so that Pagelets can define their own Cookies.
  • Some JavaScript magic, that would have modules appear smoothly instead of them just popping in. This is entirely up to the Frontend department and does not require changes to BigPipe.
comments powered by Disqus