In the past, its analytics team used Cron to manage workflows, but they quickly realized sleep statements were not enough for managing complex dependency hierarchies. Learn how Airbnb front-end and back-end engineering came together to build a distributed system for scheduling data pipelines by using Scala with Mesos.
Our analytics team depends on very complex data pipelines for analysis, necessitating having a Data Infrastructure team for building systems and tools to support this function. We built an in-house called Chronos, which is a distributed system for scheduling data pipelines, which is easier to manage and debug.
Inspired by this story? Contact us to learn more about what Lightbend can do for your organization.