Date: September 10, 2015
- Interestingly, even though the Mesos cluster controller was commercialized by techies Twitter and Airbnb, who started Mesosphere, Airbnb is not using Mesos very much today.
- The infrastructure at Airbnb is not anywhere approaching that scale, but it is still large and growing fast.
- To make embedding machine learning into its applications easier, Airbnb created a tool called Aerosolve, which is available open source on GitHub, that plugs into the Apache Spark in-memory processing engine, which Airbnb runs as a standalone platform for its early experiments in machine learning.
- As far as machine learning training goes, we train on CPUs right now.
- The two Hadoop clusters used by Airbnb today are kept in synch using Kafka for log collection and streaming data to both clusters at the same time.
Related NLP Articles
Build smarter apps with our natural language processing API.