A group of all-star computer scientists is behind OctoML, a new University of Washington spinout that aims to help companies deploy machine learning models on various hardware configurations.
The startup launched in July and today announced a $3.9 million seed round led by Madrona Venture Group, with participation from Amplify Partners.
OctoML is led by the creators of Apache TVM, an open source “deep learning compiler stack” that started as a research project at the UW’s Paul G. Allen School of Computer Science & Engineering a few years ago. It has attracted a thriving community of users including tech giants such as Amazon and Facebook that want to optimize and automate their deep learning models for IoT/edge or cloud deployment on an increasing number of platforms such as phones, cars, health devices, and other use cases.
“We formed this company to make Apache TVM available to more users,” said CEO and co-founder Luis Ceze, a UW computer science professor who previously started Corensic, a debugging startup that F5 Networks acquired in 2012.
Ceze likened the OctoML technology to how an operating system such as iOS or Windows acts as a bridge between an application and hardware.
“TVM is analogous to that, but for machine learning models running on a variety of hardware,” he said.
The company’s motto is “machine learning on your machine learning.” It aims to reduce the amount of cost and time it takes companies to develop and deploy deep learning software for specific hardware — something made even more difficult as security and privacy concerns arise with the data being processed.
OctoML plans to sell a SaaS-based, turnkey service.
“Our target is everyone who likes optimized models, but doesn’t have the resources or knowledge to [build them],” Ceze said.
Ceze, also a venture partner at Madrona, is joined by four other co-founders:
- Tianqi Chen, who just received his Ph.D. from the Allen School and is taking a year off to work on OctoML before joining the faculty at Carnegie Mellon University
- Jason Knight, a former principal engineer and AI leader at Intel who earned a Ph.D. in electrical engineering at Texas A&M
- Thierry Moreau, who earned his Ph.D. in 2018 from the Allen School and taught a graduate level machine learning class with Ceze
- Jared Roesch, currently a fourth year Ph.D. student at the Allen School who worked at Zentopy, Invoca, and Mozilla Research
Advising the company is Arvind Krishnamurthy, a UW computer science professor since 2005; Zachary Tatlock, an assistant computer science professor at the UW since 2013; and Carlos Guestrin, the Amazon Professor of Machine Learning at the UW who sold Seattle machine learning startup Turi to Apple in 2016. Guestrin is currently Apple’s senior director of machine learning and AI.
“It’s a really fantastic team,” said Ceze, a Sao Paulo, Brazil native who earned his Ph.D. from the University of Illinois at Urbana-Champaign.
Ceze said the idea to launch a company happened after his team helped host the first TVM and Deep Learning Compiler Conference in Seattle last year.
“Almost 200 people came from around the world to talk about how they use TVM,” he told GeekWire. “That was a big moment — there was something interesting here.”
Ceze said there is “significant” interest from prospective customers. The company is also in talks with hardware vendors.
The startup is somewhat similar to fellow Seattle company Xnor.ai, a spinout from the Allen Institute for Artificial Intelligence that has been working with partners on low-cost, low-power AI monitoring devices.
“Xnor offers tuned computer vision models for some edge devices. OctoML offers automatic optimizations that apply to a broad set models and to a broad set of hardware targets for edge and cloud,” Ceze explained. “So while there is some overlap, the offerings are fairly distinct.”
OctoML doesn’t have much direct competition, though it has a similar “optimization as a service” model to Latent AI, which spun out of SRI International and raised seed funding this past June. Other startups building AI/ML optimization platforms include Determined AI and Ople.
Amazon also this year rolled out its Sagemaker Neo service, though that is focused on tuning models for deployment on Amazon Elastic Compute Cloud, Ceze said.
Before they launched OctoML, some of the founding team previously partnered with Amazon Web Services to introduce an NNVM compiler for AI frameworks.
And as for OctoML’s name origination?
“It was a collaborative process that started with connection to octopi: incredibly intelligent creatures with distributed and efficient brains,” Ceze said. “But now for some of us ‘Octo’ refers to octopus, and for others the number 8 or octet.”
OctoML currently employs 10 people. Ceze said he’s plans on keeping his teaching gig at the UW, but is taking a sabbatical soon to focus on OctoML.
Madrona’s investment in OctoML aligns with its focus on investing in “intelligent applications.” It’s also the firm’s 18th portfolio company that has spun out of the UW.
“Intelligent applications are changing the landscape of software – and in fact blending again the roles of software and hardware,” Madrona Managing Director Matt McIlwain said in a statement. “The work that the OctoML team has done to build this technology into a powerhouse in the open source community is outstanding and we are excited to back this team.”
OctoML is another example of the connection between the UW’s engineering expertise, Madrona’s financial support, and startup creation. It’s similar to how Turi, the startup founded by Guestrin and also backed by Madrona, got off the ground. Turi sold to Apple, which opened a Seattle engineering center as a result and just leased a 630,000 square-foot building.
Oren Etzioni and Jacob Colker of the Allen Institute for Artificial Intelligence wrote a comprehensive analysis last week about the Seattle startup ecosystem and used Turi’s story as an example that shows why they believe “an unprecedented acceleration in high-tech startup creation” is on the horizon in the Emerald City.
“The acquisition of Turi sets the stage for more company creation over time,” they wrote. “The financial windfall from the exit begets angel investors who invest in new startups. The successful exit also whets the appetite for Turi employees to become startup founders of their own. The concentration of talent brought together by Apple creates a talent pool that become early hires of those new startups. And the cycle repeats.
“All the components are now in place for this story to repeat and accelerate.”
