News

Oracle Open Sources GraphPipe ML Model Serving Tool

• By John K. Waters
• 08/20/2018

Oracle has open sourced its GraphPipe tool for simplifying and standardizing machine learning (ML) model serving, the company announced last week.

GraphPipe is both a software tools bundle and a protocol. It's designed to simplify the deployment of the model artifacts created during the process of ML training known as machine learning models by decoupling that process from framework-specific model implementations.

The tool was created to solve a vexing problem associated with ML model serving: there is no industry-standard protocol for transmitting tensor-like data among components in a deep learning architecture. The lack of a standard protocol forces developers to build custom clients for each workload. GraphPipe aims to solve this problem by standardizing on an efficient communication protocol and providing simple model servers for the major ML frameworks.

"In the enterprise, machine-learning models are often trained individually and deployed using bespoke techniques," wrote Cloud Development Architect Vish Abrams in a post on the Oracle Developers Blog. "This impacts an organization's ability to derive value from its machine learning efforts. If marketing wants to use a model produced by the finance group, they will have to write custom clients to interact with the model. If the model becomes popular sales wants to use it as well, the custom deployment may crack under the load."

It's common for developers to use protocols such JSON, Abrams explained, which is extremely inefficient, or TensorFlow-serving's protocol buffers, which carry "the baggage of TensorFlow, a large and complex piece of software."

"GraphPipe is designed to bring the efficiency of a binary, memory-mapped format while remaining simple and light on dependencies," he wrote.

GraphPipe provides simple implementations of clients and servers that simplify deploying and querying machine learning models from a range of frameworks, including TensorFlow, PyTorch, mxnet, CNTK and caffe2.

Along with the frameworks support, GraphPipe provides a minimalist machine learning transport specification based on flatbuffers, a cross platform serialization library for C++, C#, C, Go, Java, JavaScript, Lobster, Lua, TypeScript, PHP and Python. The library was created originally by Google for game development. It also offers client implementations in three of the leading AI/ML development languages: Go, Python and Java.

"We hope that open sourcing GraphPipe makes the model serving landscape a friendlier place," Abrams wrote on GitHub, where the project and its documentation can be accessed now. Details for contributing to the project are posted here. There's also a bullet-list overview and a useful graphic posted here.