News
OpenAI Migrates to Facebook's PyTorch
OpenAI, the research organization focused on the impact of artificial intelligence technologies on humanity, recently announced it is standardizing its deep learning framework on PyTorch, the open source machine learning (ML) library for Python.
"In the past, we implemented projects in many frameworks depending on their relative strengths," the organization said in a blog post. "We've now chosen to standardize to make it easier for our team to create and share optimized implementations of our models."
OpenAI is an independent research lab based in San Francisco. It was launched in 2015 as a non-profit by Elon Musk, Sam Altman, Ilya Sutskever and Greg Bockman. It now includes a for-profit organization of the same name. "Our mission is to ensure that artificial general intelligence benefits all of humanity," the Web site reads.
Facebook's AI Research Lab (FAIR) released the first version of the PyTorch ML library to open source in 2016 under a modified BSD license. The latest version, PyTorch 1.4, was released in January with new capabilities, including the ability to do fine grain build-level customization for PyTorch Mobile, and new experimental support for model parallel training and Java language bindings.
OpenAI decided to standardize on PyTorch mainly to increase its research productivity at scale on GPUs, its announcement stated, because it's easy to try and execute new research ideas in PyTorch. Switching to PyTorch decreased the lab's iteration time on research ideas in generative modeling from weeks to days, for example.
"We're also excited to be joining a rapidly-growing developer community," the blog reads, "including organizations like Facebook and Microsoft, in pushing scale and performance on GPUs… Going forward, we'll primarily use PyTorch as our deep learning framework, but sometimes use other ones when there's a specific technical reason to do so. Many of our teams have already made the switch, and we look forward to contributing to the PyTorch community in upcoming months."
OpenAI also announced the release of a PyTorch-enabled version of Spinning Up in Deep RL, an open-source educational resource created by the organization designed to "let anyone learn to become a skilled practitioner in deep reinforcement learning." It includes examples of RL code, educational exercises, documentation, and tutorials. The organization said that it's also developing PyTorch bindings for its block-sparse GPU kernels, which are aimed at networks with block-sparse weights.
About the Author
John K. Waters is the editor in chief of a number of Converge360.com sites, with a focus on high-end development, AI and future tech. He's been writing about cutting-edge technologies and culture of Silicon Valley for more than two decades, and he's written more than a dozen books. He also co-scripted the documentary film Silicon Valley: A 100 Year Renaissance, which aired on PBS. He can be reached at jwaters@converge360.com.