News

Open Source Deep Learning Tool PyTorch Gets Update

• By Becky Nagel
• 05/14/2019

Earlier this month Facebook announced a number of new features and better interop for its increasingly popular open source "Python-first" deep learning tool PyTorch.

One major improvement in the new 1.1 version, available here, is native support for TensorBoard, a data visualization toolkit for Google Research's open source machine learning library TensorFlow

PyTorch 1.1 now also supports two brand-new machine learning tools that Facebook also open-sourced earlier this month: BoTorch and Ax. BoTorch is a PyTorch-based Bayesian optimization library aimed at researchers creating black-box functions, and Ax is a brand-new open source, modular platform for machine learning that allows for plug-and-play algorithms and A/B testing. (They are also designed to be used together.)

"With the release of Ax and BoTorch, we will be sharing some of our core algorithms, including meta-learning for efficiently optimizing hyperparameters based on historical tasks. We are excited to see this work open-sourced for the community to build on," Facebook AI said in its announcement of these products.

"We've also continued to partner with the community to foster projects and tools aimed at supporting ML engineers for needs ranging from improved model understanding to auto-tuning using AutoML methods. AutoML approaches have been valuable for applications at Facebook including computer vision and personalization."

Other upgrades to 1.1 include:

• Bug fixes and improvements to the JIT compiler.
• Better API support for Boolean tensors and custom reoccurrent neural networks.
• Performance improvements distributed training (tutorials here).
• Two new "experimental" features: support for quantized datatypes and limited support for MKLDNN tensors.
• A host of new operators.
• Support for synchronous batch normalization, circular padding and trainable "per sample weights."

A complete list of changes is available here.

PyTorch went live in December 2018; this is its first major upgrade since that release. In April, a module was released that allows PyTorch to support "extremely large" graphs.