•A Mathematics Library Built for Deep Learning
TensorFlow is described as a free and open-source software library for machine learning, with a specific focus on the training and inference of deep neural networks. The source explains that it is a symbolic mathematics library based on data flow and differentiable programming, used within Google for both research and production purposes. The name itself derives from the operations the library performs on multi-dimensional arrays of data called tensors, which is how such neural networks represent their data.
Origins in DistBelief
The source traces TensorFlow’s lineage to a system called DistBelief, which Google Brain began developing in 2011 as a proprietary machine learning system specialized for deep learning neural networks. Its use spread rapidly across various Alphabet companies in both research and commercial applications. To simplify and reorganize DistBelief’s codebase, Google assigned multiple computer scientists, including Jeff Dean, to the effort, and TensorFlow emerged from this work. The source also notes that in 2009, a team led by Geoffrey Hinton implemented improvements enabling neural network production with significantly higher accuracy, alongside other improvements that reduced errors in speech recognition by 25 percent.
The Tensor Processing Unit
In May 2016, Google announced the Tensor Processing Unit (TPU), an application-specific integrated circuit (ASIC) specifically designed for TensorFlow and machine learning. The source states that the TPU was designed to deliver high performance in low-precision arithmetic, such as 8-bit computation. Google described it as a programmable AI accelerator intended for use rather than training models, noting it had operated TPUs in its data centers for over a year and expected better-per-watt optimization for machine learning.
The source provides specific performance figures across TPU generations. Second-generation TPUs, announced in May 2017 along with their availability on Google Compute Engine, deliver up to 180 teraflops, reaching 11.5 petaflops when arranged in clusters of 64 TPUs. Third-generation TPUs, announced in May 2018, offer 420 teraflops and 128 GB of high bandwidth memory (HBM), while Cloud TPU v3 Capsules provide over 100 petaflops and 32 TB of HBM. Google had made TPUs available in beta on the Google Cloud Platform in February 2018.
Edge and Mobile Compute
The source describes how TensorFlow’s flexible architecture allows computation across various platforms including CPUs, GPUs, and TPUs, enabling deployment from desktops to server clusters and mobile or edge devices. In July 2018, Google announced the Edge TPU, an ASIC designed to run TensorFlow Lite machine learning models on edge computing devices such as smartphones.
TensorFlow Lite, announced in May 2017 as a software stack for mobile development, uses FlatBuffers as its data serialization format for network models, deliberately avoiding the Protocol Buffers format used by standard TensorFlow models. In January 2019, the TensorFlow team released a developer preview of its mobile GPU inference engine using OpenGL ES 3.1 Compute Shaders on Android devices and Metal Compute Shaders on iOS devices.
The source also covers the Pixel Visual Core (PVC), a fully programmable image, picture, and AI processor included in the Google Pixel 2, launched in October 2017. PVC supports TensorFlow for machine learning and also supports the Halide programming language for image processing.
Software Ecosystem and Applications
The source notes that in June 2016, at the Google I/O conference, Jeff Dean stated that 1,500 libraries on GitHub referenced TensorFlow, of which only five were created by Google. In December 2017, developers from Google, Cisco, Red Hat, CoreOS, and CaiCloud introduced Kubeflow, which allows TensorFlow to run and be deployed on Kubernetes. TensorFlow.js version 1.0 was announced in March 2018, and TensorFlow Graphics for deep learning in computer graphics was announced in May 2019.
Regarding applications, the source states that Google officially released RankBrain, powered by TensorFlow, on October 26, 2015. The library underpins applications such as DeepDream, an automatic image captioning software, and includes a website called TensorFlow Playground that allows users to observe and understand the concept of neural networks.
This explanation is adapted from Wikipedia and is available under CC BY-SA 4.0.
1 comment