\n\n\n\n
PyTorch and TensorFlow are far and away the two most popular Deep Learning frameworks today. The debate over which framework is superior is a longstanding point of contentious debate, with each camp having its share of fervent supporters.<\/p>\n\n\n\n
Both PyTorch and TensorFlow have developed so quickly over their relatively short lifetimes that the debate landscape is ever-evolving. Outdated or incomplete information is abundant, and further obfuscates the complex discussion of which framework has the upper hand in a given domain.<\/p>\n\n\n\n
While TensorFlow has a reputation for being an industry-focused framework and PyTorch has a reputation for being a research-focused framework, we\u2019ll see that these notions stem partially from outdated information. The conversation about which framework reigns supreme is much more nuanced going into 2023 – let\u2019s explore these differences now.<\/p>\n\n\n\n
#<\/a>Practical Considerations<\/h2>\n\n\n\nPyTorch and TensorFlow alike have unique development stories and complicated design-decision histories. Previously, this has made comparing the two a complicated technical discussion about their current features and speculated future features. Given that both frameworks have matured exponentially since their inceptions, many of these technical differences are vestigial at this point.<\/p>\n\n\n\n
Luckily for those of us who don\u2019t want their eyes glazing over, the PyTorch vs TensorFlow debate currently comes down to three practical considerations:<\/p>\n\n\n\n
\n- Model Availability: <\/strong>With the domain of Deep Learning expanding every year and models becoming bigger in turn, training State-of-the-Art (SOTA) models from scratch is simply not feasible anymore. There are fortunately many SOTA models publicly available, and it is important to utilize them where possible.<\/li>\n\n\n\n
- Deployment Infrastructure: <\/strong>Training well-performing models is pointless if they can\u2019t be put to use. Lowering time-to-deploy is paramount, especially with the growing popularity of microservice business models; and efficient deployment has the potential to make-or-break many businesses that center on Machine Learning.<\/li>\n\n\n\n
- Ecosystems: <\/strong>No longer is Deep Learning relegated to specific use cases in highly controlled environments. AI is injecting new power into a litany of industries, so a framework that sits within a larger ecosystem which facilitates development for mobile, local, and server applications is important. Also, the advent of specialized Machine Learning hardware, such as Google\u2019s Edge TPU<\/a>, means that successful practitioners need to work with a framework that can integrate well with this hardware.<\/li>\n<\/ol>\n\n\n\n
We\u2019ll explore each of these three practical considerations in turn, and then provide our recommendations for which framework to use in different areas.<\/p>\n\n\n\n
Looking for more posts like this?<\/p>\n\n\n\n
Subscribe to our newsletter!<\/p>\n\n\n\n
Subscribe Now<\/a><\/p>\n\n\n\n#<\/a>PyTorch vs TensorFlow – Model Availability<\/h2>\n\n\n\nImplementing a successful Deep Learning model from scratch can be a very tricky task, especially for applications such as NLP where engineering and optimization are difficult. The growing complexity of SOTA models makes training and tuning simply impractical, approaching impossible, tasks for small-scale enterprises. Startups and researchers alike simply do not have the computational resources to utilize and explore such models on their own, so access to pre-trained models<\/strong> for transfer learning, fine-tuning, or out-of-the-box inference is invaluable.<\/p>\n\n\n\nIn the arena of model availability, PyTorch and TensorFlow diverge sharply. Both PyTorch and TensorFlow have their own official model repositories, as we\u2019ll explore below in the Ecosystems section, but practitioners may want to utilize models from other sources. Let\u2019s take a quantitative look at model availability for each framework.<\/p>\n\n\n\n
HuggingFace<\/h3>\n\n\n\n
HuggingFace<\/a> makes it possible to incorporate trained and tuned SOTA models into your pipelines in just a few lines of code.<\/p>\n\n\n\nWhen we compare HuggingFace model availability for PyTorch vs TensorFlow, the results are staggering. Below we see a chart of the total number of models available on HuggingFace that are either PyTorch or TensorFlow exclusive, or available for both frameworks. As we can see, the number of models available for use exclusively in PyTorch absolutely blows the competition out of the water. Almost 92% of models are PyTorch exclusive<\/strong>, <\/strong>up from 85% last year<\/a>. In contrast, only about 8% being TensorFlow exclusive, with only about 14% of all models available for TensorFlow (down from 16% last year<\/a>). Further, over 45 thousand PyTorch exclusive<\/strong> models were added in 2022, whereas only about 4 thousand TensorFlow exclusive models were added.<\/p>\n\n\n\n![\"\"\/](\"https:\/\/www.assemblyai.com\/blog\/content\/images\/2023\/01\/num_hf_models_2023.png\")
<\/figure>\n\n\n\nWe see interesting results if we relegate our purview to only the 30<\/strong> most popular<\/strong> models <\/strong>on HuggingFace. While all models are available in PyTorch and none are TensorFlow exclusive, like last year, the number of models available for both has increased from 19 to 23<\/a>, indicating that there may be an effort for TensorFlow coverage on the most popular models.<\/p>\n\n\n\n![\"\"\/](\"https:\/\/www.assemblyai.com\/blog\/content\/images\/2023\/01\/num_top_models_2023.png\")
<\/figure>\n\n\n\nResearch Papers<\/h3>\n\n\n\n
For research practitioners especially, having access to models from recently-published papers is critical. Attempting to recreate new models that you want to explore in a different framework wastes valuable time, so being able to clone a repository and immediately start experimenting means that you can focus on the important work.<\/p>\n\n\n\n
Given that PyTorch is the de facto<\/em> research framework, we would expect the trend we observed on HuggingFace to continue into the research community as a whole; and our intuition is correct.<\/p>\n\n\n\nWe\u2019ve aggregated data from eight top research journals over the past several years into the below graph, which shows the relative proportion of publications that use PyTorch or TensorFlow. As you can see, the adoption of PyTorch was extremely rapid and, in just a few years, grew from use in just about 7% to use in almost 80% of papers that use either PyTorch or TensorFlow.<\/p>\n\n\n\n![\"\"\/](\"https:\/\/www.assemblyai.com\/blog\/content\/images\/2021\/12\/Fraction-of-Papers-Using-PyTorch-vs.-TensorFlow.png\")
Data source<\/a><\/figcaption><\/figure>\n\n\n\nMuch of the reason for this rapid adoption was due to difficulties with TensorFlow 1 that were exacerbated in the context of research, leading researchers to look to the newer alternative PyTorch. While many of TensorFlow\u2019s issues were addressed with the release of TensorFlow 2 in 2019, PyTorch\u2019s momentum has been great enough for it to maintain itself as the established research-centric framework, at least from a community perspective.<\/p>\n\n\n\n
We can see the same pattern if we look at the fraction of researchers who migrated frameworks. When we look at publications by authors that were using either PyTorch or TensorFlow in 2018 and 2019, we find that the majority<\/em> of authors who used TensorFlow in 2018 migrated to PyTorch in 2019 (55%), while the vast majority of authors who used PyTorch in 2018 stayed with PyTorch 2019 (85%). This data is visualized in the Sankey diagram below, where the left side corresponds to 2018 and the right side to 2019. Note that the data represent proportions of users of each framework in 2018, not total numbers.<\/p>\n\n\n\n![\"\"\/](\"https:\/\/www.assemblyai.com\/blog\/content\/images\/2021\/12\/Sankey.png\")
Data source<\/a><\/figcaption><\/figure>\n\n\n\nCareful readers will notice that this data is from before<\/em> the release of TensorFlow 2, but as we will see in the next section, this fact is irrelevant in the research community.<\/p>\n\n\n\nPapers with Code<\/h3>\n\n\n\n
Lastly, we look at data from Papers with Code<\/a> – a website whose mission it is to create a free and open resource with Machine Learning papers, code, datasets, etc. We\u2019ve plotted the percentage of papers which utilize PyTorch, TensorFlow, or another framework over time, with data aggregated quarterly, from late 2017 to the current quarter. We see the steady growth of papers utilizing PyTorch – out of the 3,319 repositories created this quarter, nearly 70%<\/em> of them are implemented in PyTorch, with just 4% <\/em>implemented in TensorFlow <\/strong>(down from 11% last year<\/a>).<\/p>\n\n\n\n![\"\"\/](\"https:\/\/www.assemblyai.com\/blog\/content\/images\/2023\/01\/percentage_repo_2023.png\")
<\/figure>\n\n\n\nConversely, we see the steady decline in use of TensorFlow. Even the release of TensorFlow 2 in 2019, which addressed many of the issues that made using TensorFlow 1 painful for research, was not enough to reverse this trend. We see a near-monotonic decline<\/strong> in the popularity of TensorFlow, even after the release of TensorFlow 2.<\/p>\n\n\n\nModel Availability – Final Words<\/h3>\n\n\n\n
It is obvious from the above data that PyTorch currently dominates the research landscape. While TensorFlow 2 made utilizing TensorFlow for research a lot easier, PyTorch has given researchers no reason to go back and give TensorFlow another try. Furthermore, backward compatibility issues between old research in TensorFlow 1 and new research in TensorFlow 2 only exacerbate this issue.<\/p>\n\n\n\n
For now, PyTorch is the clear winner in the area of research simply for the reason that it has been widely adopted by the community, and most publications\/available models use PyTorch.<\/p>\n\n\n\n
There are a couple of notable exceptions \/ notes:<\/p>\n\n\n\n
\n- Google Brain: <\/strong>Google Brain makes heavy use of JAX and makes use of Flax<\/a> – Google’s neural network library for JAX<\/a>.<\/li>\n\n\n\n
- DeepMind: <\/strong>DeepMind standardized the use of TensorFlow in 2016, although they announced in 2020<\/a> that they were using JAX to accelerate their research. In this announcement they also give an overview of their JAX ecosystem, most notably Haiku<\/a>, their JAX-based neural network library.
DeepMind created Sonnet<\/a>, which is a high-level API for TensorFlow that is tailored towards research and sometimes called \u201cthe research version of Keras\u201d, that may be useful to those considering using TensorFlow for research. Note that its development has slowed down. Further, DeepMind\u2019s Acme<\/a> framework may be essential to Reinforcement Learning practitioners.<\/li>\n\n\n\n- OpenAI: <\/strong>On the other hand, OpenAI standardized the usage of PyTorch internally in 2020; but, once again for those in Reinforcement Learning, their older baselines<\/a> repository is implemented in TensorFlow. Baselines provides high-quality implementation of Reinforcement Learning algorithms, so TensorFlow may be the best choice for Reinforcement Learning practitioners.<\/li>\n\n\n\n
- JAX: <\/strong>Google has another project called JAX<\/a> that is growing in popularity in the research community. There is, in some sense, a lot less overhead in JAX compared to PyTorch or TensorFlow; but it\u2019s underlying philosophy is different than both PyTorch and TensorFlow, and for this reason migrating to JAX may not be a good option for most. There are a growing number of models\/papers that utilize JAX, and it is being developed at a very rapid pace<\/a>. While last year the future of JAX was less clear, it is now apparent that JAX is here to stay.<\/li>\n<\/ul>\n\n\n\n
Want to learn more about JAX?<\/p>\n\n\n\n
Check out our introductory guide to learn more about JAX, including recommendations on when to use it.<\/p>\n\n\n\n
Check it out<\/a><\/p>\n\n\n\nTensorFlow has a long and arduous, if not impossible, journey ahead if it wants to reestablish itself as the dominant research framework.<\/p>\n\n\n\n
Round 1<\/strong> in the PyTorch vs TensorFlow debate goes to PyTorch<\/strong>.<\/p>\n\n\n\n#<\/a>PyTorch vs TensorFlow – Deployment<\/h2>\n\n\n\nWhile employing state-of-the-art (SOTA) models for cutting-edge results is the holy grail of Deep Learning applications from an inference perspective, this ideal is not always practical or even possible to achieve in an industry setting. Access to SOTA models is pointless if there is a laborious, error-prone process of making their intelligence actionable. Therefore, beyond considering which framework affords you access to the shiniest models, it is important to consider the end-to-end Deep Learning process in each framework.<\/p>\n\n\n\n
TensorFlow has been the go-to framework for deployment-oriented applications since its inception, and for good reason. TensorFlow has a litany of associated tools that make the end-to-end Deep Learning process easy and efficient. For deployment specifically, TensorFlow Serving<\/strong> and TensorFlow Lite<\/strong> allow you to painlessly deploy on clouds, servers, mobile, and IoT devices.<\/p>\n\n\n\nPyTorch used to be extremely lackluster from a deployment perspective, but it has worked on closing this gap in recent years. The introduction of TorchServe <\/strong>last year and PyTorch Live<\/strong> just a couple of weeks ago has afforded much-needed native deployment tools, but has PyTorch closed the deployment gap enough to make its use worthwhile in an industry setting? Let\u2019s take a look.<\/p>\n\n\n\nTensorFlow<\/h3>\n\n\n\n
TensorFlow offers scalable production with static graphs which are optimized for inference performance. When deploying a model with TensorFlow, you use either TensorFlow Serving or TensorFlow Lite depending on the application.<\/p>\n\n\n\n
TensorFlow Serving<\/h4>\n\n\n\n
TensorFlow Serving<\/a> is for use when deploying TensorFlow models on servers, be them in-house or on the cloud, and is used within the TensorFlow Extended<\/a> (TFX) end-to-end Machine Learning platform. Serving makes it easy to serialize models into well-defined directories with model tags, and select which model is used to make inference requests while keeping server architecture and APIs static.<\/p>\n\n\n\nServing allows you to easily deploy models on specialized gRPC servers, which run Google\u2019s open-source framework for high-performance RPC<\/a>. gRPC was designed with the intent on connecting a diverse ecosystem of microservices, so these servers are well suited for model deployment. Serving as a whole is tightly integrated with Google Cloud<\/a> via Vertex AI<\/a> and integrates with Kubernetes<\/a> and Docker<\/a>.<\/p>\n\n\n\nTensorFlow Lite<\/h4>\n\n\n\n
TensorFlow Lite<\/a> (TFLite) is for use when deploying TensorFlow models on mobile or IoT\/embedded devices. TFLite compresses and optimizes models for these devices, and more widely addresses 5 constraints for on-device Artificial Intelligence – latency<\/strong>, connectivity<\/strong>, privacy<\/strong>, size<\/strong>, <\/strong>and power consumption<\/strong>. The same pipeline is used to simultaneously export both standard Keras-based SavedModels<\/code> (used with Serving) and TFLite models, so model quality can be compared.<\/p>\n\n\n\nTFLite can be used for both Android and iOS, as well as microcontrollers (ARM with Bazel<\/a> or CMake<\/a>) and embedded Linux (e.g. a Coral<\/a> device). TensorFlow’s APIs for Python, Java, C++, JavaScript, and Swift (archived as of this year), give developers a wide array of language options.<\/p>\n\n\n\nPyTorch<\/h3>\n\n\n\n
PyTorch has invested in making deployment easier, previously being notoriously lackluster in this arena. Previously, PyTorch users would need to use something like Flask or Django to build a REST API on top of the model, but now they have native deployment options in the form of TorchServe <\/strong>and PyTorch Live<\/strong>.<\/p>\n\n\n\nTorchServe<\/h4>\n\n\n\n
TorchServe<\/a> is an open-source deployment framework resulting from a collaboration between AWS and Facebook (now Meta) and was released in 2020. It has features like endpoint specification, model archiving, and observing metrics; but is still older than the TensorFlow alternative. Both REST and gRPC APIs are supported with TorchServe.<\/p>\n\n\n\nPyTorch Live<\/h4>\n\n\n\n
PyTorch first released PyTorch Mobile<\/a> in 2019, which was designed to create an end-to-end workflow for the deployment of optimized machine learning models for Android, iOS, and Linux.<\/p>\n\n\n\nPyTorch Live<\/a> was released in late 2022 to build upon Mobile. It uses JavaScript and React Native to create cross-platform iOS and Android AI-powered apps with associated UIs. The on-device inference is still performed by PyTorch Mobile. Live comes with example projects to bootstrap from, and has plans to support audio and video input in the future<\/p>\n\n\n\nDeployment – Final Words<\/h3>\n\n\n\n
Currently, TensorFlow still wins on the deployment front. Serving and TFLite are more robust than the PyTorch competitors, and the ability to use TFLite for local AI in conjunction with Google\u2019s Coral devices is a must-have for many industries. In contrast, PyTorch Live focuses on mobile only, and TorchServe is still in its infancy. The playing field is more even for applications where models run in the cloud instead of on edge devices. It will be interesting to see how the deployment arena changes in the coming years, but for now Round 2<\/strong> in the PyTorch vs TensorFlow debate goes to TensorFlow<\/strong>.<\/p>\n\n\n\nA final note on the issues of model availability and deployment: For those who want to use the TensorFlow deployment infrastructure but want access to models that are only available in PyTorch, consider using ONNX<\/a> to port the models from PyTorch to TensorFlow<\/em><\/p>\n\n\n\n#<\/a>PyTorch vs TensorFlow – Ecosystems<\/h2>\n\n\n\nThe final important consideration that separates PyTorch and TensorFlow in 2023 is the ecosystems in which they are situated. Both PyTorch and TensorFlow are capable frameworks from a modeling perspective, and their technical differences at this point are less important than the ecosystems surrounding them, which provide tools for easy deployment, management, distributed training, etc. Let\u2019s take a look at each framework\u2019s ecosystem now.<\/p>\n\n\n\n
PyTorch<\/h3>\n\n\n\nHub<\/h4>\n\n\n\n
Beyond platforms like HuggingFace, there is also the official PyTorch Hub<\/a> – a research-oriented platform for sharing repositories with pre-trained models. Hub has a wide range of models, including those for Audio, Vision, and NLP. It also has generative models, including a GAN<\/a> for generating high-quality images of celebrity faces.<\/p>\n\n\n\n![\"\"\/](\"https:\/\/lh4.googleusercontent.com\/Q_STziz6TIpPtb2lxgLA3G7v9ugxDcX_5D5ZnLBl0ZroHrzfBKy-FXP4KNnUsYrGJn4M8fg32JQWgsX-opK5rNqX0dOxrLffKYib7wKnCAP7wiV2PoCQSH1G9362BHXfkwEMlyjN\")
Image source<\/a><\/figcaption><\/figure>\n\n\n\nPyTorch-XLA<\/h4>\n\n\n\n
If you want train PyTorch models on Google’s Cloud TPUs, then PyTorch-XLA<\/a> is the tool for you. PyTorch-XLA is a Python package that connects the two with the XLA compiler<\/a>. You can check out PyTorch-XLA’s GitHub repo here<\/a>.<\/p>\n\n\n\nTorchVision<\/h4>\n\n\n\n
TorchVision<\/a> is PyTorch’s official Computer Vision library. It includes everything you need for your Computer Vision projects, including model architectures<\/a> and popular datasets<\/a>. For more vision models, you can check out TIMM<\/a> (pyTorch IMage Models). You can check out the TorchVision GitHub repo here<\/a>.<\/p>\n\n\n\nTorchText<\/h4>\n\n\n\n
If your area of expertise is Natural Language Processing rather than Computer Vision, then you might want to check out TorchText<\/a>. It contains datasets frequently seen in the NLP domain, as well as data processing utilities to operate on these and other datasets. You might want to check out fairseq<\/a> too, so you can perform tasks like translation and summarization on your text. You can check out the TorchText GitHub repo here<\/a>.<\/p>\n\n\n\nTorchAudio<\/h4>\n\n\n\n
Perhaps before you can process text, you need to extract it from an audio file with ASR<\/a>. In this case, check out TorchAudio<\/a> – PyTorch’s official audio library. TorchAudio includes popular audio models<\/a> like DeepSpeech and Wav2Vec, and provides walkthroughs<\/a> and pipelines<\/a> for ASR and other tasks. You can check TorchAudio’s GitHub repo here<\/a>.<\/p>\n\n\n\nSpeechBrain<\/h4>\n\n\n\n
If TorchAudio isn’t quite what you’re looking for, then you might want to check out SpeechBrain<\/a> – an open-source speech toolkit for PyTorch. SpeechBrain supports ASR, speaker recognition, verification and diarization, and more! If you don\u2019t want to build any models and instead want a plug-and-play tool with features like Auto Chapters<\/a>, Sentiment Analysis<\/a>, Entity Detection<\/a>, and more, check out AssemblyAI\u2019s own Speech-to-Text API<\/a>.<\/p>\n\n\n\nESPnet<\/h4>\n\n\n\n
ESPnet<\/a> is a toolkit for end-to-end speech processing which uses PyTorch in conjunction with Kaldi<\/a>‘s style of data processing. With ESPnet, you can implement end-to-end speech recognition, translation, diarization, and more!<\/p>\n\n\n\nAllenNLP<\/h4>\n\n\n\n
If you’re looking for even more NLP tools, you might want to check out AllenNLP<\/a>, an open-source NLP research library built on PyTorch and backed by the Allen Institute for AI<\/a>.<\/p>\n\n\n\nEcosystem Tools<\/h4>\n\n\n\n
Check out PyTorch\u2019s Tools<\/a> page for other libraries that may be useful, such as those tailored to Computer Vision or Natural Language Processing. This includes fast.ai<\/a> – a popular library for producing neural networks using modern best practices.<\/p>\n\n\n\nTorchElastic<\/h4>\n\n\n\n
TorchElastic<\/a> was released in 2020 and is the result of collaboration between AWS and Facebook. It is a tool for distributed training which manages worker processes and coordinates restart behaviors so that you can train models on a cluster of compute nodes which can change dynamically without affecting training. Therefore, TorchElastic prevents catastrophic failures from issues such as server maintenance events or network issues so you do not lose training progress. TorchElastic features integration with Kubernetes and has been incorporated into PyTorch 1.9+.<\/p>\n\n\n\nTorchX<\/h4>\n\n\n\n
TorchX<\/a> is an SDK for the quick building and deployment of Machine Learning applications. TorchX includes the Training Session Manager API to launch distributed PyTorch applications onto supported schedulers. It is responsible for launching the distributed job while natively supporting jobs that are locally<\/em> managed by TorchElastic.<\/p>\n\n\n\nLightning<\/h4>\n\n\n\n
PyTorch Lightning<\/a> is sometimes called the Keras of PyTorch. While this comparison is slightly misleading, Lightning is a useful tool for simplifying the model engineering and training processes in PyTorch, and it has matured significantly since its initial release in 2019. Lightning approaches the modeling process in an object-oriented way, defining reusable and shareable components that can be utilized across projects. For more information on Lightning and a comparison of how its workflow compares to vanilla PyTorch, you can check out this<\/a> tutorial.<\/p>\n\n\n\nTensorFlow<\/h3>\n\n\n\nHub<\/h4>\n\n\n\n
TensorFlow Hub<\/a> is a repository of trained Machine Learning models ready for fine-tuning, allowing you to use a model like BERT with just a few lines of code. Hub contains TensorFlow, TensorFlow Lite, and TensorFlow.js models for different use cases, with models available for image, video, audio, and text problem domains. Get started with a tutorial here<\/a>, or see a list of models here<\/a>.<\/p>\n\n\n\nModel Garden<\/h4>\n\n\n\n
If ready-to-use pre-trained models aren\u2019t going to work for your application, then TensorFlow\u2019s Model Garden<\/a> is a repository that makes the source code for SOTA models available. It is useful if you want to go under-the-hood to understand how models work, or modify them for your own needs – something that is not possible with serialized pre-trained models beyond transfer learning and fine tuning.<\/p>\n\n\n\nModel Garden contains directories for official models maintained by Google, research models maintained by researchers, and curated community models maintained by the community. TensorFlow\u2019s long term goal is to provide pre-trained versions of models from Model Garden on Hub, and for pretrained models on Hub to have available source code in Model Garden.<\/p>\n\n\n\n
Extended (TFX)<\/h4>\n\n\n\n
TensorFlow Extended<\/a> is TensorFlow’s end-to-end platform for model deployment. You can load, validate, analyze, and transform data; train and evaluate models; deploy models using Serving or Lite; and then track artifacts and their dependencies. TFX can be used with Jupyter or Colab, and can use Apache Airflow<\/a>\/Beam<\/a> or Kubernetes for orchestration. TFX is tightly integrated with Google Cloud<\/a> and can be used with Vertex AI Pipelines.<\/p>\n\n\n\nVertex AI<\/h4>\n\n\n\n
Vertex AI<\/a> is Google Cloud\u2019s unified Machine Learning platform. It was released this year and seeks to unify services on GCP<\/a>, AI Platform<\/a>, and AutoML<\/a> into one platform. Vertex AI can help you automate, monitor, and govern Machine Learning systems by orchestrating workflows in a serverless manner. Vertex AI can also store artifacts of a workflow, allowing you to keep track of dependencies and a model\u2019s training data, hyperparameters, and source code.<\/p>\n\n\n\nMediaPipe<\/h4>\n\n\n\n
MediaPipe<\/a> is a framework for building multimodal, cross-platform applied Machine Learning pipelines which can be used for face detection, multi-hand tracking, object detection, and more. The project is open-source<\/a> and has bindings in several languages including Python, C++, and JavaScript. More information on getting started with MediaPipe and its ready-to-use solutions can be found here<\/a>.<\/p>\n\n\n\nCoral<\/h4>\n\n\n\n
While there are a variety of SaaS companies that rely on cloud-based AI, there is a growing need for local AI in many industries. Google Coral<\/a> was created to address this need, and is a complete toolkit to build products with local AI. Coral was released in 2020 and addresses the difficulties of implementing on-board AI mentioned in the TFLite portion of the Deployment section, including privacy and efficiency.<\/p>\n\n\n\nCoral offers an array of hardware products for prototyping, production, and sensing, some of which are essentially more powerful Raspberry Pis created specifically for AI applications. Their products utilize their Edge TPU<\/a>s for high performance inference on low-power devices. Coral also offers pre-compiled models for image segmentation, pose estimation, speech recognition, and more to provide scaffolding for developers looking to create their own local AI systems. The essential steps to create a model can be seen in the flowchart below.<\/p>\n\n\n\n![\"\"\/](\"https:\/\/www.assemblyai.com\/blog\/content\/images\/2021\/12\/Coral2.png\")
Image source<\/a><\/figcaption><\/figure>\n\n\n\nTensorFlow.js<\/h4>\n\n\n\n
TensorFlow.js<\/a> is a JavaScript library for Machine Learning that allows you to train and deploy models both in the browser and server-side with Node.js. They provide documentation<\/a> with examples and information on how to import Python models, pre-trained models<\/a> ready for out-of-the-box use, and live demos<\/a> with associated code.<\/p>\n\n\n\nCloud<\/h4>\n\n\n\n
TensorFlow Cloud<\/a> is a library that allows you to connect your local environment to Google Cloud. The provided APIs are designed to bridge the gap from model building and debugging on your local machine to distributed training and hyperparameter tuning on Google Cloud, without the need to use Cloud Console.<\/p>\n\n\n\nColab<\/h4>\n\n\n\n
Google Colab<\/a> is a cloud-based notebook environment, very similar to Jupyter. It is easy to connect Colab to Google Cloud for GPU or TPU training. Note that PyTorch can also be used with Colab.<\/p>\n\n\n\nPlayground<\/h4>\n\n\n\n
Playground<\/a> is a small but polished educational tool for understanding the basics of neural networks. It offers a simple dense network visualized inside a clean UI. You can change the number of layers in the network and their sizes to see in real time how features are learned. You can also see how changing hyperparameters like learning rate and regularization strength affects the learning process on different datasets. Playground allows you to play the learning process in real time to see in a highly visual way how inputs are transformed during the training process. Playground even comes with an open-source small neural network library<\/a> on which it was built so you can understand the nuts and bolts of the network.<\/p>\n\n\n\nDatasets<\/h4>\n\n\n\n
Google Research\u2019s Datasets<\/a> is a dataset resource on which Google periodically releases datasets. Google also has a Dataset Search<\/a> for access to an even wider dataset database. PyTorch users can of course take advantage of these datasets as well.<\/p>\n\n\n\nEcosystems – Final Words<\/h3>\n\n\n\n
This round is easily the closest of the three. Google has invested heavily in ensuring that there is an available product in each relevant area of an end-to-end Deep Learning workflow, although how well-polished these products are varies across this landscape. Even still, the close integration with Google Cloud along with TFX make the end-to-end development process efficient and organized, and the ease of porting models to Google Coral devices hands a landslide victory to TensorFlow for some industries. That having been said, it is clear TensorFlow is a fading framework, and the rapid development of ONNX only exacerbates this process given that the TensorFlow ecosystem can be leveraged by teams who build with PyTorch for many applications. For this reason, unlike last year, PyTorch will win this battle.<\/p>\n\n\n\n
Round 3<\/strong> in the PyTorch vs TensorFlow debate goes to PyTorch<\/strong>.<\/p>\n\n\n\n#<\/a>PyTorch 2.0 and JAX<\/h2>\n\n\n\nPyTorch 2.0<\/h3>\n\n\n\n
At the end of 2022, PyTorch 2.0<\/a> was announced, and it marks a very substantial shift in the story of Deep Learning frameworks. Just as PyTorch had the benefit of learning from TensorFlow’s mistakes, PyTorch 2 has the benefit of learning from PyTorch 1’s “mistakes”. These “mistakes” are really design decisions that made sense in 2016, but less so in 2023. Deep Learning is much more mature than it was when PyTorch 1 was created – just in the last year we’ve seen substantial progress in models like DALL-E 2<\/a>, Stable Diffusion<\/a>, and ChatGPT<\/a>.<\/p>\n\n\n\nThe flagship feature of PyTorch 2.0 (if you had to pick) is model compilation<\/strong>, which will allow models to be ahead-of-time compiled for lightning-fast execution. PyTorch 2.0 has made efforts to make distributed training simpler too, which means that the lifecycle for PyTorch Deep Learning projects may be reduced.<\/p>\n\n\n\nPyTorch 2.0’s first stable release is slated for early March of 2023<\/strong><\/p>\n\n\n\nJAX<\/h3>\n\n\n\n
Google’s JAX<\/a> has quickly been growing in popularity. While not a Deep Learning framework itself, it is a numerical computing library with autograd and easy distributed training. Combined with the fact that it is built on XLA<\/a>, JAX is clearly a very strong candidate for the future of Deep Learning.<\/p>\n\n\n\nThe big caveat is that much of its power stems from the fact that JAX takes a functionally-pure<\/a> approach that may be foreign to PyTorch and TensorFlow practitioners. That having been said, the Deep Learning infrastructure surrounding JAX matures every month; and if<\/em> the functionally-pure approach does not prove to make JAX a nonstarter, we expect a rapid adoption in the coming years.<\/p>\n\n\n\nAt this point, it is apparent that the battle for the future of Deep Learning will be PyTorch 2.0 vs JAX<\/strong>, and only time will tell who the victor is.<\/p>\n\n\n\n#<\/a>Should I Use PyTorch or TensorFlow?<\/h2>\n\n\n\nAs you probably expect, the PyTorch vs TensorFlow debate does not have a single correct answer – it is only sensible to say that one framework is superior to another with respect to a specific use-case. To help you decide which framework is best for you, we\u2019ve compiled our recommendations into flow charts below, with each chart tailored to a different area of interest.<\/p>\n\n\n\n
What if I\u2019m in Industry?<\/h3>\n\n\n\n![\"\"\/](\"https:\/\/www.assemblyai.com\/blog\/content\/images\/2023\/01\/2---PyTorch-vs-TensorFlow.png\")
<\/figure>\n\n\n\nIf you perform Deep Learning engineering in an industry setting, you\u2019re likely using TensorFlow. In this case, you should probably stick with it. TensorFlow\u2019s robust deployment framework and end-to-end TensorFlow Extended platform are invaluable for those who need to productionize models. Easy deployment on a gRPC server along with model monitoring and artifact tracking are critical tools for industry use. You may want to start thinking about migrating to PyTorch in the future, but for now you can stick with your current workflow.<\/p>\n\n\n\n
On the other hand, if you’re starting a project from scratch, it may be advisable to build with PyTorch and then deploy with TensorFlow’s tools. If you want to use one ecosystem and don’t want to deal with the hassle of ONNX, then TensorFlow remains a good option.<\/p>\n\n\n\n
Bottom line: if you have to choose one framework, choose TensorFlow<\/strong>.<\/p>\n\n\n\nWhat if I\u2019m a Researcher?<\/h3>\n\n\n\n![\"\"\/](\"https:\/\/www.assemblyai.com\/blog\/content\/images\/2021\/12\/Research.png\")
<\/figure>\n\n\n\nIf you\u2019re a researcher, you\u2019re likely using PyTorch and should probably stick with it for now. PyTorch is the de facto <\/em>research framework, and the announcement of PyTorch 2.0 only makes its future more promising. We certainly encourage you to check out JAX<\/a> (especially if you train on TPUs) and think about being an early adopter, but most researchers will probably want to stick with PyTorch this year.<\/p>\n\n\n\nThere are a couple of notable exceptions to this rule, the most notable being that those in Reinforcement Learning should consider using TensorFlow. TensorFlow has a native Agents<\/a> library for Reinforcement Learning, and DeepMind\u2019s Acme<\/a> framework is implemented in TensorFlow. OpenAI\u2019s Baselines<\/a> model repository is also implemented in TensorFlow, although OpenAI\u2019s Gym<\/a> can be used with either TensorFlow or PyTorch. If you plan to use TensorFlow for your research, you should also check out DeepMind\u2019s Sonnet<\/a> for higher-level abstractions. That having been said, Reinforcement Learning<\/a> is finding its way into more and more Deep Learning research, so a stronger PyTorch ecosystem is likely to develop around it in the coming years.<\/p>\n\n\n\nWhichever framework you choose, you should keep your eye on JAX in 2023, especially as its community grows and more publications begin utilizing it.<\/p>\n\n\n\n
Bottom line: if you need to choose one framework, choose PyTorch<\/strong><\/p>\n\n\n\nWhat if I\u2019m a Professor?<\/h3>\n\n\n\n![\"\"\/](\"https:\/\/www.assemblyai.com\/blog\/content\/images\/2021\/12\/Professor.png\")
<\/figure>\n\n\n\nIf you\u2019re a professor, which framework to use for a Deep Learning course depends on the goals of the course. If the focus of your course is to produce industry-ready Deep Learning engineers who can hit the ground running with competency in the entire end-to-end Deep Learning process, not just Deep Learning theory, then you should use TensorFlow. In this case, exposure to the TensorFlow ecosystem and its tools along with end-to-end practice projects will be very instructive and valuable.<\/p>\n\n\n\n
If the focus of your course is on Deep Learning theory and understanding the under-the-hood of Deep Learning models, then you should use PyTorch. This is especially true if you are teaching a high-level undergraduate course or an early graduate-level course which intends to prepare students to perform Deep Learning research.<\/p>\n\n\n\n
Ideally, students should get exposure to each framework, and dedicating some time to understanding the differences between the frameworks is probably valuable despite the time constraints of a single semester. If the course is part of a larger program in Machine Learning with many classes dedicated to different topics, it may be better to stick with the framework best suited for the course material rather than try to give exposure to both.<\/p>\n\n\n\n
What if I\u2019m a Hobbyist?<\/h3>\n\n\n\n![\"\"\/](\"https:\/\/www.assemblyai.com\/blog\/content\/images\/2021\/12\/Hobbyist_cropped.png\")
<\/figure>\n\n\n\nIf you\u2019re a hobbyist who\u2019s interested in Deep Learning, which framework you use will depend on your goals. If you\u2019re implementing a Deep Learning model as part of some larger project, then TensorFlow is likely what you want to use, especially if you are deploying to an IoT\/embedded device. While you could use PyTorch for mobile applications given the release of PyTorch Live, TensorFlow + TFLite is still the preferred methodology for now.<\/p>\n\n\n\n
If your goal is to learn about Deep Learning for its own sake, then which framework is best in this case depends on your background. In general, PyTorch is probably the better option here, especially if you\u2019re used to working in Python. If you\u2019re a total beginner who\u2019s just getting started learning about Deep Learning, see the next section.<\/p>\n\n\n\n
What if I\u2019m a Total Beginner?<\/h3>\n\n\n\n![\"\"\/](\"https:\/\/www.assemblyai.com\/blog\/content\/images\/2021\/12\/Beginner_cropped.png\")
<\/figure>\n\n\n\nIf you\u2019re a total beginner who\u2019s interested in Deep Learning and just wants to get started, we recommend using Keras<\/strong>. Using its high-level components, you can easily get started understanding the basics of Deep Learning. Once you are prepared to start understanding more thoroughly the nuts-and-bolts of Deep Learning, you have a couple of options:<\/p>\n\n\n\nIf you do not want to install a new framework and are worried about how well your competency will translate to a new API, then you can try \u201cdropping down\u201d from Keras to TensorFlow. Depending on your background, TensorFlow may be confusing, in which case try moving to PyTorch.<\/p>\n\n\n\n
If you want a framework that more natively feels like Python, then moving to PyTorch may be your best move. In this case, be aware that you\u2019ll have to install a new framework and potentially rewrite custom scripts. Further, if PyTorch seems a bit cumbersome to you, you can compartmentalize your code and get rid of some boilerplate by using PyTorch Lightning<\/a>.<\/p>\n\n\n\nIf you\u2019re a complete beginner, consider watching some YouTube tutorials<\/a> in both TensorFlow and PyTorch to decide which framework feels more intuitive to you.<\/p>\n\n\n\n#<\/a>Final Words<\/h2>\n\n\n\nAs you can see, the PyTorch vs TensorFlow debate is a nuanced one whose landscape is constantly changing, and out-of-date information makes understanding this landscape even more difficult. In 2023, both PyTorch and TensorFlow are very mature frameworks, and their core Deep Learning features overlap significantly. Today, the practical considerations of each framework, like their model availability, time to deploy, and associated ecosystems, supersede their technical differences<\/strong>.<\/p>\n\n\n\nYou\u2019re not making a mistake in choosing either framework, for both have good documentation, many learning resources, and active communities. While PyTorch has become the de facto <\/em>research framework after its explosive adoption by the research community and TensorFlow remains the legacy industry framework, there are certainly use cases for each in both domains.<\/p>\n\n\n\nHopefully our recommendations have helped you navigate the complicated PyTorch vs TensorFlow landscape! For more helpful information, check out other content on our blog<\/a>!<\/p>\n","protected":false},"excerpt":{"rendered":"
We\u2019ll explore each of these three practical considerations in turn, and then provide our recommendations for which framework to use in different areas.<\/p>\n\n\n\n
Looking for more posts like this?<\/p>\n\n\n\n
Subscribe to our newsletter!<\/p>\n\n\n\n
Subscribe Now<\/a><\/p>\n\n\n\n Implementing a successful Deep Learning model from scratch can be a very tricky task, especially for applications such as NLP where engineering and optimization are difficult. The growing complexity of SOTA models makes training and tuning simply impractical, approaching impossible, tasks for small-scale enterprises. Startups and researchers alike simply do not have the computational resources to utilize and explore such models on their own, so access to pre-trained models<\/strong> for transfer learning, fine-tuning, or out-of-the-box inference is invaluable.<\/p>\n\n\n\n In the arena of model availability, PyTorch and TensorFlow diverge sharply. Both PyTorch and TensorFlow have their own official model repositories, as we\u2019ll explore below in the Ecosystems section, but practitioners may want to utilize models from other sources. Let\u2019s take a quantitative look at model availability for each framework.<\/p>\n\n\n\n HuggingFace<\/a> makes it possible to incorporate trained and tuned SOTA models into your pipelines in just a few lines of code.<\/p>\n\n\n\n When we compare HuggingFace model availability for PyTorch vs TensorFlow, the results are staggering. Below we see a chart of the total number of models available on HuggingFace that are either PyTorch or TensorFlow exclusive, or available for both frameworks. As we can see, the number of models available for use exclusively in PyTorch absolutely blows the competition out of the water. Almost 92% of models are PyTorch exclusive<\/strong>, <\/strong>up from 85% last year<\/a>. In contrast, only about 8% being TensorFlow exclusive, with only about 14% of all models available for TensorFlow (down from 16% last year<\/a>). Further, over 45 thousand PyTorch exclusive<\/strong> models were added in 2022, whereas only about 4 thousand TensorFlow exclusive models were added.<\/p>\n\n\n\n We see interesting results if we relegate our purview to only the 30<\/strong> most popular<\/strong> models <\/strong>on HuggingFace. While all models are available in PyTorch and none are TensorFlow exclusive, like last year, the number of models available for both has increased from 19 to 23<\/a>, indicating that there may be an effort for TensorFlow coverage on the most popular models.<\/p>\n\n\n\n For research practitioners especially, having access to models from recently-published papers is critical. Attempting to recreate new models that you want to explore in a different framework wastes valuable time, so being able to clone a repository and immediately start experimenting means that you can focus on the important work.<\/p>\n\n\n\n Given that PyTorch is the de facto<\/em> research framework, we would expect the trend we observed on HuggingFace to continue into the research community as a whole; and our intuition is correct.<\/p>\n\n\n\n We\u2019ve aggregated data from eight top research journals over the past several years into the below graph, which shows the relative proportion of publications that use PyTorch or TensorFlow. As you can see, the adoption of PyTorch was extremely rapid and, in just a few years, grew from use in just about 7% to use in almost 80% of papers that use either PyTorch or TensorFlow.<\/p>\n\n\n\n Much of the reason for this rapid adoption was due to difficulties with TensorFlow 1 that were exacerbated in the context of research, leading researchers to look to the newer alternative PyTorch. While many of TensorFlow\u2019s issues were addressed with the release of TensorFlow 2 in 2019, PyTorch\u2019s momentum has been great enough for it to maintain itself as the established research-centric framework, at least from a community perspective.<\/p>\n\n\n\n We can see the same pattern if we look at the fraction of researchers who migrated frameworks. When we look at publications by authors that were using either PyTorch or TensorFlow in 2018 and 2019, we find that the majority<\/em> of authors who used TensorFlow in 2018 migrated to PyTorch in 2019 (55%), while the vast majority of authors who used PyTorch in 2018 stayed with PyTorch 2019 (85%). This data is visualized in the Sankey diagram below, where the left side corresponds to 2018 and the right side to 2019. Note that the data represent proportions of users of each framework in 2018, not total numbers.<\/p>\n\n\n\n Careful readers will notice that this data is from before<\/em> the release of TensorFlow 2, but as we will see in the next section, this fact is irrelevant in the research community.<\/p>\n\n\n\n Lastly, we look at data from Papers with Code<\/a> – a website whose mission it is to create a free and open resource with Machine Learning papers, code, datasets, etc. We\u2019ve plotted the percentage of papers which utilize PyTorch, TensorFlow, or another framework over time, with data aggregated quarterly, from late 2017 to the current quarter. We see the steady growth of papers utilizing PyTorch – out of the 3,319 repositories created this quarter, nearly 70%<\/em> of them are implemented in PyTorch, with just 4% <\/em>implemented in TensorFlow <\/strong>(down from 11% last year<\/a>).<\/p>\n\n\n\n Conversely, we see the steady decline in use of TensorFlow. Even the release of TensorFlow 2 in 2019, which addressed many of the issues that made using TensorFlow 1 painful for research, was not enough to reverse this trend. We see a near-monotonic decline<\/strong> in the popularity of TensorFlow, even after the release of TensorFlow 2.<\/p>\n\n\n\n It is obvious from the above data that PyTorch currently dominates the research landscape. While TensorFlow 2 made utilizing TensorFlow for research a lot easier, PyTorch has given researchers no reason to go back and give TensorFlow another try. Furthermore, backward compatibility issues between old research in TensorFlow 1 and new research in TensorFlow 2 only exacerbate this issue.<\/p>\n\n\n\n For now, PyTorch is the clear winner in the area of research simply for the reason that it has been widely adopted by the community, and most publications\/available models use PyTorch.<\/p>\n\n\n\n There are a couple of notable exceptions \/ notes:<\/p>\n\n\n\n Want to learn more about JAX?<\/p>\n\n\n\n Check out our introductory guide to learn more about JAX, including recommendations on when to use it.<\/p>\n\n\n\n Check it out<\/a><\/p>\n\n\n\n TensorFlow has a long and arduous, if not impossible, journey ahead if it wants to reestablish itself as the dominant research framework.<\/p>\n\n\n\n Round 1<\/strong> in the PyTorch vs TensorFlow debate goes to PyTorch<\/strong>.<\/p>\n\n\n\n While employing state-of-the-art (SOTA) models for cutting-edge results is the holy grail of Deep Learning applications from an inference perspective, this ideal is not always practical or even possible to achieve in an industry setting. Access to SOTA models is pointless if there is a laborious, error-prone process of making their intelligence actionable. Therefore, beyond considering which framework affords you access to the shiniest models, it is important to consider the end-to-end Deep Learning process in each framework.<\/p>\n\n\n\n TensorFlow has been the go-to framework for deployment-oriented applications since its inception, and for good reason. TensorFlow has a litany of associated tools that make the end-to-end Deep Learning process easy and efficient. For deployment specifically, TensorFlow Serving<\/strong> and TensorFlow Lite<\/strong> allow you to painlessly deploy on clouds, servers, mobile, and IoT devices.<\/p>\n\n\n\n PyTorch used to be extremely lackluster from a deployment perspective, but it has worked on closing this gap in recent years. The introduction of TorchServe <\/strong>last year and PyTorch Live<\/strong> just a couple of weeks ago has afforded much-needed native deployment tools, but has PyTorch closed the deployment gap enough to make its use worthwhile in an industry setting? Let\u2019s take a look.<\/p>\n\n\n\n TensorFlow offers scalable production with static graphs which are optimized for inference performance. When deploying a model with TensorFlow, you use either TensorFlow Serving or TensorFlow Lite depending on the application.<\/p>\n\n\n\n TensorFlow Serving<\/a> is for use when deploying TensorFlow models on servers, be them in-house or on the cloud, and is used within the TensorFlow Extended<\/a> (TFX) end-to-end Machine Learning platform. Serving makes it easy to serialize models into well-defined directories with model tags, and select which model is used to make inference requests while keeping server architecture and APIs static.<\/p>\n\n\n\n Serving allows you to easily deploy models on specialized gRPC servers, which run Google\u2019s open-source framework for high-performance RPC<\/a>. gRPC was designed with the intent on connecting a diverse ecosystem of microservices, so these servers are well suited for model deployment. Serving as a whole is tightly integrated with Google Cloud<\/a> via Vertex AI<\/a> and integrates with Kubernetes<\/a> and Docker<\/a>.<\/p>\n\n\n\n TensorFlow Lite<\/a> (TFLite) is for use when deploying TensorFlow models on mobile or IoT\/embedded devices. TFLite compresses and optimizes models for these devices, and more widely addresses 5 constraints for on-device Artificial Intelligence – latency<\/strong>, connectivity<\/strong>, privacy<\/strong>, size<\/strong>, <\/strong>and power consumption<\/strong>. The same pipeline is used to simultaneously export both standard Keras-based TFLite can be used for both Android and iOS, as well as microcontrollers (ARM with Bazel<\/a> or CMake<\/a>) and embedded Linux (e.g. a Coral<\/a> device). TensorFlow’s APIs for Python, Java, C++, JavaScript, and Swift (archived as of this year), give developers a wide array of language options.<\/p>\n\n\n\n PyTorch has invested in making deployment easier, previously being notoriously lackluster in this arena. Previously, PyTorch users would need to use something like Flask or Django to build a REST API on top of the model, but now they have native deployment options in the form of TorchServe <\/strong>and PyTorch Live<\/strong>.<\/p>\n\n\n\n TorchServe<\/a> is an open-source deployment framework resulting from a collaboration between AWS and Facebook (now Meta) and was released in 2020. It has features like endpoint specification, model archiving, and observing metrics; but is still older than the TensorFlow alternative. Both REST and gRPC APIs are supported with TorchServe.<\/p>\n\n\n\n PyTorch first released PyTorch Mobile<\/a> in 2019, which was designed to create an end-to-end workflow for the deployment of optimized machine learning models for Android, iOS, and Linux.<\/p>\n\n\n\n PyTorch Live<\/a> was released in late 2022 to build upon Mobile. It uses JavaScript and React Native to create cross-platform iOS and Android AI-powered apps with associated UIs. The on-device inference is still performed by PyTorch Mobile. Live comes with example projects to bootstrap from, and has plans to support audio and video input in the future<\/p>\n\n\n\n Currently, TensorFlow still wins on the deployment front. Serving and TFLite are more robust than the PyTorch competitors, and the ability to use TFLite for local AI in conjunction with Google\u2019s Coral devices is a must-have for many industries. In contrast, PyTorch Live focuses on mobile only, and TorchServe is still in its infancy. The playing field is more even for applications where models run in the cloud instead of on edge devices. It will be interesting to see how the deployment arena changes in the coming years, but for now Round 2<\/strong> in the PyTorch vs TensorFlow debate goes to TensorFlow<\/strong>.<\/p>\n\n\n\n A final note on the issues of model availability and deployment: For those who want to use the TensorFlow deployment infrastructure but want access to models that are only available in PyTorch, consider using ONNX<\/a> to port the models from PyTorch to TensorFlow<\/em><\/p>\n\n\n\n The final important consideration that separates PyTorch and TensorFlow in 2023 is the ecosystems in which they are situated. Both PyTorch and TensorFlow are capable frameworks from a modeling perspective, and their technical differences at this point are less important than the ecosystems surrounding them, which provide tools for easy deployment, management, distributed training, etc. Let\u2019s take a look at each framework\u2019s ecosystem now.<\/p>\n\n\n\n Beyond platforms like HuggingFace, there is also the official PyTorch Hub<\/a> – a research-oriented platform for sharing repositories with pre-trained models. Hub has a wide range of models, including those for Audio, Vision, and NLP. It also has generative models, including a GAN<\/a> for generating high-quality images of celebrity faces.<\/p>\n\n\n\n If you want train PyTorch models on Google’s Cloud TPUs, then PyTorch-XLA<\/a> is the tool for you. PyTorch-XLA is a Python package that connects the two with the XLA compiler<\/a>. You can check out PyTorch-XLA’s GitHub repo here<\/a>.<\/p>\n\n\n\n TorchVision<\/a> is PyTorch’s official Computer Vision library. It includes everything you need for your Computer Vision projects, including model architectures<\/a> and popular datasets<\/a>. For more vision models, you can check out TIMM<\/a> (pyTorch IMage Models). You can check out the TorchVision GitHub repo here<\/a>.<\/p>\n\n\n\n If your area of expertise is Natural Language Processing rather than Computer Vision, then you might want to check out TorchText<\/a>. It contains datasets frequently seen in the NLP domain, as well as data processing utilities to operate on these and other datasets. You might want to check out fairseq<\/a> too, so you can perform tasks like translation and summarization on your text. You can check out the TorchText GitHub repo here<\/a>.<\/p>\n\n\n\n Perhaps before you can process text, you need to extract it from an audio file with ASR<\/a>. In this case, check out TorchAudio<\/a> – PyTorch’s official audio library. TorchAudio includes popular audio models<\/a> like DeepSpeech and Wav2Vec, and provides walkthroughs<\/a> and pipelines<\/a> for ASR and other tasks. You can check TorchAudio’s GitHub repo here<\/a>.<\/p>\n\n\n\n If TorchAudio isn’t quite what you’re looking for, then you might want to check out SpeechBrain<\/a> – an open-source speech toolkit for PyTorch. SpeechBrain supports ASR, speaker recognition, verification and diarization, and more! If you don\u2019t want to build any models and instead want a plug-and-play tool with features like Auto Chapters<\/a>, Sentiment Analysis<\/a>, Entity Detection<\/a>, and more, check out AssemblyAI\u2019s own Speech-to-Text API<\/a>.<\/p>\n\n\n\n ESPnet<\/a> is a toolkit for end-to-end speech processing which uses PyTorch in conjunction with Kaldi<\/a>‘s style of data processing. With ESPnet, you can implement end-to-end speech recognition, translation, diarization, and more!<\/p>\n\n\n\n If you’re looking for even more NLP tools, you might want to check out AllenNLP<\/a>, an open-source NLP research library built on PyTorch and backed by the Allen Institute for AI<\/a>.<\/p>\n\n\n\n Check out PyTorch\u2019s Tools<\/a> page for other libraries that may be useful, such as those tailored to Computer Vision or Natural Language Processing. This includes fast.ai<\/a> – a popular library for producing neural networks using modern best practices.<\/p>\n\n\n\n TorchElastic<\/a> was released in 2020 and is the result of collaboration between AWS and Facebook. It is a tool for distributed training which manages worker processes and coordinates restart behaviors so that you can train models on a cluster of compute nodes which can change dynamically without affecting training. Therefore, TorchElastic prevents catastrophic failures from issues such as server maintenance events or network issues so you do not lose training progress. TorchElastic features integration with Kubernetes and has been incorporated into PyTorch 1.9+.<\/p>\n\n\n\n TorchX<\/a> is an SDK for the quick building and deployment of Machine Learning applications. TorchX includes the Training Session Manager API to launch distributed PyTorch applications onto supported schedulers. It is responsible for launching the distributed job while natively supporting jobs that are locally<\/em> managed by TorchElastic.<\/p>\n\n\n\n PyTorch Lightning<\/a> is sometimes called the Keras of PyTorch. While this comparison is slightly misleading, Lightning is a useful tool for simplifying the model engineering and training processes in PyTorch, and it has matured significantly since its initial release in 2019. Lightning approaches the modeling process in an object-oriented way, defining reusable and shareable components that can be utilized across projects. For more information on Lightning and a comparison of how its workflow compares to vanilla PyTorch, you can check out this<\/a> tutorial.<\/p>\n\n\n\n TensorFlow Hub<\/a> is a repository of trained Machine Learning models ready for fine-tuning, allowing you to use a model like BERT with just a few lines of code. Hub contains TensorFlow, TensorFlow Lite, and TensorFlow.js models for different use cases, with models available for image, video, audio, and text problem domains. Get started with a tutorial here<\/a>, or see a list of models here<\/a>.<\/p>\n\n\n\n If ready-to-use pre-trained models aren\u2019t going to work for your application, then TensorFlow\u2019s Model Garden<\/a> is a repository that makes the source code for SOTA models available. It is useful if you want to go under-the-hood to understand how models work, or modify them for your own needs – something that is not possible with serialized pre-trained models beyond transfer learning and fine tuning.<\/p>\n\n\n\n Model Garden contains directories for official models maintained by Google, research models maintained by researchers, and curated community models maintained by the community. TensorFlow\u2019s long term goal is to provide pre-trained versions of models from Model Garden on Hub, and for pretrained models on Hub to have available source code in Model Garden.<\/p>\n\n\n\n TensorFlow Extended<\/a> is TensorFlow’s end-to-end platform for model deployment. You can load, validate, analyze, and transform data; train and evaluate models; deploy models using Serving or Lite; and then track artifacts and their dependencies. TFX can be used with Jupyter or Colab, and can use Apache Airflow<\/a>\/Beam<\/a> or Kubernetes for orchestration. TFX is tightly integrated with Google Cloud<\/a> and can be used with Vertex AI Pipelines.<\/p>\n\n\n\n Vertex AI<\/a> is Google Cloud\u2019s unified Machine Learning platform. It was released this year and seeks to unify services on GCP<\/a>, AI Platform<\/a>, and AutoML<\/a> into one platform. Vertex AI can help you automate, monitor, and govern Machine Learning systems by orchestrating workflows in a serverless manner. Vertex AI can also store artifacts of a workflow, allowing you to keep track of dependencies and a model\u2019s training data, hyperparameters, and source code.<\/p>\n\n\n\n MediaPipe<\/a> is a framework for building multimodal, cross-platform applied Machine Learning pipelines which can be used for face detection, multi-hand tracking, object detection, and more. The project is open-source<\/a> and has bindings in several languages including Python, C++, and JavaScript. More information on getting started with MediaPipe and its ready-to-use solutions can be found here<\/a>.<\/p>\n\n\n\n While there are a variety of SaaS companies that rely on cloud-based AI, there is a growing need for local AI in many industries. Google Coral<\/a> was created to address this need, and is a complete toolkit to build products with local AI. Coral was released in 2020 and addresses the difficulties of implementing on-board AI mentioned in the TFLite portion of the Deployment section, including privacy and efficiency.<\/p>\n\n\n\n Coral offers an array of hardware products for prototyping, production, and sensing, some of which are essentially more powerful Raspberry Pis created specifically for AI applications. Their products utilize their Edge TPU<\/a>s for high performance inference on low-power devices. Coral also offers pre-compiled models for image segmentation, pose estimation, speech recognition, and more to provide scaffolding for developers looking to create their own local AI systems. The essential steps to create a model can be seen in the flowchart below.<\/p>\n\n\n\n TensorFlow.js<\/a> is a JavaScript library for Machine Learning that allows you to train and deploy models both in the browser and server-side with Node.js. They provide documentation<\/a> with examples and information on how to import Python models, pre-trained models<\/a> ready for out-of-the-box use, and live demos<\/a> with associated code.<\/p>\n\n\n\n TensorFlow Cloud<\/a> is a library that allows you to connect your local environment to Google Cloud. The provided APIs are designed to bridge the gap from model building and debugging on your local machine to distributed training and hyperparameter tuning on Google Cloud, without the need to use Cloud Console.<\/p>\n\n\n\n Google Colab<\/a> is a cloud-based notebook environment, very similar to Jupyter. It is easy to connect Colab to Google Cloud for GPU or TPU training. Note that PyTorch can also be used with Colab.<\/p>\n\n\n\n Playground<\/a> is a small but polished educational tool for understanding the basics of neural networks. It offers a simple dense network visualized inside a clean UI. You can change the number of layers in the network and their sizes to see in real time how features are learned. You can also see how changing hyperparameters like learning rate and regularization strength affects the learning process on different datasets. Playground allows you to play the learning process in real time to see in a highly visual way how inputs are transformed during the training process. Playground even comes with an open-source small neural network library<\/a> on which it was built so you can understand the nuts and bolts of the network.<\/p>\n\n\n\n Google Research\u2019s Datasets<\/a> is a dataset resource on which Google periodically releases datasets. Google also has a Dataset Search<\/a> for access to an even wider dataset database. PyTorch users can of course take advantage of these datasets as well.<\/p>\n\n\n\n This round is easily the closest of the three. Google has invested heavily in ensuring that there is an available product in each relevant area of an end-to-end Deep Learning workflow, although how well-polished these products are varies across this landscape. Even still, the close integration with Google Cloud along with TFX make the end-to-end development process efficient and organized, and the ease of porting models to Google Coral devices hands a landslide victory to TensorFlow for some industries. That having been said, it is clear TensorFlow is a fading framework, and the rapid development of ONNX only exacerbates this process given that the TensorFlow ecosystem can be leveraged by teams who build with PyTorch for many applications. For this reason, unlike last year, PyTorch will win this battle.<\/p>\n\n\n\n Round 3<\/strong> in the PyTorch vs TensorFlow debate goes to PyTorch<\/strong>.<\/p>\n\n\n\n At the end of 2022, PyTorch 2.0<\/a> was announced, and it marks a very substantial shift in the story of Deep Learning frameworks. Just as PyTorch had the benefit of learning from TensorFlow’s mistakes, PyTorch 2 has the benefit of learning from PyTorch 1’s “mistakes”. These “mistakes” are really design decisions that made sense in 2016, but less so in 2023. Deep Learning is much more mature than it was when PyTorch 1 was created – just in the last year we’ve seen substantial progress in models like DALL-E 2<\/a>, Stable Diffusion<\/a>, and ChatGPT<\/a>.<\/p>\n\n\n\n The flagship feature of PyTorch 2.0 (if you had to pick) is model compilation<\/strong>, which will allow models to be ahead-of-time compiled for lightning-fast execution. PyTorch 2.0 has made efforts to make distributed training simpler too, which means that the lifecycle for PyTorch Deep Learning projects may be reduced.<\/p>\n\n\n\n PyTorch 2.0’s first stable release is slated for early March of 2023<\/strong><\/p>\n\n\n\n Google’s JAX<\/a> has quickly been growing in popularity. While not a Deep Learning framework itself, it is a numerical computing library with autograd and easy distributed training. Combined with the fact that it is built on XLA<\/a>, JAX is clearly a very strong candidate for the future of Deep Learning.<\/p>\n\n\n\n The big caveat is that much of its power stems from the fact that JAX takes a functionally-pure<\/a> approach that may be foreign to PyTorch and TensorFlow practitioners. That having been said, the Deep Learning infrastructure surrounding JAX matures every month; and if<\/em> the functionally-pure approach does not prove to make JAX a nonstarter, we expect a rapid adoption in the coming years.<\/p>\n\n\n\n At this point, it is apparent that the battle for the future of Deep Learning will be PyTorch 2.0 vs JAX<\/strong>, and only time will tell who the victor is.<\/p>\n\n\n\n As you probably expect, the PyTorch vs TensorFlow debate does not have a single correct answer – it is only sensible to say that one framework is superior to another with respect to a specific use-case. To help you decide which framework is best for you, we\u2019ve compiled our recommendations into flow charts below, with each chart tailored to a different area of interest.<\/p>\n\n\n\n If you perform Deep Learning engineering in an industry setting, you\u2019re likely using TensorFlow. In this case, you should probably stick with it. TensorFlow\u2019s robust deployment framework and end-to-end TensorFlow Extended platform are invaluable for those who need to productionize models. Easy deployment on a gRPC server along with model monitoring and artifact tracking are critical tools for industry use. You may want to start thinking about migrating to PyTorch in the future, but for now you can stick with your current workflow.<\/p>\n\n\n\n On the other hand, if you’re starting a project from scratch, it may be advisable to build with PyTorch and then deploy with TensorFlow’s tools. If you want to use one ecosystem and don’t want to deal with the hassle of ONNX, then TensorFlow remains a good option.<\/p>\n\n\n\n Bottom line: if you have to choose one framework, choose TensorFlow<\/strong>.<\/p>\n\n\n\n If you\u2019re a researcher, you\u2019re likely using PyTorch and should probably stick with it for now. PyTorch is the de facto <\/em>research framework, and the announcement of PyTorch 2.0 only makes its future more promising. We certainly encourage you to check out JAX<\/a> (especially if you train on TPUs) and think about being an early adopter, but most researchers will probably want to stick with PyTorch this year.<\/p>\n\n\n\n There are a couple of notable exceptions to this rule, the most notable being that those in Reinforcement Learning should consider using TensorFlow. TensorFlow has a native Agents<\/a> library for Reinforcement Learning, and DeepMind\u2019s Acme<\/a> framework is implemented in TensorFlow. OpenAI\u2019s Baselines<\/a> model repository is also implemented in TensorFlow, although OpenAI\u2019s Gym<\/a> can be used with either TensorFlow or PyTorch. If you plan to use TensorFlow for your research, you should also check out DeepMind\u2019s Sonnet<\/a> for higher-level abstractions. That having been said, Reinforcement Learning<\/a> is finding its way into more and more Deep Learning research, so a stronger PyTorch ecosystem is likely to develop around it in the coming years.<\/p>\n\n\n\n Whichever framework you choose, you should keep your eye on JAX in 2023, especially as its community grows and more publications begin utilizing it.<\/p>\n\n\n\n Bottom line: if you need to choose one framework, choose PyTorch<\/strong><\/p>\n\n\n\n If you\u2019re a professor, which framework to use for a Deep Learning course depends on the goals of the course. If the focus of your course is to produce industry-ready Deep Learning engineers who can hit the ground running with competency in the entire end-to-end Deep Learning process, not just Deep Learning theory, then you should use TensorFlow. In this case, exposure to the TensorFlow ecosystem and its tools along with end-to-end practice projects will be very instructive and valuable.<\/p>\n\n\n\n If the focus of your course is on Deep Learning theory and understanding the under-the-hood of Deep Learning models, then you should use PyTorch. This is especially true if you are teaching a high-level undergraduate course or an early graduate-level course which intends to prepare students to perform Deep Learning research.<\/p>\n\n\n\n Ideally, students should get exposure to each framework, and dedicating some time to understanding the differences between the frameworks is probably valuable despite the time constraints of a single semester. If the course is part of a larger program in Machine Learning with many classes dedicated to different topics, it may be better to stick with the framework best suited for the course material rather than try to give exposure to both.<\/p>\n\n\n\n If you\u2019re a hobbyist who\u2019s interested in Deep Learning, which framework you use will depend on your goals. If you\u2019re implementing a Deep Learning model as part of some larger project, then TensorFlow is likely what you want to use, especially if you are deploying to an IoT\/embedded device. While you could use PyTorch for mobile applications given the release of PyTorch Live, TensorFlow + TFLite is still the preferred methodology for now.<\/p>\n\n\n\n If your goal is to learn about Deep Learning for its own sake, then which framework is best in this case depends on your background. In general, PyTorch is probably the better option here, especially if you\u2019re used to working in Python. If you\u2019re a total beginner who\u2019s just getting started learning about Deep Learning, see the next section.<\/p>\n\n\n\n If you\u2019re a total beginner who\u2019s interested in Deep Learning and just wants to get started, we recommend using Keras<\/strong>. Using its high-level components, you can easily get started understanding the basics of Deep Learning. Once you are prepared to start understanding more thoroughly the nuts-and-bolts of Deep Learning, you have a couple of options:<\/p>\n\n\n\n If you do not want to install a new framework and are worried about how well your competency will translate to a new API, then you can try \u201cdropping down\u201d from Keras to TensorFlow. Depending on your background, TensorFlow may be confusing, in which case try moving to PyTorch.<\/p>\n\n\n\n If you want a framework that more natively feels like Python, then moving to PyTorch may be your best move. In this case, be aware that you\u2019ll have to install a new framework and potentially rewrite custom scripts. Further, if PyTorch seems a bit cumbersome to you, you can compartmentalize your code and get rid of some boilerplate by using PyTorch Lightning<\/a>.<\/p>\n\n\n\n If you\u2019re a complete beginner, consider watching some YouTube tutorials<\/a> in both TensorFlow and PyTorch to decide which framework feels more intuitive to you.<\/p>\n\n\n\n As you can see, the PyTorch vs TensorFlow debate is a nuanced one whose landscape is constantly changing, and out-of-date information makes understanding this landscape even more difficult. In 2023, both PyTorch and TensorFlow are very mature frameworks, and their core Deep Learning features overlap significantly. Today, the practical considerations of each framework, like their model availability, time to deploy, and associated ecosystems, supersede their technical differences<\/strong>.<\/p>\n\n\n\n You\u2019re not making a mistake in choosing either framework, for both have good documentation, many learning resources, and active communities. While PyTorch has become the de facto <\/em>research framework after its explosive adoption by the research community and TensorFlow remains the legacy industry framework, there are certainly use cases for each in both domains.<\/p>\n\n\n\n Hopefully our recommendations have helped you navigate the complicated PyTorch vs TensorFlow landscape! For more helpful information, check out other content on our blog<\/a>!<\/p>\n","protected":false},"excerpt":{"rendered":"#<\/a>PyTorch vs TensorFlow – Model Availability<\/h2>\n\n\n\n
HuggingFace<\/h3>\n\n\n\n
<\/figure>\n\n\n\n<\/figure>\n\n\n\nResearch Papers<\/h3>\n\n\n\n
Papers with Code<\/h3>\n\n\n\n
<\/figure>\n\n\n\nModel Availability – Final Words<\/h3>\n\n\n\n
\n
DeepMind created Sonnet<\/a>, which is a high-level API for TensorFlow that is tailored towards research and sometimes called \u201cthe research version of Keras\u201d, that may be useful to those considering using TensorFlow for research. Note that its development has slowed down. Further, DeepMind\u2019s Acme<\/a> framework may be essential to Reinforcement Learning practitioners.<\/li>\n\n\n\n#<\/a>PyTorch vs TensorFlow – Deployment<\/h2>\n\n\n\n
TensorFlow<\/h3>\n\n\n\n
TensorFlow Serving<\/h4>\n\n\n\n
TensorFlow Lite<\/h4>\n\n\n\n
SavedModels<\/code> (used with Serving) and TFLite models, so model quality can be compared.<\/p>\n\n\n\nPyTorch<\/h3>\n\n\n\n
TorchServe<\/h4>\n\n\n\n
PyTorch Live<\/h4>\n\n\n\n
Deployment – Final Words<\/h3>\n\n\n\n
#<\/a>PyTorch vs TensorFlow – Ecosystems<\/h2>\n\n\n\n
PyTorch<\/h3>\n\n\n\n
Hub<\/h4>\n\n\n\n
PyTorch-XLA<\/h4>\n\n\n\n
TorchVision<\/h4>\n\n\n\n
TorchText<\/h4>\n\n\n\n
TorchAudio<\/h4>\n\n\n\n
SpeechBrain<\/h4>\n\n\n\n
ESPnet<\/h4>\n\n\n\n
AllenNLP<\/h4>\n\n\n\n
Ecosystem Tools<\/h4>\n\n\n\n
TorchElastic<\/h4>\n\n\n\n
TorchX<\/h4>\n\n\n\n
Lightning<\/h4>\n\n\n\n
TensorFlow<\/h3>\n\n\n\n
Hub<\/h4>\n\n\n\n
Model Garden<\/h4>\n\n\n\n
Extended (TFX)<\/h4>\n\n\n\n
Vertex AI<\/h4>\n\n\n\n
MediaPipe<\/h4>\n\n\n\n
Coral<\/h4>\n\n\n\n
TensorFlow.js<\/h4>\n\n\n\n
Cloud<\/h4>\n\n\n\n
Colab<\/h4>\n\n\n\n
Playground<\/h4>\n\n\n\n
Datasets<\/h4>\n\n\n\n
Ecosystems – Final Words<\/h3>\n\n\n\n
#<\/a>PyTorch 2.0 and JAX<\/h2>\n\n\n\n
PyTorch 2.0<\/h3>\n\n\n\n
JAX<\/h3>\n\n\n\n
#<\/a>Should I Use PyTorch or TensorFlow?<\/h2>\n\n\n\n
What if I\u2019m in Industry?<\/h3>\n\n\n\n
<\/figure>\n\n\n\nWhat if I\u2019m a Researcher?<\/h3>\n\n\n\n
<\/figure>\n\n\n\nWhat if I\u2019m a Professor?<\/h3>\n\n\n\n
<\/figure>\n\n\n\nWhat if I\u2019m a Hobbyist?<\/h3>\n\n\n\n
<\/figure>\n\n\n\nWhat if I\u2019m a Total Beginner?<\/h3>\n\n\n\n
<\/figure>\n\n\n\n#<\/a>Final Words<\/h2>\n\n\n\n
![85% last year<\/a>. In contrast, only about 8% being TensorFlow exclusive, with only about 14% of all models available for TensorFlow (down from](\"https:\/\/github.com\/AssemblyAI-Examples\/pytorch-vs-tensorflow\/blob\/main\/2022\/num_hf_models_2022.png?utm_source=substack&utm_medium=email&ref=assemblyai.com\"){kind=link}