Among the most widely used machine learning (ML)<\/a> technologies today is the open-source PyTorch framework.<\/p>\n\n\n\n PyTorch<\/a> got its start at Facebook (now known as Meta) in 2016 with the 1.0 release debuting in 2018. In September 2022, Meta moved the PyTorch project to the new PyTorch Foundation<\/a>, which is operated by the Linux Foundation. Today, PyTorch developers took the next major step forward for PyTorch, announcing the first experimental release of PyTorch 2.0. The new release promises to help accelerate ML training and development, while still maintaining backward-compatibility with existing PyTorch application code.<\/p>\n\n\n\n \u201cWe added an additional feature called `torch.compile` that users have to newly insert into their codebases,\u201d Soumith Chintala, lead maintainer, PyTorch. told VentureBeat. \u201cWe are calling it 2.0 because we think users will find it a significant new addition to the experience.\u201d<\/p>\n\n\n\n There have been discussions<\/a> in the past about when the PyTorch project should call a new release 2.0.<\/p>\n\n\n\n Intelligent Security Summit<\/p>\n\n\n\n Learn the critical role of AI & ML in cybersecurity and industry specific case studies on December 8. Register for your free pass today<\/strong>.Register Now<\/a><\/p>\n\n\n\n In 2021, for example, there was a brief discussion on whether PyTorch 1.10 should be labeled as a 2.0 release. Chintala said that PyTorch 1.10 didn\u2019t have enough fundamental changes from 1.9 to warrant a major number upgrade to 2.0.<\/p>\n\n\n\n The most recent generally available release of PyTorch is version 1.13, which came out at the end of October. A key feature in that release came from an IBM code contribution<\/a> enabling the machine learning framework to work more effectively with commodity ethernet-based networking for large-scale workloads.<\/p>\n\n\n\n Chintala emphasized that now is the right time for PyTorch 2.0 because the project is introducing an additional new paradigm in the PyTorch user experience, called torch.compile, that brings solid speedups to users that weren\u2019t possible in the default eager mode of PyTorch 1.0.<\/p>\n\n\n\n He explained that on about 160 open-source models on which the PyTorch project validated early builds of 2.0, there has been a 43% speedup and they worked reliably with the one-line addition to the codebase. ADVERTISEMENT<\/p>\n\n\n\n \u201cWe expect that with PyTorch 2, people will change the way they use PyTorch day-to-day,\u201d Chintala said. <\/p>\n\n\n\n He said that with PyTorch 2.0, developers will start their experiments with eager mode and, once they get to training their models for long periods, activate compiled mode for additional performance.<\/p>\n\n\n\n \u201cData scientists will be able to do with PyTorch 2.x the same things that they did with 1.x, but they can do them faster and at a larger scale,\u201d Chintala said. \u201cIf your model was training over 5 days, and with 2.x\u2019s compiled mode it now trains in 2.5 days, then you can iterate on more ideas with this added time, or build a bigger model that trains within the same 5 days.\u201d<\/p>\n\n\n\n PyTorch gets the first part of its name (Py) from the open-source Python programming language that is widely used in data science.ADVERTISEMENT<\/p>\n\n\n\n Modern PyTorch releases, however, haven\u2019t been entirely written in Python \u2014 as parts of the framework are now written in the C++ programming language.<\/p>\n\n\n\n \u201cOver the years, we\u2019ve moved many parts of torch.nn<\/a> from Python into C++ to squeeze that last-mile performance,\u201d Chintala said.<\/p>\n\n\n\n Chintala said that within the later 2.x series (but not in 2.0), the PyTorch project expects to move code related to torch.nn back into Python. He noted that C++ is typically faster than Python, but the new compiler (torch.compile) ends up being faster than running the equivalent code in C++. <\/p>\n\n\n\n \u201cMoving these parts back to Python improves hackability and lowers the barrier for code contributions,\u201d Chintala said.ADVERTISEMENT<\/p>\n\n\n\n Work on Python 2.0 will be ongoing for the next several months with general availability not expected until March 2023. Alongside the development effort is the transition for PyTorch from being governed and operated by Meta to being its own independent effort.<\/p>\n\n\n\n \u201cIt is early days for the PyTorch Foundation, and you will hear more over a longer time horizon,\u201d Chintala said. \u201cThe foundation is in the process of executing various handoffs and establishing goals.\u201d<\/p>\n\n\n\n <\/p>\n","protected":false},"excerpt":{"rendered":" Link: pytorch-2-0-release-accelerates-open-source-machine-learning Publication Date: September 3 rd, 2022 Among the most widely used machine learning (ML) technologies today is the open-source PyTorch framework. PyTorch got its start at Facebook (now known as Meta)… Read more »<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[51,75,19,9],"tags":[141,161],"class_list":["post-702","post","type-post","status-publish","format-standard","hentry","category-article","category-frameworks","category-ml","category-news","tag-pytorch","tag-pytorch-2-0"],"_links":{"self":[{"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/posts\/702","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/comments?post=702"}],"version-history":[{"count":1,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/posts\/702\/revisions"}],"predecessor-version":[{"id":703,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/posts\/702\/revisions\/703"}],"wp:attachment":[{"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/media?parent=702"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/categories?post=702"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/tags?post=702"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}The new compiler in PyTorch that makes all the difference for ML<\/h2>\n\n\n\n
EVENT<\/h3>\n\n\n\n
More Python coming to PyTorch 2.x<\/h2>\n\n\n\n