The Grand Illusion: The Myth of Software Portability and Implications for ML Progress

Pushing the boundaries of machine learning often requires exploring different hardware and software combinations. However, the freedom to experiment across different tooling stacks can be at odds with the drive for efficiency, which has produced increasingly specialized AI hardware and incentivized consolidation around a narrow set of ML frameworks.

The article by Fraser Mince, Dzung Dinh, Jonas Kgomo, Neil Thompson, and Sara Hooker affirms that exploratory research can be restricted if software and hardware are co-evolving, making it even harder to stray away from mainstream ideas that work well with popular tooling stacks. While this friction increasingly impacts the rate of innovation in machine learning, to our knowledge the lack of portability in tooling has not been quantified.

In the work, is asked: How portable are popular ML software frameworks? They conduct a large-scale study of the portability of mainstream ML frameworks across different hardware types. The findings paint an uncomfortable picture -- frameworks can lose more than 40% of their key functions when ported to other hardware. Worse, even when functions are portable, the slowdown in their performance can be extreme and render performance untenable.

The results reveal how costly straying from a narrow set of hardware-software combinations can be - and suggest that specialization of hardware impedes innovation in machine learning research.

‍
Understand how these results were found getting access to the full article at Arxiv