AI Demand Drives Up Hardware Costs, Reduces Power

The consumer electronics industry is entering a phase where new hardware products are becoming more expensive and less powerful, a trend driven by the immense demand for components required for artificial intelligence infrastructure. This shift is exemplified by changes in Nvidia's forthcoming Vera CPU, where JPMorgan analysts observed a significant reduction in planned memory. Specifically, the amount of SOCAMM memory attached to each chip has been halved from 1.5 terabytes (TB) to 768 gigabytes (GB). Furthermore, Nvidia has reduced the planned High Bandwidth Memory (HBM) in its Rubin Ultra systems, transitioning from 16-high HBM4E stacks to either 8-high or 12-high versions. JPMorgan analysts characterized these adjustments as "content optimization as a coping mechanism" for constrained memory supply, indicating that the systems do not inherently require less memory but are being designed with less due to supply limitations. Runar Bjorhovde, a senior analyst at the tech research group Omdia, stated that "To this scale, we’ve never seen anything like it before," highlighting the unprecedented nature of these component reductions. This situation does not signify a halt in innovation or the obsolescence of Moore's Law, which historically predicted the doubling of computing power on chips every two years. Instead, the escalating costs associated with producing advanced components are making it unsustainable to maintain the previous pace of improvement. A primary driver of these increased costs is the substantial demand for memory and other sophisticated components from hyperscalers, which are major cloud computing providers building out extensive AI infrastructure. These hyperscalers are willing to pay premium prices for high-end components, leading major memory manufacturers to increasingly direct their production capacity towards these lucrative markets. The expansion of manufacturing capacity for these advanced components is a slow process; new semiconductor plants can take two to three years to construct. Moreover, these new facilities are increasingly designed to cater specifically to the needs of hyperscalers, further diverting resources and production away from the consumer electronics market. This strategic reallocation of resources by component suppliers means that consumer-facing products, such as Apple's upcoming M6-powered Mac minis and Mac Studios, and expected iPhone 18 Pro models, may incorporate less advanced or less memory-intensive configurations to manage costs and availability, even as their prices potentially rise. The consequence for consumers is a potential decrease in the performance-per-dollar ratio for new gadgets, as the underlying technological advancements are being channeled into AI development rather than broadly distributed across the consumer market.