One fake web page can be enough to trick AI shopping recommendations

AI recommendation systems, which drive approximately 2% of referrals to major shopping sites like Target and Walmart according to data.ai, are susceptible to manipulation through fabricated online content. A study by Minghao Luo and Liang Chen, published on arXiv, demonstrated that search-augmented AI systems can be easily influenced to promote fake brands. Researchers found that when AI models are provided with polluted search results, they can elevate invented products to appear as legitimate recommendations. Luo, a researcher at the Chinese University of Hong Kong, was prompted by a television report detailing an industry that creates fake online reviews, highlighting that "a fake brand can surface in the top recommendation of the mainstream AI system just within hours." To investigate this vulnerability, Luo and Chen developed a benchmark named FORGE (Fake Online Recommendations in Generative Environments). They simulated the typical AI recommendation process by rewriting real search results to substitute genuine products with fabricated ones. Their tests involved 12 commercial and open-weight AI models, all of which were found to be vulnerable, recommending the invented brands. The study revealed that a single manipulated page could lead to a fooling rate of up to 27%, and altering the top three search results increased this rate to 73.8%. Luo expressed surprise at the minimal effort required for manipulation, noting that "You only write one page out of 10." The research also indicated that reasoning capabilities, intended to enhance AI output, did not resolve the issue and in some instances exacerbated it.