Meta Ads Promoted AI App Creating Deepfaked Politician Porn

Meta platforms recently featured advertisements for an AI application named Kromix, which appeared to facilitate the creation of deepfaked videos resembling female United States politicians, despite Meta's stated policies prohibiting advertisements containing sexual material. This incident marks a continuation of Meta's challenges in preventing advertisements for tools that generate nonconsensual intimate imagery from appearing on its platforms. The Kromix application is marketed as an "AI image styler." One promotional video advertisement, observed by WIRED, featured a voice-over that emphasized the tool's "no restrictions" and stated that "all the characters are real people," describing it as "the AI that men actually use." In a specific advertisement, an individual closely resembling a prominent female US politician was depicted in front of American flags, accompanied by the caption "What if she moved?" The ad then transitioned to a scene of a woman with the same facial features engaged in a pornographic video. The Kromix app, which was available on Apple's App Store before WIRED's inquiry, offered paid users the ability to upload photographs for use in scenarios described as "bedroom rape" and "Disney love." Additional videos within the app showcased AI-generated pornographic content featuring women in Spider-Man costumes. Attempts to reach a listed contact for the app were unsuccessful. The app's description on the App Store, prior to its removal, also indicated that users could "make any girl a pornstar" and "make any girl a sex doll." The app's terms of service stated that users must be over 18 years old. Kromix utilized generative AI technology to alter images and videos, allowing users to create explicit content. The app's availability on the App Store raised concerns among privacy advocates and lawmakers regarding the proliferation of AI-generated nonconsensual pornography. Meta's advertising policies explicitly forbid "sexual content, including the sale or promotion of sexual services or explicit imagery." The presence of Kromix ads suggests a significant lapse in Meta's content moderation systems, particularly concerning the detection and removal of advertisements that violate these policies. This incident underscores the ongoing debate surrounding the regulation of AI-generated content and the responsibilities of social media platforms in policing advertisements that promote harmful or exploitative material. The ability of such tools to generate realistic deepfakes poses a threat to individuals, particularly public figures, by enabling the creation and dissemination of fabricated intimate content, which can lead to reputational damage and personal distress. The app's explicit mention of "bedroom rape" in its features highlights the deeply concerning nature of the content it facilitated. The fact that the app was hosted on Apple's App Store prior to WIRED's reporting further indicates potential oversight issues across major technology platforms in managing AI-driven content applications. The situation also brings to light the broader challenges faced by AI developers and platform providers in establishing ethical guidelines and robust safety measures for generative AI technologies.