San Francisco officials have ordered Apple and Google to remove apps that generate non-consensual deepfake pornography. The city claims these tech giants are profiting from illegal content, potentially exposing them to civil penalties under California law if they fail to act within 28 days.
San Francisco City Attorney David Chiu has issued a formal demand to Apple and Google to immediately remove applications that allow users to create non-consensual deepfake pornography. These so-called 'nudify' apps use artificial intelligence to digitally alter images, creating explicit content without the consent of the individuals involved. This demand follows reports that these platforms have been violating California laws that specifically criminalize the facilitation of such content.
Potential Legal and Financial Risks
The city’s move centers on the claim that Apple and Google have been profiting from these apps by collecting fees from user transactions. City officials estimate that these companies may have generated millions of dollars in revenue from these services. Under California law, there is a provision for civil action that allows victims to seek legal recourse against third-party facilitators. The letters sent to the tech companies warn of potential civil penalties for these legal violations. Apple and Google have been given a 28-day window to respond to these demands and address the presence of these applications on their stores.
History of Warnings and Oversight
This demand is not the first time the tech giants have faced scrutiny regarding this issue. According to communications from the City Attorney’s office, both companies have been notified about processing payments for these apps for nearly a year. Organizations like the Tech Transparency Project (TTP) previously alerted the companies in January and April 2026, providing details on dozens of apps that sell deepfake images for profit. Despite these reports, investigators allege that the apps remained available for download and that both companies at times directed users toward such content.
Impact of AI Regulation on Tech Giants
For investors and market observers, this development highlights the rising regulatory pressure on major technology firms regarding the misuse of generative AI. While Apple and Google provide platforms that host millions of applications, the responsibility of these 'gatekeepers' to screen for illegal or harmful content has become a significant focal point for lawmakers. As California strengthens its stance against non-consensual deepfake imagery, the ability of these companies to effectively police their app ecosystems remains a key monitorable. The financial impact of potential civil penalties will depend on the scale of the enforcement actions taken by authorities and how quickly the companies adjust their moderation policies to comply with state requirements.