Google's AI Search and Shopping Tool Faces Controversy Over Image Generation

Google's I/O 2025 announcements focused on integrating AI deeper into the search experience. AI Mode in Search uses query fan-out techniques to break down questions and provide more relevant content. Deep Search enhances this by issuing hundreds of searches to create expert-level reports. Live capabilities, leveraging Project Astra, allow users to interact with Search in real-time using their camera.

The 'Try It On' tool uses Google’s Shopping Graph to allow users to virtually try on clothes. However, issues have emerged where the tool distorts bodies, adds breasts to male figures, and generates erotic images of minors. This occurs because the tool often conforms the user's body to match idealized body standards of models in the Shopping Graph. Despite Google's policies and disclaimers, the tool's safeguards have proven inadequate, raising concerns about potential misuse and ethical implications. Google's AI has previously exhibited flaws, such as suggesting users eat rocks, indicating the challenges in developing reliable AI systems.

AI Mode is Google's most powerful AI search, featuring advanced reasoning, multimodality, and the ability to answer follow-up questions with helpful links.

The 'Try It On' feature is an AI shopping tool that allows users to upload images of themselves and virtually try on clothes online.

Concerns include the generation of inappropriate images, such as adding breasts to images of men and creating erotic images of minors, raising ethical and safety issues.

Google has implemented measures to block sensitive apparel categories and prevent uploads of images of identifiable minors, but these protections have been found to be porous.

Do you think AI tools like 'Try It On' can be ethically developed and used? Let us know your thoughts!

Share this article with others who need to stay ahead of this trend!