“Oops! Try another prompt” — Microsoft accused of selling AI tool that spews violent, sexual images to kids It looks like Microsoft may be filtering violent AI outputs flagged by engineer.
Ashley Belanger – Mar 6, 2024 10:24 pm UTC EnlargeNurPhoto / Contributor | NurPhoto reader comments 74
Microsoft’s AI text-to-image generator Copilot Designer appears to be heavily filtering outputs after a Microsoft engineer, Shane Jones, warned that Microsoft has ignored warnings that the tool randomly creates violent and sexual imagery, CNBC reported.
Jones told CNBC that he repeatedly warned Microsoft of the alarming content he was seeing while volunteering in red-teaming efforts to test the tool’s vulnerabilities. Microsoft failed to take the tool down or implement safeguards in response, Jones said, or even post disclosures to change the product’s rating to mature in the Android store.
Instead, Microsoft apparently did nothing but refer him to report the issue to OpenAI, the maker of the DALL-E model that fuels Copilot Designer’s outputs.
OpenAI never responded, Jones said, so he took increasingly more drastic steps to alert the public to issues he found in Microsoft’s tool.
He started by posting an open letter, calling out OpenAI on LinkedIn. Then, when Microsoft’s legal team told him to take it down, he did as he was told, but he also sent letters to lawmakers and other stakeholders, raising red flags in every direction. That includes letters sent today to the Federal Trade Commission and to Microsoft’s board of directors, CNBC reported.
In Jones’ letter to FTC Chair Lina Khan, Jones said that Microsoft and OpenAI have been aware of these issues since at least October and will “continue to market the product to Anyone. Anywhere. Any Device'” unless the FTC intervenes. Advertisement
Bloomberg also reviewed Jones’ letter and reported that Jones told the FTC that while Copilot Designer is currently marketed as safe for kids, it’s randomly generating inappropriate, sexually objectified image of a woman in some of the pictures it creates. And it can also be used to generate harmful content in a variety of other categories including: political bias, underaged drinking and drug use, misuse of corporate trademarks and copyrights, conspiracy theories, and religion to name a few.
In a separate letter, Jones also implored Microsoft’s board to investigate Microsoft’s AI decision-making and conduct “an independent review of Microsofts responsible AI incident reporting processes.” This is necessary after Jones took “extraordinary efforts to try to raise this issue internally,” including reporting directly to both Microsoft’s Office of Responsible AI and “senior management responsible for Copilot Designer,” CNBC reported.
A Microsoft spokesperson did not confirm whether Microsoft is currently taking steps to filter images, but Ars’ attempt to replicate prompts shared by Jones generated error messages. Instead, a Microsoft spokesperson would only share the same statement provided to CNBC:
We are committed to addressing any and all concerns employees have in accordance with our company policies and appreciate the employees effort in studying and testing our latest technology to further enhance its safety. When it comes to safety bypasses or concerns that could have a potential impact on our services or our partners, we have established in-product user feedback tools and robust internal reporting channels to properly investigate, prioritize and remediate any issues, which we recommended that the employee utilize so we could appropriately validate and test his concerns. We have also facilitated meetings with product leadership and our Office of Responsible AI to review these reports and are continuously incorporating this feedback to strengthen our existing safety systems to provide a safe and positive experience for everyone.
OpenAI did not respond to Ars’ request to comment. Page: 1 2 Next → reader comments 74 Ashley Belanger Ashley is a senior policy reporter for Ars Technica, dedicated to tracking social impacts of emerging policies and new technologies. She is a Chicago-based journalist with 20 years of experience. Advertisement Channel Ars Technica ← Previous story Next story → Related Stories Today on Ars
