ChatGPT Is Listening to You (and Might Report You)

The Great Change: OpenAI Admits to Reporting to Authorities

In September 2025, OpenAI made a revelation that shook the global tech community: ChatGPT actively monitors user conversations and reports potentially criminal content to law enforcement.

The news, which emerged almost by chance in a company blog post, revealed that when automated systems detect users who "are planning to harm others," conversations are routed to specialized pipelines where a small team trained in usage policies reviews them. If human reviewers determine that there is an "imminent threat of serious physical harm to others," the case may be referred to law enforcement.

ChatGPT cordially invites you to share your most intimate thoughts. Don't worry, it's all confidential... more or less.

Sources:

• OpenAI Says It's Scanning Users' ChatGPT Conversations and Reporting Content to the Police - Futurism
• ChatGPT privacy questioned as OpenAI confirms police access in extreme cases - Storyboard18

The Contrast with "Protected" Professions

The Privilege of Professional Secrecy

When we speak with a psychologist, a lawyer, a doctor, or a priest, our words are protected by a well-established legal mechanism: professional secrecy. This principle, rooted in centuries of legal tradition, establishes that certain conversations are inviolable, even in the face of criminal investigations.

Characteristics of traditional professional secrecy:

• Very broad protection: Communications remain confidential even in the presence of confessed crimes
• Limited and specific exceptions: Only in extreme cases defined by law can/must certain professionals break the silence
• Qualified human control: The decision to breach confidentiality always remains in the hands of a trained professional
• Ethical responsibility: Professionals are bound by codes of conduct that balance duties toward the client and society

The Real Limits of Professional Secrecy

Contrary to common perception, professional secrecy is not absolute. There are well-defined exceptions that vary by professional category:

For Lawyers (Art. 28 of the Code of Legal Professional Conduct): Disclosure is permitted when necessary for:

• Carrying out defense activities
• Preventing the commission of a particularly serious crime
• Defending oneself in a dispute against one's own client
• Disciplinary proceedings

Critical example: If a client tells the lawyer that they intend to commit murder, between protecting the right to defense and protecting the value of life, the latter must prevail and the lawyer is released from secrecy².

For Psychologists (Art. 13 of the Code of Ethics): Secrecy can be breached when:

• There is an obligation to report for crimes prosecutable ex officio
• There are serious dangers to the life or psychophysical health of the subject and/or third parties
• There is valid and demonstrable consent from the patient

Important distinction: A private psychologist has greater discretion compared to a public one, who as a public official has more stringent reporting obligations³.

Sources:

• Studio Legale Puce - Professional Secrecy of the Lawyer
• La Legge Per Tutti - Must a psychologist who knows of a crime report the patient?

AI as a "Non-Professional"

ChatGPT operates in a completely different gray area:

Lack of legal privilege: Conversations with AI enjoy no legal protection whatsoever. As Sam Altman, CEO of OpenAI, admitted: "If you talk to a therapist or a lawyer or a doctor about those problems, there's legal privilege for it. There's doctor-patient confidentiality, there's legal confidentiality, whatever. And we haven't figured that out yet for when you talk to ChatGPT"².

Automated process: Unlike a human professional who evaluates case by case, ChatGPT uses algorithms to identify "problematic" content, removing qualified human judgment from the initial screening phase.

Sources:

• Sam Altman warns there's no legal confidentiality when using ChatGPT as a therapist - TechCrunch

The Practical Implications: A New Surveillance Paradigm

The Paradox of Technological Trust

The situation creates a worrying paradox. Millions of people use ChatGPT as a digital confidant, sharing intimate thoughts, doubts, fears, and even criminal fantasies that they would never share with a human being. As Sam Altman reports: "People talk about the most personal things in their life to ChatGPT. People use it - especially young people - as a therapist, a life coach"⁴.

The risk of criminal self-censorship: Awareness that conversations can be monitored could paradoxically:

• Push criminals toward more hidden channels
• Prevent people with violent thoughts from seeking help
• Create a "chilling effect" on digital communications

Expertise vs. Algorithms: Who Decides What Is Criminal?

A crucial aspect highlighted by critics concerns the competence of those who make the final decisions.

Human professionals have:

• Years of training to distinguish between fantasies and real intentions
• Codes of ethics that define when to break confidentiality
• Personal legal liability for their decisions
• Ability to assess context and credibility

The ChatGPT system operates with:

• Automated algorithms for initial detection
• OpenAI staff who don't necessarily have clinical or criminological training
• Non-public and potentially arbitrary evaluation criteria
• Absence of external oversight mechanisms

Problematic example: How does an algorithm distinguish between:

• Someone writing a thriller and looking for inspiration for violent scenes
• Someone fantasizing without intention to act
• An individual actually planning a crime

Sources:

• TechCrunch - Sam Altman warns there's no legal confidentiality when using ChatGPT as a therapist
• Shinkai Blog - OpenAI's ChatGPT Conversations Scanned, Reported to Police, Igniting User Outrage

Sources:

• OpenAI's ChatGPT Conversations Scanned, Reported to Police, Igniting User Outrage and Privacy Fears

The OpenAI Contradiction: Privacy vs. Security

The Double Standard

OpenAI's admission creates a stark contradiction with its previous positions. The company has strongly resisted requests for user data in lawsuits, citing privacy protection. In the case against the New York Times, OpenAI argued strenuously against the disclosure of chat logs to protect user privacy.

The irony of the situation: OpenAI defends user privacy in court while simultaneously admitting to monitoring and sharing data with external authorities.

The Impact of the New York Times Case

The situation has become further complicated by the court order requiring OpenAI to retain all ChatGPT logs indefinitely, including private chats and API data. This means conversations users believed were temporary are now permanently archived⁵.

Sources:

• OpenAI slams court order to save all ChatGPT logs, including deleted chats
• OpenAI Lawsuit 2025: Appeals NYT Over ChatGPT Data

Possible Solutions and Alternatives

Towards an "AI Privilege"?

As suggested by Sam Altman, it may be necessary to develop a concept of "AI privilege" - a legal protection similar to that offered to traditional professionals. However, this raises complex questions:

Possible regulatory options:

1. Licensing Model: Only certified AI can offer "conversational privilege"
2. Mandatory Training: Those handling sensitive content must have specific qualifications
3. Professional Oversight: Involvement of qualified psychologists/lawyers in reporting decisions
4. Algorithmic Transparency: Publication of the criteria used to identify "dangerous" content

Intermediate Technical Solutions

"Compartmentalized" AI:

• Separate systems for therapeutic vs. general uses
• End-to-end encryption for sensitive conversations
• Explicit consent for each type of monitoring

"Tripartite" Approach:

• Automatic detection only for immediate and verifiable threats
• Mandatory qualified human review
• Appeal process for contested decisions

The Precedent of Digital Professionals

Lessons from other sectors:

• Telemedicine: Has developed protocols for digital privacy
• Online legal consulting: Uses encryption and identity verification
• Digital therapy: Specialized apps with specific protections

Sources:

• Simon Willison - OpenAI court order analysis

What AI Means for Businesses

Lessons for the Sector

The OpenAI case sets important precedents for the entire artificial intelligence industry:

1. Mandatory transparency: AI companies will need to be more explicit about their monitoring practices
2. Need for ethical frameworks: Clear regulation is needed on when and how AI can interfere with private communications
3. Specialized training: Those making decisions about sensitive content must have appropriate expertise
4. Legal liability: Defining who is responsible when an AI system gets an assessment wrong

Operational Recommendations

For companies developing conversational AI:

• Implement multidisciplinary teams (legal experts, psychologists, criminologists)
• Develop public and verifiable criteria for reporting
• Create appeal processes for users
• Invest in specialized training for review staff

For companies using AI:

• Assess privacy risks before implementation
• Clearly inform users about the limits of confidentiality
• Consider specialized alternatives for sensitive uses

The Future of Digital Confidentiality

The central dilemma: How do we balance preventing real crimes with the right to privacy and digital confidentiality?

The issue is not merely technical but touches on fundamental principles:

• Presumption of innocence: Monitoring private conversations implies generalized suspicion
• Right to privacy: Includes the right to have private thoughts, even disturbing ones
• Preventive effectiveness: It's not proven that digital surveillance actually prevents crimes

Conclusions: Finding the Right Balance

OpenAI's disclosure marks a watershed moment in the evolution of artificial intelligence, but the question isn't whether reporting is right or wrong in absolute terms: it's how to make it effective, fair and respectful of rights.

The need is real: Concrete threats of violence, planning of attacks or other serious crimes require intervention. The problem isn't whether to report, but how to do it responsibly.

The fundamental differences to resolve:

Training and Competence:

• Human professionals have established protocols for distinguishing between real threats and fantasies
• AI systems need equivalent standards and qualified oversight
• Specialized training is needed for those making final decisions

Transparency and Control:

• Professionals operate under the supervision of professional bodies
• OpenAI needs public criteria and external control mechanisms
• Users must know exactly when and why they might be reported

Proportionality:

• Professionals balance the duty of confidentiality with safety on a case-by-case basis
• AI systems must develop similar mechanisms, not binary algorithms

For companies in the sector, the challenge is to develop systems that effectively protect society without becoming tools of indiscriminate surveillance. User trust is essential, but it must coexist with social responsibility.

For users, the lesson is twofold:

1. Conversations with AI don't have the same protections as traditional professionals
2. This isn't necessarily negative if done transparently and proportionately, but it's important to be aware of it

The future of conversational AI requires a new framework that:

• Recognizes the legitimacy of crime prevention
• Establishes professional standards for those handling sensitive content
• Guarantees transparency in decision-making processes
• Protects individual rights without ignoring security

The right question isn't whether machines should report crimes, but how we can ensure they do so with (at least) the same wisdom, training and responsibility as human professionals.

The goal is not to return to AI that is "blind" to real dangers, but to build systems that combine technological efficiency with ethics and human expertise. Only then can we have the best of both worlds: security and protected individual rights.

References and Sources

1. Futurism - "OpenAI Says It's Scanning Users' ChatGPT Conversations and Reporting Content to the Police"
2. Studio Legale Puce - "Segreto Professionale dell'Avvocato"
3. La Legge Per Tutti - "Lo psicologo che sa di un reato deve denunciare il paziente?"
4. TechCrunch - "Sam Altman warns there's no legal confidentiality when using ChatGPT as a therapist"
5. Shinkai Blog - "OpenAI's ChatGPT Conversations Scanned, Reported to Police, Igniting User Outrage and Privacy Fears"
6. Simon Willison - "OpenAI slams court order to save all ChatGPT logs, including deleted chats"
7. Success Knocks - "OpenAI Lawsuit 2025: Appeals NYT Over ChatGPT Data"