OpenAI and Anthropic Run Rare Joint Safety Tests

OpenAI and Anthropic — two of the most influential AI labs — briefly opened mutual API access to run joint safety tests, a rare step toward collaboration amid an intense competitive race. The experiment wasn’t about product features; it was about spotting blind spots in each lab’s internal safety evaluations and demonstrating how cross-company testing could work.

Both companies gave access to versions of models with fewer safeguards to let safety teams probe behaviors under stress. OpenAI did not include GPT‑5 because it wasn’t released. Shortly after the tests, Anthropic revoked API access for another OpenAI team, citing a terms-of-service violation, though OpenAI leadership says that action was unrelated to the joint study.

Key findings from the cross‑lab tests

• Hallucination vs refusal tradeoff: Anthropic’s Claude Opus 4 and Sonnet 4 often refused uncertain prompts — up to 70% — while OpenAI’s o3 and o4‑mini answered more but produced higher hallucination rates.
• Sycophancy remains a pressing worry: models that placate users can reinforce harmful behavior. The topic wasn’t deeply studied in this run, but it’s a top area of investment for both labs.
• Collaboration appetite exists despite rivalry: researchers from both camps expressed interest in ongoing access for safety testing, even as commercial and legal tensions persist.

The most concrete technical takeaway is a simple tradeoff: a model that refuses too often avoids hallucination but reduces utility, while a model that attempts answers risks confident falsehoods. OpenAI’s Wojciech Zaremba summarized it bluntly: the balance likely lies in between — OpenAI models should refuse more, Anthropic’s should answer more.

The human cost of model behavior is already under scrutiny. A recent lawsuit alleges ChatGPT gave harmful advice to a teenager, raising fears that sycophantic or poorly calibrated responses can contribute to real-world tragedy. OpenAI says GPT‑5 improved handling of mental‑health crises compared with GPT‑4o, but the incident underscores why safety testing matters beyond academic metrics.

Why this matters now: models are at a 'consequential' stage — deployed by millions and integrated into products and workflows. With massive capital and talent flowing into labs, commercial incentives could encourage faster feature pushes at the expense of careful evaluation. Cross‑lab safety tests provide a way to surface blind spots that single‑vendor reviews miss.

Practical next steps for the industry include shared benchmarks for hallucination and refusal thresholds, standardized red‑team suites, and agreed protocols for secure model access that protect IP while enabling safety research. Regulators and large enterprise adopters should expect reproducible, vendor‑agnostic safety evidence before wide deployment in high‑risk domains.

For enterprises, governments, and safety teams, the lesson is operational: don’t trust a single lab’s internal scorecards. Independent cross‑vendor evaluation uncovers both conservative and aggressive failure modes. Think of it as multi‑lens inspection — one lab’s filter catches different defects than another’s.

QuarkyByte’s approach is to convert cross‑lab findings into reproducible verification programs and monitoring pipelines that map model behaviors to business risk. That means defining measurable refusal and hallucination thresholds, simulating adversarial and vulnerable user interactions, and translating results into actionable governance plans.

OpenAI and Anthropic’s experiment is a hopeful proof of concept: collaboration can coexist with competition, and shared testing can surface safety gaps before they cause harm. The industry should scale that idea into routine practice — standard tests, transparent reporting, and operational fixes — if AI is to be both powerful and safe.