Three things about each model

Each card answers three questions: how restrictive its 500-question results are, whether its treatment changes across case arms, and how it performs.

The Chinese questions were asked in Simplified Chinese; they are shown here in Traditional characters.

Chinese questions: DeepSeek (27.55 points) ties with Doubao (26.94 points) at the top.

English questions: DeepSeek (22.61 points) drops to about the level of the two US models, GPT and Claude.

Cases: the share of cases where it changed its treatment after a condition changed is highest in Taiwan identity: 10 of 13 cases (77%).

Performance: Not slow: median 47.1 secondsLeast stable connection: 857 of 3608 requests failed99.9% of answers usable after re-asking

Chinese questions: Qwen (24.59 points) ties with GLM and Kimi in the group just behind DeepSeek and Doubao, which tie at the top.

English questions: Qwen (25.00 points) is too close to separate from GLM, Kimi and Doubao; together they lead.

Cases: the share of cases where it changed its treatment after a condition changed is highest in Taiwan identity: 10 of 13 cases (77%).

Performance: Mostly stable connection: 131 of 2811 requests failedMiddling to slow: median 72.0 secondsOccasional small format slips: 11 answers unreadable99.6% of answers usable after re-asking

Chinese questions: GLM (24.03 points) ties with Qwen and Kimi in the group just behind DeepSeek and Doubao, which tie at the top.

English questions: GLM (26.10 points) is too close to separate from Qwen, Kimi and Doubao; together they lead.

Cases: the share of cases where it changed its treatment after a condition changed is highest in Taiwan identity: 11 of 13 cases (85%).

Performance: Fastest of the five: median 22.1 secondsMost format mistakes: 25 answers unreadableMost questions blocked by its platform: 11799.0% of answers usable after re-asking

Chinese questions: Kimi (24.32 points) ties with Qwen and GLM in the group just behind DeepSeek and Doubao, which tie at the top.

English questions: Kimi (24.95 points) is too close to separate from GLM, Qwen and Doubao; together they lead.

Cases: the share of cases where it changed its treatment after a condition changed is highest in Responses to criticism and Taiwan identity, tied at 9 of 12 cases each (75%).

Performance: Cleanest format: 2 answers unreadableIts platform often blocks questions: 70 times99.9% of answers usable after re-asking

Chinese questions: Doubao (26.94 points) ties with DeepSeek (27.55 points) at the top.

English questions: Doubao (25.05 points) is too close to separate from GLM, Qwen and Kimi; together they lead.

Cases: the share of cases where it changed its treatment after a condition changed is highest in Taiwan identity: 10 of 13 cases (77%).

Performance: Most stable connection: 56 of 2854 requests failedSlowest: median 177.7 secondsThinks at length before answering: one reply in ten takes over 404.2 seconds99.4% of answers usable after re-asking

Chinese questions: GPT (20.47 points) and the other US model, Claude (22.77 points), both score below all five Chinese models.

English questions: GPT (20.73 points) is about level with DeepSeek and Claude, behind GLM, Qwen, Kimi and Doubao.

Cases: the share of cases where it changed its treatment after a condition changed is highest in Taiwan identity: 11 of 13 cases (85%).

Performance: Stable connection: 0 of 2617 requests failedMedian 12.2 seconds per question100.0% of answers usable after re-asking

Chinese questions: Claude (22.77 points) and the other US model, GPT (20.47 points), both score below all five Chinese models.

English questions: Claude (22.02 points) is about level with DeepSeek and GPT, behind GLM, Qwen, Kimi and Doubao.

Cases: the share of cases where it changed its treatment after a condition changed is highest in Economic policy: 9 of 15 cases (60%).

Performance: Fast: median 9.1 secondsNo recorded connection failures: 0 of 2560 requests failedOften needs format cleanup: 462 answers99.7% of answers usable after re-asking