To put it simply, as of November 2026, the six main models are nearly equal in terms of handling everyday tasks, but the differences lie in response time, output token count, and whether the request returns filled content. Even with just three questions, there's a 30x gap between the fastest and slowest.

This article is aimed at people who are about to embed API keys into their code. It's for those in positions where issues like pay-as-you-go, batch processing, and timeout settings arise. If you're just occasionally chatting in the browser, the following numbers won't be very helpful. The measurement was completed on November 12, 2026, in one day.

Verification environment: 6 models, 18 requests, one API key

First, I'll write down the environment. This is because differences in the environment change the meaning of the numbers.

Item	Content
Measurement date	November 12, 2026
Endpoint	Single OpenAI-compatible endpoint (to be described later)
API key	One. Shared by all six models
Number of requests	18 (3 tasks × 6 models, one each)
temperature	0.3 for inference and code, 0.7 for writing
Prompt language	Chinese

The models are the following six. I'll note both the displayed name and the actual request ID.

Vendor	Model ID
Anthropic	claude-sonnet-5-5
OpenAI	gpt-5.3-codex
Google	gemini-3.1-pro
DeepSeek	deepseek-v4-pro
ByteDance（Doubao）	doubao-seed-2-1-turbo-260628
Tencent（Hunyuan）	hy4

The prompt was written in Chinese. It was intended for the target users to write in Chinese, and it is not an experiment to measure the quality of prompts in Japanese. This point also applies to the section "What This Validation Cannot Say" mentioned later.

What Was Measured

Three tasks were prepared and the same was given to six models. Judging was done not subjectively, but mechanically via a script.

Reasoning. Three questions with a unique answer. The number of integers between 1 and 1000 that are divisible by 3 or 5 but not by 15, the weekday of March 1, 2027 given that November 12, 2026 is a Thursday, and the conditional probability that the first five tosses are all tails given that at least one head appears in ten tosses of a coin. The correct answers were pre-calculated by a script (401, Monday, 31/1023).

Writing. A 140- to 160-character promotional copy for a robot vacuum cleaner, family-oriented. Must include three selling points, avoid four specific terms (Chinese: "不仅", "极致", "让您", "智能生活"), and prohibit subheadings and emojis. The constraints themselves are part of the test. Anyone can write a copy, but it's not guaranteed that they will adhere to the character limit.

Code. Implement parse_duration(s) in Python. Convert durations like "1h30m" or "90s" into seconds, and return None for invalid inputs. More than six assert statements should be added. The assert statements written by the model were not used for grading; instead, we used thirteen hidden cases we prepared ourselves. These include cases with duplicate units (e.g., "1h1h"), units only (e.g., "h"), no units (e.g., "10"), inputs that are cut off (e.g., "1h30"), negative numbers (e.g., "-5s"), and inputs with swapped order but valid syntax (e.g., "1h90s").

Reasoning: Only one model got the date wrong by one day.

Five models answered all three questions correctly. Codex got two correct answers.

The one that failed was the weekday question. It counted the number of days from November 12, 2026 to March 1, 2027 as 108 days, divided by 7, and concluded that it was a Sunday, advancing three days from Thursday. The correct number of days is 109, with a remainder of 4, which means it should be a Monday. The model missed counting March 1st itself.

The problem was that the output looked perfect. It had numbered steps, neat formatting, and a confident conclusion. There was no "dubious" written anywhere. If we hadn't pre-calculated the correct answer with a script, it would have passed as is.

Response time was significantly larger than accuracy.

Measured Results. The code task for Gemini was not completed, so it was not measured.

• Codex：6.1 seconds, 547 tokens
• Claude：6.6 seconds, 612 tokens
• Gemini：39.1 seconds, 3,531 tokens
• Hunyuan：45.8 seconds, 3,690 tokens
• Doubao：186.0 seconds, 11,958 tokens
• DeepSeek：195.0 seconds, 16,138 tokens

Two models are in the 6-second range, two are around 40 seconds, and two take over 3 minutes. If it's a chat scenario, it's just a matter of waiting, but for batch processing at night with thousands of requests, it directly affects the queue length and timeout rate.

Also check the output token counts. For the same three questions, the slowest model consumes about 30 times more tokens than the fastest one. A low cost per request does not necessarily mean a low cost per request.

Writing: The fastest model fell short by 23 characters.

The five models produced content of the same quality as the original. They stayed within 145-150 characters, covered all three selling points, and had zero prohibited words. The tone varies by media, not by quality. Claude starts with a prompt that asks about the reader's return time, Doubao starts with a discount as if it's an advertisement, and Hunyuan starts from a sense of dissatisfaction about "dust accumulating on the floor in a day."

Codex had 117 characters. It met all the selling points and prohibited words criteria, but only lowered the minimum character count. Its response time was 3.0 seconds and output 104 tokens, making it the lightest among the six models. Sometimes speed and straightforwardness cannot be achieved together. In this case, the output that looks like a finished product was actually a result that got rejected by the character counter.

The cost for the same 150-character output would be as follows (converted using each company's published price per token).

• Codex：$0.00017
• Claude：$0.00122
• DeepSeek：$0.00164
• Gemini：$0.00457

27 times the cost for the same number of characters. This difference is not coming from the price per token. It's the difference in how much each model thinks about the 150-character copy. If you estimate costs based on "price per million tokens," you will make errors in writing-related tasks.

Code：5 models passed the same hidden test, 1 model did not finish

I thought there would be a clear winner here, but there wasn't.

Five models—Claude, Codex, Hunyuan, DeepSeek, and Doubao—passed all 13 hidden cases. They handled all cases, including unit duplication, missing elements, and negative numbers. At this difficulty level, the question "which model writes code well" does not make sense as a question. The difference comes from delivery speed.

• Codex：4.8 seconds
• Claude：8.7 seconds
• Hunyuan：39.3 seconds
• DeepSeek：75.3 seconds
• Doubao：139.3 seconds

A function that can be specified in four lines has a 29-fold difference between the fastest and slowest.

Gemini did not return usable answers. When the output limit is set to 4,096, it continuously returns 429 errors, but when lowered to 2,048, it returns 200, though the code cuts off in the middle of the function. It's not a proof of impossibility, but a record that the model couldn't produce stable results under this route and limit. It's a property that should be known before sending live traffic.

Reproduction Steps: Why 6 companies were compared in half a day

The reason why the same comparison was completed in half a day is because all six companies had the same endpoint, same key, and same request format. Switching models is just a field change, not adding new SDKs, new charges, or new authentication.

If you keep the body the same and only replace the model, since other conditions aren't changing, the comparison won't be muddied. The 18 requests this time are also being sent in this format. The current pricing can be confirmed on the pricing page.Pricing Page

This level of comparison, the cost is less than one dollar. Since conditions were aligned, the next four traps I will mention were also encountered under the same conditions.

Four traps actually encountered during implementation

From here on, it's not about model performance, but about whether the first week will be chaotic. I actually encountered all four.

1. Thinking-type models return 200 with empty content. When Hunyuan and DeepSeek are given an output limit of 4,096, they use the entire budget for the thinking process, leaving the body empty while returning finish_reason as 'length'. Answers were only obtained when raising the limit to 16,384. Writing a client that only checks HTTP status codes would result in shipping a feature that returns nothing.

2. Rate limits appear as 'instability' rather than 'clean errors'. Gemini's 429 only occurs at specific output limits. Changing the limit makes it disappear. Exponential backoff with retries isn't arbitrary here.

3. None of these six APIs perform web searches. When I asked for the previous day's US market closing price, all six models answered 'unable to retrieve.' It's commendable that they didn't fabricate numbers. When declaring the web_search tool in the next request, five models returned 400 (Claude's path only supports function tools, Gemini was judged as having an empty tool name, the remaining three had parameter errors). Codex alone returned 200, fabricated specific news headlines, and attached the homepage of a news site as the source.

4. The model's version is 2026, but its knowledge is even older. When asked about the cutoff of training data, the six models' answers ranged from June 2024 to January 2025. Even if the name is the latest, facts may be up to half a year behind.

Which task to assign to which model

What you want to do	First choice	Second choice	Basis
Write text that people would read	Claude	Doubao	Copy that can be published if you answer 3 questions correctly, run in under 6 seconds, and fix 2 words
Just run it quickly and in large quantities	Codex	Claude	All tasks completed at the fastest speed, writing at 3.0 seconds
Write a single small function	Codex	Claude	Pass through 13 hidden cases, 4.8 seconds
Spoken Chinese copy	Doubao	Hunyuan	Most natural among 6 models, but inference takes 186 seconds
Just want to save as much money as possible	Codex	DeepSeek	Writing cost is 0.17 cents. However, mechanical verification of the output is required
Safe choice when unsure	Hunyuan	Claude	No breakdown in any task, no obvious weaknesses

What cannot be said from this verification

One sample per task, one day, one endpoint. If you repeat the same prompt 20 times, the failure rate will also change. Especially for the inference question, it is a single data point and not a ranking. We have not verified long text context, image input, or tool calls at all. The cost is calculated based on the publicly announced unit prices of each company, and actual charges may vary depending on contract terms.

That's all we can say. At this difficulty level, accuracy is no longer a differentiating factor. What differentiates is response time, output token count, and the initial value of the request output limit. Choose based on these criteria and decide based on your own prompts.