Skip to content

Commit b09f0be

Browse files
dihydroJenoxidesunbrye
dihydroJenoxide
and
sunbrye
authored
Updating model comparison table for recent model ships (#58557)
Co-authored-by: sunbrye <sunbrye@github.com> Co-authored-by: Sunbrye Ly <56200261+sunbrye@users.noreply.github.com>
1 parent c269a9d commit b09f0be

File tree

3 files changed

+155
-17
lines changed

3 files changed

+155
-17
lines changed

‎content/copilot/reference/ai-models/model-comparison.md‎

Lines changed: 6 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -29,23 +29,12 @@ contentType: reference
2929

3030
Use this table to find a suitable model quickly, see more detail in the sections below.
3131

32-
| Model | Task area | Excels at (primary use case) | Additional capabilities | Further reading |
33-
|-------------------------------------------------------|--------------------------------------------------|-------------------------------------------------------------------------|-------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------|
34-
| {% data variables.copilot.copilot_gpt_41 %} | General-purpose coding and writing | Fast, accurate code completions and explanations | Agent mode, vision | [{% data variables.copilot.copilot_gpt_41 %} model card](https://openai.com/index/gpt-4-1/) |
35-
| {% data variables.copilot.copilot_gpt_52 %} | Deep reasoning and debugging | Multi-step problem solving and architecture-level code analysis | Agent mode | Not available |
36-
| {% data variables.copilot.copilot_gpt_51 %} | Deep reasoning and debugging | Multi-step problem solving and architecture-level code analysis | Agent mode | Not available |
37-
| {% data variables.copilot.copilot_gpt_5_codex %} | General-purpose coding and writing | Fast, accurate code completions and explanations | Agent mode | [{% data variables.copilot.copilot_gpt_5_codex %} model card](https://cdn.openai.com/pdf/97cc5669-7a25-4e63-b15f-5fd5bdc4d149/gpt-5-codex-system-card.pdf) |
38-
| {% data variables.copilot.copilot_gpt_5_mini %} | General-purpose coding and writing | Fast, accurate code completions and explanations | Agent mode, reasoning, vision | [{% data variables.copilot.copilot_gpt_5_mini %} model card](https://cdn.openai.com/gpt-5-system-card.pdf) |
39-
| {% data variables.copilot.copilot_gpt_5 %} | Deep reasoning and debugging | Multi-step problem solving and architecture-level code analysis | Reasoning | [{% data variables.copilot.copilot_gpt_5 %} model card](https://cdn.openai.com/gpt-5-system-card.pdf) |
40-
| {% data variables.copilot.copilot_claude_haiku_45 %} | Fast help with simple or repetitive tasks | Fast, reliable answers to lightweight coding questions | Agent mode | Not available |
41-
| {% data variables.copilot.copilot_claude_sonnet_45 %} | General-purpose coding and agent tasks | Complex problem-solving challenges, sophisticated reasoning | Agent mode | [{% data variables.copilot.copilot_claude_sonnet_45 %} model card](https://assets.anthropic.com/m/12f214efcc2f457a/original/Claude-Sonnet-4-5-System-Card.pdf) |
42-
| {% data variables.copilot.copilot_claude_opus_41 %} | Deep reasoning and debugging | Complex problem-solving challenges, sophisticated reasoning | Reasoning, vision | [{% data variables.copilot.copilot_claude_opus_41 %} model card](https://assets.anthropic.com/m/4c024b86c698d3d4/original/Claude-4-1-System-Card.pdf) |
43-
| {% data variables.copilot.copilot_claude_sonnet_40 %} | Deep reasoning and debugging | Performance and practicality, perfectly balanced for coding workflows | Agent mode, vision | [{% data variables.copilot.copilot_claude_sonnet_40 %} model card](https://www-cdn.anthropic.com/6be99a52cb68eb70eb9572b4cafad13df32ed995.pdf) |
44-
| {% data variables.copilot.copilot_gemini_25_pro %} | Deep reasoning and debugging | Complex code generation, debugging, and research workflows | Reasoning, vision | [{% data variables.copilot.copilot_gemini_25_pro %} model card](https://storage.googleapis.com/model-cards/documents/gemini-2.5-pro.pdf) |
45-
| {% data variables.copilot.copilot_gemini_3_flash %} | Fast help with simple or repetitive tasks | Fast, reliable answers to lightweight coding questions | Agent mode | Not available |
46-
| {% data variables.copilot.copilot_grok_code %} | General-purpose coding and writing | Fast, accurate code completions and explanations | Agent mode | [{% data variables.copilot.copilot_grok_code %} model card](https://data.x.ai/2025-08-20-grok-4-model-card.pdf) |
47-
| {% data variables.copilot.copilot_qwen_25 %} | General-purpose coding and writing | Code generation, reasoning, and code repair / debugging | Reasoning | [{% data variables.copilot.copilot_qwen_25 %} model card](https://arxiv.org/pdf/2409.12186) |
48-
| {% data variables.copilot.copilot_raptor_mini %} | General-purpose coding and writing | Fast, accurate code completions and explanations | Agent mode | Coming soon |
32+
33+
| Model | Task area | Excels at (primary use case) | Further reading |
34+
|----------------------------------------------------|-----------------------|------------------------------|-----------------------------|
35+
| {% for model in tables.copilot.model-comparison %} |
36+
| {{ model.name }} | {{ model.task_area }} | {{ model.excels_at }} | {{ model.further_reading }} |
37+
| {% endfor %} |
4938

5039
## Task: General-purpose coding and writing
5140

‎data/tables/copilot/model-comparison.yml‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# Please keep this list sorted in the following order:
2+
# 1. By provider, in this order:
3+
# - OpenAI
4+
# - Anthropic
5+
# - Google
6+
# - xAI
7+
# 2. Within each provider group, alphabetically by model name.
8+
9+
# OpenAI
10+
- name: GPT-4.1
11+
task_area: General-purpose coding and writing
12+
excels_at: Fast, accurate code completions and explanations
13+
further_reading: '[GPT-4.1 model card](https://openai.com/index/gpt-4-1/)'
14+
15+
- name: GPT-5
16+
task_area: Deep reasoning and debugging
17+
excels_at: Multi-step problem solving and architecture-level code analysis
18+
further_reading: '[GPT-5 model card](https://cdn.openai.com/gpt-5-system-card.pdf)'
19+
20+
- name: GPT-5-Codex
21+
task_area: General-purpose coding and writing
22+
excels_at: Fast, accurate code completions and explanations
23+
further_reading: '[GPT-5-Codex model card](https://cdn.openai.com/pdf/97cc5669-7a25-4e63-b15f-5fd5bdc4d149/gpt-5-codex-system-card.pdf)'
24+
25+
- name: GPT-5 mini
26+
task_area: General-purpose coding and writing
27+
excels_at: Fast, accurate code completions and explanations
28+
further_reading: '[GPT-5 mini model card](https://cdn.openai.com/gpt-5-system-card.pdf)'
29+
30+
- name: GPT-5.1
31+
task_area: Deep reasoning and debugging
32+
excels_at: Multi-step problem solving and architecture-level code analysis
33+
further_reading: '[GPT-5.1 model card](https://cdn.openai.com/pdf/4173ec8d-1229-47db-96de-06d87147e07e/5_1_system_card.pdf)'
34+
35+
- name: GPT-5.1-Codex
36+
task_area: Deep reasoning and debugging
37+
excels_at: Multi-step problem solving and architecture-level code analysis
38+
further_reading: 'Not available'
39+
40+
- name: GPT-5.1 Codex Max
41+
task_area: Agentic software development
42+
excels_at: Agentic tasks
43+
further_reading: '[GPT-5.1-Codex-Max model card](https://cdn.openai.com/pdf/2a7d98b1-57e5-4147-8d0e-683894d782ae/5p1_codex_max_card_03.pdf)'
44+
45+
- name: GPT-5.1-Codex-Mini
46+
task_area: Deep reasoning and debugging
47+
excels_at: Multi-step problem solving and architecture-level code analysis
48+
further_reading: 'Not available'
49+
50+
- name: GPT-5.2
51+
task_area: Deep reasoning and debugging
52+
excels_at: Multi-step problem solving and architecture-level code analysis
53+
further_reading: '[GPT-5.2 model card](https://cdn.openai.com/pdf/3a4153c8-c748-4b71-8e31-aecbde944f8d/oai_5_2_system-card.pdf)'
54+
55+
# Anthropic
56+
- name: Claude Haiku 4.5
57+
task_area: Fast help with simple or repetitive tasks
58+
excels_at: Fast, reliable answers to lightweight coding questions
59+
further_reading: '[Claude Haiku 4.5 model card](https://assets.anthropic.com/m/99128ddd009bdcb/Claude-Haiku-4-5-System-Card.pdf)'
60+
61+
- name: Claude Opus 4.1
62+
task_area: Deep reasoning and debugging
63+
excels_at: Complex problem-solving challenges, sophisticated reasoning
64+
further_reading: '[Claude Opus 4.1 model card](https://assets.anthropic.com/m/4c024b86c698d3d4/original/Claude-4-1-System-Card.pdf)'
65+
66+
- name: Claude Opus 4.5
67+
task_area: Deep reasoning and debugging
68+
excels_at: Complex problem-solving challenges, sophisticated reasoning
69+
further_reading: '[Claude Opus 4.5 model card](https://assets.anthropic.com/m/64823ba7485345a7/Claude-Opus-4-5-System-Card.pdf)'
70+
71+
- name: Claude Sonnet 4.0
72+
task_area: Deep reasoning and debugging
73+
excels_at: Performance and practicality, perfectly balanced for coding workflows
74+
further_reading: '[Claude Sonnet 4.0 model card](https://www-cdn.anthropic.com/6be99a52cb68eb70eb9572b4cafad13df32ed995.pdf)'
75+
76+
- name: Claude Sonnet 4.5
77+
task_area: General-purpose coding and agent tasks
78+
excels_at: Complex problem-solving challenges, sophisticated reasoning
79+
further_reading: '[Claude Sonnet 4.5 model card](https://assets.anthropic.com/m/12f214efcc2f457a/original/Claude-Sonnet-4-5-System-Card.pdf)'
80+
81+
# Google
82+
- name: Gemini 2.5 Pro
83+
task_area: Deep reasoning and debugging
84+
excels_at: Complex code generation, debugging, and research workflows
85+
further_reading: '[Gemini 2.5 Pro model card](https://storage.googleapis.com/model-cards/documents/gemini-2.5-pro.pdf)'
86+
87+
- name: Gemini 3 Flash
88+
task_area: Fast help with simple or repetitive tasks
89+
excels_at: Fast, reliable answers to lightweight coding questions
90+
further_reading: '[Gemini 3 Flash model card](https://storage.googleapis.com/deepmind-media/Model-Cards/Gemini-3-Flash-Model-Card.pdf)'
91+
92+
- name: Gemini 3 Pro
93+
task_area: Deep reasoning and debugging
94+
excels_at: Complex code generation, debugging, and research workflows
95+
further_reading: '[Gemini 3 Pro model card](https://storage.googleapis.com/deepmind-media/Model-Cards/Gemini-3-Pro-Model-Card.pdf)'
96+
97+
# xAI
98+
- name: Grok Code Fast 1
99+
task_area: General-purpose coding and writing
100+
excels_at: Fast, accurate code completions and explanations
101+
further_reading: '[Grok Code Fast 1 model card](https://data.x.ai/2025-08-20-grok-4-model-card.pdf)'
102+
103+
# Other providers (alphabetized by model name)
104+
- name: Qwen2.5
105+
task_area: General-purpose coding and writing
106+
excels_at: Code generation, reasoning, and code repair / debugging
107+
further_reading: '[Qwen2.5 model card](https://arxiv.org/pdf/2409.12186)'
108+
109+
- name: Raptor mini
110+
task_area: General-purpose coding and writing
111+
excels_at: Fast, accurate code completions and explanations
112+
further_reading: 'Coming soon'

‎src/data-directory/lib/data-schemas/tables/copilot/model-comparison.ts‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
// This schema enforces the structure in model-comparison.yml
2+
3+
const modelComparisonSchema = {
4+
type: 'object',
5+
additionalProperties: false,
6+
required: ['models'],
7+
properties: {
8+
models: {
9+
type: 'array',
10+
items: {
11+
type: 'object',
12+
additionalProperties: false,
13+
required: ['name', 'task_area', 'excels_at', 'further_reading'],
14+
properties: {
15+
name: {
16+
type: 'string',
17+
lintable: true,
18+
},
19+
task_area: {
20+
type: 'string',
21+
lintable: true,
22+
},
23+
excels_at: {
24+
type: 'string',
25+
lintable: true,
26+
},
27+
further_reading: {
28+
type: 'string',
29+
lintable: true,
30+
},
31+
},
32+
},
33+
},
34+
},
35+
}
36+
37+
export default modelComparisonSchema

0 commit comments

Comments
 (0)