LLM SQL Generation Benchmark Results

We assessed the ability of popular LLMs to generate accurate and efficient SQL from natural language prompts. Using a 200 million record dataset from the GH Archive uploaded to Tinybird, we asked the LLMs to generate SQL based on 50 prompts. The results are shown below and can be compared to a human baseline.

Show metrics relative to human baseline

--

human

human

--

--

--

--

--

332.6 ms

31,006,852

759.83 MB

#1

anthropic

claude-opus-4.7

82.55

99.61

65.48

4.738

1.00

2,800.88 ms

34,437,823

811.31 MB

#2

openai

gpt-chat-latest

82.47

99.90

65.05

1.793

1.00

818.72 ms

39,159,059

941.02 MB

#3

anthropic

82.46

99.86

65.05

7.349

1.14

633.94 ms

38,050,598

949.38 MB

#4

google

gemini-3-pro-preview

80.04

99.78

60.30

49.440

1.02

682.04 ms

46,150,625

1,055.25 MB

#5

qwen

qwen3.5-plus-02-15

79.90

99.63

60.17

67.609

1.00

1,201 ms

46,773,375

984.67 MB

#6

google

gemini-3.5-flash

79.87

99.69

60.05

9.736

1.00

1,484.88 ms

50,048,646

1,060.75 MB

#7

moonshotai

kimi-k2-0905:exacto

79.40

99.75

59.06

4.547

1.02

1,397.327 ms

46,877,411

848.80 MB

#8

google

gemini-2.5-pro-preview-05-06

79.33

99.73

58.94

21.273

1.00

825.714 ms

41,502,252

947.30 MB

#9

google

gemini-3.1-pro-preview-customtools

79.26

99.03

59.48

20.360

1.02

3,833.3 ms

49,317,675

1,056.97 MB

#10

google

gemini-3.6-flash

79.16

98.84

59.48

7.875

1.00

5,507.52 ms

41,838,459

1,045.51 MB

#11

google

78.95

99.91

57.98

19.100

1.06

443.22 ms

42,878,115

826.88 MB

#12

anthropic

claude-opus-4.8

78.85

99.93

57.77

2.780

1.00

585.18 ms

34,526,743

811.92 MB

#13

google

gemini-3-flash-preview

78.76

99.18

58.35

2.406

1.00

6,401.3 ms

42,874,619

927.15 MB

#14

openai

gpt-4o-search-preview

78.59

99.97

57.21

3.757

1.40

525.58 ms

41,771,932

394.36 MB

#15

bytedance-seed

78.37

99.39

57.35

11.170

1.06

2,954.24 ms

46,000,305

964.12 MB

#16

openrouter

78.32

99.85

56.78

1.401

1.02

1,202.46 ms

72,041,510

1,141.62 MB

#17

anthropic

claude-3.7-sonnet:thinking

78.27

99.60

56.93

40.858

1.00

1,425.82 ms

40,046,627

947.37 MB

#18

z-ai

78.22

99.48

56.95

16.908

1.00

2,150.16 ms

45,225,460

1,036.61 MB

#19

anthropic

claude-3.7-sonnet

78.01

99.93

56.08

3.149

1.10

374.224 ms

40,099,998

824.57 MB

#20

z-ai

77.96

99.70

56.21

32.466

1.02

968.06 ms

45,783,003

1,078.88 MB

#21

deepseek

deepseek-v4-pro

77.91

99.91

55.91

20.368

1.02

2,224.64 ms

42,412,877

299.77 MB

#22

qwen

qwen3.6-35b-a3b

77.85

99.73

55.97

20.423

1.12

1,065.78 ms

50,827,717

997.11 MB

#23

moonshotai

77.84

99.92

55.77

4.265

1.06

589.22 ms

49,539,148

903.11 MB

#24

x-ai

77.78

99.75

55.81

61.602

1.00

677.06 ms

49,360,869

1,145.95 MB

#25

openai

gpt-5.6-terra-pro

77.73

99.53

55.92

9.575

1.02

2,624.06 ms

45,682,344

923.23 MB

#26

x-ai

77.53

99.92

55.14

0.896

1.02

794.26 ms

35,500,350

846.01 MB

#27

qwen

qwen3.5-397b-a17b

77.47

99.71

55.23

78.487

1.02

809.92 ms

45,777,471

1,046.79 MB

#28

meta

77.46

99.51

55.42

5.558

1.00

2,944.1 ms

44,070,185

969.39 MB

#29

x-ai

grok-4.20-multi-agent

77.45

99.79

55.12

16.323

1.06

998.54 ms

42,709,671

910.85 MB

#30

x-ai

77.43

99.49

55.37

17.513

1.00

2,168.32 ms

44,026,726

1,009.68 MB

#31

openai

77.38

99.92

54.83

9.886

1.14

448.84 ms

49,432,133

844.29 MB

#32

google

gemini-2.5-pro-preview

77.34

98.43

56.25

20.550

1.00

2,995.771 ms

51,942,301

1,111.37 MB

#33

openai

77.32

99.69

54.95

3.188

1.00

1,898.78 ms

57,024,811

1,137.19 MB

#34

deepseek

deepseek-v3.2-exp

77.29

99.82

54.77

4.786

1.00

1,371.82 ms

42,265,330

838.05 MB

#35

xiaomi

77.20

99.67

54.74

23.199

1.06

1,395.92 ms

43,289,653

927.15 MB

#36

moonshotai

77.16

99.77

54.54

8.252

1.00

1,468.84 ms

46,652,241

877.99 MB

#37

openai

77.11

99.85

54.36

19.254

1.04

625.82 ms

43,602,960

960.85 MB

#38

mistralai

mistral-medium-3-5

77.10

99.71

54.50

5.209

1.02

1,331.286 ms

52,340,224

974.02 MB

#39

anthropic

claude-opus-4.5

77.07

99.88

54.26

2.907

1.02

4,677.22 ms

42,411,762

294.93 MB

#40

moonshotai

77.06

99.46

54.67

68.072

1.00

1,175.633 ms

44,153,829

997.26 MB

#41

qwen

qwen3.6-max-preview

76.97

99.69

54.26

50.610

1.00

1,024.12 ms

40,394,423

964.23 MB

#42

sakana

76.96

99.63

54.30

58.429

1.06

1,059.8 ms

38,309,339

990.27 MB

#43

qwen

76.95

99.50

54.40

35.404

1.00

1,217.265 ms

42,908,205

1,021.26 MB

#44

deepseek

deepseek-r1-0528

76.93

99.71

54.16

61.820

1.02

682.673 ms

42,768,724

937.48 MB

#45

nex-agi

76.89

99.48

54.29

30.772

1.10

1,749.78 ms

54,863,792

1,079.00 MB

#46

openai

76.88

99.67

54.09

10.389

1.00

1,022.388 ms

53,083,092

1,134.93 MB

#47

anthropic

claude-sonnet-4

76.80

99.30

54.30

2.799

1.02

3,187.48 ms

39,847,072

984.97 MB

#48

qwen

76.79

99.70

53.88

51.377

1.00

1,144.52 ms

45,274,360

904.00 MB

#49

openai

gpt-5.1-codex-mini

76.76

99.87

53.65

9.049

1.02

639.54 ms

41,901,897

971.73 MB

#50

qwen

qwen3-max-thinking

76.71

99.38

54.05

2.089

1.00

6,131.14 ms

38,332,239

776.42 MB

#51

deepseek

deepseek-v4-flash

76.64

99.77

53.51

6.753

1.08

1,397.02 ms

46,005,139

899.56 MB

#52

qwen

qwen3-235b-a22b-thinking-2507

76.61

99.58

53.64

93.889

1.04

1,244.28 ms

42,483,080

932.90 MB

#53

kwaipilot

kat-coder-pro-v2

76.57

99.44

53.71

2.012

1.00

5,157.86 ms

51,659,367

909.16 MB

#54

deepcogito

cogito-v2.1-671b

76.54

99.57

53.50

1.165

1.00

4,554.34 ms

36,881,329

806.31 MB

#55

stepfun

76.53

99.04

54.03

35.870

1.06

1,416.312 ms

46,041,589

1,111.22 MB

#56

qwen

76.51

99.61

53.41

5.172

1.17

550.043 ms

40,419,858

895.38 MB

#57

openai

76.50

99.77

53.23

5.045

1.00

1,374.06 ms

47,296,114

1,000.48 MB

#58

tencent

76.42

99.43

53.41

34.712

1.20

1,251.122 ms

45,276,609

982.15 MB

#59

anthropic

claude-sonnet-5

76.40

99.83

52.97

7.003

1.00

3,823.7 ms

41,901,088

384.21 MB

#60

openai

76.38

98.74

54.01

6.711

1.02

6,556.7 ms

45,940,212

1,029.91 MB

#61

qwen

76.36

99.77

52.95

37.553

1.06

761.347 ms

44,676,197

795.72 MB

#62

qwen

76.34

99.54

53.14

23.787

1.00

1,869.66 ms

38,823,710

936.95 MB

#63

openai

gpt-5.6-luna-pro

76.20

98.94

53.47

9.382

1.00

5,260.18 ms

39,558,560

969.10 MB

#64

anthropic

claude-3.5-sonnet

76.19

99.96

52.41

3.234

1.02

388.96 ms

37,145,042

684.44 MB

#65

openai

76.18

99.78

52.59

76.620

1.04

746.8 ms

52,804,037

936.55 MB

#66

z-ai

76.17

99.86

52.49

56.839

1.14

1,280.245 ms

44,533,261

368.54 MB

#67

perceptron

76.14

99.47

52.81

4.372

1.02

3,587.02 ms

66,915,860

1,055.43 MB

#68

qwen

qwen3-235b-a22b-07-25

76.11

99.90

52.31

8.620

1.18

397.755 ms

38,751,330

781.80 MB

#69

anthropic

claude-opus-4.6

76.09

99.93

52.25

3.423

1.02

2,997 ms

37,271,990

261.77 MB

#70

z-ai

76.05

99.62

52.48

12.758

1.02

1,676.94 ms

45,577,866

1,035.83 MB

#71

kwaipilot

kat-coder-air-v2.5

75.94

99.03

52.85

4.666

1.08

6,085.74 ms

43,355,598

915.92 MB

#72

qwen

75.87

99.87

51.86

13.982

1.02

2,741.04 ms

50,261,089

362.24 MB

#73

openai

75.86

99.85

51.87

1.534

1.02

1,386.16 ms

60,323,408

988.77 MB

#74

openai

75.79

99.89

51.69

10.228

1.08

613.66 ms

52,581,751

940.75 MB

#75

deepseek

75.76

99.91

51.62

3.690

1.14

729.48 ms

42,566,756

789.60 MB

#76

anthropic

claude-opus-4.1

75.75

99.85

51.65

6.342

1.04

580.51 ms

39,294,543

936.76 MB

#77

qwen

qwen3.5-plus-20260420

75.75

99.69

51.81

47.981

1.02

1,044.3 ms

46,539,910

999.42 MB

#78

qwen

75.73

99.98

51.48

4.571

1.04

457.224 ms

46,666,126

333.42 MB

#79

qwen

qwen3-coder-plus

75.68

99.97

51.39

2.303

1.02

679.163 ms

53,829,416

387.08 MB

#80

openai

gpt-oss-120b:exacto

75.68

99.76

51.60

5.261

1.08

1,351.14 ms

51,060,147

1,021.13 MB

#81

openrouter

75.67

99.84

51.51

2.754

1.00

1,160.78 ms

71,383,218

1,133.85 MB

#82

poolside

75.66

99.75

51.56

7.356

1.14

1,255.76 ms

46,603,715

962.52 MB

#83

openrouter

75.62

99.83

51.40

1.362

1.02

1,358.24 ms

67,797,316

1,137.85 MB

#84

qwen

qwen3-vl-235b-a22b-thinking

75.59

99.79

51.40

29.091

1.00

782.62 ms

52,951,289

1,073.90 MB

#85

z-ai

75.58

99.40

51.77

18.891

1.00

2,731.18 ms

50,021,624

994.13 MB

#86

qwen

qwen3.5-122b-a10b

75.56

99.48

51.65

13.946

1.06

2,107 ms

45,772,163

1,056.49 MB

#87

openai

gpt-4o-2024-08-06

75.54

99.79

51.29

1.375

1.02

1,946.04 ms

49,630,116

923.17 MB

#88

qwen

qwen3-coder:exacto

75.52

99.78

51.26

1.292

1.02

1,383.265 ms

46,171,856

959.85 MB

#89

moonshotai

75.45

99.63

51.28

24.452

1.00

1,374.34 ms

36,653,795

986.71 MB

#90

openai

75.44

99.86

51.01

11.124

1.00

596.52 ms

48,389,329

1,097.99 MB

#91

aion-labs

75.41

98.89

51.92

24.331

1.00

4,130.38 ms

57,308,234

1,133.15 MB

#92

anthropic

claude-opus-4.6-fast

75.36

99.91

50.82

2.813

1.02

2,053.426 ms

34,047,552

206.91 MB

#93

nvidia

nemotron-3-super-120b-a12b

75.30

99.44

51.17

22.996

1.06

2,175.16 ms

45,879,249

994.61 MB

#94

google

gemini-3.5-flash-lite

75.29

99.78

50.80

1.556

1.02

1,936.7 ms

57,137,045

1,002.74 MB

#95

google

gemini-3.1-flash-lite-preview

75.29

99.19

51.39

1.517

1.00

7,438.06 ms

53,824,558

965.34 MB

#96

aion-labs

75.26

99.70

50.81

59.689

1.06

847.02 ms

45,570,498

1,048.70 MB

#97

openai

gpt-4.5-preview

75.22

99.94

50.51

2.955

1.00

442.98 ms

41,636,677

756.28 MB

#98

google

gemma-4-26b-a4b-it

75.20

99.76

50.63

5.060

1.06

6,086.8 ms

56,772,354

393.05 MB

#99

x-ai

grok-3-mini-beta

75.19

99.87

50.52

7.127

1.06

651.74 ms

55,296,404

869.75 MB

#100

openai

75.18

99.60

50.77

1.503

1.02

3,542.34 ms

42,878,575

910.97 MB

#101

mistralai

mistral-medium-3

75.17

99.85

50.50

2.621

1.06

1,349.88 ms

54,701,026

878.39 MB

#102

openai

75.06

99.86

50.26

21.133

1.04

702.64 ms

68,364,075

1,005.01 MB

#103

openai

75.03

99.99

50.08

2.074

1.00

421.6 ms

52,027,773

246.69 MB

#104

qwen

75.01

99.73

50.28

45.733

1.04

846.5 ms

49,315,858

1,050.31 MB

#105

tencent

75.01

98.46

51.55

15.523

1.04

6,150.5 ms

49,211,334

1,078.85 MB

#106

openai

gpt-4o-2024-11-20

75.01

99.77

50.24

0.876

1.06

1,866.388 ms

55,625,851

880.60 MB

#107

qwen

qwen3-next-80b-a3b-instruct

74.98

99.95

50.00

1.474

1.00

421.14 ms

38,561,447

879.98 MB

#108

x-ai

74.95

99.76

50.15

10.611

1.00

762.122 ms

50,156,062

1,088.51 MB

#109

deepseek

deepseek-v3.1-terminus

74.95

99.79

50.11

3.681

1.04

1,430.3 ms

43,504,553

917.37 MB

#110

openai

74.95

99.89

50.00

2.190

1.06

818.38 ms

54,736,481

995.56 MB

#111

bytedance-seed

74.91

99.65

50.18

26.392

1.04

1,477.74 ms

46,878,338

944.20 MB

#112

openai

74.91

99.98

49.84

16.292

1.04

549.54 ms

53,315,039

303.04 MB

#113

anthropic

claude-4.5-sonnet

74.88

99.92

49.84

3.702

1.02

585.62 ms

43,365,288

907.62 MB

#114

openai

74.86

99.80

49.93

1.158

1.00

1,920.74 ms

61,160,789

1,027.66 MB

#115

qwen

qwen3-next-80b-a3b-thinking

74.85

99.75

49.96

17.344

1.02

720.837 ms

54,897,195

1,106.61 MB

#116

qwen

qwen3-235b-a22b-2507

74.85

98.95

50.75

3.224

1.08

8,653.66 ms

44,233,987

815.52 MB

#117

google

74.83

98.84

50.82

5.276

1.00

7,081.24 ms

56,772,354

1,050.70 MB

#118

xiaomi

74.83

99.49

50.17

11.241

1.00

2,737.28 ms

47,678,693

957.11 MB

#119

openai

gpt-5.6-sol-pro

74.83

99.05

50.61

18.564

1.04

4,009.1 ms

45,921,518

989.20 MB

#120

z-ai

74.81

99.44

50.18

55.855

1.04

1,656.82 ms

50,110,333

1,071.67 MB

#121

qwen

74.80

99.67

49.93

31.219

1.04

1,253.26 ms

47,306,065

990.21 MB

#122

nex-agi

nex-n2-pro:free

74.76

99.61

49.92

57.277

1.02

819.755 ms

40,479,420

1,012.16 MB

#123

openai

gpt-oss-20b:free

74.75

99.68

49.82

14.122

1.27

610.104 ms

46,853,009

965.28 MB

#124

perplexity

sonar-deep-research

74.72

99.63

49.82

46.034

1.36

862.54 ms

53,631,153

1,119.54 MB

#125

nvidia

nemotron-3-super-120b-a12b:free

74.72

99.69

49.75

24.582

1.08

1,274.32 ms

44,080,993

919.44 MB

#126

nousresearch

hermes-3-llama-3.1-405b:free

74.68

99.92

49.45

7.260

1.12

552.64 ms

52,968,329

862.73 MB

#127

qwen

qwen3-235b-a22b

74.63

99.91

49.34

36.262

1.04

439.38 ms

45,468,824

791.67 MB

#128

openai

74.57

99.71

49.42

4.156

1.02

1,685.42 ms

55,308,771

1,106.43 MB

#129

deepseek

deepseek-chat-v3-0324

74.53

99.96

49.10

5.366

1.24

362.62 ms

39,914,537

612.03 MB

#130

anthropic

claude-sonnet-4.6

74.53

99.23

49.83

3.549

1.00

5,336.72 ms

46,682,298

961.64 MB

#131

openai

gpt-oss-120b:free

74.45

99.95

48.95

8.630

1.06

1,214.36 ms

47,908,602

340.50 MB

#132

writer

74.44

99.58

49.30

9.108

1.18

2,368.32 ms

38,942,989

791.79 MB

#133

perplexity

sonar-reasoning-pro

74.44

99.75

49.13

8.937

1.02

1,131.3 ms

54,592,943

1,142.75 MB

#134

google

gemini-3.1-flash-lite

74.42

99.85

49.00

1.281

1.00

1,465.88 ms

53,680,420

964.11 MB

#135

qwen

qwen-plus-2025-07-28:thinking

74.38

99.04

49.72

58.591

1.06

2,758.8 ms

45,622,670

1,037.56 MB

#136

mistralai

mistral-medium-3.1

74.37

99.93

48.81

2.088

1.04

666.02 ms

53,051,447

878.95 MB

#137

x-ai

74.36

99.83

48.89

17.702

1.00

686.06 ms

44,422,695

1,033.66 MB

#138

minimax

74.34

99.51

49.16

25.143

1.18

1,680.46 ms

47,270,248

996.89 MB

#139

deepseek

74.27

99.74

48.79

62.020

1.02

869.52 ms

50,574,856

983.35 MB

#140

openai

74.25

98.55

49.94

3.996

1.04

6,177.531 ms

51,609,499

1,045.21 MB

#141

deepcogito

cogito-v2-preview-llama-70b

74.21

99.90

48.51

2.851

1.10

847.14 ms

42,613,427

792.70 MB

#142

qwen

qwen-2.5-coder-32b-instruct

74.19

99.90

48.49

2.456

1.08

732.878 ms

46,841,414

767.00 MB

#143

z-ai

74.17

99.60

48.73

27.030

1.08

934.408 ms

50,746,043

1,101.56 MB

#144

aion-labs

74.14

99.68

48.60

74.917

1.04

885.66 ms

48,400,008

1,052.70 MB

#145

meta-llama

llama-4-maverick

74.12

99.95

48.30

3.095

1.04

410.78 ms

40,161,866

793.26 MB

#146

google

gemini-2.5-pro-preview-06-05

74.10

99.89

48.32

20.782

1.04

579.36 ms

38,815,820

806.77 MB

#147

deepseek

deepseek-chat-v3.1

74.08

99.95

48.22

3.914

1.04

608.681 ms

45,651,463

322.46 MB

#148

perplexity

74.07

99.89

48.26

2.611

1.06

1,000.28 ms

37,440,965

778.30 MB

#149

openai

74.06

99.60

48.51

10.003

1.02

1,443.469 ms

54,867,793

1,033.70 MB

#150

qwen

qwen-plus-2025-07-28

74.03

99.10

48.96

1.642

1.06

8,332.72 ms

40,598,612

833.41 MB

#151

openai

gpt-4o-2024-05-13

73.98

99.86

48.10

1.366

1.04

1,364.88 ms

49,226,881

913.08 MB

#152

poolside

laguna-s-2.1:free

73.94

99.98

47.91

16.848

1.12

788.96 ms

38,618,194

228.47 MB

#153

qwen

qwen2.5-vl-72b-instruct

73.93

99.79

48.08

7.308

1.06

4,844.7 ms

46,407,274

370.56 MB

#154

x-ai

73.93

98.89

48.97

7.519

1.00

6,369.62 ms

57,388,979

1,035.02 MB

#155

openai

gpt-3.5-turbo-0613

73.92

99.75

48.08

1.101

1.04

1,539.143 ms

47,348,539

989.91 MB

#156

inclusionai

73.90

99.21

48.59

8.477

1.00

3,858 ms

51,586,642

1,089.58 MB

#157

google

gemini-2.0-flash-lite-001

73.88

99.80

47.95

1.286

1.02

1,285.714 ms

50,589,629

969.49 MB

#158

mistralai

mistral-small-3.2-24b-instruct

73.85

98.88

48.82

2.728

1.09

1,068.091 ms

41,646,956

860.77 MB

#159

poolside

73.84

99.98

47.69

5.713

1.10

719.58 ms

42,657,871

296.97 MB

#160

openai

73.76

99.01

48.52

1.926

1.06

4,263.938 ms

53,931,690

930.29 MB

#161

arcee-ai

trinity-large-thinking

73.72

99.53

47.90

8.240

1.20

2,119.44 ms

48,245,941

996.87 MB

#162

anthropic

claude-sonnet-4.5

73.67

99.17

48.18

3.491

1.00

6,180.14 ms

43,114,181

899.14 MB

#163

nousresearch

73.65

98.26

49.04

2.094

1.02

9,009.143 ms

47,704,275

1,008.86 MB

#164

cohere

73.65

99.01

48.29

1.612

1.04

6,164.959 ms

44,226,012

899.85 MB

#165

mistralai

73.59

99.96

47.22

0.855

1.00

775.14 ms

42,657,411

620.15 MB

#166

qwen

qwen3-30b-a3b-thinking-2507

73.59

99.88

47.30

19.002

1.02

602.48 ms

45,928,106

890.50 MB

#167

z-ai

73.54

99.17

47.92

10.920

1.14

4,103.58 ms

44,778,371

908.40 MB

#168

z-ai

73.53

99.73

47.33

13.738

1.10

5,415.82 ms

52,774,370

366.60 MB

#169

openai

73.51

99.66

47.36

9.993

1.02

1,506.08 ms

53,506,834

1,119.95 MB

#170

openrouter

73.50

98.15

48.85

82.452

1.22

673.4 ms

37,498,477

1,044.59 MB

#171

mistralai

73.48

99.40

47.56

2.237

1.04

6,024.58 ms

44,102,484

778.48 MB

#172

deepseek

73.47

98.77

48.17

4.662

1.04

6,194.184 ms

46,010,640

876.42 MB

#173

openai

gpt-5.1-codex-max

73.37

99.73

47.00

4.821

1.04

1,703 ms

59,148,959

1,016.79 MB

#174

nvidia

nemotron-3-ultra-550b-a55b:free

73.33

99.31

47.35

75.122

1.10

1,321.306 ms

42,687,966

986.25 MB

#175

openrouter

73.27

99.56

46.97

17.596

1.04

1,977.66 ms

47,054,403

968.46 MB

#176

openai

73.25

99.70

46.80

1.300

1.02

1,981.02 ms

36,763,125

1,070.25 MB

#177

qwen

qwen3-coder-flash

73.25

99.93

46.56

2.453

1.02

556.74 ms

42,185,121

868.81 MB

#178

openai

73.22

99.75

46.68

3.205

1.09

533.957 ms

41,234,766

980.50 MB

#179

x-ai

grok-code-fast-1

73.15

99.84

46.46

6.570

1.02

830.98 ms

51,488,460

1,077.86 MB

#180

z-ai

73.13

99.59

46.68

10.752

1.04

838.771 ms

48,329,302

1,130.98 MB

#181

baidu

ernie-4.5-vl-424b-a47b

73.08

99.02

47.13

4.773

1.22

5,744.94 ms

40,362,238

856.35 MB

#182

inception

73.06

99.24

46.88

0.966

1.02

6,492.7 ms

49,413,821

1,050.30 MB

#183

openai

73.04

99.02

47.07

9.276

1.04

4,358.28 ms

40,287,810

1,027.15 MB

#184

google

gemini-2.5-flash

73.04

99.99

46.09

2.126

1.02

337.4 ms

36,295,667

262.45 MB

#185

perplexity

73.04

99.89

46.19

2.310

1.06

902.08 ms

37,171,993

846.08 MB

#186

moonshotai

73.01

99.50

46.51

30.577

1.00

2,093.58 ms

37,188,519

875.48 MB

#187

anthropic

claude-3.5-haiku

72.96

99.99

45.93

2.731

1.08

522.38 ms

47,370,988

297.58 MB

#188

poolside

laguna-xs.2:free

72.95

98.74

47.15

8.874

1.10

4,107.306 ms

45,822,565

1,011.69 MB

#189

minimax

72.90

98.96

46.83

6.706

1.14

5,725.68 ms

51,112,554

952.89 MB

#190

qwen

qwen3.5-35b-a3b

72.84

99.69

45.99

8.095

1.20

1,400.5 ms

49,184,648

1,009.49 MB

#191

thedrummer

cydonia-24b-v4.1

72.80

99.93

45.68

1.966

1.10

412.306 ms

36,265,794

823.24 MB

#192

google

72.77

99.85

45.70

5.021

1.06

694.878 ms

41,212,019

869.17 MB

#193

deepseek

deepseek-chat-v3-0324:free

72.77

99.52

46.01

5.875

1.11

383.682 ms

38,010,973

813.72 MB

#194

qwen

72.75

99.47

46.04

1.800

1.02

4,211.94 ms

41,619,269

938.67 MB

#195

qwen

72.71

99.15

46.26

30.579

1.12

1,997.531 ms

53,919,075

1,064.75 MB

#196

bytedance-seed

72.59

99.84

45.34

8.785

1.06

2,173 ms

44,485,951

312.69 MB

#197

minimax

72.58

99.71

45.45

8.115

1.04

3,911.204 ms

46,994,209

415.62 MB

#198

qwen

qwen3-vl-235b-a22b-instruct

72.56

98.88

46.24

2.308

1.06

6,441.857 ms

35,227,076

816.64 MB

#199

poolside

laguna-xs-2.1:free

72.50

99.75

45.25

20.571

1.28

730.26 ms

38,514,356

1,011.07 MB

#200

meta-llama

72.48

100.00

44.96

2.048

1.04

289.875 ms

39,101,618

134.66 MB

#201

mistralai

mistral-large-2512

72.45

99.24

45.66

2.166

1.06

6,698.14 ms

44,059,393

840.20 MB

#202

openai

72.44

99.78

45.10

1.945

1.00

919.562 ms

44,180,776

913.74 MB

#203

bytedance-seed

72.43

98.70

46.16

22.165

1.10

2,919.347 ms

54,494,944

1,182.43 MB

#204

qwen

72.42

99.60

45.23

31.462

1.04

1,591.3 ms

53,444,620

993.79 MB

#205

openai

72.41

99.99

44.83

2.145

1.04

690.28 ms

54,131,214

193.58 MB

#206

meituan

72.38

99.62

45.15

26.224

1.04

964.837 ms

46,725,377

1,047.87 MB

#207

qwen

72.37

99.47

45.26

18.700

1.06

2,507.64 ms

44,319,995

887.56 MB

#208

google

gemini-2.5-flash-image

72.36

99.99

44.74

1.301

1.00

408.34 ms

39,751,249

369.30 MB

#209

x-ai

72.32

99.89

44.76

1.701

1.04

633.612 ms

42,572,577

720.40 MB

#210

z-ai

72.27

99.86

44.67

39.634

1.10

1,245.061 ms

47,340,745

410.78 MB

#211

openai

72.23

99.82

44.64

25.613

1.04

643.3 ms

61,356,069

1,161.59 MB

#212

mistralai

mixtral-8x22b-instruct

72.10

99.67

44.53

0.875

1.13

1,680 ms

37,772,037

814.39 MB

#213

openai

72.01

99.79

44.24

10.635

1.00

976.7 ms

46,652,690

1,054.90 MB

#214

z-ai

71.92

99.24

44.60

16.719

1.02

3,428.94 ms

47,166,161

978.76 MB

#215

nex-agi

71.87

98.93

44.81

18.496

1.06

2,941.5 ms

45,188,794

1,294.56 MB

#216

qwen

qwen3.5-flash-02-23

71.81

99.48

44.14

14.099

1.14

2,444.84 ms

46,373,865

913.66 MB

#217

anthropic

claude-haiku-4.5

71.75

99.93

43.57

1.671

1.12

553.44 ms

49,735,847

918.65 MB

#218

google

gemini-2.5-flash-lite-preview-09-2025

71.71

99.84

43.57

1.611

1.08

1,410.02 ms

43,979,391

864.07 MB

#219

openai

gpt-4o-mini-2024-07-18

71.67

99.94

43.39

2.137

1.04

2,280.84 ms

54,031,752

342.74 MB

#220

qwen

qwen3-coder-next

71.55

98.92

44.18

1.423

1.08

6,878.225 ms

39,065,170

842.06 MB

#221

mistralai

mistral-large-2407

71.51

99.40

43.63

2.117

1.04

6,525.08 ms

46,866,961

759.81 MB

#222

qwen

qwen3-coder-30b-a3b-instruct

71.46

99.30

43.62

3.073

1.06

4,973 ms

34,608,542

848.96 MB

#223

thedrummer

71.42

98.76

44.07

2.614

1.04

4,995.833 ms

39,136,379

842.22 MB

#224

kwaipilot

kat-coder-pro-v2.5

71.35

98.71

43.99

3.518

1.06

5,266.775 ms

50,403,744

1,075.42 MB

#225

relace

71.30

99.64

42.95

1.236

1.08

3,351.32 ms

34,859,649

819.65 MB

#226

openai

gpt-4o-mini-search-preview

71.27

99.97

42.58

2.766

1.06

748.437 ms

31,706,247

214.85 MB

#227

nvidia

nemotron-3-nano-30b-a3b:free

71.27

99.22

43.32

10.142

1.12

4,084.56 ms

48,768,188

919.86 MB

#228

sao10k

l3.3-euryale-70b

71.26

96.36

46.17

29.030

1.00

5,946.468 ms

43,271,705

869.27 MB

#229

mistralai

mistral-small-3.1-24b-instruct

71.26

98.50

44.02

1.809

1.00

376.5 ms

37,893,118

912.60 MB

#230

tencent

71.25

94.21

48.30

25.097

1.50

4,016.065 ms

43,534,014

956.02 MB

#231

nvidia

nemotron-nano-9b-v2

71.22

99.85

42.59

12.717

1.33

483.347 ms

40,823,966

813.29 MB

#232

mistralai

mistral-small-24b-instruct-2501

71.15

97.17

45.14

1.856

1.02

2,076.14 ms

34,303,920

846.49 MB

#233

anthropic

71.11

99.42

42.80

3.657

1.22

761.778 ms

36,590,481

232.57 MB

#234

openai

71.06

99.67

42.45

1.792

1.14

1,843.9 ms

45,378,797

1,138.96 MB

#235

openai

71.03

99.37

42.69

2.211

1.04

5,273.02 ms

57,313,535

959.36 MB

#236

mistralai

voxtral-small-24b-2507

71.02

99.92

42.12

0.902

1.02

610.163 ms

41,840,418

837.25 MB

#237

google

gemini-2.0-flash-001

71.02

99.99

42.04

1.426

1.02

350.146 ms

44,547,543

181.54 MB

#238

mistralai

devstral-medium

71.01

99.95

42.08

1.405

1.08

420.714 ms

44,380,748

715.11 MB

#239

nousresearch

70.98

99.85

42.10

1.140

1.12

7,162.714 ms

36,607,875

216.41 MB

#240

openai

70.89

99.66

42.12

27.104

1.08

1,015.46 ms

45,844,074

1,126.98 MB

#241

minimax

70.86

99.64

42.08

8.530

1.08

1,863.64 ms

43,702,513

933.60 MB

#242

qwen

qwen3-30b-a3b-instruct-2507

70.70

99.96

41.44

2.806

1.09

308 ms

31,184,916

374.88 MB

#243

inclusionai

70.62

99.62

41.62

2.387

1.02

3,063.36 ms

42,522,173

892.15 MB

#244

anthropic

70.50

98.76

42.24

1.544

1.31

4,766.958 ms

34,045,851

774.89 MB

#245

mistralai

magistral-small-2506

70.43

99.89

40.96

12.425

1.18

522.531 ms

39,072,130

681.69 MB

#246

qwen

qwen3-vl-8b-instruct

70.40

99.74

41.05

1.229

1.06

938.298 ms

46,924,142

852.80 MB

#247

mistralai

mistral-small-2603

70.06

99.59

40.54

0.913

1.07

841.891 ms

34,809,725

941.72 MB

#248

google

gemini-2.5-flash-preview

69.93

99.95

39.90

1.622

1.00

384.551 ms

42,309,547

735.32 MB

#249

poolside

69.80

99.03

40.56

17.002

1.33

745.043 ms

41,118,378

1,012.30 MB

#250

poolside

laguna-m.1:free

69.78

99.45

40.10

57.933

1.02

612.319 ms

37,472,080

934.91 MB

#251

inclusionai

69.77

99.82

39.72

1.591

1.19

3,602.521 ms

57,601,522

327.10 MB

#252

mistralai

69.71

99.85

39.58

2.412

1.09

372.696 ms

44,597,846

757.05 MB

#253

nvidia

nemotron-nano-9b-v2:free

69.69

99.47

39.91

27.559

1.16

2,047.6 ms

41,923,462

880.96 MB

#254

z-ai

69.66

99.73

39.58

15.652

1.17

1,055.304 ms

51,805,270

403.62 MB

#255

openai

gpt-3.5-turbo-16k

69.63

99.97

39.30

1.272

1.33

673.958 ms

37,688,322

273.55 MB

#256

minimax

69.50

98.87

40.12

8.381

1.10

4,454.68 ms

44,511,023

1,127.07 MB

#257

mistralai

ministral-14b-2512

69.44

99.94

38.95

1.251

1.04

1,188.128 ms

45,278,108

285.40 MB

#258

mistralai

69.30

99.67

38.93

0.942

1.10

2,089.51 ms

34,677,350

856.21 MB

#259

nvidia

nemotron-3-nano-30b-a3b

69.12

97.78

40.47

9.416

1.21

3,259.936 ms

53,660,951

1,062.85 MB

#260

microsoft

68.96

97.43

40.50

1.074

1.16

4,285.511 ms

35,244,379

793.67 MB

#261

ai21

jamba-large-1.7

68.85

99.73

37.98

2.841

1.18

6,607.449 ms

40,653,494

295.37 MB

#262

mistralai

ministral-8b-2512

68.84

99.64

38.05

1.397

1.21

1,240.521 ms

47,439,900

1,011.42 MB

#263

meta-llama

llama-3.1-70b-instruct

68.77

99.88

37.66

1.786

1.14

772.49 ms

52,201,709

857.02 MB

#264

openrouter

68.71

94.82

42.60

42.194

1.35

3,291.913 ms

43,647,901

998.75 MB

#265

amazon

68.54

98.74

38.33

1.147

1.02

4,954.766 ms

40,098,946

795.77 MB

#266

minimax

68.52

98.91

38.13

11.278

1.59

4,790.717 ms

49,453,309

320.59 MB

#267

openai

68.45

99.69

37.20

20.092

1.02

906.12 ms

61,641,565

1,386.39 MB

#268

google

gemini-2.5-flash-lite

68.40

99.98

36.83

1.512

1.04

850.96 ms

39,578,736

261.53 MB

#269

deepseek

deepseek-v3.1-terminus:exacto

68.39

94.87

41.91

3.206

1.56

950.439 ms

53,138,787

1,064.91 MB

#270

qwen

68.06

99.36

36.76

13.037

1.12

2,157.531 ms

46,780,909

924.89 MB

#271

cognitivecomputations

dolphin-mistral-24b-venice-edition

67.91

95.34

40.47

2.454

1.26

7,461.022 ms

31,506,499

812.88 MB

#272

ibm-granite

67.77

95.77

39.77

0.985

1.12

3,330.326 ms

54,786,273

990.76 MB

#273

amazon

67.73

99.22

36.24

1.141

1.04

5,754.449 ms

46,477,940

865.11 MB

#274

upstage

67.56

97.12

38.00

6.896

1.20

1,715.727 ms

41,230,113

939.81 MB

#275

mistralai

67.41

99.97

34.86

0.925

1.00

385.911 ms

40,043,041

257.63 MB

#276

meta-llama

llama-3.3-70b-instruct

66.36

97.15

35.56

3.501

1.21

445.242 ms

38,658,489

992.39 MB

#277

openrouter

sherlock-dash-alpha

66.22

94.19

38.24

1.237

1.22

932.811 ms

56,016,230

1,309.17 MB

#278

openai

65.61

92.94

38.29

241.404

1.11

1,570.629 ms

43,522,261

426.47 MB

#279

meta-llama

llama-3-70b-instruct

65.56

90.63

40.49

9.571

1.00

3,343.071 ms

38,164,715

870.23 MB

#280

openrouter

64.98

88.70

41.26

25.916

1.06

1,119.273 ms

34,595,145

1,015.30 MB

#281

tencent

hunyuan-a13b-instruct

64.97

97.38

32.57

3.219

1.17

6,919.255 ms

65,822,716

941.41 MB

#282

mistralai

64.97

99.94

30.00

3.307

1.09

680.644 ms

48,641,279

222.69 MB

#283

openai

63.96

99.99

27.93

1.538

1.06

445.694 ms

52,428,071

239.26 MB

#284

alibaba

tongyi-deepresearch-30b-a3b

63.53

94.77

32.29

20.128

1.13

553.026 ms

57,570,886

1,153.90 MB

#285

perplexity

sonar-pro-search

61.63

88.30

34.95

3.717

2.45

683.182 ms

56,964,923

1,347.49 MB

#286

moonshotai

kimi-linear-48b-a3b-instruct

61.54

97.81

25.27

1.305

1.57

348.686 ms

18,091,302

755.15 MB

#287

baidu

ernie-4.5-21b-a3b-thinking

61.53

92.38

30.68

54.468

1.15

847.854 ms

48,091,065

1,108.87 MB

#288

openai

61.39

99.39

23.39

1.150

1.80

1,126.878 ms

36,206,482

218.84 MB

#289

bytedance

61.01

97.62

24.39

0.928

1.07

7,555.711 ms

44,234,522

601.89 MB

#290

google

61.00

99.24

22.76

2.972

1.26

2,179.022 ms

31,151,872

532.00 MB

#291

deepseek

deepseek-r1-distill-llama-70b

60.79

94.05

27.53

23.636

1.36

491.97 ms

51,371,378

1,121.80 MB

#292

mistralai

ministral-3b-2512

60.13

95.59

24.67

0.917

1.08

898.5 ms

34,734,695

1,081.71 MB

#293

arcee-ai

59.50

98.40

20.60

5.629

1.75

1,228.208 ms

54,051,699

278.08 MB

#294

cohere

command-r7b-12-2024

58.66

98.51

18.82

1.907

1.13

3,657.15 ms

63,295,865

217.31 MB

#295

aion-labs

aion-rp-llama-3.1-8b

57.86

99.71

16.00

3.787

1.22

320.135 ms

19,347,326

209.48 MB

#296

moonshotai

57.17

94.82

19.51

39.871

1.71

1,145.429 ms

39,286,658

419.18 MB

#297

openrouter

andromeda-alpha

57.15

99.30

15.00

28.858

1.82

358.176 ms

37,959,195

240.96 MB

#298

google

gemma-3n-e4b-it

56.55

93.85

19.24

2.848

1.21

1,773.879 ms

24,522,122

586.93 MB

#299

meta-llama

llama-3-8b-instruct

56.39

99.92

12.86

1.256

2.35

452.45 ms

16,735,360

53.18 MB

#300

meta-llama

llama-3.2-3b-instruct

54.91

99.20

10.61

0.564

1.27

1,001.962 ms

69,010,677

394.73 MB

#301

google

53.34

99.61

7.08

4.106

1.67

954.867 ms

12,995,860

87.26 MB

#302

liquid

52.93

99.85

6.00

1.393

1.07

428.357 ms

77,197,005

224.10 MB

#303

nvidia

nemotron-nano-12b-v2-vl:free

52.81

87.59

18.03

23.357

1.86

487.333 ms

57,185,740

1,558.26 MB

#304

qwen

qwen2.5-coder-7b-instruct

51.07

99.83

2.30

5.664

1.00

264 ms

29,050,068

205.58 MB

#305

liquid

50.97

99.93

2.00

1.389

1.08

294.462 ms

37,560,298

133.52 MB

#306

inclusionai

50.88

99.76

2.00

26.389

1.00

293 ms

7,321,600

116.24 MB

#307

meta-llama

llama-3.2-1b-instruct

50.10

99.96

0.25

1.557

1.08

274.769 ms

86,662,200

136.94 MB

#309

openai

49.91

99.81

0.00

4.652

1.08

1,130.64 ms

40,869,345

918.65 MB

#310

poolside

49.77

99.53

0.00

11.675

1.04

2,422.32 ms

42,920,255

910.97 MB

#311

thinkingmachines

49.75

99.49

0.00

9.172

1.04

2,728.54 ms

45,832,574

945.40 MB

#312

nvidia

nemotron-3-ultra-550b-a55b

49.74

99.48

0.00

6.639

1.02

3,119.28 ms

38,819,411

890.54 MB

#313

openai

49.71

99.41

0.00

3.255

1.02

4,224.68 ms

49,033,170

956.22 MB

#314

liquid

lfm-2.5-1.2b-instruct:free

49.43

98.29

0.57

0.865

1.48

3,961.652 ms

153,028,682

288.32 MB

#315

meta-llama

llama-3.2-11b-vision-instruct

48.11

89.45

6.78

1.693

1.32

1,332.88 ms

69,602,499

1,484.41 MB

#316

meta-llama

llama-3.1-8b-instruct

34.45

60.04

8.86

3.030

1.15

3,823.8 ms

50,342,319

1,595.67 MB

#317

nvidia

nemotron-3-nano-omni-30b-a3b-reasoning:free

30.39

51.27

9.50

8.782

2.33

5,478.167 ms

105,344,275

957.80 MB

#318

google

3.71

0.00

7.42

1.609

1.83

3,631.083 ms

56,145,491

2,570.86 MB