← All transcripts

China just dropped a new frontier model... and nobody can explain how Transcript, AI Summary & Key Points

Fireship · 16 days ago · Science & Technology · 06:26 · EN

Watch on YouTube

Answer

Kimi K3.1 from Moonshot AI reaches frontier-level performance — near-Opus Terminal Bench scores, independently verified — which seems impossible if Anthropic's distillation-blocking claims are true; the mystery isn't fully explained, though distillation attacks were the suspected path.

AI Summary

Moonshot AI released Kimi K3.1, a free 3-trillion-parameter open-weight model downloadable on November 4th, and it lands within one point of Opus 5.5 on Terminal Bench 4.0 — Artificial Analysis independently scored it 67%, the highest of any model it tested. That performance is puzzling because Anthropic claims 90 to 99% of distillation attacks from China are now blocked, and big AI labs have long attributed Chinese models' rapid gains to stealing their outputs. Moonshot deliberately stripped cybersecurity data from training: an internal unfiltered checkpoint solved 90.5% of its illegal exploit tests while the release model solves 81%, likely to avoid a US ban like the one just imposed on DeepSeek for government use. On the informal 3D-horse-with-Three.js test, Kimi scored 4.20 — the most visually beautiful horse of the group, but its animations warp the mesh instead of using skeletal rigging like Claude (95,000 vertices, 35 independently animated bones) or Gemini Argon, placing it behind Opus (6.39) and GPT6 Astra (5.21).

Key Points

  • An anonymous frontier-level model called Kibitz appeared on Chapot Arena last week, widely identified as Kimi before Moonshot officially dropped Kimi K3.1 last Friday.
  • Kimi K3.1 is a 3-trillion-parameter open-weight model, fully downloadable on November 4th.
  • Kimi K3.1 scores within one point of Opus 5.5 on Terminal Bench 4.0; Artificial Analysis independently ran it and got 67%, the highest score of any model they tested.
  • Harness choice causes huge differences in benchmark results across models.
  • Kimi also wins on SWE marathon, a test of multi-hour software tasks that would take humans weeks without AI.
  • Moonshot deliberately removed cybersecurity data from training: an internal unfiltered checkpoint solved 90.5% of illegal exploit tests, the release model only 81%.
  • The US just banned China's DeepSeek model from all federal civilian agency government use.
  • Despite the censorship, Kimi ranked on top of all publicly available models on Artificial Analysis's new cyber index — mostly because Claude and GPT6 declined every prompt on safety grounds.

AI in practice

Used for

Tools & resources

8 items

ANo. 0939
AIAINotes.us AI product

Artificial Analysis

artificialanalysis.ai/

An independent platform for evaluating and comparing generative AI models. It publishes third-party assessments and comparative data on model capabilities and performance.

Mentioned in
15 videos
Kind
AI
BNo. 5361
AIAINotes.us Tool

Bytes

bytes.dev

Bytes is a JavaScript newsletter from Fireship, delivered twice a week for free. It covers JavaScript and related web-development topics in an entertaining and informative format.

Mentioned in
3 videos
Kind
Other
CNo. 0016
AIAINotes.us AI product

ChatGPT

chatgpt.com

ChatGPT is a general-purpose AI assistant that can generate movie ideas and expand them into story and storyboard prompts. It can also produce software projects from natural-language prompts, such as a Python dating application with user profiles and swiping logic, and help explain electronics and hardware-development concepts. It is also cited as a general-purpose AI capability that consumer products can package into more personal, character-based experiences.

Mentioned in
120 videos
Kind
AI
FNo. 5362
AIAINotes.us Tool

Fireship

fireship.dev

Fireship is an online course library covering the JavaScript ecosystem. Its courses use explanatory videos, historical context, examples, storytelling, and interactive presentation to teach how JavaScript technologies work. The site offers access to the full course library through a yearly Fireship Pro subscription.

Mentioned in
3 videos
Kind
Other
GNo. 0098
AIAINotes.us AI product

Gemini

gemini.google.com

Gemini is Google's AI model and assistant accessible through gemini.google.com. It processes mixed media including video, audio, images, and text, and can convert recordings into SOPs, summarize meetings, extract decisions, and draft follow-up emails. In research and development contexts, it serves as an alternative LLM backend for tasks such as adversarial audits of requirements and resume evaluation.

Mentioned in
44 videos
Kind
AI
LNo. 6870
AIAINotes.us AI product

LMArena

In the AINotes directory

LMArena is an anonymous model-testing arena where users compare AI models and where the video says a previously unidentified model first appeared at frontier-level performance.

Mentioned in
1 video
Kind
AI
MNo. 6871
AIAINotes.us Tool

Mobbin Pro

mobbin.com/signup?referrer_creator_id=ed

Mobbin Pro is a UI and UX research library with more than 600,000 searchable screenshots from mobile and web applications. Its MCP server can provide these real-world interface references to coding agents such as Claude Code, Cursor, and Codex while they are designing interfaces.

Mentioned in
1 video
Kind
Other
TNo. 2097
AIAINotes.us Tool

Three.js

Open source · mrdoob/three.js

Three.js is an open-source JavaScript 3D library developed in the mrdoob/three.js project for assembling and rendering procedural scenes in browser applications, including WebGPU recreations such as playable versions of Halo, Fable 51 Worlds, Human Atlas, browser FPS worlds, CAD workspaces, and physics games. It creates scenes containing cameras, geometry, materials, meshes, and animations, with WebGL and WebGPU renderers and SVG and CSS3D renderers available as addons. Its official site and repository provide documentation, examples, tools, an editor, community resources, and a JavaScript module distribution.

Mentioned in
9 videos
Kind
Other

Links mentioned

🔒 Full analysis locked

Unlock more videos and the full analysis

A credit unlocks one video's full analysis for good — the build steps, the tools and how each was used, the methods behind every use case. Pro opens the whole library instead, and raises how many videos you can analyse a day.

Unlock full analysis — free

Transcript

Searchable transcript of China just dropped a new frontier model... and nobody can explain how — Fireship (06:26). Search for a phrase, then click its timestamp to jump straight to that moment in the video.

Captions sourced from the original video on YouTube, published by Fireship. The video, its captions and all related intellectual property remain the property of their respective owners; AINotes claims no ownership. Provided for research, accessibility and search — see the Transcript Notice and Copyright Policy.

00:00 Last week, an anonymous mystery model named Kibitz showed up randomly on Chapot Arena performing at the frontier level. Kibitz is the Yiddish word for that guy standing behind you at a chess game, it giving you tips on how to blunder your queen, which to be honest is the perfect analogy for a coding agent. But it was pretty obvious who was behind this mystery model from the beginning because the first two letters of this word are a dead giveaway.

00:21 Everybody knew it was Kimmy. Then right on Q last Friday, Moonshot dropped Kimmy K3.1, officially giving us a new three trillion parameter openweight model that will be fully downloadable on November 4th. Or so they say. And just when you thought the AI bubble was popping and the Trust Me Bro benchmarks could not possibly get any better, Kimmy made them better.

00:40 >> I can go lower. >> In today's video, we'll find out why Kimmy K3.1's performance should be impossible and put it to the test on the only benchmark that matters, the building 3D horses with 3JS. It is October 26, 2026, and you're watching the Code Report. Earlier this year, Anthropic accused Moonshot AI, the company behind Kimmy, alongside a bunch of other Chinese companies like Deep Seek and Miniax of running large-scale distillation attacks against Claude by sending swarms of fake users to steal Claude's output and

01:07 reasoning tokens, which could then be used to train these open models that they give to us for free. And now, I believe distillation is a fundamental human right because big AI companies have literally distilled a collective human brain. but by taking our internet web blogs, our GitHubs, our scientific papers, and they even destructively scanned our books without asking.

01:25 Just Google Project Panama. But OpenAI and Anthropic don't really see distillation as a human right like I do. To them, it's grand theft autocomplete. And they've invested millions and millions of dollars into stopping it. And just a few weeks ago, Anthropic said it pretty much solved the distillation problem with 90 to 99% of distillation attacks from China now being blocked.

01:44 And over at OpenAI, they've proved that their tech can even stop small-scale distilleries like PewDiePie. But if that's true, that means an open Chinese model with Kimmy's performance should not be possible. Because for years, Big AI has insinuated that the reason Chinese models get so good so fast is that they steal Big AI's outputs. But if we can trust this Trust Me Bro distillation data from Anthropic, that would mean that these Trust Me Bro benchmarks from Kimmy should not be possible.

02:09 like Kimmy K 3.1 is within one point of opus 5.5 on terminal bench 4.0. That's pretty crazy. And no other open model is remotely close. And we might even be able to actually trust these numbers because artificial analysis ran it independently and got 67% which is the highest score of any model they tested. But one thing to keep in mind is that the harness plays a big role here.

02:31 Running different models with different harnesses will produce huge differences in benchmark results. In addition, Kimmy also wins on S. marathon, which is a test for multi-our software requirements, or in other words, jobs that would take humans weeks of work to finish without AI assistance. That's all pretty impressive, but there is one big caveat when it comes to coding and specifically cyber security.

02:50 Moonshot deliberately ripped out a bunch of cyber security data from its training data set, where the goal was to make the model even worse at hacking than pirate software. >> I'm a genius, dude. And it worked because internally at Moonshot, an unfiltered checkpoint solved 90.5% of their illegal exploit tests. But the final release model only solves 81%.

03:09 And that's a smart business move on their part because just a few days ago, the US banned China's Deepseek model from government use, including all federal civilian agencies. And they would just love a good reason to ban all Chinese models across the board to hand Open AAI and Anthropic a huge win and make all the insiders even more unbelievably rich than they already are.

03:29 And that's why Moonshot really has no choice but to lobomtomize its hacking capabilities. But despite that, Kimmy still performed on top of all publicly available models on the new cyber index at artificial analysis. Although that's mostly because Claude and GBT6 declined every prompt on safety grounds and just took the L. But nowadays, the only AI capability that I really care about is how well it can generate a 3D horse with 3JS.

03:50 And to analyze the skill objectively and scientifically, I use a formal benchmark. It's a state-of-the-art eval I've developed after thousands of hours of research, which I call thoroughbench. The way it works is I literally just go into the chat window and type create a realistic interactive 3D horse with 3JS and then press enter. From there, I rank them from 0 to 10 based on a proprietary loss function I keep in my brain.

04:12 Using this technique, Google Argon achieved a 0.69. GPT6 Astra achieved a 5.21 and Opus achieved a 6.39. But now it's time to try Kimmy K3.1. I tried to be cheap and use the free tier, but got denied. So now I'm paying 250 yuan per month to show you guys this demo. It was worth it though because Kimmy produced what is easily the most beautiful horse in this sample.

04:36 Its coloring is spot-on. It looks realistic. And most importantly, its copulatory organ is a reasonable size. I was blown away by its beauty at first glance, but then I pushed play on the animation and found a problem. The animations are absolutely terrible. If we look closely, you can see it achieves the animation by simply warping the original mesh, which is a very unsophisticated and half-assed way to animate a 3D object.

04:58 If we compare that to Claude, the horse itself may not look as beautiful, but it actually built a mesh with 95,000 vertices with each shape tied to one of 35 bones, and all those bones are animated independently. Even Gemini Argon used bones, but for some reason, it added an extra one. The skeletal animation is more difficult, but it's why the horse doesn't look like it's being torn apart by a black hole when it walks.

05:20 So, the final score for Kimmy K 3.1 on Thoroughbench is a 4.20, which is pretty dang impressive, but still behind the other Frontier models. But if you really want to build an amazing 3D horse or user interface, you need to check out Mobin, the sponsor of today's video. I've been using Mobin for over 5 years and its MCP server now lets your coding agent study its library of 600,000 real UI screens before it designs anything.

05:45 So instead of asking for a login page and getting the same purple gradient vibe slop, you start by telling Claude Code, Cursor, or Codeex to study how the best apps built that complete user flow. It'll use Mobin to gather references and build a visual report of the patterns they all share. Then it'll recreate that flow and adapt it to your app. Every result points back to the original screen.

06:06 This so you can check the source and after you connect Mobin, everything your agent designs will feel like it has actual taste. But try Mobin for free today and get a 20% discount at the link below. This has been the code report. Thanks for watching and I will see you in the next one.