Tag: Field-Notes

The MCP maintainers published an updated roadmap.

Since March, MCP went stateless (no more session handshakes), turned Tasks into an official extension for long-running work, and shipped enterprise auth pieces like issuer validation and Client ID Metadata Documents.

Next up: one HTTP-native transport instead of separate stdio and HTTP paths, real agent identity (workload federation, token exchange), and progressive tool discovery for large catalogs. No firm dates yet.

MCP is leaving its single-session, request/response origins behind for something built for long-running, multi-agent systems.

🚢 Marin 535B-A23B started training this week! As usual, the whole process is open.

Voyage plan: pretraining (80%) + midtraining (20%) on 18.75T tokens on 11 x GB200 NVL72 for ~3 months (2.7e24 FLOPs). Post-training will follow.

Before kicking off the run, we trained a 4-rung scaling ladder from 1.6B-A61M (48B tokens) to 27.7B-A1.2B (926B tokens) to debug issues, and to make a forecast of our hero run. This is by far our biggest run, so definitely expecting the unexpected.

Percy Liang, announcing the run on X.

The Marin project is a great example of being open to AI. The whole process is public: the scaling ladder they ran to debug the pipeline before committing GPU-months to the real thing, the exact token counts, the exact FLOPs, even the admission that they’re “expecting the unexpected” on their biggest run yet.

Most labs treat a run like this as a trade secret until the model ships. It’s great we see another public model build. The last public run at this scale was BLOOM, BigScience’s 176B model. Marin’s 535B total parameters (23B active) puts it well past that, the biggest public pretraining run that I’m aware of.

Surya trained Qwen 3.5 to have taste in watercolor painting. The results are stunning.

It’s purely RL:

The system is a four-step loop, run thousands of times during training.

The model receives a prompt, something like draw a peach hibiscus in watercolour, and writes a complete p5.brush JavaScript sketch. The sketch is rendered in a sandboxed Puppeteer environment, which produces a PNG. The PNG is judged against two random reference paintings sampled from a hand-rated pool, with a separate judge model picking the better watercolour. The judgment is converted into a reward signal, GRPO updates the model, and the loop runs again.

[…297 words]

And this was a debug session from hell, enormously helped by an AI doing much of the grunt-work.

I’d like to call it my tireless helper, but the AI several times stated flat out that this was impossible and unsolvable and that we should just write a report about it.

I suspect those things have been trained by people who may not be quite as stubborn as I am.

But while the AI was ready to give up several times, it did keep adding debug code and analyzing it faithfully when I pushed. So credit where credit is due and I let the AI write the commit message above.

This is basically a one-liner fixing a bogus round_up() to a round_down(), but there were 24 patches adding more and more debug information to this, and 18 kernel boot to finally narrow it down to this. — Linus

Linus Torvalds, in the commit message for a kernel fix.

I troubleshoot every now and then myself, and it’s way harder than programming, tbh. AI can’t always find the root cause. But it’s still a tireless companion to have on-call — it surfaces details I’d have missed on my own, and some of them are the ones that end up pointing the way.

You already know ChatGPT’s style, but it moved away from the old chat interface some time ago. A chat interface works like ping-pong: one message bubble, then another. Agentic UI does not work that way, because it often runs a long process behind the scenes. Between your prompt and the agent’s reply, many details happen. The interface must show you these details.

Here is what I believe agentic UI must give you:

[…562 words]

Thariq at Anthropic posted that it’s a skill people there have been using a lot recently: /eli5 <what you want to explain>.

I tried it on neural networks. Here’s what it made.

The first pass had bad colors, low contrast, hard to read. One follow-up prompt asking for a color fix and it was done.

No formulas, no sigmoid, no softmax, none of the math that actually makes a neural network work. It’s missing for good. What’s left is the dataflow — inputs go in, get combined, come out the other end as a decision — drawn simply enough to follow at a glance.

That’s the trade the skill makes, and it’s the right one for someone who isn’t about to read a textbook. eli5 is good exactly because it throws out the complexity most explanations lead with. I imagine this isn’t just for self-learning — it’s a genuinely useful tool for collaboration, communication, meetings. Conveying an idea isn’t easy. ;P

Updated at 2026-08-23. The neural network example is ported to julin.ai now.

Vizuara AI Labs trained a miniature Kimi K3 from scratch: 1.02B parameters, 145M active, 5B tokens, one H200, $252.35.

Not simplifying the architecture like Karpathy’s microgpt, they kept Kimi K3’s MoE and attention design intact.

That’s a surprisingly cheap way to learn pretraining (in real-world). They worked through expert collapse, data-mixing bugs, distributed-training bugs, kernels, and GPU utilization on a modern MoE architecture.

A few things worth noting:

• 5B tokens is probably too little for a 1B model. The authors agree the run was budget constrained. So the cheap cost might due to the training stopped early.
• Beating GPT-2 isn’t particularly meaningful when Mini K3 has roughly 10× the parameters.
• MoE at this scale is debatable. A smaller dense model trained on more tokens would likely be better if the goal was capability.