I’m a software engineer interested in AI and how it changes the way we build software.
I use julin.ai to write about what I learn from building with AI.
Most of my work here is about AI models, agents, developer tools, infrastructure, and enterprise AI.
I test new ideas and tools, build small projects, and write about what works and what does not.
Some posts are short notes and links. Others are experiments or longer articles.
This is my notebook for learning AI in public.
Vizuara AI Labs trained a miniature Kimi K3 from scratch: 1.02B parameters, 145M active, 5B tokens, one H200, $252.35. Rather than simplify the architecture, they kept Kimi K3’s MoE and attention design intact. Some pushback from the LocalLLaMA discussion is worth noting: 5B tokens is probably too little for a 1B model. The authors agree the run was budget constrained. Beating GPT-2 isn’t particularly meaningful when Mini K3 has roughly 10× the parameters. MoE at this scale is debatable. A smaller dense model trained on more tokens would likely be better if the goal was capability. For $252, they worked through expert collapse, data-mixing bugs, distributed-training bugs, kernels, and GPU utilization on a modern MoE architecture. ...