Today's edition highlights diverse approaches to self-improvement, from multimodal models verifying their own outputs to optimizers evolving their own diagnostic tools.
The loop: A unified multimodal model generates images and writes programs to evaluate them, using the verified results to train its own visual understanding and generation.
A unified multimodal model uses program execution to verify its own generated images, creating a reliable training loop that improves its visual and text capabilities over multiple rounds.
The loop: The VERSE optimizer tests draft edits and replays failures to revise an agent harness alongside its own prompts and tools, improving its ability to diagnose and fix future errors.
An agent optimizer improves its own diagnostic tools and workflows while evolving an executor's harness, using execution-based verification to prevent regressions.
The loop: A data synthesis system recursively improves its own generation harness by converting intermediate solver failures into reusable skills, producing progressively harder reasoning tasks.
A recursive self-improvement framework co-evolves its reasoning-data synthesis harness alongside the tasks it generates, significantly increasing task difficulty and downstream model performance.
The loop: An advisor model reflects on completed interactions to propose corrections, then uses targeted self-distillation to improve the advice it issues to a frozen language-model executor.
A small trainable advisor improves its ability to steer a frozen language model by selectively self-distilling from its own reflection on past interactions.
The loop: A synthetic document generator produces training data with reduced markers, which finetunes a model to internalize false facts that evade middle-layer linear probes.
Researchers demonstrate that reducing synthetic markers in generated training documents allows finetuning to implant false facts that are indistinguishable from pretraining-acquired knowledge.
The loop: The Jagged Flash Attention kernel improves inference efficiency on Blackwell, which makes the Generative Ads Model faster.
TL;DR In this blog post, we present our work on Jagged Flash Attention (JFA) - the attention kernel behind Meta’s Generative Ads Model (GEM) - on NVIDIA Blackwell (B200), built...
The loop: Olmo-core 3 provides an open training stack that improves the efficiency of training MoE models, which are then used to build better versions of the stack.
Olmo-core 3 introduces a redesigned, fully open training stack for efficiently scaling mixture-of-experts models into the trillion-parameter range.
