Vizuara is hiring

New to this topic? That is completely fine. It is completely fine if you have never studied some of these topics before. We want to see how fast you can learn something new, grasp it, and teach it from first principles.

This page is a quick primer, not a script. Use it to get your bearings, then read the links below and anything else you find: other blogs, papers, videos, code. We are judging the story you build, your examples and metaphors, your first-principles thinking, and any experiments you run yourself.

A quick primer

In January 2025, DeepSeek reported something remarkable. They trained a model called DeepSeek-R1-Zero with pure reinforcement learning: no human-written examples of reasoning, only a reward for getting maths and coding answers right in the right format. During training, the model started to stop, re-evaluate and correct itself in the middle of its answers. The paper calls it an “aha moment”.

The algorithm underneath: GRPO

Group Relative Policy Optimization. Sample a group of answers to the same question, score each one, and push the model towards the answers that did better than the group average. No separate critic network is needed.

For each question $q$ , sample a group of $G$ answers $o_1,\dots,o_G$ from the current policy and score them with rewards $r_i$ . The advantage of each answer is how much better it did than its group:

\hat{A}_i = \frac{r_i - \mathrm{mean}(r_1,\dots,r_G)}{\mathrm{std}(r_1,\dots,r_G)}

The model is then updated with a clipped objective, much like PPO, plus a penalty that keeps it close to a reference model:

\mathcal{J}(\theta) = \mathbb{E}\left[\frac{1}{G}\sum_{i=1}^{G} \min\!\Big(\rho_i \hat{A}_i,\ \mathrm{clip}(\rho_i, 1-\varepsilon, 1+\varepsilon)\,\hat{A}_i\Big) - \beta\, \mathbb{D}_{KL}\big(\pi_\theta \,\|\, \pi_{\mathrm{ref}}\big)\right], \quad \rho_i = \frac{\pi_\theta(o_i\mid q)}{\pi_{\theta_{\mathrm{old}}}(o_i\mid q)}

What the paper reports, drawn as a sketch: as RL training goes on, responses get longer as the model learns to spend more time thinking, and reflective behaviour appears.

Questions worth exploring

Where does this story really begin? Policy gradients, PPO, RLHF, and why DeepSeek dropped the critic.
Why do simple rule-based rewards work for maths and code, and where would they fail?
Was it really an “emergent” moment? Read the critiques below, which find self-reflection already present in base models.
What does each term in the objective do? What happens if you remove the KL term, or the std normalisation?

Read

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning · DeepSeek-AI, 2025The primary source: R1-Zero, the "aha moment" passage, and the R1 recipe.
DeepSeekMath · Shao et al., 2024Where GRPO was introduced.
The Illustrated DeepSeek-R1 · Jay AlammarA visual breakdown of the training pipeline.
There May Not be Aha Moment in R1-Zero-like Training · Sea AI Lab (Oat team)The critique: self-reflection already appears in base models.
Understanding R1-Zero-Like Training: A Critical Perspective · Liu et al., 2025Biases hiding in GRPO, and a fix (Dr. GRPO).

Watch

▶ GRPO Explained: DeepSeekMath · Yannic Kilcher
▶ DeepSeek's GRPO, Reinforcement Learning for LLMs · Julia Turc
▶ GRPO: The RL Trick Behind DeepSeek-R1's Reasoning · Jia-Bin Huang

Go beyond this list. These are starting points. The best submissions usually bring in something we did not link: an experiment, a paper, a story of their own.

How to submit Next: Q4 The GPU roofline plot →

DeepSeek's "aha moment"

A quick primer

The algorithm underneath: GRPO

Questions worth exploring

Read

Watch

Any doubts or questions?