Update README.md

README.md

@@ -196,19 +196,23 @@ $$\text{GRPO} = -\mathbb{E}_{(s,a)}\left[\frac{\pi(a|s)}{\pi_{\text{old}}(a|s)}
 
 1. **Concatenate** `prompt_ids + completion_ids`
 
-2. **Run forward pass through old policy** to compute $$\pi_{\text{old}}(a|s)$$
+2. **Run forward pass through old policy** to compute
+
+   $$\pi_{\text{old}}(a|s)$$
+
+   
    - This actually happens only once at the first iteration when we create the rollout
 
-3. **Run forward pass through ref policy** to compute $$\pi_{\text{ref}}(a|s)$$
+4. **Run forward pass through ref policy** to compute $$\pi_{\text{ref}}(a|s)$$
    - This actually happens only once at the first iteration when we create the rollout
    - Ref model is the original model without LoRA adapters
 
-4. **Run forward pass through current policy** to compute $$\pi(a|s)$$
+5. **Run forward pass through current policy** to compute $$\pi(a|s)$$
    - Needed only if `num_iterations > 1`; otherwise the same as old policy
 
-5. **Compute KL loss** between $$\pi(a|s)$ and $\pi_{\text{ref}}(a|s)$$
+6. **Compute KL loss** between $$\pi(a|s)$ and $\pi_{\text{ref}}(a|s)$$
 
-6. **Compute advantage-weighted logprobs:** $$\frac{\pi(a|s)}{\pi_{\text{old}}(a|s)} \times A(s,a)$$
+7. **Compute advantage-weighted logprobs:** $$\frac{\pi(a|s)}{\pi_{\text{old}}(a|s)} \times A(s,a)$$
 
 ## Workflow Summary
 
@@ -305,11 +309,6 @@ Mark wants 12 total pieces of fruit. He already has 3 apples and 4 bananas, whic
 </answer>
 ```
 
-## Performance
-
-- **Dataset**: GSM8K test split (1,319 examples)
-- **Evaluation Metric**: Exact match accuracy on final numerical answers
-- **Performance**: [Insert actual accuracy from your evaluation]
 
 ## Technical Details