Hugging Face's logo
Activity Feed

AI & ML interests

None defined yet.

Recent Activity
Doubiiuย  authored a paper 1 day ago
WanPE: Towards Cinematic Prompt Enhancement for Modern Text-to-Video Generation
Doubiiuย  authored a paper 1 day ago
Wan-Image: Pushing the Boundaries of Generative Visual Intelligence
Doubiiuย  authored a paper 1 day ago
ReCA: Multi-Shot Long Video Extrapolation via Recursive Context Allocation
View all activity
sergiopaniegoย 
posted an update 6 days ago
view post
Post
238
ICYMI, Async GRPO in TRL now supports LoRA and we wrote a looong blog testing it

> the adapter is a few megabytes, so the weight sync is a file instead of an NCCL transfer
> 3 HF Jobs: 1 trainer and 2 vLLM replicas
> the adapter travels through an HF Storage Bucket mounted in all 3 at the same path
> a proxy in front of the replicas routes each rollout to the one already holding its KV prefix

https://huggingface.co/blog/asyncgrpo-lora-hfjobs
sergiopaniegoย 
posted an update 17 days ago
view post
Post
3268
while preparing the last class of the Training Agents live series during the summer, i spent some time reading the post-training sections of many frontier model reports, to learn how they use RL environments to improve their models, and wrote a blog about it

if you use any kind of coding harness, or you saw the Blender scenes that went viral recently, this might be interesting to you

Blog: https://huggingface.co/blog/sergiopaniego/rl-environments-2026
yuntian-dengย 
authored 2 papers 24 days ago
yuntian-dengย 
submitted a paper to Daily Papers 25 days ago
codelionย 
in zero-gpu-explorers/README 25 days ago

ZeroGPU clone fails during tensor packing with Errno 28 โ€” 82.1 GB pack target

1
#185 opened 26 days ago by
Thananon
sergiopaniegoย 
posted an update 26 days ago
view post
Post
2135
Can you do RL over taste?

I've spent some time reproducing, in the open, Surya N's idea of training a model to paint with code. It's a coding model that learns to paint watercolours by writing JS code, trained with GRPO. I used TRL and OpenEnv for this, with the whole pipeline running on Hugging Face.

The interesting part is that the reward has no correct answer, unlike a math problem. In this case it's based on the artistic preferences of the person who builds the dataset.

Everything is published: the environment, the reference pool, the trained adapters, every painting of every run with the code that made it, and a write-up with all the decisions, including the ones that went wrong.

Blog post: https://huggingface.co/blog/train-to-paint-with-code
yuntian-dengย 
authored a paper 27 days ago
yuntian-dengย 
submitted a paper to Daily Papers 27 days ago
sergiopaniegoย 
posted an update about 1 month ago
view post
Post
521
catching up on some bookmarked reads from the summer, reading Antidoom from @liquidai

small reasoning models get stuck more easily when the task involves a long thinking trace and a hard problem. It starts repeating the same word over and over again ("Wait", "Alternatively"โ€ฆ), each repetition makes the next one likelier, and the generation is spent before it reaches an answer

they measured it, 10.2% of completions for an early LFM2.5-2.6B checkpoint and 22.9% for Qwen3.5-4B at greedy. After training those drop to 1.4% and 1.0%

the fix is FTPO (final token preference optimization). What I like is how narrow it is, it only touches the single token where the loop starts

three ways it differs from DPO:
> trains one token position, mid-generation, instead of whole sequences
> spreads probability across ~20 plausible alternatives instead of swapping one overtrained token for another
> keeps the regularizer in logit space, no softmax, so the rest of the vocabulary stays put

the third one is what makes it usable. If you want to edit one position without disturbing the model, you can't have a loss that reshuffles the other 150k logits on the way

and their explanation abt the result: the training teaches the model nothing new about math or code, it clears the failure mode that was blocking answers the model could already produce

full blog > https://www.liquid.ai/blog/antidoom

FTPO itself comes from Antislop, where it was built to strip overused phrasing. LiquidAI retargeted it to doom loops

and under the hood it's a subclass of TRL's DPOTrainer with compute_loss overridden, around 90 lines of loss and no new trainer

we documented that pattern in TRL's docs
https://huggingface.co/docs/trl/main/en/customization#change-the-training-objective
  • 1 reply
ยท
sergiopaniegoย 
posted an update about 1 month ago
view post
Post
360
super interesting new paper from Microsoft "Agent Lightning v1.0: Towards Harnessed Agentic RL" by Zhiyuan He et al.

same idea we've seen already several times: you train the agent inside the real harness it ships with, instead of a reimplementation of it

now that recipe has a name โ†’ harnessed agentic RL

paper: huggingface.co/papers/2608.17528

the tricky bit they nail down: one rollout is not one training sample

the harness calls the model many times, so a single episode โ†’ a variable number of (prompt, response) rows

you don't even know the batch size until the episode finishes running

its real contribution is being first to systematically map the four problems that fall out of that:

> retokenization + sample merging
> advantage calculation over a variable sample count
> loss normalization at the rollout level, not per sample
> backend scheduling when the batch size is dynamic

and it actually works โ†’ plain RL inside the real harness, no reimplementation

Qwen3.5-9B on SWE-bench Verified 41.8 โ†’ 56.4 (+14.6), with only ~6k examples

the whole thing is ~3,500 lines, any harness, self-hosted k8s

from our side, we've shared some materials on the same line you may want to check out :)

> Agentic RL: Token-In, Token-Out Done Right: https://huggingface.co/blog/huggingface/tito
> a full worked example, opencode owning its loop trained with GRPO: https://huggingface.co/blog/sergiopaniego/trl-openenv-harness-training
> Harness, Scaffold, and the AI Agent Terms Worth Getting Right: https://huggingface.co/blog/agent-glossary

on a similar line:

https://x.com/SergioPaniego/status/2062911580564496576
dreamerdeoย 
authored a paper about 1 month ago
RioShiinaย 
in zero-gpu-explorers/README about 1 month ago

All ZeroGPU Spaces stuck on "Starting" with clean logs โ€” CPU and paid GPU unaffected

๐Ÿ‘ 2
4
#183 opened about 1 month ago by
luna0805
Nymboย 
posted an update about 2 months ago
view post
Post
2338
Anthropic gave me six months of Claude Max 20x through the Claude for Open Source program, granted based on my Hugging Face work. Thank you
Anthropic for supporting open source.

So far I've been pointing it at Markdown Minimap, an Obsidian plugin that adds a scrollable IDE-style minimap to your notes. This week I've been clearing a backlog of user-reported issues on it, with Claude often handling them end to end.

https://github.com/Nymbo/Markdown-Minimap โ€” issues and PRs welcome.
sergiopaniegoย 
posted an update about 2 months ago
view post
Post
780
Something I really like when I study a subject is understanding its history, how it reached the point where it is today

I did that exercise for RL in post-training: from RLHF and PPO, to verifiable rewards, to the GRPO family of variants, to agents acting in environments. Everything is backed by what the labs themselves say in their public reports (DeepSeek, Qwen, Kimi, GLM-5, Nemotron, Mistral and more), in their own words

This is the companion piece to Class 3 of our Training Agents series with @burtenshaw . The class explains how GRPO works, with three hands-on experiments. The article shows where the same ideas appear at frontier scale

https://huggingface.co/blog/sergiopaniego/agentic-rl-2026
  • 1 reply
ยท
BramVanroyย 
posted an update about 2 months ago
view post
Post
2772
**I benchmarked HF buckets against https access for Common Crawl.**

Took me a while to get round to do this but I benchmarked access to Common Crawl via https vs hf buckets. Both experiments were run at night in Europe. I do not think other hardware problems were impacting the speeds since CPU processing time of the non-download pipeline components were highly similar (within 2% identical) and below only the WarcReader speeds of datatrove are used.

Experiment: selected 5 disjoint samples of 64 files each (randomly from the latest crawl; 20,499 docs/file). Those five batches were then processed by 32 single-core tasks with 4GB/core (five batches to calculate CIs). Paired experiment between using https and hf bucket.

- https: 40.0 [39.3-40.6] (seconds per WARC file)
- hf bucket: 172.0 [122.7-221.2]

That is a difference of about 4x in streaming speed. You'll see that https is also more stable (smaller CI).

I also ran raw throughput tests to the endpoints to measure rate limiting (64MiB transfer at 8/32/128/256 concurrent readers) and rate limiting seems not an issue for either: at any of those parallel reader numbers, their respective speeds stay about the same.

Note that, given CC scale, this is still a small test. Rate limiting may become more obvious when processing a full crawl. I do not know whether the https endpoint vs HF bucket will shut you out earlier with which limits.
sergiopaniegoย 
posted an update about 2 months ago
view post
Post
352
we just released a new blog "Training a coding agent using the OpenCode harness in remote HF sandboxes with TRL and OpenEnv"

you can take a real coding agent (OpenCode), let it run its own tool loop against real coding problems, and train it with RL on the exact tokens it produced

and every rollout runs in its own remote HF sandbox, so rollouts scale out beyond one machine

the loop:
- OpenCode owns its tool loop inside an OpenEnv sandbox
- an in-sandbox proxy records the real token ids + logprobs, per turn
- a hidden-test verifier scores the result, and that is the reward
- TRL trains with AsyncGRPO, weights sync back to vLLM over NCCL

blog + runnable example: https://huggingface.co/blog/sergiopaniego/trl-openenv-harness-training
  • 3 replies
ยท
Lin-Chenย 
authored a paper about 2 months ago
sergiopaniegoย 
posted an update about 2 months ago
view post
Post
2674
LFM2.5-2.6B just dropped!

and the @liquidai blog comes with some nice details about the training procedure, so let's analyze it.

basically, a full agent training pipeline but compressed into 2.6B

base model โ†’ SFT โ†’ specialized teachers per domain (SFT + RLVR) โ†’ on-policy distillation back into one student โ†’ agentic RL

the two most interesting stages

โ†’ MOPD: the student generates, each prompt routes to its domain teacher for token-level feedback. teachers branch from the same SFT checkpoint, so their signal stays close to the student's distribution

โ†’ agentic RL: multi-turn GRPO inside real harnesses (OpenClaw, Hermes Agent), one sandbox per rollout, a proxy captures token-level trajectories while the harness stays a black box

this makes a 2.6B that beats much larger models on instruction following and tool use

SFT, distillation, RL, RL envs: exactly what we're covering in our Training Agents livestream series (next one coming soon!)

โ†’ model: LiquidAI/LFM2.5-2.6B
โ†’ blog: https://www.liquid.ai/blog/lfm2-5-2-6b
โ†’ live series: https://www.youtube.com/playlist?list=PLo2EIpI_JMQvQZm-kVlz4wY1vWF0LBcf5
  • 3 replies
ยท
Lin-Chenย 
authored a paper about 2 months ago