AI & ML interests

πŸ€— Hugging Face x 🌸 BigScience initiative to create open source community resources for LAMs.

Recent Activity

albertvillanovaΒ 
posted an update 19 days ago
view post
Post
348
Day-0 Mistral Large 4 support in TRL πŸš€

- LoRA SFT of the 1T-A52B model on one 8xB300 node, on top of the FP8 checkpoint
- Experts split 8 ways with expert parallelism, everything else sharded with FSDP, all set at load time
- Assistant-only loss out of the box with its training chat template
- ~32.7k tokens per step in ~8 s

Script + Slurm launcher πŸ‘‡
https://github.com/huggingface/trl/tree/main/examples/sft_mistral_large_4
https://huggingface.co/posts/sergiopaniego/408182860646834
alielfilali01Β 
posted an update about 2 months ago
view post
Post
634
multi-agents orchestration is old news by now. Try multi-teams of agents ! that is a whole different nightmare ... This is the real signal about the sparks of AGI.
davanstrienΒ 
updated a bucket 3 months ago
davanstrienΒ 
published a bucket 3 months ago