stuck repeating "actually let me act"

#7
by nabelekm - opened

Using pi harness with 256k context, single stream, running SGLang it degrades over time (after few compactions) to constant repetition within CoT. In my case it ends up infinitely repeating

 Let me actually check the actual ACT β€” the actual issue β€” actually the actual issue β€” actually the ACT β€” actually ACT  
 β€” "ACT" OK actually let me ACT β€” actually the actual issue β€” actually the actual ACT β€” actually the ACT β€” ACT          
 β€”Actually the ACTACT ACTACT ACT ACTACT ACTACT ACT the ACTACT ACT ACT ACT ACT ACT ACT ACT ACT ACT ACTACT ACT ACTACT     
 ACTACT ACTACT ACT ACT ACT ACT"ACT"OK actually the ACT β€” ACT actually ACTACT ACT ACT ACT ACT ACT ACT ACTACT ACT ACT ACT 
 ACT ACT 

increase temp or rep penalty πŸ€—

Actually act ACT ACT ACT ACT β€” actually πŸ€“β˜οΈ

I've also seen it get stuck looping in the thinking blocks running the original weights, but with much a much longer repeated segment than yours.
Like:

OK β€” I'll extend the box left to x=0 (it's already at x=0). The 'p' descender β€” looking at the image: the 'p' β€” the descender extends to the LEFT of the box β€” the orange border line at x=0 β€” the 'p' descender β€” looking at the image: the 'p' β€” the descender extends to x=0 β€” at the very edge of the box!

Hmm β€” wait β€” actually looking at the image: the orange box β€” the left edge β€” the 'p' descender β€” looking at the image: the 'p' β€” the descender extends to the LEFT of the box β€” the orange border line at x=0 β€” the 'p' descender β€” looking at the image: the 'p' β€” the descender extends to x=0 β€” at the very edge of the box!

OK β€” I'll extend the box left to x=0 (it's already at x=0). The 'p' descender β€” looking at the image: the 'p' β€” the descender extends to the LEFT of the box β€” the orange border line at x=0 β€” the 'p' descender β€” looking at the image: the 'p' β€” the descender extends to x=0 β€” at the very edge of the box!
...

So it clearly has a repetition problem. Guess they really meant it when they talked about a looping transformer. πŸ˜„

What quants do you guys

Sign up or log in to comment