LiquidAI/LFM2.5-230M-GGUF
Text Generation • 0.2B • Updated • 3.47k • 40
Sergei Tilga
tilgasergey
AI & ML interests
None yet
Recent Activity
liked a model 2 days ago
LiquidAI/LFM2.5-230M-GGUF liked a model 2 days ago
LiquidAI/LFM2.5-230M liked a model about 1 month ago
LiquidAI/LFM2.5-8B-A1BOrganizations
liked 2
models 2 days ago
liked a
model about 1 month ago
updated a
dataset 6 months ago
published a
dataset 6 months ago
updated a
dataset 10 months ago
liked a
dataset 10 months ago
updated a
dataset about 1 year ago
liked a
dataset over 1 year ago
reacted to cogwheelhead's post with 👍 over 1 year ago
Post
2545
Me and my team have performed an in-depth investigation comparing o1 to R1 (and other reasoning models)
Link: https://toloka.ai/blog/r1-is-not-on-par-with-o1-and-the-difference-is-qualitative-not-quantitative
It started with us evaluating them on our own university-math benchmarks: U-MATH for problem-solving and μ-MATH for judging solution correctness (see the HF leaderboard: toloka/u-math-leaderboard)
tl;dr: R1 sure is amazing, but what we find is that it lags behind in novelty adaptation and reliability:
* performance drops when updating benchmarks with fresh unseen tasks (e.g. AIME 2024 -> 2025)
* R1-o1 gap widens when evaluating niche subdomains (e.g. university-specific math instead of the more common Olympiad-style contests)
* same with going into altogether unconventional domains (e.g. chess) or skills (e.g. judgment instead of problem-solving)
* R1 also runs into failure modes way more often (e.g. making illegal chess moves or falling into endless generation loops)
Our point here is not to bash on DeepSeek — they've done exceptional work, R1 is a game-changer, and we have no intention to downplay that. R1's release is a perfect opportunity to study where all these models differ and gain understanding on how to move forward from here
Link: https://toloka.ai/blog/r1-is-not-on-par-with-o1-and-the-difference-is-qualitative-not-quantitative
It started with us evaluating them on our own university-math benchmarks: U-MATH for problem-solving and μ-MATH for judging solution correctness (see the HF leaderboard: toloka/u-math-leaderboard)
tl;dr: R1 sure is amazing, but what we find is that it lags behind in novelty adaptation and reliability:
* performance drops when updating benchmarks with fresh unseen tasks (e.g. AIME 2024 -> 2025)
* R1-o1 gap widens when evaluating niche subdomains (e.g. university-specific math instead of the more common Olympiad-style contests)
* same with going into altogether unconventional domains (e.g. chess) or skills (e.g. judgment instead of problem-solving)
* R1 also runs into failure modes way more often (e.g. making illegal chess moves or falling into endless generation loops)
Our point here is not to bash on DeepSeek — they've done exceptional work, R1 is a game-changer, and we have no intention to downplay that. R1's release is a perfect opportunity to study where all these models differ and gain understanding on how to move forward from here
liked a
Space over 1 year ago
liked a
dataset over 1 year ago
authored 2
papers over 1 year ago
upvoted a collection over 1 year ago
upvoted a paper over 1 year ago
liked 2
datasets over 1 year ago
liked a
dataset almost 2 years ago