Open SLM Leaderboard
🏆
96
Open Small Language Model Leaderboard
👽 Can AI go rogue?
Ash
Hoglet-33
AI & ML interests
Open source AI, datasets, parameter efficiency, SLMs, AI for the betterment of humanity. Contact at ash@basicallyai.co
Recent Activity
liked a Space 3 days ago
AxiomicLabs/Open_SLM_Leaderboard new activity 3 days ago
AxiomicLabs/Open_SLM_Leaderboard:Adding Pebble. repliedto their post 3 days ago
Today, we planned to release Pebble-50M and Pebble-50M-Chat to the world. Unfortunately, due to a few issues, that didn't go quite as planned.
What happened:
- Some data and benchmark results were lost or corrupted
- The models performed worse on benchmarks than our other Pebble models
Despite that, you can still find both models here:
Pebble-50M-beta: https://huggingface.co/basically-experimental/Pebble-50M-beta
Pebble-50M-Chat-beta: https://huggingface.co/basically-experimental/Pebble-50M-Chat-beta
There are still some interesting improvements in these models:
- Compatible with non-CUDA devices
- Vocabulary increased to 16K tokens
- Context length increased to 16K tokens
For now, there won't be any more Pebble releases for a while. We're going to take some time to experiment with other approaches and hopefully make the next generation a monumental leap over this one.
Follow for updates:
@Hoglet-33
https://huggingface.co/basically-ai
https://huggingface.co/basically-experimentalOrganizations
liked a
Space 3 days ago
Adding Pebble.
1
#132 opened 3 days ago
by Hoglet-33
replied to their post 3 days ago
I mean most SLM labs are doing the same thing though, go look at BananaMind, Axiomic Labs, and FromZiro. So i wasnt doing anything out of the normal, and its the same training data as Pebble-25M and Pebble-10M so it isnt 100% the dataset.
replied to their post 4 days ago
that could work, we are going to start a series of tests that will hopefully give some useful data, simplifying data will probably be part of it.
updated a
Space 4 days ago
reacted to their post with 😔 4 days ago
Post
3153
Today, we planned to release Pebble-50M and Pebble-50M-Chat to the world. Unfortunately, due to a few issues, that didn't go quite as planned.
What happened:
- Some data and benchmark results were lost or corrupted
- The models performed worse on benchmarks than our other Pebble models
Despite that, you can still find both models here:
Pebble-50M-beta: basically-experimental/Pebble-50M-beta
Pebble-50M-Chat-beta: basically-experimental/Pebble-50M-Chat-beta
There are still some interesting improvements in these models:
- Compatible with non-CUDA devices
- Vocabulary increased to 16K tokens
- Context length increased to 16K tokens
For now, there won't be any more Pebble releases for a while. We're going to take some time to experiment with other approaches and hopefully make the next generation a monumental leap over this one.
Follow for updates:
@Hoglet-33
basically-ai
basically-experimental
What happened:
- Some data and benchmark results were lost or corrupted
- The models performed worse on benchmarks than our other Pebble models
Despite that, you can still find both models here:
Pebble-50M-beta: basically-experimental/Pebble-50M-beta
Pebble-50M-Chat-beta: basically-experimental/Pebble-50M-Chat-beta
There are still some interesting improvements in these models:
- Compatible with non-CUDA devices
- Vocabulary increased to 16K tokens
- Context length increased to 16K tokens
For now, there won't be any more Pebble releases for a while. We're going to take some time to experiment with other approaches and hopefully make the next generation a monumental leap over this one.
Follow for updates:
@Hoglet-33
- 4 replies
posted an update 4 days ago
Post
3153
Today, we planned to release Pebble-50M and Pebble-50M-Chat to the world. Unfortunately, due to a few issues, that didn't go quite as planned.
What happened:
- Some data and benchmark results were lost or corrupted
- The models performed worse on benchmarks than our other Pebble models
Despite that, you can still find both models here:
Pebble-50M-beta: basically-experimental/Pebble-50M-beta
Pebble-50M-Chat-beta: basically-experimental/Pebble-50M-Chat-beta
There are still some interesting improvements in these models:
- Compatible with non-CUDA devices
- Vocabulary increased to 16K tokens
- Context length increased to 16K tokens
For now, there won't be any more Pebble releases for a while. We're going to take some time to experiment with other approaches and hopefully make the next generation a monumental leap over this one.
Follow for updates:
@Hoglet-33
basically-ai
basically-experimental
What happened:
- Some data and benchmark results were lost or corrupted
- The models performed worse on benchmarks than our other Pebble models
Despite that, you can still find both models here:
Pebble-50M-beta: basically-experimental/Pebble-50M-beta
Pebble-50M-Chat-beta: basically-experimental/Pebble-50M-Chat-beta
There are still some interesting improvements in these models:
- Compatible with non-CUDA devices
- Vocabulary increased to 16K tokens
- Context length increased to 16K tokens
For now, there won't be any more Pebble releases for a while. We're going to take some time to experiment with other approaches and hopefully make the next generation a monumental leap over this one.
Follow for updates:
@Hoglet-33
- 4 replies
published 2
models 4 days ago
updated 3
models 5 days ago
updated a
Space 6 days ago
liked a
model 6 days ago
replied to their post 6 days ago
Thank you for that link, i might train future models with mamba3 right now though im going to stick with mamba2, mamba2 vs mamba3 will probably be one of the things i test in the hunt for the optimal mamba models.
replied to their post 7 days ago
I also basically live under a rock, i somehow miss a lot of things in the AI space..
replied to their post 7 days ago
tbh i didnt actually know that mamba3 was out... but thanks for letting me know.
updated a
collection 8 days ago
replied to their post 8 days ago
Thank you ContextReq!