Is ckpt000 the initialized model?
#6
by jasonrqh - opened
Really appreciate this great work on opensourcing all intermediate checkpoints! Just a quick question, is ckpt000 the initialized model or the model trained for one chunk of data? The model output from ckpt000 seems to have some patterns (e.g., 'the first one is the first one is...') rather than pure nonsense.