oga-macronizer-char-code
Training, hyperparameter search and evaluation code for the char-level Ancient Greek macronizer (Ericu950/oga-macronizer-char), as revised for the reviewed version of Automatic Annotation of Ancient Greek Vowel Length (Thörn Cleland & Cullhed).
Compared to the original code in
grc-macronizer/macron_model
(commit 456c970):
- Decontaminated training data: training uses Urdatorn/oga-macronized-no-norma, from which every sentence overlapping the benchmark has been removed.
- Current Norma layout: evaluation reads the jsonl version of
Urdatorn/norma
(
data/test.jsonl,data/validation.jsonl) and reports the metrics of the paper's Table 1 (per-label accuracy, P, R, F1 and coverage) and per-position predictions for error analysis. - Model selection on gold data: Norma validation, which is gold-annotated, is scored at every evaluation and can drive checkpoint selection, early stopping and hyperparameter search. The test split is used only in the final evaluation.
- Hyperparameter search: an Optuna search over learning rate and weight decay, with dropout optional.
- Faster training input: the corpus is encoded once into flat arrays, and padding and diacritic masking are vectorized.
model.py and tokenizer.py are unchanged from the original, so checkpoints
are compatible both ways.
Files
| file | purpose |
|---|---|
model.py, tokenizer.py |
model and char-plane tokenizer (unchanged) |
prepare_data.py |
encode the silver corpus into cache/ (fixed 90/10 split, --data_seed 0) |
train.py |
train one model, logging silver-val and Norma-val metrics at every evaluation |
hpo.py |
Optuna study shared by several slurm workers through a journal file |
evaluate.py |
score checkpoints (local or Hub) on Norma, writing metrics and per-position jsonl |
norma.py |
Norma loading and scoring |
predict.py |
batched inference (MacronPredictor) |
slurm/ |
jobs for the Arrhenius GH200 nodes: prepare.slurm (venv + data cache), then hpo.slurm (search) |
Usage
pip install -r requirements.txt
python prepare_data.py --out cache/oga-no-norma
python train.py --cache cache/oga-no-norma --output_dir runs/seed1 --seed 1
python evaluate.py --models runs/seed1/best --splits test validation --out results/seed1
Scoring
The scored positions are the gold-marked dichrona in open syllables. In test, openness comes from the parallel syllabify lines, giving 1,916 positions. In validation, which has no syllabify lines, every gold mark is scored, giving 2,498 positions.
- Accuracy: an unmarked position counts as wrong.
- Per label: accuracy equals recall; precision is computed over that label's predictions.
- Mixed (micro-averaged): recall = accuracy; precision = correct / marked positions; F1 is their harmonic mean. This reproduces Table 1.
- Defaults-to-short accuracy: an unmarked position counts as "short". This is the metric reported on the model cards.
License
GNU GPL v3.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support