oga-macronizer-char-code

Training, hyperparameter search and evaluation code for the char-level Ancient Greek macronizer (Ericu950/oga-macronizer-char), as revised for the reviewed version of Automatic Annotation of Ancient Greek Vowel Length (Thörn Cleland & Cullhed).

Compared to the original code in grc-macronizer/macron_model (commit 456c970):

  • Decontaminated training data: training uses Urdatorn/oga-macronized-no-norma, from which every sentence overlapping the benchmark has been removed.
  • Current Norma layout: evaluation reads the jsonl version of Urdatorn/norma (data/test.jsonl, data/validation.jsonl) and reports the metrics of the paper's Table 1 (per-label accuracy, P, R, F1 and coverage) and per-position predictions for error analysis.
  • Model selection on gold data: Norma validation, which is gold-annotated, is scored at every evaluation and can drive checkpoint selection, early stopping and hyperparameter search. The test split is used only in the final evaluation.
  • Hyperparameter search: an Optuna search over learning rate and weight decay, with dropout optional.
  • Faster training input: the corpus is encoded once into flat arrays, and padding and diacritic masking are vectorized.

model.py and tokenizer.py are unchanged from the original, so checkpoints are compatible both ways.

Files

file purpose
model.py, tokenizer.py model and char-plane tokenizer (unchanged)
prepare_data.py encode the silver corpus into cache/ (fixed 90/10 split, --data_seed 0)
train.py train one model, logging silver-val and Norma-val metrics at every evaluation
hpo.py Optuna study shared by several slurm workers through a journal file
evaluate.py score checkpoints (local or Hub) on Norma, writing metrics and per-position jsonl
norma.py Norma loading and scoring
predict.py batched inference (MacronPredictor)
slurm/ jobs for the Arrhenius GH200 nodes: prepare.slurm (venv + data cache), then hpo.slurm (search)

Usage

pip install -r requirements.txt
python prepare_data.py --out cache/oga-no-norma
python train.py --cache cache/oga-no-norma --output_dir runs/seed1 --seed 1
python evaluate.py --models runs/seed1/best --splits test validation --out results/seed1

Scoring

The scored positions are the gold-marked dichrona in open syllables. In test, openness comes from the parallel syllabify lines, giving 1,916 positions. In validation, which has no syllabify lines, every gold mark is scored, giving 2,498 positions.

  • Accuracy: an unmarked position counts as wrong.
  • Per label: accuracy equals recall; precision is computed over that label's predictions.
  • Mixed (micro-averaged): recall = accuracy; precision = correct / marked positions; F1 is their harmonic mean. This reproduces Table 1.
  • Defaults-to-short accuracy: an unmarked position counts as "short". This is the metric reported on the model cards.

License

GNU GPL v3.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support