Evaluation of Grammatical Annotation Tools for Slovenian

Compare commonly used Slovenian grammatical annotation tools on manually annotated test sets. Select a tool and test set to view detailed per-label results.
Tool
Test set
Details

Overall Accuracy —

Dependency relation scores

Part-of-speech tag scores

Most common dependency parsing errors

This table shows the most frequent dependency-relation errors made by the selected tool, compared with the manually annotated test set. Click a row to see examples; use ↑/↓ to navigate and Esc to close the example view.

Most common part-of-speech tagging errors

This table shows the most frequent UPOS tagging errors made by the selected tool, compared with the manually annotated test set. Click a row to see examples.
Error pair Error count

Most common MULTEXT-East tagging errors

This table shows the most frequent MULTEXT-East / JOS morphosyntactic tagging errors made by the selected tool, compared with the manually annotated test set. Hover over or focus a row to see the MSD tag decoded; click a row to see examples.
Error pair Error count

Most common lemmatisation errors

This table shows the most frequent lemmatisation errors made by the selected tool, compared with the manually annotated test set. Click a row to see examples.
Error pair Error count

Acknowledgement

This work was partially supported by the research projects SPOT, LLM4DH and MAPCASE (ARIS grant nos. Z6-4617, GC-002 and J6-70213), with infrastructural support from the Centre for Language Resources and Technologies at the University of Ljubljana (CJVT UL) and the Common Language Resources and Technology Infrastructure, Slovenia (CLARIN.SI). The interactive website was developed by Nives Hüll under the supervision of Kaja Dobrovoljc Zor.

Recommended citation

If you use this evaluation table, please cite:

Dobrovoljc, Kaja. 2025. “Treebanking Spoken Slovenian: New Data, Models, and Lessons Learned.” Contributions to Contemporary History 65(3). https://doi.org/10.51663/pnz.65.3.01

BibTeX 
@article{dobrovoljc2025treebanking,
  author  = {Dobrovoljc, Kaja},
  title   = {Treebanking Spoken Slovenian: New Data, Models, and Lessons Learned},
  journal = {Contributions to Contemporary History},
  year    = {2025},
  volume  = {65},
  number  = {3},
  doi     = {10.51663/pnz.65.3.01}
}
ARIS — Slovenian Research and Innovation Agency CJVT UL — Centre for Language Resources and Technologies CLARIN.SI