LT4HALA 2026

--Home--  --CFP--  --EvaLatin--  --EvaHan--  --EvaCun--  --Program--  --Organization--

EvaLatin

INTRODUCTION

The LT4HALA 2026 workshop will also be the venue of the forth edition of EvaLatin, the campaign totally devoted to the evaluation of NLP tools for Latin. The campaign is designed with the aim of answering two questions:

EvaLatin 2026 edition will have 2 tasks, i.e. Dependency Parsing and Named Entity Recognition.

Shared test data and an evaluation script will be provided to the participants who will choose to participate in either one or all tasks.

IMPORTANT DATES

DATA

Dependency parsing

The dependency parsing task is based on the Universal Dependencies (UD) framework. No specific training set is released but participants are free to make use of any (kind of) data/resource they consider useful for the task, including the Latin treebanks already available in the UD collection. In this regard, one of the challenges of this task is to understand which treebank (or combination of treebanks) is the most suitable to deal with new test data.

Test data will be distributed in the CoNLL-U format with gold tokenization, lemmatization, part-of-speech tagging and morphological annotation.

Named Entity Recognition

In the Named Entity Recognition task Participants are required to develop systems capable of automatically identifying and classifying proper names in provided Classical Latin texts. The goal is to detect the span of the mention and assign it to a pre-defined category. A small training set (plain text HIPE IOB format) will be made available in advance, together with the participation guidelines.

Test data will be distributed in the same HIPE IOB format with the values for the NER predictions obscured.

EVALUATION

Dependency parsing

The output .conllu file provided by the participants shall have the indications of the syntactic head and of the dependency relation in the fields 7 (HEAD) and 8 (DEPREL) respectively. We will provide an official scorer and we will evaluate dependency relations with and without subtypes separately, e.g., “advcl:abs” (ablativus absolutus/ablative absolute) and “advcl” (adverbial clause modifier), providing two separate rankings. However, the use of subtypes is not mandatory: participants who do not use subtypes will not be penalized but they will still be evaluated for dependency relations without subtypes.

Named Entity Recognition

The output IOB file that the participants will provide must have the column of the NER predictions filled in. System performance will be assessed using standard Information Retrieval metrics: Precision, Recall, and F-1 score. To account for the complexities of Latin morphology and potential ambiguity in text segmentation, two distinct evaluation settings will be employed. Two evaluation settings will be considered considered: strict (exact boundary matching) and relaxed (fuzzy boundary matching). Results will be computed using the CLEF HIPE Scorer, an evaluation toolkit designed for historical named entity recognition.

HOW TO PARTICIPATE

Participants will be required to submit their runs using specific email addresses and to provide a technical report that should include a brief description of their approach, focusing on the adopted algorithms, models and resources, a summary of their experiments, and an analysis of the obtained results.

Technical reports will be included in the proceedings as short papers: the maximum length is 4 pages (excluding references) and they should follow the LREC 2026 official format). Reports will receive a light review (we will check for the correctness of the format, the exactness of results and ranking, and overall exposition). Reports should be submitted using the START submission page of the workshop (TBA).

Participants are allowed to use any approach (e.g. from traditional machine learning algorithms to Large Language Models) and any resource (annotated and non-annotated data, embeddings): all approaches and resources are expected to be described in the systems’ reports.

Back to the Main Page