Llamipa: An Incremental Discourse Parser

Llamipa is Llama3-8B finetuned on the Minecraft Structured Dialogue Corpus (MSDC) https://huggingface.co/datasets/linagora/MinecraftStructuredDialogueCorpus.

	Link F1	Link+Rel F1
Llamipa + gold structure	0.9004	0.8154
Llamipa + predicted structure (incremental)	0.8830	0.7951

For a given speaker turn, Llamipa was trained to predict the discourse relations which connect the elementary units of the turn to the units of the previous dialogue turns, given the text of the previous dialogue turns and the previous discourse structure, or the relations that connect those turns. For training, the gold annotated structure was used. The model was then tested using gold structure, and gave state of the art results on the MSDC (see above table). However, for a discourse parser to be truly incremental, it should be able to predict the relations for each new turn using the structure it predicted in previous steps. We tested the model using its predicted structure and found the results were robust to this change.

Model Description

Language(s) (NLP): English
Finetuned from model: Llama3-8B

Running Llamipa

Training from scratch

The training data are provided in the \data folder. They contain a maximum context window of 15 elementary units (EDUs). For training parameters see the paper cited below.

Reproducing test results

The \model folder contains the adapters for the parser trained on Llama3-8B, as well as the scripts for generating structures using both gold (parse_gold.py) and predicted structure (parse_incremental.py). Be sure to use either the gold or incremental version of the test data, found in \data.

Using Llamipa on new data

In order to re-generate the Llamipa data from the original MSDC files, or to format new data to be parsed using Llamipa, we provide data formatting scripts and instructions in the \bespoke folder.

Evaluation

Get F1 scores using \evaluation\evaluation.py, and produce a friendlier version of Llamipa output using \evaluation\output_formatter.py.

Citations

Paper: https://aclanthology.org/2024.findings-emnlp.373/

Video: https://www.youtube.com/watch?v=yerUotx3QZY

Please cite the EMNLP Findings paper if you use Llamipa in your work:

@inproceedings{thompson-etal-2024-llamipa,
    title = "Llamipa: An Incremental Discourse Parser",
    author = "Thompson, Kate  and
      Chaturvedi, Akshay  and
      Hunter, Julie  and
      Asher, Nicholas",
    editor = "Al-Onaizan, Yaser  and
      Bansal, Mohit  and
      Chen, Yun-Nung",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2024",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.findings-emnlp.373/",
    doi = "10.18653/v1/2024.findings-emnlp.373",
    pages = "6418--6430"
}

Acknowledgements

We acknowledge support from the National Interdisciplinary Artificial Intelligence Institute, ANITI (Artificial and Natural Intelligence Toulouse Institute), funded by the French ‘Investing for the Future–PIA3’ program under the Grant agreement ANR-19-PI3A-000. We also thank the ANR project COCOBOTS (ANR-21-FAI2-0005), the ANR/DGA project DISCUTER (ANR21-ASIA-0005), and the COCOPIL “Graine” project funded by the Région Occitanie of France. This work was granted access to the HPC resources of CALMIP supercomputing center under the allocation 2016-P23060.

linagora
/

Llamipa