Center for Artificial Intelligence and Machine Learning along with The Data Science Research Unit of the Institute of Information Systems Engineering are organising a hackathon focused on LLM hallucinations scheduled on January 11th, 2024. You can register here.

About the Event

The modern Natural Language Generation (NLG) landscape is plagued by two interlinked problems: On the one hand, our current neural models have a propensity to produce inaccurate but fluent outputs; on the other hand, our metrics are most apt at describing fluency, rather than correctness. This leads neural networks to “hallucinate”, i.e., produce fluent but incorrect outputs that we currently struggle to detect automatically. For many NLG applications, the correctness of an output is however mission critical. For instance, producing a plausible-sounding translation that is inconsistent with the source text puts in jeopardy the usefulness of a machine translation pipeline. With the following hackathon, we hope to foster the growing interest in this topic in the community.

The hackathon is aligned with the SemEval-2024 Task-6 - SHROOM shared task, coordinated by the Helsinki NLP group of the University of Helsinki.

The hackathon provides a great opportunity to delve deeper into the realm of LLMs if you are passionate about NLP and machine learning. The primary objective of the shared task is a binary classification for detecting hallucinations in LLMs. The dataset required for the hackathon is readily available, and you can access it through the download link on the shared task webpage.

Participation in the hackathon is open to everyone, regardless of affiliation with TU Wien or student status. All you need to bring is your laptop and a positive attitude. :) Food and beverages will be provided.

The hackathon will commence on January 11th, 2024, at 2 p.m. and is expected to conclude around 11 p.m.

You are strongly encouraged to brainstorm ideas related to the SHROOM shared task before the event. While it’s beneficial to explore the data beforehand, it is not mandatory. During the hackathon, we plan to form groups of approximately 5 participants each, based on the goals we aim to achieve.

Additionally, there is a possibility to submit your solutions as part of the SemEval-2024 Task-6 - SHROOM shared task and even work on a publication. Engaging in such a shared task provides a valuable experience on how current research in NLP works, both in academia and industry.

For any inquiries, we urge you to reach out to the organizers’ team. We look forward to your participation!

The hackathon will take place in the contact room, Gußhausstraße 25 - 29, 1040 Wien, 6.OG.

Registration

You can use the following link to register: https://events.htu.at/events/a1503813-8ecc-4619-8cd9-d4af68ac4345