The Slovenian Language Technologies Society (SDJT), the Centre for Language Resources and Technologies at the University of Ljubljana (CJVT), and the research infrastructures CLARIN.SI (Jožef Stefan Institute) and DARIAH-SI (Institute of Contemporary History and ZRC SAZU) organized the biennial international conference “Language Technologies and Digital Humanities.” The conference boasts a tradition spanning over 20 years, with its thematic scope expanded to include digital humanities in 2016. The Institute of Contemporary History (INZ) managed the organizing committee for the 2026 edition, and the conference took place at the Faculty of Computer and Information Science at the University of Ljubljana.
As part of the event, a panel titled “Benchmarking Large Language Models for Slovenian: Methods, Challenges and Perspectives” took place. Participants included Dr. Simon Krek, Dr. Špela Vintar, and Dr. Nikola Ljubešić. The discussion was moderated by Dr. Ajda Pretnar Žagar. The central theme of the discussion was evaluation tests, which are used to assess how well language models perform specific tasks and understand various domains.
Most evaluation tests are Anglocentric, much like a large portion of language models, which are developed using English-language data. This is reflected in the models’ knowledge, values, biases, and understanding of cultural context. The panelists agreed on the need for tests tailored to individual languages and their specific cultural and social nuances.
Large language models require continuous assessment, and such tests must become an integral part of language infrastructure. In this process, it is crucial to be clear about our own values—and for the tests to reflect those values.
