VOXReality | Gerasimos (Jerry) Spanakis

VOXReality’s goal is to conduct research and develop new AI models to drive future XR interactive experiences, and to deliver these models to the wider European market. These new models will address human-to-human interaction in unidirectional (theatre) and bidirectional (conference) settings, as well as human-to-machine interaction by building the next generation of personal assistants.

VOXReality will develop large-scale self-supervised models that will be fine-tuned to specific downstream tasks with minimal re-training. At the same time, we will rely on modern training approaches for designing models that include subnetworks with a common representation power but are more targeted towards specific architectures. By leveraging the once-model-for-all concept from the model training (large-scale self-supervision) and deployment (jointly learning sub-networks) perspective, we will be able to provide a catalogue of highly generic models with high representation capacity that will be efficiently specialized for downstream tasks.

UM (DACS) is the scientific coordinator of the project and our team is responsible for developing state-of-the-art automatic speech recongition (ASR) and neural machine translation (NMT) models with emphasis on non-native speech and the efficient use of context.

Team members are:

Dr. Jerry Spanakis (co-PI)
Dr. Konstantia Zarkogianni (co-PI)
Dr. Yusuf Can Semerci (scientific and technical coordinator)
Abderrahmane Issam (PhD candidate)
Pawel Maka (PhD candidate)

(Maka et al., 2024) (Issam et al., 2024) (missing reference)

References

2024

Sequence Shortening for Context-Aware Machine Translation

Paweł Maka, Yusuf Semerci, Jan Scholtes, and Gerasimos Spanakis

In Findings of the Association for Computational Linguistics: EACL 2024, Mar 2024

HTML
Fixed and Adaptive Simultaneous Machine Translation Strategies Using Adapters

Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, and Gerasimos Spanakis

In Proceedings of the 21st International Conference on Spoken Language Translation (IWSLT 2024), Aug 2024

Abs DOI HTML

Simultaneous machine translation aims at solving the task of real-time translation by starting to translate before consuming the full input, which poses challenges in terms of balancing quality and latency of the translation. The wait-k policy offers a solution by starting to translate after consuming words, where the choice of the number k directly affects the latency and quality. In applications where we seek to keep the choice over latency and quality at inference, the wait-k policy obliges us to train more than one model. In this paper, we address the challenge of building one model that can fulfil multiple latency levels and we achieve this by introducing lightweight adapter modules into the decoder. The adapters are trained to be specialized for different wait-k values and compared to other techniques they offer more flexibility to allow for reaping the benefits of parameter sharing and minimizing interference. Additionally, we show that by combining with an adaptive strategy, we can further improve the results. Experiments on two language directions show that our method outperforms or competes with other strong baselines on most latency values.