Midas: A Declarative Multi-Touch Interaction Framework
Multi-touch technology allows users to use their hands to manipulate digital information. We have observed that mainstream software frameworks do not offer support to deal with the complexity of these new devices. Current multi-touch frameworks only provide a narrow range of hardcoded functionality. Therefore, the development of new multi-touch gestures and their integration with other gestures is notoriously hard. The main goal of the Midas framework is to provide developers adequate software engineering abstractions to close the gap between the evolution in the multi-touch technology and software detection mechanisms.
Current frameworks force the programmer into an event driven programming model where the programmer has to register and compose event handlers manually. This results in applications where the control flow of the application is driven by external events and no longer by the sequential structure of the program. The reuse, composition and understanding of program code gets difficult when using such frameworks.
In this work, we propose a solution based on research conducted in the complex event processing domain. We advocate the use of a rule language which allows programmers to express gestures in a declarative way. The advantage of such an approach is that the programmer no longer needs to be concerned about how to derive gestures but only about describing the gesture. We present a first step in that direction in the form of a domain-specific language supporting spatio-temporal operators.
Complex gestures which are extremely hard to be implemented in traditional approaches can be expressed in one or multiple rules which are easy to understand. The use of a rule language has the benefit that the developed gestures are reusable and easy to compose. Further, a strong connection to application-level entities allows developers to activate and deactivate gestures depending on their graphical context.
Engineering Gestures for Multimodal User Interfaces,
Florian Echtler, Dietrich Kammer, Davy Vanacken, Lode Hoste and Beat Signer,
Proceedings of EICS 2014, 6th International Conference on Engineering Interactive Computing Systems,
Rome, Italy, June, 2014
Midas: A Declarative Multi-Touch Interaction Framework,
Christophe Scholiers, Lode Hoste, Beat Signer and Wolfgang De Meuter,
Proceedings of TEI 2011, 5th International Conference on Tangible, Embedded and Embodied Interaction,
Funchal, Portugal, January 2011 (acceptance rate: 32%)
Available:
Mudra: A Unified Multimodal Interaction Framework,
Lode Hoste, Bruno Dumas and Beat Signer,
Proceedings of ICMI 2011, 13th International Conference on Multimodal Interaction,
Alicante, Spain, November 2011 (acceptance rate: 39%)
