Objectives of the user tests

The findings of the experiments were manifold. Generally, the usability test with the real end-users was rather complicated, as they had to be picked out from the running assembly line. Therefore, the time schedule for usability testing was very tight, making it impossible to extend the tests when required. On the other hand, it was not possible to recruit the amount of users needed in order to obtain statistically significant results. Dealing with real end-users, it was not feasible to try to apply a user-centred approach through a human translator. In fact, all end-users were unable to understand and speak English, which required the services of a professional Czech/English translator. These constraints prevented the usage of some effective techniques like ‘thinking aloud’, thus, interaction with the workers turned out to be unnatural and difficult. Additionally, it had to be dealt with the fact that the design/developing team consisted of remotely located partners, what made it very complicated to guarantee a real iterative design process. Apart from these aspects, it was difficult to measure system performance. As a result, it was not easy to quantitatively evaluate the training process. Due to the “learning effects”, it was not legitimate to use the same worker to measure the same training process before and after the introduction of wearable technology. Furthermore, it was difficult to compare two different users because of their different skills and learning capabilities.
Nevertheless, the first results of the experiment confirmed that the wearable system was well accepted. Regarding all theoretical concerns about lack of privacy and loss in autonomy, it was a surprise that one of the favourite features of the workers was the ability of the wearable system to monitor the task completion. In fact, when one of the workers made a mistake in the assembling process, the system detected it, and triggered an error message. Later on, during the post-questionnaire, it was one of the most valuable features the worker identified. It has to be underlined as well, that video support was not requested by the workers; on the contrary, they preferred pictures with aggregated information in comparison to simple text.
The main outcome of the Vrachlabi Usability Experiments was the decision to perform new usability experiments in Spain. The purpose was to involve a larger group of end-users located near the research team in order to overcome the constraints experienced at Skoda: difficulty for the recruitment of users located in the production line, language constraints and geographic distance which doesn’t makes economically feasible a long-time research.
Two different experiments were carried out:

• The aim of the first experiment was to extend the initial findings of the experiment made with the workers of Skoda at Vrachlabi: to measure the acceptance of the system, the performance in terms of memo ability (how fast workers get trained), and in terms of task completion (time consumed and errors made).
• The second experiment was aimed to compare the benefits of using Head Mounted Displays (HMDs) to access information, versus the presentation of the same information on a large screen near the working place.