• Hardware
• Software
• Applications
• • Healthcare
  • • Objectives of the user tests
    • Participants in the tests
    • Task scenario
    • Experiment setup
    • Testing methodology and procedure
    • Data collection and analysis
    • Results
  • Emergency response
  • Maintenance
  • Production

Results

In the following, the main results of the evaluation are summarized. In a first paragraph, overall system results are presented from doctors and nurses point of view. A second paragraph steps into the findings using gestures as an interaction method.

Overall System Results
When asked if they would use a system such as represented by the prototype if it was available, the doctors were hesitant, and gave on average neutral answers (3+/5 on Likert scale). The doctors were not fully convinced that such a system would make the ward round easier and more efficient. Most likely, the scepticism can be explained by the novelty of the gesture interaction and by the fact that the gesture interaction is a central part of the prototype. Several doctors were reluctant to the idea of performing gestures in front of the patient, concerned that these would consider them crazy or strange, waving around with their hands in the air. According to the doctors, attention for interacting with the proposed system would decrease the patient contact. The doctors were only mildly thrilled (3+/5 on Likert scale) to perform extensive system training. However, several doctors could imagine using gesture interaction if the gestures were smaller and/or more elegant. Furthermore, some doctors commented that hands-free interaction was good because of hygiene reasons and that the negative implications of the gesture interaction would most probably decrease with more practice.

The possibility to view the latest patient documents electronically at all and to be able to enter the examination request, fully authorized, immediately into the system, was seen as very positive and helpful for the efficiency of work, because it will reduce unnecessary paper work (post ward round) and duplication of information. The doctors on average (4/5 on Likert scale) found the audio feedback (start/end/error dictate audio recording) very helpful. Almost all doctors agreed that speech input was a suitable method to perform the examination request (in average 4+/5 on Likert scale). This probably has its cause in the fact that the doc-tors currently dictate the examination request to the nurses who write down the information. Basically, there is no great principal difference dictating to the system instead of the nurse.

Generally, the nurses were very positive. It should be said, however, that the nurses had quite small role in the experiment and their tasks were not very extensive, although still representative. Further, the time pressure that is often present during a real ward round was not part of the experiment setting. When asked if they would use the system in this case the PDA if it was available, the nurses answered with a clear yes (4/5 on Likert scale). They felt that their ward round related work would go faster and be easier with the use of such system (4+/5 on Likert scale) connected to the hospitals backend. Today, thus, the nurses are performing several post ward round processing of information, which could disappear with the introduction of a fully functional version of the tested system. The nurses felt to have the system under control (4 /5 on Likert scale). The comments revealed that they found the interface easy to understand and the actual touch screen interaction simple to perform. According to the nurses, they would have no problem controlling the interface perfectly (meaning a 5/5 answer to the control question) with a little more training.

The contact with the patient is to all nurses extremely important. Thus, several of them are worried that the new technology would take position in between themselves and the patient, and that they would be too busy with the PDA, especially at the beginning before the interaction becomes more automatic, but to a certain extent also after the initial stage. Especially the nurses from the paediatric department thought the system, when in use, would be a change to the worse and even unacceptable. Today, the only responsibility of the paediatric nurses is to talk to the children and comfort them during the ward round. Any further tasks that would take attention away from the children are not welcome. On the other hand, some of the nurses (non-paediatrics) felt that there was no greater difference using the PDA compared to taking notes like today, and that both tasks would require about as much attention. In average, the nurses answered slightly negatively to the question if the new system could help improve the patient contact (0-/5 on Likert scale).

Gesture Interaction Results
The gesture interaction was by far the most controversial aspect of the prototype, which is not surprising considering the novelty of the concept and that none of the test participants had had experience with anything similar before. Below, the most frequently occurring issues and their potential causes are discussed. Most of the problems will most probably become less frequent when the test participants were given the possibility to practice the gestures for a longer period of time. Due to the short training period included in the experiments, most of the participants did not reach a level of proficiency where they could work comfortably. However, a certain learning effect can already be observed from the tests results.

Shape: 6/9 doctors had problems performing a gesture without striking out in the other direction first in order to get power for the movement. This often led to a situation where the opposite gesture was recognized, i.e. the cursor moved up instead of down ore a document that was meant to be opened remained closed, since the command close will have no effect on an already closed document. This issue was mainly caused by the fact, that doctors had no direct feedback, i.e. the effect of striking out is not perceived while doing it only the end-result is seen. 4/9 doctors sometimes forgot to perform the second part of the gestures, i.e. the movement back to the original position. One reason for this mistake is that doctors found the movement back less intuitive. This is because the movement goes in the direction opposite of the effect wanted. This is the case at least for the up-down gestures, i.e. in order to move the cursor up, one must move the hand up but then also back down.

Timing: 6/9 of the doctors had problems performing the gestures in the right tempo. Most frequently, they carried them out too slow, i.e. under the threshold speed defined in the gesture recognizer. However, a clear learning effect was observed from the beginning of the experiment to the end. In the initial practice phase, 5 doctors performed the gestures too slow, while during the third and last experiment task, only one doctor made this mistake. Some doctors also performed the gestures too fast to be recognized (3/9).

Conceptual: Most of the doctors at some point confused the gestures. 2/9 doctors at some point forget completely how to perform a gesture, or at least had to think for a long time before remembering it. 7/9 confused the paired gestures with one another for example open with close or activate with deactivates. After a bit of practice, however, the confusion decreased, and in the last phase only 1 doctor confused the gestures with each other. The up and down gestures were rarely confused at all.

As the most frequent problem while de-/activating the system, the doctors completely forgot the gestures, i.e. the doctors tried to interact with the system without activating it or started with non-system-related activities (like examining the patient) without deactivating. A possible reason for this is the fact that doctors did not really see the benefit of being able to move their hands freely without their movements being randomly recognized. Probably, this was because the experiment context was not realistic enough in terms of activities involving the hands. For example the test did not contain a full patient examination (most doctors only symbolically touched the dummy patient when asked to do so) or lively communication with neither nurses, colleagues, nor the patient. Consequently, doctors did not really see a need for the de-/activation action and thus forgot about it. All 9 doctors at some point in time had this issue.

In summary, the use of arm gestures was the most controversial part of the tested system. As doctors had never used such an interaction technology before many beginners mistakes occurred, like performing gestures to fast or to slow as well as confusing gestures or just forgetting how to perform a particular gesture.

Three issues regarding the gestures were found to be important for the further development of the prototype:

• Gestures have to be socially acceptable: There were several comments that doctors do not want to perform gestures in front of patients because of the fact that these might be distracting or doctors just felt stupid performing such gestures.
• Gestures should be easy and simple to perform in a way that those gestures require not to much of the doctor’s attention. This issue became apparent when doctors were asked whether gesture interaction was comfortable and making their job easier.
• There should be some sort of feedback, whether a gesture was performed right or not: As there was no direct visual feedback showing what the systems interpretation of the performed gesture was (i.e. when performing a gesture in the wrong way) doctors felt not to be in control of the system.