R4 was “in which ways can users be involved in evaluation and design of the Few Touch application to ensure its usability?” The fourth finding was the basis to answer R4.
“Usability of the Few Touch application is strongly influenced by individual user’s needs and various types of backgrounds both directly and indirectly relevant to the problem domain that the application addresses”.
As written in 2.2.1, usability is defined as the “extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [ISO 9241-11: 1998, definition 3.1]” [23]. This definition indicates the dependency of usability on individual users in terms of both their personal backgrounds related to the specified goals and specified context of use. Effectiveness is about how
accurately a user can achieve the goal by using the product. Therefore, in the case of mHealth technology, what concerns effectiveness from user perspective is mostly user’s specified goals in relation to his/her health status that the mHealth technology is used for. In this research, the goal is supporting daily self-management of T2DM. On the other hand,
efficiency is about recourses required to conduct a task by using the product. Therefore, what concerns efficiency from user perspective is mostly user’s specified context of use and user’s backgrounds that influence his/her resources to conduct a task using the product. In this research, the context of use is in user’s everyday life as a general citizen, and the backgrounds are user’s characteristics relevant to use of the application including familiarity with it.
Finding 2 was “motivation to use the application is a result of balancing between the expected effort required to use it and the expected benefit, mainly learning about user’s T2DM, by using the application”. Considering what effectiveness and efficiency mostly concern from user perspective, the factors that determine motivation to use the application are both associated with qualities explaining usability. Results in the two trials indicated actual
experiences over expectation determine user’s evaluation of usability, especially satisfaction.
Here the actual experiences mean results of balancing between obtained benefit by using the application and experienced effort to use the application.
First, the SUS scores given to Diabetes Diary version 2 did not significantly improve from those given to Diabetes Diary version1 in spite of updates of the user-interface designs to improve usability by following established design guidelines. The SUS questionnaire on Diabetes Diary version 1 was administered at Meeting 4, when already six months had passed since the start of Trial I. They experienced benefit out of using it and they were used to the design of it. On the other hand, SUS questionnaire on Diabetes Diary version 2 was
administered at Meeting 6 when Diabetes Diary version 2 was introduced. The participants tried version 2 for a short time at the meeting, but they had not experienced any benefit out of using it in their daily life yet. In addition, the two minor updates before Meeting 6 already
149
solved problems they had experienced with the Diabetes Diary version 1. Therefore, updates in version 2 might have addressed relatively minor usability issues for the participants in Trial I, and the new and thus unfamiliar user interface might not be so attractive.
Second, the evaluation regarding satisfaction with design elements of Diabetes Diary version 3 in Trial II was more severe than that of Diabetes Diary version 1 in Trial I (7.2.2). The participants in Trial I were involved in the design process of the Few Touch application. In the case of a tool to support self-management for behavior change, not only usage of a tool in real-life setting but also participation in design process involve participants’ empowerment as a patient: they are more motivated to take an active part of treatment and use of the tool as a part of it than before the participation [146]. We received feedback expressing “the Few Touch application as a learning tool until a user understands him/herself” at Meeting 4, only six months after the start of Trial I. Given this fact, the empowerment by participating in the design process would partly explain the enhanced motivation for self-management, and thereby it contributed to the motivation to use the application. Naturally, they could obtain benefit, namely learning effects, by using the application. On the other hand in Trial II, the participants were categorized into three groups and they had different level of needs for self-management and needs for the Few Touch application in their self-self-management. Therefore, it can be concluded that when seen as the whole group, obtained benefit by using the application was, at average, more in the participant-group of Trial I than that of Trial II. This caused the difference in results of usability evaluation between the groups.
Third, “experienced effort to use the application” means that the effort cannot be known before being experienced. Results from Phase 1 showed that mismatches between design concepts and reality can happen even though the same participants are involved in both design-concept making and a trial. The finding here is supported by an argument by Jensen and Larsen [177]: “some issues will not be apparent before the user has used the service for a while and incorporated it into the daily routine”. As written in 9.3.1, integration with
everyday life is one of the factors associated with usability of the Few Touch application. This concerns the context of use that varies a lot from a user to another, depending on user’s
lifestyle as a general citizen rather than people with T2DM.
The finding has three implications to answer R4.
The first is the importance of involving prospective users with various needs and backgrounds, both directly and indirectly relevant to self-management of T2DM in design process.
Regarding users’ needs and background directly relevant to self-management, it is mainly concerning their understanding about; severity of their T2DM, their motivation and needs for self-management, and their skills in self-management. As found in both trials, user needs regarding self-management differed a lot among individuals. Each user’s needs change over time as well. Tailoring intervention to the psychological status including motivation or skills of a patient has been strongly advocated and investigated in many studies
[41,43,44,47,270,271]. Therefore, mHealth technology for self-management should also be designed both flexible and robust enough to be capable of being adapted in individual care processes.
Regarding users’ needs and background indirectly relevant to self-management, it is mainly concerning familiarity with ICT-use and lifestyle as a general citizen. Factors such as age [235,236,272]and prior experience of a similar interface [236] were known to influence
150
absorption of new knowledge. In any phase of this study, we could not observe any clear influence by age on usability of the application or the prototypes. However, familiarity within ICT-use was found to be highly associated with satisfaction. As written in 9.3.1, integration with everyday life means integration with an entire everyday life, because as the result of Inquiry 1 in Phase 3 showed, users may want to use the tool whenever and wherever. In addition, although lifestyle is strongly relevant to T2DM, background of patients with T2DM could vary a lot including occupation and education. Image-recognition based interface was suggested as a design that can be used by people with low literacy [208]. However, the findings in the pilot usability testing implied that image-recognition based interface is not necessarily perceived as easy by people with high literacy, because they are much more used to reading or recognizing text. In addition, while List View type of user interaction design was a design that most of the participants in the pilot usability testing were used to, Food Browser design was unfamiliar as both search and comparison on a mobile phone. The design principle of “match between system and real world” here well apply if the “real world” means what users are used to in the context of ICT use. A significant change in ICT system must be significantly better [220]. The Food Browser was not inferior to List View in terms of effectiveness, represented by error rate and task completion rate, and efficiency, represented by task completion time, but it was so in terms of satisfaction. Given that the test tasks are of no interest for the healthy volunteer testers, experienced effort to carry out the task, which was not limited to only time as their resource, was major contributor to satisfaction. As
Krug’s second law of usability [62] says, “it doesn’t matter how many times I have to click, as long as each click is a mindless, unambiguous choice”, the unfamiliar and new user interface might have been the major factor that deteriorated satisfaction.
Karapanos et al. [273] studied users’ engagement with iPhone over time. They argued that a tool engaging many users over long time has “designs that are specific enough to address one single need, but flexible enough to enable the artful appropriation in diverse contexts”. Design of a personal-use based mHealth technology for self-management of T2DM involving target users can be carried out efficiently and effectively when the users’ lifestyle, familiarity with ICT-use, needs, motivation and skills regarding self-management are diverse. In the case of the Few Touch application, the participants were not selected based on such condition but their back grounds were diverse, as confirmed by results of Trial I. The developed tool
resulted in focusing on simple and essential functions for people with T2DM. The Few Touch application was generally found to be easy to use in both trials. In other words, no-use of the application due to anxiety or fear of using the application was not observed, which was the case in other studies [274,275]. As a consequence, the developed tool was considered useful to a greater or lesser extent. A challenge here would be how to incorporate diverse user needs that might be incompatible. Introduction of the notion of “design for me” and “design for us”
might help the participants understand their role clearly and give constructive opinions on the design process [276].
The second implication is the value of pilot usability testing by people without needs for self-management of T2DM to identify usability flaws to fix before testing by target users.
We recruited healthy volunteers for the pilot usability testing for ethical reasons as written in 1.3.3. Healthy volunteers basically don’t have goals to achieve by using the subject artifact, thereby, there is not any benefit out of using it. On the other hand, they are also general citizens that may have some common backgrounds with people with T2DM as general citizens, especially in terms of familiarity with ICT-use. Therefore, effort required to conduct tasks and accompanying satisfaction level are the sole contributors to usability. This implies that pilot usability testing involving healthy volunteers would efficiently and effectively
151
identify usability flaws, when tasks are very simple with clear goals. A challenge here would be specification of tasks. In UCD, it is important to focus on tasks and users throughout the process [277]. When involving healthy volunteers, researchers need to explain well about background of the design. It is also essential to make simple- and fundamental-navigation based tasks that they easily understand. Success of pilot usability testing depends on a lot of factors for test design. People must also check wording of tasks and user interface for testing with preferably different backgrounds for their comprehensibility and error-proneness.
The last implication is the importance of inclusion of a long-term testing of resulted design by people who participated in design process.
As we argued in Paper 2 and 3, I would advocate the importance of involving the same participants until a long-term testing before testing on other target users. Because of their empowerment and feeling of ownership, they are motivated to try the designed tool [278].
They will therefore keep using it for a certain period at high probability. In order to find out if the implemented design concepts work out well to achieve their goals in their context of use, namely in their everyday life, the tool needs to be used for a certain period. During this period, they would notice any mismatches and problems that stem from design or specification. In our experience from Trial I and Trial II, participants’ feedback to design was quite consistent in both positive and negative ways. It is important to obtain feedback about any problems stemming from the design at an early stage before even users get used to them or lose interest in further engagement.