Unlike Phases 1 and 2, Phase 3 is about the very early stage of a design process until the first working prototype was made and pilot tested.
The design process basically followed the HCD process defined in ISO13407 [23].
First, inquires were made to “understand and specify the context of use” and “specify the user requirements” [23]. “Producing design solutions” [23] followed. As a part of this design activity “producing design solutions”, working prototypes were made for a pilot test to find usability flaws.
At this early stage of design process, results of data collection and analysis influence the next step to take in terms of concrete methods to take for both data collection and analysis. In order to enhance readability by keeping the flow, I will describe concrete methods and materials as well as corresponding results in Chapter 8 and will introduce methods employed and the rationale in this section. Overview of research and design activities is described in 1.3.2.2.
43
5.2.1 Initial requirement identification
Inquiries 1-3 were made to the participants in Trial I. Inquiry 4 was made to the two project leaders of Lifestyle project [1].
At Inquiry 1, a custom-made questionnaire was administered to collect user needs regarding methods for access to information relevant to T2DM and situations in which they would need the information. Four questions were made to ask the participants’ needs regarding
information function of the Few Touch application (APPENDIX 5). Questions were made in the multiple-choice format, for two questions with an option to add free-text comment as well.
Intention of using multiple-choice style was to investigate whether or not user needs that only a few participants had explicitly expressed were actually common user needs. Therefore, the items in each question reflected feedback we obtained in Trial I. The results were simply summarized into a table.
At Inquiry 2, card sorting technique was used to understand users’ mental model about organization of information and concepts in order to design information architecture enabling an efficient access to information [58,220]. This technique was used to redesign structure of the information function of the Diabetes Diary. In this particular case, the attention was paid to keep the “Few Touch” principle: to organize information so that users don’t need to scroll the page. The intention was thus to know: what types of categories the users expect to find at the menu page (the top level) of the information function; what types of information they expect to see immediately under each category (the second level); and what types of
information they would like to see or read in case they are interested in details of the topic by pressing “more info” button (the third level).
An open card sorting method [58] was employed with a modification so that the participants could make their own grouping of information and hierarchy in each group.
Results of card sorting were analyzed using Hierarchical Cluster (HC) analysis and Multi-Dimensional Scaling (MDS) [58]. Although I employed hierarchical structure for each group in this card sorting, it is still non-parametric and difficult to quantify. Thus a distance between cards was defined as follows: a distance between cards in a same group is 0, and a distance between cards in different groups is 1, regardless of a level in which each card was placed at.
For HC analysis and MDS, I used the summed data of results by all the participants. Statistic software program R was used for calculation and plotting.
At Inquiry 3, paper prototyping technique was used as a basis to gather ideas and preference on information to display in a detail view of a food item as a part of food-information
database module of the information function. Hand-drawn sketches were made in the actual size of the display so that the participants could visualize how reasonable the size of text and the amount of information would be [221]. Hand-drawn sketches were used instead of realistic graphics by following the strategy taken by Årsand et al. to the same participants [222]. A visual analogue scale (VAS) was used for rating of each design idea. This scale is typically using a 10 cm-long straight continuous line with two verbal descriptors at the extremes on each edge [223]. An answerer is expected to make a cross on the line where s/he considers most appropriately describing what s/he perceives about the issue in question. VAS has been historically used as a valid measure of intensity of pain [224–226], but there are some attempts to use it for evaluation of perception of object in HCI field [223,227]. The study by Van Shaik and Ling [223] concluded that 7-point Likert scale and VAS are equally good for psychometric online questionnaire. Because the participants in Trial I mentioned that
44
it is difficult to decide on which score to rate at 7-point Likert scale, for example either 5 or 6, I decided to use VAS instead.
To compare the scores given to the four designs, Kruskal-Wallis one way analysis of variance by ranks was used. As a post-hoc test of Kruskal-Wallis, I used Bonferroni correction [228]
on the results of Wilcoxon test on each pair of samples. Bonferroni correction is criticized for its main weakness explained as “the interpretation of a finding depends on the number of other tests performed” [228]. The same author [228] also states that “a final situation in which Bonferroni adjustments may be acceptable is when searching for significant associations without pre-established hypotheses”. At Inquiry 3, we did not have any pre-established hypothesis. Considering this fact and the relatively small number of total Wilcoxon test (all combination of four designs generates six pairs), we employed Bonferroni correction. Details of each test method can be found for example in a book by Wohlin et al. [229]. Software program R was used for calculation.
Focus group sessions were carried out after the participants completed the questionnaire to hear their opinions about suggested design alternatives and their needs for food-relevant information. The sessions were led by the project leaders of Lifestyle project [1], but I also asked supplemental questions accordingly. Collating audio records, their opinions written in the free form on the questionnaire, and the notes taken during the focus group sessions, their needs and opinions were categorized and summarized.
Inquiry 4 Based on findings from Inquiries 1-3 and results of relevant literature, an inquiry was made to gather requirements to design the food-information database module as an education tool. The topic was concentrated around user interaction design and usage scenario to design tasks at usability testing. Inquiry was made to the two project leaders of Lifestyle project [1] as both domain experts and stakeholders of the project. Both two leaders had been diagnosed with T1DM for more than 10 years at the time of 2010, and they were very skilled in terms of self-management. For approximately 10 years, they had been engaged in research projects focusing on patients with diabetes (both T1DM and T2DM) and people caring them. As patients with T1DM, they were also users of the Few Touch application. In all senses that they were skilled patients with T1DM, users, and research project leaders, they were regarded as domain experts. Due to an exploratory nature of the aim of this inquiry, unstructured individual interview was administered. Notes were taken and the main issues of interest were summarized, following the manner introduced by Sharp et al. [57].
5.2.2 Concept design 5.2.2.1 Conceptual model
Johnson and Henderson [230] define conceptual model as “a high-level description of how a system is organized and operates”. In the interactive system context, it shows an “idealized view of the how the system works” [230].
I developed conceptual models of user-interaction designs for functions to search and compare food items in a food-information database module by specifying and describing the followings:
The major design metaphor (and analogy, if any)
The concepts (task-domain data objects, attributes and operations)
The relationships between concepts
45
The mapping between the concepts and the task-domain I followed the framework that Johnson and Henderson suggested [230].
5.2.2.2 Prototyping for presentation
Houde and Hill [231] define three types of prototypes according to the focus:
Role: referring to “questions about the function that an artifact serves in a user’s life – the way in which it is useful to them”
Look and feel: denoting “questions about the concrete sensory experience of using an artifact – what the user looks at, feels and hears while using it”
Implementation: referring to “questions about the techniques and components through which an artifact performs its function” – “how it actually works”
Here artifact is defined as “the interactive system being designed” [231]. Prototypes may explore design space with a single focus or more foci.
To gather opinions to the design concepts of a food-information database module from the participants in Trial I, a role prototype of the design concept with partial focus on look and feel was made and presented. It was a low-fidelity role prototype [231] with a moderate resolution. A variety of animation functions of Microsoft Office PowerPoint 2007® (PPT) were utilized to help the participants concretely visualize the design concepts for user interaction.
Focus group sessions with semi-structured interview were held after the presentation of design concepts to the participants in Trial I. Notes was taken and the sessions were audio recorded. Audio records were listened and main issues of interest were summarized.
5.2.3 Prototyping for pilot usability testing
Two interactive prototypes were made based on HTML and JavaScript for the pilot usability testing described below. For these prototypes, combination of “role” and “implementation”
with partial focus on “look and feel” was used due to the aim of the testing with focus on testing fundamental functions. One was to implement the user-interaction design concepts.
The other is to implement more traditional user interaction designs, i.e., text- and number- based design with hierarchical data structure using a list for the purpose of comparison.
5.2.4 Pilot usability testing
A pilot usability testing was administered to find usability flaws of the proposed design concepts as well as to examine advantages and disadvantages of it with regard to fundamental functions.
Usability testing is normally conducted in a controlled environment unlike field testing such as long-term testing described above. The purpose is typically to measure performance of certain tasks and to elicit users’ opinions to make them as basis for improvement of the design [57].
Pilot study is run to make sure the proposed methods are viable before going into the main study. This enables avoiding wasting resources and time of the participants as well as
46
avoiding frustrations of the participants. Therefore, when we found any flaws in methods, we made changes in methods accordingly. In pilot study, convenient samples such as colleagues or peers are asked to participate, if access to the real target users is limited [57].
As data collection methods; pre- and post-test questionnaires, automatic data recording of task completion time and task answers by implemented program in prototypes, post-test semi-structured interview were used.
For questionnaires, both custom-made questionnaires and off-the-shelf questionnaires were used.
Pre-test questionnaire was to ask participants’ demographic information (gender and age-bracket [20<40, 40<60, 60+]) and the previous experiences relevant to the tested prototypes.
Post-test questionnaires (APPENDIX 6) included a custom-made questionnaire made to ask participant’s preference on tested prototypes in terms of tested functions and as a whole system. It also asked reasons for preferences. Off-the-shelf questionnaires, SUS and pragmatic quality dimension of AttrakDiff™ [60], were also included.
AttrakDiff™ [60] “is an instrument for measuring the attractiveness of interactive products”.
Leuteritz et al. showed that AttrakDiff™ is a good measure of satisfaction of a system [232].
Questionnaire comprises of many items using semantic differential scale divided into seven points with range of -3 to 3. An answerer is expected to place a cross at the point which s/he thinks describes best the product or system to evaluate. Four dimension; “pragmatic quality”,
“hedonic quality – stimulation”, “hedonic quality – identity” and “attractiveness” can be evaluated in AttrakDiff™. Due to the focus on functionality, only pragmatic quality
dimension was used at the usability testing in Phase 3. This decision is supported by the study by Büring et al. [233] which evaluated functionality of zoomable user interfaces (ZUI) on a hand-held device using scatter plot concept to search films. The following shows the word pairs used to evaluate pragmatic quality dimension (left: -3, right: +3).
Technical – Human
Complicated – Simple
Impractical – Practical
Cumbersome – Straightforward
Unpredictable – Predictable
Confusing – clearly structured
Unruly – Manageable
Non-parametric tests based on ranks were used to compare; scores of SUS and pragmatic quality dimension of AttrakDiff™ given to the two prototypes, and task completion time by the two prototypes. Regarding the scores given to the questionnaires, the reasons for choice of non-parametric methods are described in 5.3. Regarding direct measure, such as task
completion time, normality could not be assumed in distribution of samples because many different reasons for time-loss were observed. When samples are repeated measures, such as scores of questionnaires, Wilcoxon test was used. On the other hand, when measurements in two samples could not be paired, Mann-Whitney test was employed. Such cases are typically when a participant completed a task with one prototype but did not with the other.
47