7.1 Trial II
8.1.2 Inquiry 2
Inquiry 2 was done to understand users’ mental model of information architecture so that the results could be used to redesign structure of the information function of the Diabetes Diary.
Card sorting technique was used (5.2.1). Fifteen cards were prepared as follows together with blank cards where the participants could make their own cards or duplicate other cards (APPENDIX 5, p.4).
a. Food
b. Physical activity c. Disease
d. Diabetes in general e. Blood glucose
f. Information to show others (e.g. acute information, foods that are not recommended to eat)
g. Picture of a food item in an amount that contains 10 gram of carbohydrates h. Glycaemic Index (GI)
i. Amount of carbohydrates in a normal portion of a food item j. Nutrition contents of a food item
k. Reference book about diabetes l. Quiz about diabetes
96 m. Bookmarks
n. List of items in alphabetical order
o. Search by word with manual typing/writing p. Others – write your own category on a Post-it™
Contents of the cards could be divided into: categories for the updated version of the
information (tips) function of Diabetes Diary version 1 (Figure 6.12) (cards “a”-“e”); topics or types of information reflecting feedback from the participants (cards “f”-“j”); contents that were suggested after a discussion in the Lifestyle project team (cards “k” and “l”); and options to enable quick access to information (cards “m”-“o”). The answer sheet31 had a matrix with four columns and three rows. Each row represented the level (the top, the second, and the third, as described above). The participants were instructed to choose at maximum four cards to place at each cell of the top row. The rest of the cards were expected to be placed at the second or the third level, but it was not necessary to use all the cards.
As same as for Inquiry 1, 11 out of the 12 participants in Trial I carried out the card sorting at Meeting 5. The results are summarized into the tables shown in APPENDIX 11.
The cards were sorted into either three (by five participants) or four (by six participants) groups (APPENDIX 11, Results of Inquiry 2, Table 1). Ten out of 11 participants used “a.
Food” and nine used “b. Physical activity” while “c. Disease”, “d. Diabetes in general” and “e.
Blood glucose” were used moderately and evenly (each by five participants). All the 11 participants used at least one of these three cards (“c”-“e”) for the top level, and four participants used two of them for the top level. This illustrates that the information
represented by the names of the cards “c”-“e” are regarded as important as food and physical activity, but not regarded as independent as food and physical activity. Thus, it is plausible to combine them into one category.
While information relevant to foods were clearly sorted under card “a”, not any clear
tendency was observed for the information sorted under cards “b”-“e” (APPENDIX 11, Results of Inquiry 2, Table 2).
Only one participant (P12) used card “p. Others – write your own category on a Post-it™” to write suggestions. P12 gave several concrete suggestions in a questionnaire held in previous user-meetings as well. And there was only one participant (P05) who duplicated a card (card
“m. Bookmarks”). P05 used card “m” at the third level of all the groups whose top levels were: card “a”, “b”, and “e”.
The result of HC analysis (Figure 8.1) well confirms the results described above. “Height” on the Figure 8.1 expresses dissimilarity. The higher a pair of cards meet, the less similar the participants on average regarded they were.
31 The answer sheet for Inquiry 2 was distributed separately from the questionnaire sheet. This is not attached as appendix.
97
Figure 8.1 Result of a hierarchical cluster (HC) analysis of the summed data of all the card sorting results using “average” method option (left) and “complete” method option (right) Although the two cluster dendrograms show slightly different results in terms of the heights of meeting points of cards or clusters, both show that there are three major clusters, each of which consists of same cards in both dendrograms. Considering the three major clusters, match between the grouping by HC analysis and the grouping by the participants was at average 79.7% (SD: 10.9, Range: 60.0% - 97.1%). When dividing participants into two groups depending on the number of groups they sorted cards into (e.g, either three groups and four groups, not considering “unnecessary” as a group) and compare these two groups of participants, the average of match is better for the participants who sorted cards into three groups (84.6%, SD: 7.7%, Range: 69.5% - 97.1%) than four groups (75.7%, SD: 10.0%, Range: 60.0% - 84.8%), but this difference was not statistically significant by Students’ t-test32.
Figure 8.2 shows result of a MDS analysis of summed distance matrices for all the
participants. How well an MDS plot reflects the original data is measured by a stress value:
the smaller, the better; and according to Tullis and Albert [58], “a good rule of thumb is that stress values under 10% are excellent, whereas stress values above 20% are poor”. The stress value for Figure 8.2 was 10.8%, which is satisfactory. The plot illustrates relative distance between each other. The axes don’t mean anything special, and the plot area can be rotated.
Figure 8.2 illustrates the contrast between similarity among cards “a”, “f-j” and dissimilarity among the other cards. This result therefore implies a rather low consistency among the participants when it comes to grouping of the prepared cards.
32 Shapiro-Wilk normality test could not discard the null-hypothesis that samples come from a normally distributed population for both groups. F-test could not discard the null-hypothesis that variances between the two groups are not different. Therefore, Student’s t-test was applied with assumption of equal variances.
98
Figure 8.2 Result of a Multidimentional Scaling (MDS) analysis
Lessons Learned Against my anticipation, cards “k”-“o” were not used for the top level except one case that P07 used “n. List of items in alphabetical order”. On the contrary, card “n.
List of items in alphabetical order” and “m. Bookmarks” were regarded as unnecessary by three and two participants, respectively. The reason for this might be because of unfamiliarity with them: these had never been used as a part of the information (tips) function while cards a-e showed the categories used in the updated version of the function. Another phenomenon which needs to be noted is that card “o. Search by word with manual typing/writing” was placed at the second level under card “b. Physical activity” by five participants. This seemed quite odd to me, but the Norwegian wording “Skrive inn manuelt fysisk form33”, of which
“fysisk form” means “physical fitness”, is a sound reason for the participants to
misunderstand what card “o” meant. In order to avoid leading the participants to choose certain cards that I (and the research project team) expected them to use for the top category, I did not use any concrete examples such as illustration-based mockups to show images of possible redesigns of user interface for the information function. However, these results above well illustrate that only oral explanation and the wordings on the cards might not have been explanatory enough.
One participant (P09) used cards “h. Glycaemic Index (GI)”, “i. Amount of carbohydrates in a normal portion of a food item”, “j. Nutrition contents of a food item” for the top level in addition to card e”. (APPENDIX 11, Results of Inquiry 2, Table 3) Given the fact that P09 explicitly expressed his/her wish for information represented by cards “h”-“j”, it is
understandable that s/he used them at the top level. However, considering the cards used for the second and third level of each group, it is very questionable if s/he understood the concept of the card sorting. This question is applicable to two other participants (P01 and P03),
33 This wording was made by a native Norwegian-speaking researcher as a modification of my suggestion “skriv inn form”. “Form” in Norwegian mean fitness. There must have been a miscommunication between me and him.
99
although the cards they chose for the top levels are consistent with most of the other participants.