Haptic Displays Used for Exploration of Works of Art at Museums
6. Evaluation within a CAVE-like environment
A CAVE-like room (CAVE Automatic Virtual Environment [10]), i.e. a room with floor and 3-4 walls back-projected with a stereoscopic visualization of a virtual computer-generated scenario, was used for the experiments. The tests were conducted within the ReaCToR at the Department of Computer Science, University College London. The PURE-FORM exoskeleton haptic display was installed. Visitors could navigate through a virtual museum, select a virtual works of art from the Petrie Museum or OPAE and inter-act with it (Figure 6). In total 2 statues were chosen to be made available during the tests.
c The Eurographics Association 2005.
Massimo Bergamasco / Application, Cultural heritage
Figure 6: The interaction with statues during the simulation in the CAVE-like environment.
6.1. Methods
A main part of the evaluation concerned co-location of stereo graphics and haptic display information. A group of 6 computer-literate adults (50% male, 50% female) were invited to take part to the experiment. A simple task was set, the users being asked to explore the shape of two stat-ues wearing the exoskeleton haptic device respectively with force feedback enabled and disabled. Each task took lasted five minutes and statues were presented in different orders.
After the completion of the exploration, the users were asked to complete a questionnaire with 30 questions, of which 7 dealt with the sense of touch and 9 with the sense of pres-ence. Two other additional questions dealt with the user’s observation of the environment.
The answers were given on the basis of a 1 to 7 (Likert) scale.
6.2. Results and discussion
The analysis of results, even if based on a limited sample, evidenced a significant difference between the means of an-swers related to the sense of touch for the condition with and without haptic feedback (Mann-Whitney U test, p 0.001).
All the scores in the touch-related set of questions were above the midpoint of the scale for the haptics on condi-tion. For instance Question 8 ("Did you feel you were touch-ing the statue") scored a mean of 4.4 with SD = 1,12 vs. 1 with SD = 0, while Question 11 ("To what extent could you feel the different curves and shapes of the facial features?") reaches a mean of 5 with SD = 1.1 vs. 1 with SD = 0, re-spectively for the two conditions of haptics on and haptics off.
No significant differences were observed in the set of questions related either to the observation of the environ-ment and to overall sense of presence, even if higher scores where obtained in questions related to presence and a higher
number of errors were committed in the description of the environment in the condition with haptics on.
Informal observations [2] indicated that co-location was very important to enhance the experience, but it was also found that some participants had problems to visualize a 3D stereo object and focused on the front wall instead of the 3D position of the stereo model. This led to a discrepancy between visual and haptic information and loss of the stereo effect. A planned solution was to place the haptic interaction as close as possible to the projection wall.
6.3. The exhibition at CGAC
The PURE-FORM exhibitions in Santiago de Compostela and Stockholm had much in common both in the general ar-rangements and in the questionnaires presented. However, there were also differences and the evaluations are partly presented separately. A more detailed report of both eval-uations was given in [2].
Methods The participants had to choose two statues for exploration among seven available ones. The questionnaire was presented, in Spanish, Galician and English versions, and collected from 127 visitors, 69 women and 58 men with a mean age of 28 and 31 years, respectively, in both cases SD
= 9 years. A majority (56 All statistical evaluation was car-ried out using non-parametric tests. Correlation factors were computed using the Spearman rank test, while comparison of means was made through Mann-Whitney U statistics.
6.4. The exhibition at NM
Methods Two statues were used in the evaluation, Saint John by Andrea Pisano. and the "The thorn extractor" by an un-known artist ( marble copy of a Roman bronze sculpture), and all visitors explored both, with the order of presentation varied between them. The exploration time was between 3 and 28 min (M = 8 min, SD = 3 min). The questionnaires (presented in Swedish and English versions) were given af-ter the explorationand filled in individually by the visitors in their own pace outside the exhibition area.
Usable questionnaires were obtained from 115 sighted visitors (Mean age = 33 years, Range = 13 - 71, SD = 14 years). About two thirds were men, and nearly the same pro-portion had a university degree. Only a minority (13 %) had earlier experience of virtual reality or similar installations.
Two-thirds reported never or rarely playing computer games, while the self-reported experience of computers had a mean of 5.5 on a 7-degree scale.
The questionnaires from six blind visitors will be ana-lyzed, as well as those from 13 visitors who took part when only one of the two fingers of the display was functioning.
6.5. Results
6.5.1. Haptic experience of specific parts of the statues.
The questions on this subject were typically of the following form: "To what extent could you feel the shape of X" with the answer given on a 7-point scale. The specific parts con-cerned at NM hair, beard, face, breast, back and clothes and the means varied between 3.8 and 5.3 with SDs between 1.5 and 2.0. At CGAC the features were pressure and volume, curves and shapes of the facial features, and the shape of the dress and body features with means between 4.1 and 5.1 and SDs between 1.6 and 1.8.
In sum, all the means except one were above the mid-dle point (4) of the scale. The exception was the back of a sculpture that apparently was more difficult than the other features; the visitor had to explore from behind.
6.5.2. The feeling of touching a statue.
A general question at both museums about the exploration with the haptic display was "Did you feel that you were touching the statue?" with an answer on a 7-point scale. The means of the answers for the four statues at the two museums varied between 4.2 and 5.0 and the SDs between 1.7 and 1.9, thus again over the middle point of the scale.
6.5.3. Factor analysis of questionnaire answers at CGAC.
A factor analysis of a selection of answers at CGAC was undertaken in order to identify some main factors underlying the visitor experience and determining the final judgment of the overall exhibition (Table 1, Table 2; for details, see [2]).
The questions can be grouped in several sets and reduced in number according to the results from the factor analysis that was conducted through a Principal Component Analysis (Varimax with Kaiser Normalization).
This division in sets led to the identification of six fac-tors that were able to account for the 75% of the variability observed among subjects’ answers. The expression of the 6 factors is reported in the tables below, together with the re-lationship of each factor with the items that compose it.
The above 6 factors allows to carry out the analysis of results on a reduced set of variables and to better highlight the relationships among them.
The most important factors found were related to Haptic experience (F1), Device usability (F2) and Added value of haptic experience (F3) as detailed in Table 1.
Factor F1 mainly refers to the ability of persons in getting a realistic feeling of touch in the VE, while F2 represents the usability and satisfaction of usage of the device, and F3 the added value that visitors perceived from the experience in the VR. Factors F4, F5 and F6 are instead related to the visitors profile.
F1: Haptic experience
Q5 Were you able to feel the volume?
Q6 Did you feel you were touching the sculpture?
Q10 To what extent could you feel the shape of X?
Q11 To what extent were you able to perceive the specific parts of the statue X, i.e. facial features?
F2: Device usability and satisfaction Q13 It was easy to learn to use the device Q14 I felt comfortable using the device Q15 Overall, I am satisfied with the device F3: Added value of haptic experience
Q8 Does the feeling of touch improve your experience?
Q16 How much do you improve your knowledge about arts?
F4: Visitor’s age related profile
Q18 How would you describe you visit to the museum of pure form?
Q21 Do you play videogame?
AGE Age of the visitor F5: Previous VR Experience
Q19 Have you had previous experience of virtual reality?
F6: Familiarity with pc
Q4 Did you need to ask more information?
Q20 Please rate your degree of familiarity with computers Table 1: List of the principal components identified from the factor analysis
Table 2: Composition of principal components as derived from factor analysis
c The Eurographics Association 2005.
Massimo Bergamasco / Application, Cultural heritage
Table 3: Correlation between principal factors, age and Q22 (Spearman rank test)
6.5.4. General Judgement.
their general opinion on the system (Q22: "Write your gen-eral opinion about the Museum of Pure Form"). In CGAC answers, the general opinion Q22 that persons have ex-pressed was positively correlated with F1 and F3, but with F2, and negatively correlated with age (Table 3). The evalu-ation results provided a strong indicevalu-ation that the final judg-ment of the system was expressed on the basis of the quality and added value of the haptic experience; moreover younger people expressed in general more positive comments. Ques-tions related to the usability of the device (F2) did not pro-vide any significant influence on Q22.
6.5.5. Importance of haptic, in addition to visual, experience.
At NM for both statues the visitors were asked to judge if touching the statue added anything to the visual experience.
The Ms of the answers were 4.8 and 4.9, respectively, on the 7-degree scale with SDs of 1.8.
The analysis of Q16 ("How much do you improve your knowledge about arts?") in CGAC answers highlighted the elements that almost provided an additional value to the ex-perience in VR with respect to a traditional one. A significant correlation was found with answers to Q5, Q6, Q8, Q10 and Q11 that are all related to the haptic experience (F1). This indicates that the haptic experience by itself was considered as an added value.
6.5.6. The usability of the haptic display.
Three questions at both museums concerned general aspects on the usefulness of the display:
1. It was easy to learn to use the device (Q13).
2. I felt comfortable using the device (Q14).
3. Overall, I am satisfied with the device (Q15).
The answers were given on a 7-point scale from "strongly disagree" (1) to "strongly agree" (7). The result is given in Table 4.
Table 4: Results of usability questions at the two museums)
From the analysis of answers of CGAC test, it is found that the answer to Q15 was significantly correlated with those to questions Q5 (R=-0.294, p=0.001), Q6, (R=-0.256, p 0.004 (1), R=-0.230, p 0.02 (2)) and with Q13 and Q14, too. This means that in general the user’s satisfaction was strictly dependent on their haptic experience.
The wish of finding similar installation (Q17: "Would you like to find similar installations?") was found to be strongly dependent on the overall satisfaction with the system (Q15, R=-0.259, p=0.004) and the overall learning from the expe-rience (Q16, R=0.199, p=0.027).
6.5.7. Amusement/ instructiveness of the experience.
At CGAC 70 % of the visitors answered that they found the experience amusing and 39 % that they found it instructive.
At NM the answer was given on a 7-point scale with 1 as low judgment and 7 as high judgment. The experience was evaluated on average as amusing (mean score 6.2 with SD 1.0) and instructive (mean score 5.6 with SD 1.2).
6.5.8. Suggestion and wishes from visitors.
At CGAC 96 % of the visitors reported that they wanted sim-ilar devices at other museum. At NM the answers were given on a 7-point scale as in the earlier questions. On average vis-itors expressed their willingness of suggesting the visit to friends (mean score 5.7 with SD 1.5) and wanting similar devices (mean score 5.7 with SD 1.6).
6.5.9. Effect of age.
Age influenced the general opinion expressed on the system, as it was found to negatively correlated with general judg-ment (Q22) in answers at CGAC.
Moreover the visitor profile (factors F4, F5 and F6) was dependent on age, as it is shown from the correlation factors in Table 2.
The habit of playing video-games was negatively corre-lated to the age, and this determined also the level of amuse-ment that was derived from the experience (Q18). A posi-tive correlation was observed between the habit of playing videogames Q21 (that was measured on a 7 points scale) and Q8 (R=0.202 N=125 p<0.025), meaning that people ac-quainted with interactive pc games better appreciated the usefulness of the haptic experience.
Familiarity with VR systems and with PCs was instead positively correlated with age. People that were more famil-iar with PCs needed also to ask additional information on the system.
However there was no significant relationship between the level of familiarity with computers and virtual reality, and the scores reported in the other evaluation categories.
6.5.10. Effect of training.
In the CGAC exhibit two sculptures were randomly presented to the visitors according to the preferences that they expressed. This allowed us to compare the answers re-lated to the haptic experience of the first and second sculp-ture, and to estimate the effect of training in the interac-tion with haptic device. It is evident the effect of training for getting acquainted with the application: if we compare the values of Q6 (1 and 2) and Q11 (1 and 2): there is a significant difference (Mann-Withney test) of the means of Q6 (p<0.0001) and Q11 (p<0.005) for the first and the sec-ond sculpture, but the difference is not significant for Q10.
Greater scores were achieved during the exploration of sec-ond sculpture, thus indicating an improvement in performing with the device (see Figure 7).
6.5.11. One finger or two fingers used.
In real life more than one finger is used at the same time in most tasks. That number of exploring fingers is important has been shown experimentally concerning identification of objects [3].
The results were much improved already when the num-ber was increased to two. However, the size of the contact area is also important [4][5]. Only increasing the number of contact areas has been demonstrated to be insufficient [6].
This was also shown in a study of the performance of 13 visitors at the exhibition at NM that had information from only one finger available (because of temporary failure of the second finger of the device). There answers were not significantly different from the answers of the visitors with information from two fingers.
6.5.12. Visually impaired visitors.
Visually impaired people, especially those with total loss of vision, have severe problems of accessing information from 2D pictures, even if much effort has been devoted to get them well functioning [7].
One of the main obstacles is the difficulty of haptically getting 3D information from such pictures, and it has been suggested that haptic display may be a partial solution [8][9].
It was planned to include visually impaired visitors in the evaluation at NM, but, unfortunately, it was possible to get only six participants. They are also older than the sighted visitors (M = 47, Range = 29-59). The answers from such a small group may, however, provide indications of a result.
Figure 7: Effect of training during the interaction with two subsequent sculptures.
For most of the questions the answers for the visually im-paired are roughly the same as those for the sighted, but there are tendencies for lower reported experience of having felt the statues and satisfaction with the display. The importance for the experience of the statue was judged at Nationalmu-seum to be somewhat lower for the visually impaired (M = 4.0) than the added value for the sighted (M = 4.9), which is against the expectations of a higher value for the visually impaired.
The suggestions for improvements are about the same as those given by the sighted. However, one additional sugges-tion was given by the blind: complesugges-tion with a verbal de-scription.
The last suggestion is probably a most important one for making the haptic display useful for the visually impaired.
The display provides a direct access to 3D aspects of the work of art and is thereby an important improvement in comparison with 2D pictures. However, a major problem re-mains: to get a rapid overview. Verbal information may con-tribute to a solution of this problem. This information may be given in an introduction including, among other things, sug-gestions about suitable exploratory procedures. Verbal infor-mation may also be given when specific parts of work of art is touched, similar to the information provided at tactile maps on a touch tablet [14].
6.5.13. The importance of training.
It was shown above that already the experience of one statue provided greater values for the next one. More experience can be expected to provide still greater values, as it has been demonstrated experimentally [11] that a considerable im-provement can be expected after a few hours of training to use the device. It was not suitable in the present context to ar-range with long training sessions and long exploration times
c The Eurographics Association 2005.
Massimo Bergamasco / Application, Cultural heritage with the display, but such arrangements would probably have
improved both self-rated and measured performance.