Evaluation Proposal

In answering “What could be a more effective evaluation method?”, the researcher pulled valuable insights from the literature review and formed new evaluation guidelines. These guidelines are specifically for designers and developers where the top portion acts as a pre-test check, and the bottom flow chart is meant to be used during a user testing session. The evaluation method was sketched out based on the information acquired from the literature review. It was first illustrated on a whiteboard, then taken to a paper prototype, and then refined in Figma.

Figure 6: Sketches of Evaluation Prototypes

The evaluation method proposed is a tool for designers to use and learn from to improve their knowledge of haptics. The evaluation proposal is divided into two sections: Examine and Evaluate. The ‘Examine’

section is a checklist for designers to perform before evaluating their haptic feedback with a participant.

The ‘Evaluate’ section is for designers to use when performing a user testing session with a coworker or anyone they can get a hold of. The evaluation method can be found on the next page.

Figure 7: Evaluation Method

4.2.3.1 Examine

The ‘Examine’ section of the evaluation proposal is divided into two sections: communication and accessibility. The two sections have five questions in total for designers. To better understand the evaluation proposal, an explanation of each step is as follows:

Communication

1. Does the haptic have a clear relationship with its trigger?

Making sure that the user is aware of the cause and effect relationship with the haptic feedback is essential. Feedback is used to explain information to the user which indicates haptic feedback should be used when a message is meant for the user. If the user completes and action and receives feedback based on the action and is unable to associate the feedback with the action, the feedback is ineffective.

The relationship between the trigger and the haptic should feel natural and should not be experience too often. This is important for designers to be aware of as haptics are

2. Is there another form of feedback (audio, visual) paired with the haptic feedback?

Haptic interfaces are almost always multimodal. This question is to confirm that the designer has visual or audio feedback in tandem with haptic feedback. If the phone starts buzzing randomly without visual or audio feedback, the experience becomes unnatural. The user is able to learn more through multimodal feedback and is able interpret it in different environments.

3. Does this haptic feedback add to the user’s experience?

Haptics should only be used to improve the user’s experience. If they do not improve the experience for the user’s, then there is not point adding haptic feedback into the mobile application. Haptic feedback should provide long lasting value so that the user will be able to associate certain action with a specific type of haptic feedback.

Accessibility

1. Can the user turn the haptic feedback off?

Having the ability to turn haptic feedback off gives the user more control. Everyone uses their phone different and has different conditions when using it, giving control gives the user more freedom. More freedom makes it more likely that they will keep using your app in the future. The haptic feedback is not the main feature of the app and should be able to enjoy the application without the haptic feedback.

2. Is the haptic being used consistently?

When building relationships between the haptic and their trigger, they should always be consistent.

Consistency is how we build mental models and are able to make associations. Designers are normally familiar with associations and building mental models, but it should always be checked. A success action should always trigger the same haptic. If this is not the case, then the haptic is not being used consistently.

4.2.3.2 Evaluate

In the ‘Evaluate’ section of the evaluation proposal, the researcher created a flow chart based on Android source’s subjective assessment. The flow chart is aimed for designers to use when performing a user test.

The flow chart can act as a loose script when user testing. The flow chart evaluation consists of three yes or no questions with one conditional step that could be added.

The flow chart evaluation begins in the user test session after the participant has completed a task on the application that they are testing. After the participant has completed the task with the haptic feedback, the evaluator then asks the question in the first step, “Did you clearly feel the haptic impulse?” The

participant then has the option to answer yes or no. Depending on either answer, the flow chart will lead the evaluator to the next step. If the participant answers no to feeling the haptic impulse, it leads the evaluator to reevaluate the haptic and increase the amplitude.

If the participant’s answer is yes to feeling the haptic impulse, then the evaluator is able to move on to the next step. The next step in the evaluation proposal is to ask the participant if they understood what the haptic feedback was trying to communicate. As with the previous question, if the participant were to answer no, then the evaluator would be led to reevaluate, but in this instance, the feedback method should be reevaluated. This is because the participant says they cannot understand what the feedback is trying to communicate. For this question, there is a conditional step. The conditional step is to aid in the

confirmation of the combination of the haptic feedback. The conditional step is an exercise that can be done with the participant to confirm that the feeling of the haptic feedback was what the evaluator or designer was trying to convey. The exercise would consist of the evaluator providing the participant with a series of adjectives. The participant would have to select the adjective they believe relates the most to the received vibration pattern. The researcher does not recommend more than three adjectives. If the participant does not select the correct adjective related to the feedback received, the evaluator can then ask the participant to explain their answer and justify.

Figure 8: Example of the conditional step in the ‘evaluate’ section

After completing the conditional step, the evaluator then would ask the participant, “Do you think the haptic feedback is necessary for the user experience?” If the participant answers yes, then the assessment is complete. However, if the participant answers no, then the evaluator is led to reevaluate the use case or the need for adding haptics into the mobile application. After the evaluation has been completed, the designer can give the developer to implement the haptic feedback in the app.