This heuristic evaluation is based on the scenario where children are playing outside in the playground with iBot.
Feedback:
iBot can give information and feedback to the children.
Scenario: iBot tracked the score of the rounders game and informed the children.
Everyday Language:
Currently iBot can only speak English. In the future, iBot will be capable of using multiple languages. In order to do this, iBot would be able to download languages when required. This would allow iBot to help children learning other languages, or communicate with a child who speaks a different language.
Scenario: When a child communicates in a different language iBot can respond.
Undo:
iBot doesn't malfunction when a minor accident occurs.
Scenario: If a rounders ball hit iBot, it would continue playing.
Consistency:
Sometimes iBot performs actions that are unpredictable. Therefore, improvements must be made so that iBot can perform consistently. In the future iBot will respond in similar ways to children’s actions.
Scenario: If a child hit the ball up in the air iBot would try and catch it.
Recognition not Recall:
As mentioned above, iBot is not consistent all of the time making it difficult for children to predict its actions. Children should be able to familiarise themselves with it's behaviour and predict its actions. The children would be able to predict iBot’s behaviour and actions in the same way as they would predict their teacher’s.
Scenario: If iBot caught a ball the child could expect iBot to catch the next ball thrown.
Simple Design:
iBot’s appearance is now robotic and available in several colours.
Scenario: The children were very enthusiastic about the robotic appearance of iBot and found it much more interesting than its previous appearance.
Expert Use:
Teachers in control of iBot are able to assign new tasks to iBot’s schedule, which then can be used in children’s lessons.
Scenario: A teacher can set iBot a new task like playing hide and seek in the playground.
Error Recovery:
iBot has an in-built recovery system to handle serious malfunctions. First the teacher is notified and then iBot connects to its server and attempts to repair itself.
Scenario: If iBot fell over while running to a base, iBot can get back up.
Documentation:
Only part of the required documentation is currently available to the teacher, but in the future detailed online documentation will be available.
Scenario: N/A
Other:
In the future children would be able to communicate with iBot after school hours, while iBot is charging. An instant messenger service could be used to enable this communication.
Farhad Bahramy
Matthew Walker
Thursday 8 March 2007
Cooperative evaluation
On making our refinements we have decided to evaluate our new design using a cooperative evaluation method in collaboration with our personas. We conducted out cooperative evaluation in an individual environment with our target age group of 5 to 9.J
This individual environment is similar to that constructed during the scenario phase the project (Posted: http://hcigroup2007.blogspot.com/2007/02/scenarios.html)
Interacting in an individual environment
During the process, we observed the personas interact with iBot to see how they would react to the Teaching Assistant.
Personas: -
Serena Sondhi
Ben Manvir
Jason Powers
Tom Green
Summary of interaction
Serena seemed excited when being introduced to the iBot in her reading tutorial. She was eager to read to the iBot to impress the iBot and to see how the robot would rate her reading skills. This attitude seems to be in contrast to our initial humanoid design, which did not receive this desired response.
Her reading tutorial, however, did make her late for her next lesson. This raised concerns regarding iBot’s synchronisation with the local school database timetable.
Ben thought it would be a good feature if he could communicate with iBot outside school hours via instant messenger.
Tom raised issues regarding iBot’s interactivity in relation to help/support.
The following questions were raised to us: -
1. Serena asked, “There was a part in the book which iBot did not understand because it was written in French. Why is this?”
We have noted this concern as iBot only supports the English language.
2. Jason asked, “Why doesn’t the iBot move its mouth when it is talking?”
At this point of time, there is a speaker inside iBot’s mouth, which outputs the sound. This issue was not raised in the previous prototype design, as it was humanoid robot.
3. Tom asked, “Why didn’t iBot help me when I was struggling through my reading?”
iBot only offers help if the child requests help. This raises issues of interactivity between iBot and its interactions.
4. Ben asked, “Why doesn’t iBot wear clothes?”
iBot’s outer shell is its main form of clothing. We questioned Ben further; he compared the iBot to Miss Dawson (the original prototype design which had clothing).
The following questions were raised to iBot: -
1. Ben asked, “What are we doing next week?”
iBot replied that information was not available at this time. This raises issues regarding iBot’s access to future scheduling information.
2. Ben asked, “Are you on Hi5 or Bebo?”
iBot questioned what Hi5 and Bebo was.
This illustrates the enthusiasm the personas had for iBot outside school hours.
3. Serena asked, “Can you help me with playing my violin?”
iBot replied that it was unable to play the violin. This issue was similar to the issue raised by Ben which relates to information / data download.
4. Tom asked, “Do you have any iBot friends?”
iBot replied no. This raised issues regarding synchronisation with other iBots to gain information in any environment.
Conclusion
From this cooperative evaluation a number of issues have been raised: -
- iBot should be more interactive in terms of helping children even when help has not been requested
- iBot should support different languages in order to cater to different children’s needs and to teach foreign languages.
- iBot should download lessons / tutorials which further expands its knowledgebase.
- iBot should synchronise with other iBots to share information / knowledge.
- iBot needs better synchronisation with the school timetable database
- Online interface to iBot through instant messaging services. Will allow children to communicate with iBot at home.
Gaurav Chander
Bobby Biran Singh
This individual environment is similar to that constructed during the scenario phase the project (Posted: http://hcigroup2007.blogspot.com/2007/02/scenarios.html)
Interacting in an individual environment
During the process, we observed the personas interact with iBot to see how they would react to the Teaching Assistant.
Personas: -
Serena Sondhi
Ben Manvir
Jason Powers
Tom Green
Summary of interaction
Serena seemed excited when being introduced to the iBot in her reading tutorial. She was eager to read to the iBot to impress the iBot and to see how the robot would rate her reading skills. This attitude seems to be in contrast to our initial humanoid design, which did not receive this desired response.
Her reading tutorial, however, did make her late for her next lesson. This raised concerns regarding iBot’s synchronisation with the local school database timetable.
Ben thought it would be a good feature if he could communicate with iBot outside school hours via instant messenger.
Tom raised issues regarding iBot’s interactivity in relation to help/support.
The following questions were raised to us: -
1. Serena asked, “There was a part in the book which iBot did not understand because it was written in French. Why is this?”
We have noted this concern as iBot only supports the English language.
2. Jason asked, “Why doesn’t the iBot move its mouth when it is talking?”
At this point of time, there is a speaker inside iBot’s mouth, which outputs the sound. This issue was not raised in the previous prototype design, as it was humanoid robot.
3. Tom asked, “Why didn’t iBot help me when I was struggling through my reading?”
iBot only offers help if the child requests help. This raises issues of interactivity between iBot and its interactions.
4. Ben asked, “Why doesn’t iBot wear clothes?”
iBot’s outer shell is its main form of clothing. We questioned Ben further; he compared the iBot to Miss Dawson (the original prototype design which had clothing).
The following questions were raised to iBot: -
1. Ben asked, “What are we doing next week?”
iBot replied that information was not available at this time. This raises issues regarding iBot’s access to future scheduling information.
2. Ben asked, “Are you on Hi5 or Bebo?”
iBot questioned what Hi5 and Bebo was.
This illustrates the enthusiasm the personas had for iBot outside school hours.
3. Serena asked, “Can you help me with playing my violin?”
iBot replied that it was unable to play the violin. This issue was similar to the issue raised by Ben which relates to information / data download.
4. Tom asked, “Do you have any iBot friends?”
iBot replied no. This raised issues regarding synchronisation with other iBots to gain information in any environment.
Conclusion
From this cooperative evaluation a number of issues have been raised: -
- iBot should be more interactive in terms of helping children even when help has not been requested
- iBot should support different languages in order to cater to different children’s needs and to teach foreign languages.
- iBot should download lessons / tutorials which further expands its knowledgebase.
- iBot should synchronise with other iBots to share information / knowledge.
- iBot needs better synchronisation with the school timetable database
- Online interface to iBot through instant messaging services. Will allow children to communicate with iBot at home.
Gaurav Chander
Bobby Biran Singh
Evaluation
At this point the refinement will be evaluated using cooperative and heuristic evaluation to discover any further refinements. The evaluations will place the prototype into one of the scenarios developed with a number of personas.
There are many types of evaluation that could have been followed:
Heuristic Evaluation – is an evaluation technique which analyses a product against a set of principles known as heuristics.
We have chosen to use heuristic evaluation due to a number of reasons:
We have chosen to use co-operative e evaluation due to a number of reasons:
Think-Aloud - “is a form of observation where the user is asked to talk through what he is doing as he is being observed; for example, describing what he believes is happening, why he takes an action, what he is trying to do”[2].
Interviews - is a technique used to gather information about the user's thoughts and preferences on particular issues. This technique is not suitable as the children may have difficulty in expressing themselves, hence, invalid results.
[1] Source
[2] Source
There are many types of evaluation that could have been followed:
Heuristic Evaluation – is an evaluation technique which analyses a product against a set of principles known as heuristics.
We have chosen to use heuristic evaluation due to a number of reasons:
- The evaluation can be based upon the personas and applied to the scenarios which is a key component of user-centred design
- The evaluation offers a quick indication into the current state of the system/product
- The evaluation can be conducted early in the development cycle
We have chosen to use co-operative e evaluation due to a number of reasons:
- User involvement at the evaluation stage.
- If any problem arises they can be clarified and solved with the users' help.
Think-Aloud - “is a form of observation where the user is asked to talk through what he is doing as he is being observed; for example, describing what he believes is happening, why he takes an action, what he is trying to do”[2].
Interviews - is a technique used to gather information about the user's thoughts and preferences on particular issues. This technique is not suitable as the children may have difficulty in expressing themselves, hence, invalid results.
[1] Source
[2] Source
