Cooperative evaluation

On making our refinements we have decided to evaluate our new design using a cooperative evaluation method in collaboration with our personas. We conducted out cooperative evaluation in an individual environment with our target age group of 5 to 9.J

This individual environment is similar to that constructed during the scenario phase the project (Posted: http://hcigroup2007.blogspot.com/2007/02/scenarios.html)

Interacting in an individual environment
During the process, we observed the personas interact with iBot to see how they would react to the Teaching Assistant.

Personas: -
Serena Sondhi
Ben Manvir
Jason Powers
Tom Green


Summary of interaction
Serena seemed excited when being introduced to the iBot in her reading tutorial. She was eager to read to the iBot to impress the iBot and to see how the robot would rate her reading skills. This attitude seems to be in contrast to our initial humanoid design, which did not receive this desired response.

Her reading tutorial, however, did make her late for her next lesson. This raised concerns regarding iBot’s synchronisation with the local school database timetable.

Ben thought it would be a good feature if he could communicate with iBot outside school hours via instant messenger.

Tom raised issues regarding iBot’s interactivity in relation to help/support.

The following questions were raised to us: -

1. Serena asked, “There was a part in the book which iBot did not understand because it was written in French. Why is this?”

We have noted this concern as iBot only supports the English language.

2. Jason asked, “Why doesn’t the iBot move its mouth when it is talking?”

At this point of time, there is a speaker inside iBot’s mouth, which outputs the sound. This issue was not raised in the previous prototype design, as it was humanoid robot.

3. Tom asked, “Why didn’t iBot help me when I was struggling through my reading?”

iBot only offers help if the child requests help. This raises issues of interactivity between iBot and its interactions.

4. Ben asked, “Why doesn’t iBot wear clothes?”

iBot’s outer shell is its main form of clothing. We questioned Ben further; he compared the iBot to Miss Dawson (the original prototype design which had clothing).


The following questions were raised to iBot: -

1. Ben asked, “What are we doing next week?”

iBot replied that information was not available at this time. This raises issues regarding iBot’s access to future scheduling information.


2. Ben asked, “Are you on Hi5 or Bebo?”

iBot questioned what Hi5 and Bebo was.

This illustrates the enthusiasm the personas had for iBot outside school hours.


3. Serena asked, “Can you help me with playing my violin?”

iBot replied that it was unable to play the violin. This issue was similar to the issue raised by Ben which relates to information / data download.


4. Tom asked, “Do you have any iBot friends?”

iBot replied no. This raised issues regarding synchronisation with other iBots to gain information in any environment.


Conclusion
From this cooperative evaluation a number of issues have been raised: -

- iBot should be more interactive in terms of helping children even when help has not been requested

- iBot should support different languages in order to cater to different children’s needs and to teach foreign languages.

- iBot should download lessons / tutorials which further expands its knowledgebase.

- iBot should synchronise with other iBots to share information / knowledge.

- iBot needs better synchronisation with the school timetable database

- Online interface to iBot through instant messaging services. Will allow children to communicate with iBot at home.

Gaurav Chander
Bobby Biran Singh