Thanks to our sponsor, Informaat, last meetup was held at Restaurant LE:EN: A creatively designed Asian restaurant in Utrecht. After a short and solid story of the expertise and background of Informaat, we kicked off with our first speaker: Anke Sesink.
Anke did her graduation assignment for a particularly challenging target group: the profound intellectual disabilities (ID), for example users with a cognitive developmental age from half a year to 5 years, but in a more grown up body. Their motor skills are mostly well developed, but their social emotional development is lacking. This results in users that get overstimulated really quickly. So how do you calm down these over-stimulated clients? . Research showed that physical activity benefits the clients, so the three prototypes that Anke tested were based on this:
- A floorprojection
- A floorprojection with a fysical object
- A fysical object (pillow with led-lights)
She developed all the protoypes by herself. The pillow with led-lights was hand produced and the floorprojection was coded in an app. Testing was a challenge: finding the right users and to get permission to work with them wasn’t as much possible as she would have liked. After the testing phase, Anke continued in developing a final design of one of the three concepts: the interactive floor projection with no physical objects that can be destructed by the clients.
The preferred projections can be created by the caretaker for each individual client in a web portal, and can be controlled and reviewed in an app. The advantage of this approach was that the lead caretaker could customize the use of the app for his or her client, so the tool could be used at the level appropriate to each particular user/client. The settings can be saved in a user profile, so a caretaker can apply the app for multiple users/clients. It was really cool to see that Anke’s solution really has effect on the over-stimulated clients. Unfortunately, due to funding issues, it was not possible to keep developing Anke’s solution and to maintain a viable product. For more info on the assignment, please take a look at Anke’s portfolio.
The next speaker was Henk Westerhof, an experienced Information Architect. Henk shared with us his experience of working on a chatbot for insurance company CZ. One of the biggest problems he faced was that all possible questions they could think of had to be entered into the system manually; even the stupid questions like this:
Question: intelligent test
Answer: Test 1,2, 3. Yes I do work you know.
But they also had to enter all the sensitive questions they could think of, for example in case of a death of a client, you don’t want to answer a question like ‘what should we do now a family member has deceased and is client of you’ with something painfull as ‘you want to have a new insurance?’.
Since this was a pilot, these were the steps to improve the pilot to put to real use in the future:
Step 1: pump up the recognition rate. The max was 85%, whatever they did their goal of 90% wasn’t reachable. It wasn’t good enough and this is a big problem.
Step 2: make better answers. Puppy learning takes too much time (and money)
Step 3: optimize…. optimize…. optimize… finishing a dialog got pumped up too, from nil to almost 60%.
Step 4: conclude. Results of this pilot were not very good. In spite of all the effort put in. You cannot do without (some) AI to constantly improve you chatbot.
One of the worst decisions they made was to start the chatbot with stating ‘you can ask me anything you want’. That’s just not feasible at all, especially in insurance land. So make really clear that the client is talking to a bot and not to a person!
After this short introduction, we were ready to make our own chatbot with an online service called Converse.ai. In groups, we first had to come up with an answer and then think of all questions you could ask that would result in that answer, like Henk explained in his introduction. For example:
Answer: “We are Ladies that UX”
Questions: “Who are you?”, “What is this?”, “what group is this?”, “What are you?”
After this, we could already test the chatbot. It was a bit hard for some of us to find where to actually start the conversation with the chatbot, since there were 3 input fields and only one was correct… (hint: bottom right)
Then we got the hang of it and we could test the chatbot!
To match even more phrases, the next step was to create a context where we could add all kinds of synonyms.
But somehow it did not always work:
This part was already a bit confusing for some of the workshoppees, so we after this we ended the evening with a few announcements: We created a new facebook page (please like it here) and usability testing at Night of the Nerds: they are looking for usability testers or observers on may 16th in Eindhoven! If you want to join, please contact us at firstname.lastname@example.org.
For an impression of the evening, take a look at our Flickr page.