UNDER ARMOUR COACHING CHATBOT
An emerging idea within Under Armour's Connected Fitness division was to provide coaching feedback to the users of our running and nutrition apps—MapMyRun and MyFitnessPal. To validate the potential of in-app coaching, I built a chatbot and used it to test ideas with real users. The chatbot helped us understand how best to design for our users. The principles we discovered continue to be used to implement coaching features in Under Armour's suite of apps today. Below is the article I wrote on the Under Armour Maker's blog detailing the project and learnings.
My Role
As a masters student at Carnegie Mellon University, we had used SMS texting as a research tool to test the design of push notifications. To explore in-app coaching at Under Armour, I suggested we employ a similar process. I built upon this idea with our exercise scientist to build a realistic coaching experience, first using Apple iMessage, and later with an automated SMS chatbot. Myself and our exercise scientist collaborately closely on research planning, user recruitment, and user interviews. I built the chatbot, designed the visuals, wrote the blog article, and presented our final findings with our team to Under Armour's Connected Fitness division.
Prototyping Design Ideas with Chatbots
May 9, 2018 — 7 minutes
Original article published on Under Armour Maker's Blog here.
At Under Amour®, we’re committed to our mission: Under Armour makes you better — not just through our gear but also through our fitness apps. As software designers, it’s our job to craft digital solutions that give our users what they need to be better.
How do we know what people need? Asking users directly doesn’t always get accurate answers. Human beings can often be poor judges of their own behavior. As anthropologist Margaret Mead put it: “What people say, what people do, and what people say they do are entirely different things.”
A more fruitful tactic for learning about customer needs is to make ideas tangible. When we build working prototypes, we can let our users interact with them. Rather than asking users what they would do, we can observe what they actually do. However, building high-fidelity prototypes can take a lot of time. As part of a recent research study, we came up with an idea to speed up the prototyping timeline: we used a chatbot to prototype our ideas. What a visual prototype might uncover after months of development, a chatbot can discover in weeks or even days.
Chatbot intelligence can range from choose-your-own-adventure style decision trees to more sophisticated Natural Language Processing.
A brief overview of chatbots.
A chatbot is any technology that mimics the back-and-forth communication of humans. Its interface is referred to as “conversational UI.” Today, we communicate with computers like two people speaking different languages. One person gestures or points to a picture while the other person deciphers the action. With computers and smartphones, we click on a menu item or tap an icon to signal what we want to do. The technology behind conversational UI aims to let people talk to computer naturally, as they would to other humans.
Chatbots range in intelligence from simple to very advanced. The simplest of chatbots are powered by decision-tree logic, whereas the most advanced chatbots like Siri and Alexa use natural language processing, or NLP. Chatbots are commonly used to streamline user interactions such as answering FAQs.
Chatbots aren’t exactly a conventional design prototyping tool. To test with chatbots, the designer turns visual interactions into conversational interactions. Rather than designing screens, the designer crafts a conversation.
Testing coaching via text messaging.
To experiment with fitness coaching in our apps, we built a chatbot that coached users via text messaging. We began by asking our study participants onboarding questions such as, “How often do you exercise?”, “What are your goals?” and “What kind of foods do you like to eat?”. Once we understood what our participants were trying to accomplish, we began coaching them on their fitness, nutrition, and sleep.
At first, we used SMS texting via a computer to coach our users. A designer sat at the computer and texted our participants while pretending to be the chatbot.
Before building an automated chatbot, we posed as the chatbot and sent texts directly to our participants.
We started with nutrition recommendations: participants in our study could text a command such as “POST” to get a post-workout meal suggestions. The designer-as-chatbot would reply as quickly as possible. However, sometimes users waited hours for a response. This is one of the limitations of chatbot prototyping. The designer needs to be on-call almost 24/7.
We designed meals cards in Sketch and texted them to our participants as part of their nutritional plans.
Later on, we discovered a tool called Dexter that helped us automate our texting chatbot. We could now manage all of our conversations through Slack. We could now schedule texts to be dispatched to all participants at the same time, similar to a push notification. We also created auto-replies to our commands such as “POST,” and to our participants’ questions. With our chatbot automated, we were able to experiment with all sorts of recommendations.
Rich data through conversation.
As we learned more about our participants’ goals and challenges, we came to understand which recommendations were most engaging. We asked participants to text the bot what they ate, how they slept, and what kind of workouts they were doing. In return, participants received individualized coaching from our chatbot.
We selected MyFitnessPal blog articles that were relevant to each participant. The bot sent these articles to each participant with suggestions on what the participant could do to improve.
We experimented with texting recipe ideas, links to MyFitnessPal® blog articles, workout plans, and motivational messages. We learned that context, time of day, and even mood can affect whether the participant considered the content helpful. By the end of our experiments, we not only had a better idea of what our users might need, but also which recommendations tend to be most helpful to them.
Chatbots: A new type of scratch paper.
Our bot was an incredibly valuable tool because we learned how our ideas worked in the real world. Popular design tools such as Invision mimic how a user transitions from screen to screen, and this can be helpful when we are testing flows and app usability, but when we’re digging for the core of our users’ needs, we need tools that help us test whether our screens are solving a real user problem.
Benefits of using chatbots:
- Chatbots embrace mobility – Users can interact with them as they go about their daily lives, similar to a mobile app.
- Chatbots elicit raw and immediate feedback – One of the most satisfying aspects of using chatbots is getting off-the-cuff responses from users. Users are able to text the chatbot back, voicing their frustrations or delight. Users can ask the chatbot questions which can tell us what information our users are seeking.
- Chatbots are cheap, fast, and easy to get up and running – What takes some time is planning the conversation. However, if the conversation isn’t working, it’s easy to change.
Adding the chatbot to the toolbox.
The chatbot isn’t the only method for rapidly testing ideas, but it is a tool that should be included in every designer’s toolbox. Chatbots are great for testing what type of information is valuable and when. It’s one thing to ask users how they feel about a meal recommendation when they are sitting in our office being interviewed. It’s quite another to send a recipe recommendation to a busy mom with a stroller after a run on the trail. A chatbot can ask her, “Was this recommendation helpful?” and get an off-the-cuff response. If the user ignores the text, well, that says something too. Chatbots are not, however, great for testing complex tasks that cannot be translated into a conversation—other tools are more appropriate for testing things like visual elements, interaction design, and usability.
Despite their limited functionality, chatbots can help glean information that later informs interaction and visual design. When used early enough in the product development lifecycle, chatbots can course correct and validate ideas quickly before a pixel has been drawn or a line of code has been written.
Tips for using chatbots.
- Don’t try to get it right the first time. You’re not designing screens, you’re designing a conversation – Just get it out there and iterate on the fly like you would steer a conversation.
- Test, test, then test again – Test internally, send texts to yourself, and test with different cellphone providers as network delays may affect how your sequence of messages appear in the conversation.
- You won’t prevent every error, but your errors may lead to insights – No matter how well you plan.
- The more rounds you do of testing, the better you get at using the tools – When it comes to chatbots, this is half the effort.
- The easiest way to learn how to design conversational user interfaces is to learn by doing – Find something tedious that you can translate into a conversation, such as registering to vote or RSVP-ing to a wedding. A good way to learn is to pick a use case and start playing!
- They’re fun and easy to learn!
Tools for building chatbots.
- Twine – Twine is a web tool for writing choose-your-own-adventure-stories. This tool is also useful for crafting the decision-tree of a chatbot conversation.
- iMessage – iMessage is a great way for a designer to get started with chatbots. With iMessage, the designer pretends to be a real chatbot. When quickly responding to users becomes a challenge, it may be time to use Dexter.
- Dexter – Dexter is a robust yet easy-to-learn tool for building chatbots.
- Slack – Slack can be integrated with Dexter so that conversations can be manned in one simple interface. If using Dexter with more than a few users it is highly recommended to integrate with Slack.
Credits
Senior Creative Director
Marcus Piña
Senior Design Manager
Eric Boam
Lead Exercise Scientist
Jeff Knight
Contact
Email — hello@katiesawaya.com