UNDER ARMOUR COACHING CHATBOT
An emerging idea within Under Armour's Connected Fitness division was to provide coaching feedback to the users of our running and nutrition apps—MapMyRun and MyFitnessPal. To validate the potential of in-app coaching, I built a chatbot and used it to test ideas with real users. The chatbot helped us understand how best to design for our users. The principles we discovered continue to be used to implement coaching features in Under Armour's suite of apps today. Below is the article I wrote on the Under Armour Maker's blog detailing the project and learnings.
My Role
As a masters student at Carnegie Mellon University, we had used SMS texting as a research tool to test the design of push notifications. To explore in-app coaching at Under Armour, I suggested we employ a similar process. I built upon this idea with our exercise scientist to build a realistic coaching experience, first using Apple iMessage, and later with an automated SMS chatbot. Myself and our exercise scientist collaborately closely on research planning, user recruitment, and user interviews. I built the chatbot, designed the visuals, wrote the blog article, and presented our final findings with our team to Under Armour's Connected Fitness division.
Prototyping Design Ideas with Chatbots
May 9, 2018 — 7 minutes
Original article published on Under Armour Maker's Blog here.
At Under Amour®, we’re committed to our mission: Under Armour makes you better — not just through our gear but also through our fitness apps. As software designers, it’s our job to craft digital solutions that give our users what they need to be better.
How do we know what people need? Asking users directly doesn’t always get accurate answers. Human beings can often be poor judges of their own behavior. As anthropologist Margaret Mead put it: “What people say, what people do, and what people say they do are entirely different things.”
A more fruitful tactic for learning about customer needs is to make ideas tangible. When we build working prototypes, we can let our users interact with them. Rather than asking users what they would do, we can observe what they actually do. However, building high-fidelity prototypes can take a lot of time. As part of a recent research study, we came up with an idea to speed up the prototyping timeline: we used a chatbot to prototype our ideas. What a visual prototype might uncover after months of development, a chatbot can discover in weeks or even days.
Chatbot intelligence can range from choose-your-own-adventure style decision trees to more sophisticated Natural Language Processing.
A chatbot is any technology that mimics the back-and-forth communication of humans. Its interface is referred to as “conversational UI.” Today, we communicate with computers like two people speaking different languages. One person gestures or points to a picture while the other person deciphers the action. With computers and smartphones, we click on a menu item or tap an icon to signal what we want to do. The technology behind conversational UI aims to let people talk to computer naturally, as they would to other humans.
Chatbots range in intelligence from simple to very advanced. The simplest of chatbots are powered by decision-tree logic, whereas the most advanced chatbots like Siri and Alexa use natural language processing, or NLP. Chatbots are commonly used to streamline user interactions such as answering FAQs.
Chatbots aren’t exactly a conventional design prototyping tool. To test with chatbots, the designer turns visual interactions into conversational interactions. Rather than designing screens, the designer crafts a conversation.
To experiment with fitness coaching in our apps, we built a chatbot that coached users via text messaging. We began by asking our study participants onboarding questions such as, “How often do you exercise?”, “What are your goals?” and “What kind of foods do you like to eat?”. Once we understood what our participants were trying to accomplish, we began coaching them on their fitness, nutrition, and sleep.
At first, we used SMS texting via a computer to coach our users. A designer sat at the computer and texted our participants while pretending to be the chatbot.
Before building an automated chatbot, we posed as the chatbot and sent texts directly to our participants.
We started with nutrition recommendations: participants in our study could text a command such as “POST” to get a post-workout meal suggestions. The designer-as-chatbot would reply as quickly as possible. However, sometimes users waited hours for a response. This is one of the limitations of chatbot prototyping. The designer needs to be on-call almost 24/7.
We designed meals cards in Sketch and texted them to our participants as part of their nutritional plans.
Later on, we discovered a tool called Dexter that helped us automate our texting chatbot. We could now manage all of our conversations through Slack. We could now schedule texts to be dispatched to all participants at the same time, similar to a push notification. We also created auto-replies to our commands such as “POST,” and to our participants’ questions. With our chatbot automated, we were able to experiment with all sorts of recommendations.
As we learned more about our participants’ goals and challenges, we came to understand which recommendations were most engaging. We asked participants to text the bot what they ate, how they slept, and what kind of workouts they were doing. In return, participants received individualized coaching from our chatbot.
We selected MyFitnessPal blog articles that were relevant to each participant. The bot sent these articles to each participant with suggestions on what the participant could do to improve.
We experimented with texting recipe ideas, links to MyFitnessPal® blog articles, workout plans, and motivational messages. We learned that context, time of day, and even mood can affect whether the participant considered the content helpful. By the end of our experiments, we not only had a better idea of what our users might need, but also which recommendations tend to be most helpful to them.
Our bot was an incredibly valuable tool because we learned how our ideas worked in the real world. Popular design tools such as Invision mimic how a user transitions from screen to screen, and this can be helpful when we are testing flows and app usability, but when we’re digging for the core of our users’ needs, we need tools that help us test whether our screens are solving a real user problem.
Benefits of using chatbots:
The chatbot isn’t the only method for rapidly testing ideas, but it is a tool that should be included in every designer’s toolbox. Chatbots are great for testing what type of information is valuable and when. It’s one thing to ask users how they feel about a meal recommendation when they are sitting in our office being interviewed. It’s quite another to send a recipe recommendation to a busy mom with a stroller after a run on the trail. A chatbot can ask her, “Was this recommendation helpful?” and get an off-the-cuff response. If the user ignores the text, well, that says something too. Chatbots are not, however, great for testing complex tasks that cannot be translated into a conversation—other tools are more appropriate for testing things like visual elements, interaction design, and usability.
Despite their limited functionality, chatbots can help glean information that later informs interaction and visual design. When used early enough in the product development lifecycle, chatbots can course correct and validate ideas quickly before a pixel has been drawn or a line of code has been written.
Senior Creative Director
Marcus Piña
Senior Design Manager
Eric Boam
Lead Exercise Scientist
Jeff Knight
Email — hello@katiesawaya.com