The Problem

Microsoft Advertising had no automated support on their contact page. Their support volume was surging, leading to longer response times and decreased customer satisfaction.

In addition, Microsoft Advertising had built multiple human advocate channels for their chat support. Channels included general support, sales, agency support, and so on. However, these channels experienced significant misroute traffic because their chat support system relied on customers self-selecting which channel to enter.

The Solution

In March 2022, Microsoft Advertising hired me as the conversation designer for a small team also consisting of a product manager and a data analyst. Together we planned designs for the virtual agent by researching competitor chatbots and by communicating with stakeholders about the needs of the business.

A guiding principle for us was to do what’s best for the customer. We designed self-help for topics, but never wanted to make it difficult to get to human support when desired.

We then designed, developed, and launched the virtual agent. We maintained and updated it with new feature, including NLU capabilities, intelligent routing capabilities, and advanced conversation flows for high traffic topics. We supported this virtual agent for a year and a half before it was deprecated in a post-LLM world by Copilot in Microsoft Advertising.

The Result

The virtual agent we developed saved Microsoft Advertising money by automating routine queries and by more accurately routing customers to support advocates. In addition, it helped Microsoft Advertising earn more by detecting sales opportunities and routing them to sales teams.

• Deflection: Our virtual agent deflected 40% of traffic as customers opted for the virtual agent’s self-help solutions.
• Sales: Our virtual agent increased traffic to sales teams 20%, improving revenue for Microsoft Advertising.
• Accurate routing: Misroutes significantly reduced thanks to our virtual agent’s intelligent routing capabilities.

My Role and Contributions

In my role as Conversation Designer, I was responsible for:

• Research and Planning: I analyzed competitor designs as well as our own historical support tickets and conducted comprehensive research to identify common customer pain points. This analysis informed the foundational structure of the virtual agent’s conversation flows.
• Dialogue Design: I mapped conversation flow and wrote chatbot dialogue in Vizio, ensuring sign off with component area planners and technical advisors.
• User Experience: As a designer with a user-focused mentality, it felt critical to me that I see the end user experience during development phase, I advocated for and established a prototype that mirrored the customer-facing interface. This proactive approach allowed for thorough validation of the bot’s performance prior to its official launch.
• Chatbot Development: Though originally hired to just design in Vizio, I saw that our team could work more efficiently if I also implemented my designs in our low-code development platform of Power Virtual Agents (now Copilot Studio). I taught myself PVA, then became our chatbots developer as well.

• Routing Strategy: I developed a robust routing system to handle any customer query, ensuring that the correct query gets to the correct human agent queue. This strategy involved detailed mapping of dialogue paths to appropriate support channels.

• Introducing Natural Language Understanding (NLU): Our Virtual Agent began as a button-based experience with no user typing. As it grew in complexity, we realized NLU capability was critical to enable customers to quickly navigate. I worked with our product manager to build and present a business case to leadership, then worked in Power Virtual Agents to redesign our conversation flow from the ground up to enable a seamless typing-based chatbot experience.

Challenges and Lessons

Throughout the project, I encountered and addressed several challenges:

• User Privacy in Testing: Due to privacy constraints, our team couldn’t access actual user conversations. To make up for this, we dove deep into data. We utilized advanced pivot tables to understand where users were self-helping, where they were escalating from, and where they were abandoning the chat experience before resolving their query.
• Unauthenticated User Interactions: The chatbot operated outside the authenticated environment, making it challenging to verify the legitimacy of sign-in issues. This limitation required the development of strategies to differentiate between genuine user problems and potential fraudulent activities.

These challenges provided valuable insights and informed continuous improvements to the virtual agent, enhancing its effectiveness and reliability.

In Short

The initiative not only enhanced operational efficiency for Microsoft Advertising but also delivered measurable improvements in customer experience. The lessons learned from this project have been instrumental in shaping my future chatbot initiatives, ensuring ongoing innovation and value delivery.