Kevan Lin
Case study

Bringing personalization to the community

Background

Atlassian products are either hosted in the cloud or on premise. Sometimes this distinction is known to the customer and often times it is not. In the effort of maximizing the accuracy of posts being categorized correctly, personalization was brought in to help improve these efforts.

Details

  • Context
  • Design project lead
  • Contributions
  • User Experience
  • Prototyper
  • Visual Design
  • Collaborators
  • Erika Nunotani

The problem

With a recent feature update to the way posts were being cataloged, even though the posts were getting to the right categories, the surrounding contextual information about the post were still missing or incomplete. Specifically around hosting types as a primary concern. When this information is not available to the answerer, there are extra steps involved in asking the original poster to provide these details, so that they can get an accurate response.

People who are not Atlassian staff account for 10-12% of the population but answer 60% of questions that get asked on the site. It's imperative that these people have what is necessary for them to answer questions in a timely and streamlined manner.

Business metrics

Support had made the conscious decision for community to be the preferred route for troubleshooting and resolution (for select products and hosting types). Due to this shift in strategy, the community would need to scale its ability to receive these people and ensure they have a solid support experience. Personalization of the customer’s entitlements enables the community to be able to capture the right data, so that answers can be given quickly and accurately as possible.

Success metrics

The design process

Involving our multi-disciplinary team, we built a three week schedule to deliver a confident vision.

Discovery - Discover insights into the problem by uncovering the problem space.

Definition - Defining the area to focus more effort on

Develop - Develop potential solutions

Deliver - Deliver an experience with confidence

Discovery

The business is asking for us to solve for support deflection. The user doesn't need to know about the support strategy, they just need to know how to either find their answer or ask their question so that they can get an answer.

How might we build something that elicits trust, especially because we're showing data that's personal to our customer's Atlassian product instances?

We first found a post in the community that contained missing information about the product and some back and forth between the author and an answerer.

We then used this interview as a basis for a journey mapping exercise. Participants of this exercise included the Product owner as well as some engineers.

Customer journey map created by our cross functional team

Definition

Our journey map gave the team some useful insights:

  1. Don't speak the same language - Participant_1a lacked the implied knowledge of Atlassian products so they didn't know what to include on the question creation form.
  2. "Others can't see what I see" - As a pain point, they couldn't express to the community what they were uniquely seeing. They wanted a way to show their environment and set up so that others could help.
  3. Unnecessary complicated - the back and forth nature of clarifying questions, proved to be laborious and lasted too long (a lot of time elapsed from creation to final follow up).
  4. Potentially an unsatisfactory experience - Atlassian did not give him the best evaluation experience and it may affect whether or not they will purchase in the future.

Ultimately, we decided to double-down on #2 and make the pain point of not being able to replicate my current environment to be available on every post in the community. It would reduce the number of back and forth comments/answers and drastically cut down the time to getting an answer. Win-win for everybody involved!

Develop

From the outset, we prioritized earning trust and delivering immediate and tangible value, as our pillars to this experience.

We started on the whiteboard and in Sketch making rough sketches of what the experience might be like. Early on in the process, we determined that we would need to introduce an interstitial page before the actual form to fill out details and information.

Beginning stage of personalization where were starting to figure out the shape of the experience.

This interstitial page provides immediate and tangible value to the customer by showing them the very product they had in mind when they came to ask a question to the community. But how would we ensure that we build trust into every interaction?

Enter rapid prototyping.

Instead of trying to get pixel perfect on one approach to this, we instead quickly stood something up, tested it, iterated, tested again. We repeated this process until we felt confident in where we ultimately landed.

We asked ourselves, if I was a Jira administrator and I was publishing a question that was publically available on the internet, what kinds of information would I not want to be made public? From the available sources of data, this is what we settled on. Cloud url, hosting type, permissions level and product edition version.

Prototype

Of these pieces of information, cloud url was the most important not to reveal publically for various security reasons. This was one of the first areas of focus for prototyping. We could either take a passive approach where the defaulted value upon selecting their product was in a safe position, but if they wanted to change visibility on this information they could.

Cloud url hidden by default, "visibility settings" button to change appearance of other data.

Learn and iterate

We heard from customers that the lock icon didn't make sense: Does the lock mean that I can't change it? Do I have the permission to change it?

We also discovered that many administrators of our products didn't know what the cloud url was nor did they understand the severity of displaying publically, this was evident based on how little time they spent looking at or trying to figure it out.

What if instead of being passive with the visibility settings, we were aggressive and showed your settings right after you chose your product and before you got to the form?

Well as it turns out, customers liked that they got to choose these visibility settings but ultimately wanted to get to their questions first before filling in this information.

Customers also didn't understand the red toggle switch. Does the 'x' mean that it's disabled? If I can't toggle this, why is this on here to begin with?

Learn and iterate

At this point, our hypothesis was, there's clearly something about the cloud url as a means of denoting locked or disabled by way of UI that didn't resonate with our users. None of them took any issue with selecting "their" product to move on. So why don't we remove cloud url in next screen after they have chosen their product. We can also move elements around visibility lower on the form in favor of question completion.

Reaching confidence

Customers trusted the last prototype the most. One customer said, "Despite probably being an alternative that will take me more time, it's more reassuring knowing that I can control the data that's being presented in such a fine way." More than trusting the prototype, the efficiency and confidence of which customers created questions (albiet through a prototype) seemed like an improvement. We had a winner!

Delivery

With customer feedback in the pocket, we began to put the polish on it all. We ran reviews on the copy and looped in legal for approval. This is the final result of the work.

Outcomes

This new experience has not yet shipped into production, but is in active beta, without any conclusive data points yet.