What goes into trusting an answer(er)?

Are the attributes of trusting a person and their answer definable?

Details

  • Capacity
  • Design project lead
  • Contributions
  • Design process / UX
  • Visual design
  • Researcher
  • Length of time
  • 2 weeks

Research question

When someone goes to the community to find an answer, what kinds of information are they needing to have so that they can feel confident and trust that the person who provided the answer knows what they are talking about (in other words, do they have credibility?).

The business goals

Atlassian University is revamping their entire catalog of courses, and with it comes new certifications. Naturally, Community is an extension of where someone might display these accomplishments, or the business would hope that they would. Community has 2 million active users per month coming to the site, this would be a boon for gaining traction in a very contextual market for certifications.

What's known about this space?

Currently visitors are using a combination of signals to determine credibility:

LocationSignalDescription
On the question page
Number of upvotesHow many people also have upvoted this answer
Response typesNumber of comments to answers and how many times they have been 'liked'
Mini profile card (appears when hovering on author's avatar)
Number of posts of authorHow many posts? a little or a lot?
Author's nameIs it a human name or an obscure name?
Author's avatarDo they have one? Is it a real face? Just initials?
Badges earnedWhich badges were earned specifically?
LocationWhere are they located in the world?
Company nameWhat size of company do they work for?
Job titleIs it even in the realm of someone who I'd trust? Same industry?
Accepted answersOthers who have deemed that this person's answers are 'correct' as vetted by the community
Successful answer ratioThe total number of accepted answers divided by the total number posts in the community.

We also know from our data instrumentation that the hover event for the profile card occurs, on average, 126,000 times per month, denoting that it is getting a lot of attention and usage. This is roughly 29% of all accounted interactions on the site.

* It makes total sense that the number of accepted answers in the community would elicit the most amount of trust. This has been echoed by many customers in various interviews.

Methodology

Using usertesting.com with specific criteria to target people who have experience on online forums for testing.

Test A (person to person comparison)

While on the same page, in an oversimplified question and answer view of the community (no interaction UI), there are two answers. Each avatar and name for each answer has a hover state. Thomas has more community engagement metrics in his profile card, while Mike has more of the Atlassian Certification information on display in his.

The central question asked in this experience: If all other factors are the same (both Community Leaders, same number of upvotes, both provided good quality answers) Which of these people are you more willing to believe?

Person A versus Person B

Test B (apples to apples comparison)

This test includes presenting two distinctly different profile cards, one that is more community engagement metrics and the other that’s more certification focused. This is meant to solicit feedback on which profile card creates more trust with the answer provided. The profile cards will appear on hover over a reduced and simplified experience of the community (only shows the question and an answer, none of the interaction UI will be present).

Apples to apples comparison

The results

Test A (person to person comparison

Both certifications earned and community engagement activity help shape the credibility of a person, but they must be shown together to elicit the most amount of trust.

Apart, they reveal a missing counterbalancing element. Together, they prove to be a combination that can’t be beat. Participant 20B's oscillations are clear, as he mentally juggles the merits of certifications with what it means to contribute inside the context of a community

“I think that if this guy had the certifications, or if all things being equal, I’d still gravitate towards this one [Thomas, the version that had community engagement activity]. I’m not saying that certifications don’t matter or not important, but that doesn’t mean that you’ve been effective at participating within the user community and effectively explaining and answering their questions.”

Test B (apples to apples comparison)

Both certifications earned and community engagement activities help shape the credibility of a person, but they must be shown together to elicit the most amount of trust.

Apart, they reveal a missing counterbalancing element. Together, they prove to be a combination that can’t be beat.

"These certifications give me a good sense of the amount of information this guy’s background is in. Being able to see how many time he’s posted and how many of those answers gives me that social proof that he’s done this before. Someone can be certified but could be a pretty bad communicator. They’re know what they are talking about but can’t get their point across, and so that’s not going to help me in establishing his sense of trustworthiness. but if he’s posted accepted and its been accepted so many times, he is a good communicator, and his answers do work for the vast majority of users that aren’t as tech savvy as he is.”

Research conclusion

It's evident that the inclusion of certifications is a no brainer, especially if they have them. There is no downside. However, if they do have certifiations, trustworthiness only increases when there is social proof of activity on the platform, other else there's a slight lost of trust.

What are the practical implications of this research?

It has informed the need to redesign the profile hover card in a way that's more aligned to users of the internet forums.

Previous profile card

Previous profile card

Design principles

  1. Less is more, but more of the stuff that builds trust
  2. Connect people to collaborate better

Leaning on the design principles, which piece of existing data builds trust?

CurrentProposedDeltaRationale
AvatarAvatar
Builds
Real faces or images being uploaded versus initials or abstract shapes.
NameName
Builds
Real names or close to a real name.
Job titleJob title
Neutral
Often times user generated inputs to this profile field isn’t always complete and could be read incorrectly, e.g. “Administrator” Is this someone who is a system administrator or someone who is an administrator of the community?
Community roleCommunity role
Neutral
It does build trust, but it’s already displayed next to the person’s name. Is duplication necessary? Additionally, “Community Leader” is insider-baseball language and it’s meaning isn’t totally apparent.
Company nameCompany name
Consider removal
Relevant information, yes, but adds noise that’s not part of credibility establishment. Can still be found in profile page.
LocationLocation
Consider removal
Relevant information, yes, but adds noise that’s not part of credibility establishment. Can still be found in profile page.
No. of acceptedNo. of accepted answers
Builds
Yes 💯
Total postsTotal posts
Builds
Yes 💯
BadgesBadges
Neutral
Badges are understood but the individual badges themselves may not be totally apparent. e.g. “Rising star” or “May Challenge”
View profile buttonView profile button
Neutral
Necessary action but when seen in light of the primary action (establishing trust and evaluating whether an answer is worthwhile) this can be seen as a more a neutral building activity and can be seen as adding unnecessary noise.

Before & After

After a few rounds of critique and feedback, this was the result.

before and after

What's changes?

  1. Orientation of the card went from vertical to horizontal
  2. The blue header was removed to allow the eye to rest as it takes in this information
  3. The above elements (in the table) were removed
  4. The elements that stayed in were moved to more prominent and logical positions:
    1. Avatar got moved closer to the person's name
    2. Community activity got moved to it's own section with more emphasis on the counts
    3. Badges were placed in the upper right as a way to balance out the space and give them a unique place
  5. Certifications have their own place on the profile card and are located right under the person's name and above their accomplishments in the community.

Visual changes

visually different
  1. Avatar
  2. Certifications
  3. Name of person & link to profile
  4. Recently earned badges
  5. Total posts
  6. Count of answers (creates a ratio of helpfulness)
  7. Count of answers that have been accepted
States

Outcomes

Previous profile card was shown 126,000 times per month, on average. The new profile card has averaging about 150,000 times per month. Which represents a 119% growth in hover events. More than that, the previous 30 day period before shipping, we saw a click through rate to profile around 0.25%. With the new profile the click through rate to the profile has increased to 1%. Which is rather interesting because the button is no longer there, and the name is the only clickable item to get to the profile page.