The problem I was hired to solve
When I joined SonarSource as the first UX researcher, the team had no baseline for how users were experiencing SonarCloud. We were shipping improvements with no systematic way to know if they were actually working. I built a benchmarking programme from scratch to change that — giving the team a repeatable, metrics-based view of product performance over time.
What I did
- Defined the measurement framework: Worked with the PM and designer to identify the six most critical user journeys to track. For each, I established five metrics: completion rate, error count, time on task, perceived user effort, and severity. Adapted GitLab's benchmarking methodology to calculate a single composite grade per task, making it easy to track progress across rounds.
- Ran the first study in 2022: Established the baseline across all six journeys. This gave the team, for the first time, a clear view of where the product was performing well and where users were genuinely struggling.
- Identified the biggest problem: The upgrade journey — moving from free to paid plan — was the worst performing task. Users couldn't find the entry point, spending nearly a minute just figuring out how to start. Once they found it, the billing page gave them no information about their current plan or what they'd gain by upgrading.
- Worked with the team to define the fix: Brought the findings to the PM and designer with specific recommendations. The team iterated on the experience — moving the upgrade option into the main navigation as "Billing and upgrade," introducing a tiered plan slider, and clearly surfacing the user's current plan for easy comparison.
- Re-ran the study in 2023: Measured the same journeys against the same metrics to evaluate whether the iterations had worked.
What I found
- The upgrade experience went from the worst performing task in 2022 to the best performing task in 2023, scoring 92/100.
- Overall usability of the upgrade flow improved by 56% — a direct result of the iterations the benchmarking findings had informed.
- The programme gave the team a language and a cadence for talking about product quality that hadn't existed before — moving the conversation from "we think this is better" to "here's the data showing it is."
The decisions this enabled
- Gave the product team a repeatable measurement infrastructure — not a one-off study, but an ongoing programme that tracked whether product decisions were actually moving the needle.
- Directly informed the redesign of the upgrade flow, which contributed to improved free-to-paid conversion.
- Established benchmarking as a standard practice at SonarSource, giving future PMs and designers a baseline to build on.
- Demonstrated that systematic measurement — not just intuition — was how the team should be evaluating the impact of its work.