Streamlining tracking number creation: more numbers and fewer mistakes

CallRail 2024 | Drove Adoption + Engagement

Hebe Zheng (Lead Product Designer), Pia Kendrick (Lead PM), and the Experience team

Summary

Jun - Jul 2024

One-liner

I redesigned CallRail's tracking number creation wizard, a core setup flow used by every customer, by restructuring the flow and using behavioral data to drive key decisions.

Outcome

Wizard completion increased 31%, questionnaire completion 17%, and tracking numbers per account 11%, indicating more source-specific numbers and clearer attribution.

My role

As Lead Product Designer, I led the project end to end: identifying the problem, conducting research, shaping the strategy, and executing the work in close partnership with Product and Engineering.

Context

Tracking numbers power CallRail’s attribution

CallRail helps businesses know which marketing strategies are generating phone calls and conversions. Tracking numbers power CallRail’s attribution by tying calls to campaigns

In 2024, growing tracking numbers per account was the main NRR bet: more numbers meant clearer attribution and higher retention and expansion. But the creation wizard had a persistent ~50% drop-off rate, and many created numbers were misconfigured which undermines the very attribution accuracy the business was betting on.

Problem

Too many steps leading to drop-off and inaccurate setup

The number creation wizard saw ~30% drop-off in the questionnaire alone, and many created numbers were misconfigured, undermining attribution accuracy.

The flow relied on rigid binary questions that overwhelmed new users and slowed experienced users, leading many to create one “catch-all” number instead of campaign-specific numbers.

New users: decision fatigue, leading to hesitation and drop-off.
‍Experienced users: repetitive, click-heavy setup when creating multiple numbers.

The series of binary questions, "this or that" cause decision fatigue and confusion for new users.

The drop-off rate is consistently around 30% for the questionnaire section. (FullStory)

Design thinking

Streamlining the flow has the highest impact

The issue was structural: users faced too many upfront decisions without enough context. I restructured the flow while keeping the UI largely intact so we could ship faster and measure the impact of each change.

Constraints

Couldn't redesign the whole experience from scratch, scope had to be focused
Had to optimize for both new and experienced users
Some steps had to stay in a fixed order for technical reasons

Old user flow vs. New user flow

Reduced the questionnaire to two paths

I restructured the questionnaire into two clear paths: “On my website” and “Somewhere else.” This replaced a branching set of binary questions with one decision, reducing drop-off for new users and cutting clicks for experienced users. In the most complex setups, clicks dropped 65%.

The tradeoff: Consolidating the step required more context, which led to the visual platform library in Question 2 (Somewhere else).

Using behavioral data to advocate for design change

I proposed moving destination number (plus call recording and whisper message) into a single Number Routing step after number selection, which better matched how users approached setup.

There was a valid concern from stakeholders: the destination number was used to pre-populate the area code during number selection. Moving it later risked breaking that logic.

To evaluate the tradeoff, I used Looker and found that ~85% of accounts had only one destination number. FullStory also showed users changed the area code input only 1.5% of the time. Thus, we safely pre-populated area code from account data while keeping manual override.

Additionally, I observed that many users routed numbers to call flows right after creation, so I added in-wizard routing to avoid a separate follow-up flow.

Visual platform library to drive discovery

Most users didn't know best practice was to create a separate number for each platform so they created a single number for everything. That directly limited attribution accuracy and capped numbers per account.

I introduced a visual platform library organized by popularity, with familiar logos and category groupings. This made it easier to choose the right platform and reinforced the idea of “one platform, one number”.

De-risking: I tested an early, limited version in a parallel first-run onboarding project. It outperformed the existing flow, validating the approach for the wizard.

Contextual suggestion is better than generic ones

Tracking numbers work best with external integrations, but many users never discovered them because setup lived deep in Settings. I replaced a generic "Suggested Next Steps" on the confirmation screen with contextual integration suggestions based on the tracking number type, driving 24% higher engagement.

Impact 🎉

31% boost in wizard completion rate

The updated wizard achieved a 71% completion rate, up from 54% previously, representing a 31% increase.

17% reduction in questionnaire drop-off: 77% of users completed it, up from 66% previously.

11%Increase in numbers per account: the average rose from 12.2 to 13.5 tracking numbers per account.

Reflection

This project reinforced a core belief: the most impactful design decision is often what not to change. Rather than redesign the wizard, I focused on the smallest changes that could drive the biggest impact. That focus let us ship quickly, measure clearly, and see real behavior change.

It also reinforced the value of behavioral data. The 85% destination-number reuse pattern came from what users actually did, and it directly shaped the solution.