Guiding new users to successfully optimize CallRail

CallRail 2023 | Drove Adoption & Engagement

Hebe Zheng (Lead Product Designer), Heather Lutz (Senior Content Strategist), Grant Sadowski (Senior Product Manager), and the Experience Engineering team.

Context

In 2023, CallRail faced a growing retention problem. Churn jumped from 11.6% to 14% by year-end, a 20% increase. We were losing more customers each month than we acquired.

New user onboarding became a company-level priority.

Problem

New users signed up with clear intent. They wanted call tracking and marketing attribution. But once inside the app, many felt lost.

"Have someone walk you through the set up as it can be a little confusing at first. But once you get the hang of it’s pretty easy." - Marketing agency, Feb 2023

CallRail requires meaningful setup to unlock value: Creating tracking numbers, installing tracking scripts, configuring call routing, and activating integrations. This created a catch-22: users needed to configure their account to see value, but were unlikely to invest time without experiencing that value first which results in slow activation, weak habit formation, and rising churn.

Research

Through FullStory and Looker analysis, past Growth team research, and conversations with Sales and Support, I gathered the key insights:

Initial motivation - users who trial CallRail already have an interest in call tracking and marketing attribution.
Once inside the CallRail app, users expressed confusion and uncertainty about how to proceed
CallRail's ability to integrate with users' existing tools is a key consideration for potential customers.
Experienced agency users follow the same setup process across all client accounts.

Exploration & Tradeoffs

Approach 1: Expand the First-Run Wizard

My goal is to capitalize on users' initial motivation and eagerness when they first sign up. I proposed front-loading essential setups directly into the initial onboarding wizard.

Installing the tracking code snippet (essential for CallRail's functionality)
Setting up call routing and activating integrations (both strongly correlated with retention and customer value)

This approach didn't align with our risk tolerance or timeline:

Stakeholders were risk-averse to altering the first-run experience and hesitant to add steps before users entered the app.
Trust concerns: users might be reluctant to install tracking code or configure calls into CallRail immediately upon signup
UX risk of overwhelming first-time users who are still unfamiliar with the platform

Expand the existing first-run experience: front load the essential setups

Approach 2: Customized onboarding based on user’s intended use

We explored a customized onboarding approach that would adapt based on each user's intended use. The inspiration came from our Sales team. When the team onboards new customers, they ask about goals and intended use, then tailor the setup accordingly. We brainstormed a concept that mirrored this personalized approach.

Why we moved away:

Too technically complex to implement all the necessary variations
Stakeholders weren't ready to embrace customization for the first iteration
We lacked sufficient user research to make the variations truly meaningful

Customized onboarding based on user’s intended use

Design solutions

Responsive variation (tablet and mobile)

General onboarding checklist - guiding users across various entry points

The final solution was a one-size-fits-all onboarding checklist model. This checklist focused on the most valuable features and setup tasks across all entry points. We were confident in this approach because we had been intentional in understanding our users and selecting the right onboarding tasks.

Onboarding checklist task

To identify the right onboarding tasks, I used three methods:

Observational analysis: I studied the behavior of our marketing agency customers as they set up accounts for their clients, extracting key patterns and best practices.
Internal knowledge mining: I reviewed our Account Executives' guidebook to understand how they help new customers configure their accounts.
Collaborative workshops: I ran action mapping exercises with my PM and Content Strategist, inspired by "Better Onboarding" (A Book Apart). This helped us identify which actions new users need to build similar habits to experienced users.

Location and layout

Since new users often feel uncertain and lost when first landing on CallRail's homepage, we strategically placed the onboarding checklist there for maximum impact. As they complete tasks, the homepage dashboard updates in real time, showing tangible results. The design challenge was making the checklist feel like a natural part of the homepage and not an interruption.

Designing for emotion

To address new CallRail users' feelings of uncertainty and being overwhelmed during initial account setup, I crafted the following design strategy for the onboarding checklist:

Task prioritization: We carefully sequenced the checklist items, balancing task complexity with perceived user value. This approach aims to gradually build user confidence while demonstrating the CallRail’s value.
Positive reinforcement: I incorporated visual feedback mechanisms upon task completion. This design element serves to boost user confidence and maintain motivation throughout the onboarding process.

Outcomes

In December 2023, we launched the company's first in-app onboarding guide to help new users set up and optimize CallRail for their business. To assess its long-term impact, I compared graduation rates between Q2 2023 and Q2 2024. Graduation rates, the percentage of customers who remain after the first 90 days, increased from 86.4% to 88.6%, a 2.55% improvement.

38% higher retention for accounts that completed the checklist

Accounts that completed the checklist are 38% more likely to remain active than those that didn't. While correlation ≠ causation, the signal was strong.

Unexpectedly, marketing agencies began using the checklist as a setup to-do list when configuring client accounts, extending its value beyond first-time users.

Loom Overview