Iterative Onboarding Research

ROLE: End-to-end UX research | TEAM: Product | DURATION: 2.5 months

CONTEXT

Covet Fashion has enjoyed 9 years in the mobile gaming space. To maintain competitive presence and improve early retention, we sought out to iteratively test the new user onboarding flow. Over the course of 2.5 months, I evaluated 3 different prototypes (V3, V4, and V5) to evaluate usability and comprehension of key gameplay systems which resulted in staggeringly positive KPIs upon rollout of the new flow.

OBJECTIVES

  • Understand the first time player experience of the onboarding flow

  • Identify usability issues to improve the new user flow

  • Evaluate understanding of the core loop and key gameplay systems

IMPACT

  • We saw a lift in all KPIs, especially Retention, which is the hardest KPI to move.

    • Tutorial Completion up 28% in V5 as compared to V4

    • Day 2 Retention increased by 5%

    • Day 7 Retention increased by 17%

    • Average Revenue Per User (ARPU) up by 12%

    • Sessions Per Day up 5%

APPROACH

Remote unmoderated playtest with 10 casual mobile gamers (representing our potential audience) who have never played this game, hosted on PlaytestCloud

  • Participants were given tasks to complete during their playthrough with think-aloud protocol, followed by a post-play survey

TIMELINE

Each wave of testing was conducted on a 2 week cadence:

  • Protocol Creation: 1 day

  • Recruit: 1 day

  • Analysis: 5 days

    • Consisted of watching videos, tagging, and affinity mapping themes

  • Reporting: 3 days

    • Included an executive summary, categorization of usability issues by severity (high, medium, low), video highlight reels of player pain points, key recommendations, and a benchmark comparison of where the NUF has improved across key areas we have evaluated over time.