John Lewis Finance powers ahead with CRO

As a long-time CRO partner of John Lewis Finance, we were asked to provide further strategic support to help them overcome various challenges and grow their CRO programme.

Some of the challenges they faced were:

• Lack of historical test documentation and documented process etc.

• Tracking issues and questions over the integrity of data.

• Relatively low traffic volumes to test within certain areas of the website.

• Using Monetate as a tool for quick and temporary fixes to the site that did not drive any value.

• Struggle to get support for the programme in terms of internal development, design resource etc.

Resource changes had slowed testing activity, but at the same time, new leadership brought an increased appetite for experimentation.

We revamped the existing CRO programme by introducing a faster-paced experimentation approach, supported by improved processes, documentation and delivery frameworks.

Our team worked closely with John Lewis Finance to increase test frequency, support ideation and provide ongoing research and strategic input across the user journey.

Alongside this, we helped recruit and upskill an in-house CRO team, enabling them to take ownership of testing activity while maintaining momentum and performance.

Through improving research, test planning, test build and analysis practices, we enabled John Lewis Finance to build the foundations for a more successful CRO programme.

In 2021/ 2022 we tripled the test velocity, with 26 tests running in 10 months (2.5 per month, up from just under 1 per month before the project).

• A series of iterative redesign tests on the Home Insurance landing page increased the visibility of John Lewis Finance's value proposition and key selling points. They yielded a +1.6% increase in the 'Visit to quote' rate. This redesign was also rolled out across other financial products as tests.
  

• A single test at the first step of the home insurance quote and buy journey to revamp their 'suitability page' improved start-to-quote conversion by +2.7%. A test further along the user journey to improve product presentation to mobile users achieved a 4.7% increase in progression from the quote summary page, resulting in a significant increase in sales.
  

• The business now has robust processes and frameworks for test ideation, prioritisation, briefing, development, QA, UAT, launch and analysis that underpin the experimentation programme. This has been integral in gaining the other internal teams' understanding, trust, and support.

The John Lewis Finance team grew by 300% (from 1 to 4), which has been vital for the success of the CRO programme and where it stands today. The test-and-learn culture we created has permeated through the Partnership, with internal teams embracing the potential of CRO to support their goals and taking an active interest in the test plans and results.