Getting the groceries you ordered!
Client: Kroger product teams
Role: Senior product designer
Cross department collaboration (Customer Experience and Associate Experience). Collaborated with product managers, product designers, engineers, stakeholders and design research
Background
Customer Experience and Associate & Operations Experience (Supply Chain) worked together to provide a better inventory experience. This started with a week long focused workshop to understand our respective product spaces and continued with several discovery sessions, research, testing, iterations and weekly touch point meetings.
This project took over a year to complete and involved many product teams, stakeholders and leadership buy-in to carry-out the combined 3 initiatives: 1. inventory check at checkout, 2. item replacement and 3. projected inventory.
Challenges
There was a concern that business (increased fill rate) was driving the teams to build something that wouldn’t add value to the customer. In fact, there were concerns that adding an inventory check at checkout would cause additional fallout later in the shopping experience causing frustration.
Additionally, there were questions on whether giving a customer an option to replace an item that was unavailable would cause the customer to abandon their cart because of the additional effort to replace an unavailable item.
With projected inventory, there was always a concern of accuracy. Blocking sales because the projection is inaccurate. (Ex: if an item is in stock at the time of order scheduling but will be out of stock at the time of order fulfillment; and therefore, we show it as out of stock.)
Initial Workshop Goals
Focus on instilling trust in, and accuracy of Kroger inventory so that customers have confidence they will receive the items they ordered
Share service blueprints and goals of each area
Identify pain points and opportunities
Brainstorm ideas and experiments
Inventory service blueprint with pain points from both customers and the business side
Everyone’s early ideas written down on stickies
Break out groups: Group 1 brainstorm ideas
Notifications of back in stock during & after shopping journey
Break out groups: Group 2 brainstorm ideas
Multiple ideas / paths
Break out groups: Group 3 brainstorm ideas
No dead ends - multi fulfillment methods
Background
Once the team came up with a list of experiments, we mapped them to the impact / difficulty matrix to prioritize and determine how to move forward.
Final list of experiments
Point in time (POT) inventory check at checkout.
Review of metrics to see how much fallout would occur by doing an inventory check at checkout
Allow customers to choose a backup / replacement item
Two rounds of usability testing
A/B testing for item replacement experience
Available to Promise (projected inventory) at checkout
Interviews with customers to understand their needs around inventory
Concept testing of projected inventory mockups to get customers’ feedback
Workshop 2 - Item Replacement Focus
How might we eliminate dead-ends in checkout & ensure that the experience is easy to use?
The team went through the following activities to define the item replacement experience:
Reviewed metrics for substitutions and back up items, time customers spent shopping and various inventory numbers
Competitive analysis (3 shown here). Rose , bud, thorn analysis
Pulled secondary and past primary research
Created storyboards
Rough & Ready Prototyping
Voting
Amazon competitive analysis
Walmart competitive analysis
Target competitive analysis
Word cloud of previous research: Customer’s biggest challenges
Previous customer research that talks about wanting to pick their own item for substitution
Research from Baymard Institute related to inventory
Storyboards were created to help the team get into the mindset of the customer and understand the context surrounding the experience
Wireframes were used to conceptualize the experience. There was a vote to determine the direction we should take for mockups and testing
Direction for User Experience
By the end of workshop 2, we had a general experience direction to test with customers:
We knew that we wanted to notify customers that there was an unavailable item and for them to replace the item from the scheduler page
However, we needed to test a few options, based on the above wireframes, using higher fidelity Kroger stylized mockups
Workshop 3 - Assumption Mapping
Along with testing our wireframes, we wanted to test our assumptions that were both customer facing and internal feasibility questions.
We wrote down assumptions that we had about the initiative, identified the riskiest assumptions, categorized them and discussed how we could test them
We categorized our assumptions by desirability, feasibility, viability, ethical and usability
This helped us build a list of questions and test scenarios with our usability tests.
Usability Testing
Top items to test:
Entry / Exit into experience
Does this benefit the customer and how often would it be used?
Customer sentiment
Ease of replacing items
Unmoderated testing of 3 prototypes using Userzoom
Screening for customers who do the majority of the shopping and are Kroger customers
Two tests (technical glitches in the first test)
1st test- 10 Kroger customers
2nd test- 4 Kroger customers
Top Findings
1st test: We were on the right path of replacing / removing items, but specific tasks needed to be simplified
Customers were confused with replacing a quantity of 1- 1% gallon milk with a quantity of 2- 1% 1/2 gallon milk. Therefore, in the 2nd test, we had customers. replace a 1 gallon 1% Kroger branded milk with a different brand of 1% gallon milk
There were some technical glitches with the pop-up drawer. We fixed the issue to make the transition seamless.
2nd Test: Participant Success! It's easy to replace / remove items
Customers liked the pop-up drawer the best (cherry); however, it was decided to go forward with the ‘Unavailable Items link’ for MVP because it was straight-forward technically
Customers did find value in replacing an unavailable item! This allowed us to proceed with development and A / B testing
Projected Inventory
Worked continued in refining the user experience for item replacement based on testing. Meanwhile, things were heating up with the 3rd initiative - projected inventory.
At first, our team didn’t know what the experience would look like or if customers would value projected inventory. So we did interviews with customers, asking them general questions about inventory and items being out of stock, tested a mockup of one possible area projected inventory could be used, and tested the item replacement experience one last time.
Projected Inventory Research
Moderated virtual interview sessions and concept testing with 5 customers
General inventory questions
Critique of item replacement prototype
Critique of projected inventory mockup
Affinity mapping of interview data
Initial findings, insights and HMW questions
Projected inventory experience test
The interview started with asking the customer to think about an item they really like or buy often.
The screen below is showing the product card results from a search and the gray box represents the item that they like or need. We didn’t use a specific item because everyone has different needs / likes and the item that we chose, may not be important to them. We needed to create a sense of urgency for this item and find out what customers would do if they saw the ‘Back in Stock March 8th’ message. Would they pause their order and come back when the item is back in stock? Would they pick it up at their local store etc?
*Sorry, video does not play here due to confidentiality
Conclusion
*Unfortunately, currently checkout is the place where most customers chooses a fulfillment method (pickup, Kroger delivery, Instacart delivery) and a timeslot (unless they choose an up front time slot and a very low percentage of customers do.) So, this final check is necessary because we usually do not have this information up-front and we need to know where order is being fulfilled from and the time it will need to be fulfilled to have accurate inventory. The team will continue to improve this experience over time.
According to the test, projections should not be used on the product cards because customers do not trust the message of when it will be back in stock.
It was decided that it was best to have projected inventory in the background for Kroger delivery orders. An item would show as in stock if an item was currently out of stock, but was being replenished at the time of fulfillment. Vice versa for orders that were in stock at the time of ordering but would be out of stock, due to demand, at the time of fulfillment.
Item Replacement Experience Continued
Iterations & Minor Revisions
Added the date / time within the item replacement experience to remind the customer of the day / timeslot that was chosen
Copy update for when there is not a similar item:
From: ‘There was a problem retrieving similar items. Please try again.’
To: ‘No similar items were found. Please try a different time to check items' availability.”
Removed ‘black & white’ stylized images when product was unavailable and needed to be replaced
‘qty’ to Quantity
Line was extended across product card
Final item replacement experience. (Happy path for iOS / desktop).
iOS Happy Path
Desktop Happy Path
Conclusion
There was a lot of hesitation about this project because the ideal situation is to never have unavailable items and to keep checkout as simple as possible. Our team will continuously improve supply chain and increase availability of all products. Meanwhile, item replacement is a way for customers to get out of a dead end and get an item that is needed.
All 3 of these initiatives were rolled out individually after separate A/B testings showed high engagement with the item replacement experience, increased fill rate / perfectly picked orders, and increased customer satisfaction.
A few metrics from the item replacement experience:
30% overall engagement rate
Findings from Pickup:
+9.4% lift in cart adds and +8.7% lift in orders placed within the new experience - both reaching statistical significance
Findings from Delivery:
Order revenue: Around a +25% lift
Orders and cart adds originating directly from the fallout recovery experience increased by +4% and +1%, respectively
