1030x542_merchandisetesting

Why We Test The Way We Do

Below is a message from Gwynnie Bee CEO Christine Hunsicker:

I wanted to share some details about our merchandising testing and why we do it.

Our goal is to carry the best possible mix of brands and items that will make our overall member base as happy as possible. There are hundreds of brands we could carry and thousands of items we could buy from those brands. Since the buyers have a limited pool of dollars, they need to be smart in how and what they buy. The last thing we want to do is disappoint our members by having too much of something and not enough of something else. In order to maximize the allocation, the buyers need to make the best decisions they can.

In order to remove human bias, we take a data-driven approach and take into consideration things like:

  • How popular is a brand?
  • What styles are people closeting?
  • How does a style fit?
  • How durable is the style?
  • How did people rate the style?
  • How many people purchased the style?
  • Does the brand help attract new members?
  • Does the brand impact retention of current members?

While some of the data can be collected relatively easily, the last two points are not so straightforward. Acquisition and retention metrics require looking at data over time, which then introduces “noise” into the data.

For example, let’s say we see retention rate go up when we launch Style & Co. That would seem to indicate that we should buy more Style & Co but it’s not that easy. In August, we launched our Loyalty Rewards program, released a new Android app, increased our success rate of sending out prioritized items, altered how we visually merchandise the garments and know that teachers ramp back up to use the service heavily.

In order to understand the impact of solely the brand or the style, we have to construct tests that allow us to control for all of the “noise” that is going on at any given moment in time. To do this, we randomly segment the customer base into group A and group B. Each group sees the same number of styles per launch. Some styles are common to both groups and others are different for both Group A and Group B.

Currently, we are testing the impact of workwear and 3 specific new brands. For instance, in a launch of 9 styles, everyone would see the same 5 styles and Group A would see 4 test styles (Enlo, Vince Camuto, and Kasper) while Group B would see 4 styles from brands we already carry. And yes, that means Group A is not seeing 4 styles, and Group B is not seeing 4 styles. However, once the test ends, everyone will have access to these styles and brands. If the test is successful, we will then make these new brands part of the regular rotation.

We believe that running these tests before committing millions of dollars on a new brand is the best thing to do for the member base. If there were a better way to do this and get accurate results, we would do it. If we don’t get accurate results, the assortment could shift dramatically in 6-9 months in an incorrect direction based on dirty data, ultimately disappointing our members.

But we also know that we need to maintain our customer experience. Therefore, we have to find a compromise. Today, that compromise is a New Arrivals board on Pinterest that gives members direct access to all the launched styles, regardless of group. We know this won’t satisfy everyone and is not frictionless, however, it is the best path we can offer while maintaining the integrity of the test and data.

We continue to explore all possible avenues that allow us to gain data, while better serving our members. At the same time, I am completely open to suggestions that meet statistical rigor. The long-term benefits are significant for the entire membership, and that is what we are optimizing for through these tests.

Christine

  • Stephanie Bray-Voorhes

    I understand that testing is necessary and that data points drive business decisions….
    However, what would make more sense is to have a survey on the site that let all members go and look at the collections proposed. Then each member could answer questions based on the new items. No one needs to be affected by a test.

    Alternatively, a more active test would have been to have sent out a message to members saying that you were wanting to do a trial on work wear and let people volunteer for that. Then randomly chosen from the volunteer pool, you could have sent your testers additional clothing (not impacting their normal service) and allowed testers to give unbiased opinions on the test shipments.

    There is no reason to segregate your customers. You have a dedicated fan base, but you continue to find ways to downgrade your own services in the name of progress. When you divided the group you unintentionally took value away from both groups.

    By surveying your client base you find out what percentage is interested in the new idea. By going back to those who show interest, and allowing them to give you more specific feedback you can gather actual data. If you survey 500 clients about workwear clothing and 300 of them are interested you know its something to look into further. If you randomly select 100 of the interested to test them, you then get random unbiased opinions. However, the way you have tested these newest options you have ensured that your data points will be biased by those who feel you have foisted unwanted product on them, or have witheld valued product from them.

    It doesnt have to be this complicated.

    • http://closet.gwynniebee.com Gwynnie Bee

      Hi Stephanie. Thanks for the feedback, your point is well taken. Gathering information around interest is critical – we have asked our members for feedback through surveys and focus groups on a continuous basis. And we’ve heard you, our assortment can be improved (among other things).

      Some want more workwear, others more casual. We even received a petition for gothic chic. But the results are not conclusive, we need more quantitative data to get to the right assortment for our members. We believe that before making any permanent decisions to the assortment, listening to the unbiased voices of our members – positive and negative – is our best way to ensure the quality of the Gwynnie Bee service long term.

      Thanks again for the feedback and we appreciate the time it took for you to respond. Keep the questions coming and we will continue to answer them as best as we can.

  • Heather

    I appreciate this thoughtful explanation, and the transparency of your process. Thank you for sharing this information. I’m sure you have a team dedicated to data analysis and this was the best process for data collection- and it seems like a sound decision for the business. Can you provide a link to the Pinterest page? I was unable to find it.

  • Kathy

    Please add to your list of questions how a garment is made and whether it is possible to develop a group of eco-friendly made and traded garments. I applaud gwynniebee for leading the industry in reducing the environmental impact of the number of clothes we purchase while making us look good! I believe you can increase your performance in this area by featuring designers who have made a commitment to fair trade (made) clothes and the environment. Which of your designers lead in this area?