 |
| Products | Recent Posts | Blogroll | Categories | Affiliates | Advertising |
| Affiliates |
|
Multivariate Burrito Test A friend of mine recently bought a restaurant that specializes in making burritos in a popular sub-tropical beach town. He asked for my help in improving his burrito restaurant. I thought it was funny when he said "of course, I can't use multivariate testing with this business." Why not? In fact, if he wanted to retain the title of "best damn burrito in the Caribbean", he better be doing multivariate testing. The first few times, I explained the process... his eyes glazed over. I finally broke it down simply enough that he got it. I decided that I should write it up so everyone can see how easy it is to perform multivariate testing to enhance the quality of their business. In this case, the business has only a few products and clearly the most important products are five different kinds of burritos. People come to this restaurant because they hear about the burritos how good they taste. People come back again and again because they know how good they taste. So, you don't mess with the recipes; right? WRONG!Every business is either growing or dying. Every burrito recipe is either getting better or worse. The prior owner confirmed that she had constantly tested new burrito recipes and highly recommended continuing to improve her recipe. This scared my friend a bit (the new owner) because he was concerned that any test he made would make the recipe worse because it had been optimized so well. So I actually helped him design his first multivariate test. We decided to use his chicken asada burrito as the first type of burrito to test. And I encouraged him to test each ingredient against "nothingness" because so many people see to forget to test that. The simple test was a huge winner! Here is how it worked. We chose five variables (five items that go in each chicken burrito) and tested them against "nothingness" or not having those items. So every burrito had a tortilla and chicken that had been marinaded and grilled using the current recipe (but he committed to testing that in a future test). The other ingredients that he decided to make his four variables were: 1. Pico de Gallo (a mild salsa of chopped tomatos, onions, cilantro and lime juice). 2. Natilla (a dairy product that is similar to sour cream). 3. Cheddar cheese. 4. Jalapeno sauce. 5. Spanish rice. I showed him how to lay out 32 wrappers on a work table in the kitchen of his new restaurant and number them. It was four rows of eight wrappers across the table. Then the tortillas went on top and the chicken. Now every burrito became different. They were numbered from 0 to 31 with zero in the upper left corner. Starting from the lower right corner, we put pico del gallo on the bottom two rows only. Then natilla went on only the bottom row and the 2nd row. Then cheddar cheese went on only the right side of each of he four rows. Then jalapeno sauce went on two in the lower right, then working to the left, two were skipped and then two received jalapeno sauce. That pattern repeated until the upper left was reached. Burrito 0 and 1 didn't receive any jalapeno sauce because burrito 2 and 3 did. Finally spanish rice was put on every other burrito starting with the lower right corner. The burritos were then wrapped with their numbers showing on the outside and we headed down to the beach. My friend was concerned about doing a taste test with current repeat customers because he thought it might drive them away (even though the prior owner did taste tests every week with current customers). This is a popular island that receives a lot of tourists this time of year. They can usually be found on the beach at about lunch time, so that's where we headed (tourists will be gone next week so my friend didn't have to worry about offending repeat customers). We then offered a free burrito in return for a taste test to each person we encountered. When they accepted, we explained that we were really going to give them two burritos and ask them to take a bite out of each one and tell us which one they liked better. The burritos were selected at random and the two under test were recorded for each person along with the number that the person preferred. DUring this time, we had no idea what was on the burritos being served. But we received clues from the taste testers. Some said "wow; this one is spicy, but I really like how cheesy it is." So we knew that the burrito they were tasting had cheese and jalapeno sauce. But we ignored that. We had that information back at the restaurant. We just recorded the numbers and the winner in each of the 16 tests. The entire taste test took 15 minutes to prepare and 20 minutes to give out 16 pairs of burritos and get the results. Is it statistically significant. Nope. We could repeat it several times if we wanted to get statistically significant results and pass some test in college. But you really don't need to do that until all of your results start coming very close to zero. We do have 16 data points. That is just four short of being able to do a chi squared test for statistical significance. But really, we can look at the results and see how silly that is (or how important it is) at a glance. We went back to the restaurant and tallied the results for each of the five variables. The lower received a negative 1 score. The winner received a +1 score for each element. I had already done this test dozens of times in the past and knew that: 1) The results will always be an even number, so divide by two when finished. 2) A final result of 3 (or -3) or more would almost certainly be found to be statistically significant at a power of 90 if we repeated the test. A score of significantly higher than 3 (max score can be 8) gets even more certain to be a winner (and lower than -3 a loser). We don't need to do any fancy math. We just look for scores of 3 or more (or -3 or less). We had three of those out of the five variables!!!Since the prior owner tested repeatedly, I was certain I was right. They forgot to test against nothingness. Before I explain that, let me give you the results: Pico de Gallo: +2Natilla: -4Cheese: +4Jalapeno: +4Rice: 0So Natilla was a clear loser. People didn't want this form of sour creme on their burritos. Cheese and Jalapeno sauce were clear winners. Rice and pico de gallo where in the grey area. Those results clearly show that the prior owner had focused a great deal on choosing the recipe for all of their macro-ingredients, but they had never tested to find out if they should even use that ingredient. They hadn't tested against "nothingness" before now. We confirmed the results by making 16 burritos using the old recipe and 16 burritos using that same recipe, but omitting the Natilla. The new recipe one 12 out of 16 times in the next test. Natilla costs money and had been added to every burrito, but it was actually detracting from the taste. In addition, the jalapeno sauce had previously only been added on request. When my friend heard a taste tester saying "oh my God is this good... it is so spicy and I don't usually like spicy, but it is so good." He watched in astonishment as she ate the entire burrito while exclaiming that she didn't like hot sauce usually, but this was so good that she couldn't stop. That along with the test results made him decide to put the jalepeno sauce on by default unless someone requested to have hot sauce excluded. In one day with about an hour of work, he had dramatically improved the taste of the burrito recipe with multivariate testing. He couldn't stop talking about all of the things he would be testing next. He was hooked. And so were his customers... on the best tasting burrito served on the entire island... and constantly getting even better. |
||
You May Also Like...Reverse Engineering and Statistics - Part 1 Test Results: Internet Marketing Forums |
||