Advanced Paid Search Track: Multivariate Testing

1. Advanced Paid Search Track:Multivariate Testing

2. Most internet marketers understanding A/B testing as well as multivariate testing. Take landing pages and make variations of the original until you find the exact layout that drives in the most amount of sales for your organization. Most marketers however do not understand how to properly test and track their data. Most internet marketers understanding A/B testing as well as multivariate testing. Take landing pages and make variations of the original until you find the exact layout that drives in the most amount of sales for your organization. Most marketers however do not understand how to properly test and track their data.

3. Part 1: Understanding user behavior We assume that prior user behavior is characteristic across all of the landing pages that we are testing to build a statistical model Why? All landing pages will receive users from the same source (keywords, bids, geographic, language & ad copy) Give the same users the same test Do not have Test A in Germany and Test B in France. Run both Test A and Test B in Germany for accurate data It is important that how a user comes to your landing pages needs to be the same. If you have been bidding on your most high-volume keyword at $5 for the past week, do not drop the keyword to $1 and expect the same type of users finding your landing page. You need to be consistent with the type of traffic that comes to your website. Also remember that certain types of people will require different types of landing pages. Do not run a test where your �London Tours� campaig receives one landing page and your �London Vacation� campaign receives the other version. You need to run both landing pages A & B in the same campaign �London Tours� It is important that how a user comes to your landing pages needs to be the same. If you have been bidding on your most high-volume keyword at $5 for the past week, do not drop the keyword to $1 and expect the same type of users finding your landing page. You need to be consistent with the type of traffic that comes to your website. Also remember that certain types of people will require different types of landing pages. Do not run a test where your �London Tours� campaig receives one landing page and your �London Vacation� campaign receives the other version. You need to run both landing pages A & B in the same campaign �London Tours�

4. Same users. Same intent. Here is a perfect example of why users will behave the same when they find your landing page. They are thinking of the same things They all use the same keyword on Google They see the exact same ad/creative The only difference in whether they convert is how well your landing page is optimizedHere is a perfect example of why users will behave the same when they find your landing page. They are thinking of the same things They all use the same keyword on Google They see the exact same ad/creative The only difference in whether they convert is how well your landing page is optimized

5. Part 2: Setting up a proper test There are a lot of tools and services out there. Some are good, some are not. We won�t cover all of them because the list is quite exhaustive. In the past I have developed custom solutions on-site and has been brought in to help organizations integrate various solutions with the company�s current setup. I will cover the best solutions out there from my experience of spending millions of dollars. There are a lot of tools and services out there. Some are good, some are not. We won�t cover all of them because the list is quite exhaustive. In the past I have developed custom solutions on-site and has been brought in to help organizations integrate various solutions with the company�s current setup. I will cover the best solutions out there from my experience of spending millions of dollars.

6. Track the entire funnel There are options, but many times most are not enough You need more than just basic conversion tracking that the search engines provide Track: click >> lead >> signup >> order >> follow up orders First is you need to understand that whatever solution you decide on, you need to track your funnel. You need to track in addition to initial conversions, whether any subsequent conversions occur � whether it is a follow-up (sale after an initial lead) or a returning customer (buying another item online after being pleased with their first purchase)First is you need to understand that whatever solution you decide on, you need to track your funnel. You need to track in addition to initial conversions, whether any subsequent conversions occur � whether it is a follow-up (sale after an initial lead) or a returning customer (buying another item online after being pleased with their first purchase)

7. Do not set yourself up for failure If you copy and paste existing ads and just change the destination URL�. Google can favor your older ad/url because of the quality score and history Your original destination url (older ad) may have a higher position than your new destination url (new ad) so you get different quality traffic Don�t copy and paste you ads. Instead, use one ad and have an interstitial page to redirect users for your test Click One goes to Page A, Two goes to B, Three to C, Four goes to A, � The �quickest fix� for marketers is after they find a winning creative, they immediate copy and paste the ad and start testing various landing pages. Remember that Google can still favor your old ad creative. I have seen in the past that after copying and pasting a high CTR ad for a new landing page that my keyword position dropped from Position 2 to Position 2.4 � what happened is that my old ad (and original landing page) was at Position 2 and my new ad (test landing page) was at Position 3 � this caused my test ad to receive lower quality traffic and did not allow me to perform a proper test because there was bias for my old ad. Using a redirection script for all of your ads will avoid you having to copy and paste new urls. Have your redirection script decide where to send traffic so Google does not change your quality score (QS)The �quickest fix� for marketers is after they find a winning creative, they immediate copy and paste the ad and start testing various landing pages. Remember that Google can still favor your old ad creative. I have seen in the past that after copying and pasting a high CTR ad for a new landing page that my keyword position dropped from Position 2 to Position 2.4 � what happened is that my old ad (and original landing page) was at Position 2 and my new ad (test landing page) was at Position 3 � this caused my test ad to receive lower quality traffic and did not allow me to perform a proper test because there was bias for my old ad. Using a redirection script for all of your ads will avoid you having to copy and paste new urls. Have your redirection script decide where to send traffic so Google does not change your quality score (QS)

8. Use interstitial pages for your ads A great solution for PPC tracking is Tracking202.com Made for internet marketers in mind. Free version and Pro version -- affordable for anyone�s budget A great solution for tracking ads, landing pages and keywords is Tracking 202. Check them out, I highly suggest using them. They provide much more insight about your keywords when compared to Google conversion tracking (I won�t get into too many details but if you visit tracking202.com and try them out you will see for yourself) A great solution for tracking ads, landing pages and keywords is Tracking 202. Check them out, I highly suggest using them. They provide much more insight about your keywords when compared to Google conversion tracking (I won�t get into too many details but if you visit tracking202.com and try them out you will see for yourself)

9. Commercial tools for click tracking You need to integrate all your paid marketing: AdWords, Yahoo, media buys, affiliates� HitPath.com (highly suggested) Very reliable Great customer support Excellent replacement for the basic conversion tracking that search engines provide Easy to integrate with your paid online marketing efforts Track multiple conversions (lead, signup, order) Integrating all of your paid marketing channels is key. Only Google conversion tracking will tracking Google conversions. Yahoo conversion tracking will track Yahoo� but who will track both for you? You either have to build something in-house which can be very expensive and may not perform properly if you do not know what you are doing, or you can use various solutions. There are several tracking solutions out there but my favorite is HitPath. Integrating all of your paid marketing channels is key. Only Google conversion tracking will tracking Google conversions. Yahoo conversion tracking will track Yahoo� but who will track both for you? You either have to build something in-house which can be very expensive and may not perform properly if you do not know what you are doing, or you can use various solutions. There are several tracking solutions out there but my favorite is HitPath.

10. After we setup our tracking� Ok. We have decided to use Tracking202 and HitPath� a winning combination (by the way, I do not get paid by either of these two organizations� I am suggesting them because they are rockstars at what they do)Ok. We have decided to use Tracking202 and HitPath� a winning combination (by the way, I do not get paid by either of these two organizations� I am suggesting them because they are rockstars at what they do)

11. Part 3: Statistical testing A basic test you can do yourself is G-Test G-Test calculator is available online: http://manybodythoery.com/smxlondon Very effective for search marketing Based off of spending millions of dollars in paid search Find out more info at http://manybodytheory.com A simple test you can use is called the G-Test. There is also Chi Squared and Standard Deviation � in the end all of these kinds of formulas are derived from the basics of the Central Limit Theory� I won�t bore you with the mathematics but most likely all you will need to use is the G-Test � if you are an extreme exception (ie: eBay) � contact me at nick@manybodytheory.comA simple test you can use is called the G-Test. There is also Chi Squared and Standard Deviation � in the end all of these kinds of formulas are derived from the basics of the Central Limit Theory� I won�t bore you with the mathematics but most likely all you will need to use is the G-Test � if you are an extreme exception (ie: eBay) � contact me at nick@manybodytheory.com

12. Looking at actual data� Cost too high? Lower your keyword bids Never make a decision based off of your CPA Remember: You can not change the �bid� of a destination URL CTR too low? Update your ad creative NEVER make a decision based off of what your cost-per-conversion is. If you are spending too much for a conversion you need to lower your keyword bids. The only things you care about are your clicks (trials) and conversions (successses) NEVER make a decision based off of what your cost-per-conversion is. If you are spending too much for a conversion you need to lower your keyword bids. The only things you care about are your clicks (trials) and conversions (successses)

13. Now that you have your data� Originally I ran everything in the G-Test as 3 versions but to make it easier on the eyes, I then changed the results to A/B and B/C to chart it out more clearer. You can run the test yourself at http://manybodytheory.com/smxlondonOriginally I ran everything in the G-Test as 3 versions but to make it easier on the eyes, I then changed the results to A/B and B/C to chart it out more clearer. You can run the test yourself at http://manybodytheory.com/smxlondon

14. You need to have real data Always use a reasonable size data set Golden Rule: No more than 10% of your days can have under 5 conversions to ensure some level of confidence A one week test needs 5 days with at least 5 conversions per day(5 days x 5 conversions = 25 minimum plus data from other two days) You need a reasonably sized data set. If you have 2 clicks and 2 conversions do you really expect to maintain a 100% conversion rate? You need a reasonably sized data set. If you have 2 clicks and 2 conversions do you really expect to maintain a 100% conversion rate?

15. Never stop testing� Always let the data speak for itself. Do not get caught up with what you wanted the outcome to be If you expected Test B to succeed but data shows Test A � look at the difference between the pages and expand Be realistic. Having two clicks with two conversions is nice but can you maintain a 100% conversion rate? Always have a question that you want to answer before you begin your test (ie: �What color font should we use�) Always let your data speak for itself. Figure out the best method of gaining new customers and incorporate it in your marketing strategy. Never stop testing. Keep a journal of questions that you want to know about your website/users �Do people like blue links or green?� -- �Is there more trust to signup if I have https or will http provide the same number of new users?� � �Should I have a long sales pitch or just a signup form?� Never start a test without having a question that you want answered (so always have a question and always test to find the answer)Always let your data speak for itself. Figure out the best method of gaining new customers and incorporate it in your marketing strategy. Never stop testing. Keep a journal of questions that you want to know about your website/users �Do people like blue links or green?� -- �Is there more trust to signup if I have https or will http provide the same number of new users?� � �Should I have a long sales pitch or just a signup form?� Never start a test without having a question that you want answered (so always have a question and always test to find the answer)

16. Part 4: Additional Resources My blog (lots of cool stuff) www.ManyBodyTheory.com Personal email address (for Q&A) nick@manybodytheory.com Want to work with Zoosk? nicka@zoosk.com My contact information has been included � have a great time at SMX London 2009!My contact information has been included � have a great time at SMX London 2009!