Website optimization testing sometimes resembles throwing a dart at a dartboard blindfolded. Done right, however, a company can easily hit a bull's eye. This was the case for Chicago.com, which achieved a double-digit lift in key site metrics through optimization testing.

Chicago.com, an online guide that highlights the city's neighborhoods and attractions, recently deployed a solution that analyzes numerous variables to determine which interface designs will increase clicks and show better conversion rates. Josh Metnick, CEO of Chicago.com, says the goal is to continuously increase conversions and drive revenue, because the website is primarily a commerce site. Having the ability to constantly test is essential, he says.

Chicago.com had been conducting basic testing through Google, but Metnick knew the site would benefit from an in-depth approach. In February the company deployed Amadesa's Customer Experience Platform to conduct A/B testing for placement, design, colors, and content. Now Chicago.com executes page-level tests and identifies visitors across a variety of criteria, namely demographics and buying history, to enable personalized communications (the site hasn't yet used this functionality). An analytics algorithm leverages characteristics such as screen resolution and time of day , and the platform sends automated suggestions for product add-ons to the marketing team on a regular basis. For example, after drilling down into the data for theater ticket purchases and realizing that 72 percent were bought for Wicked, the site received a suggestion to add an advertisement for Wicked tickets to the home page.

With multivariate testing, Chicago.com can test 15 variables at a time and examine 225 different combinations; for example, changing colors, content, and item placement. One test might include a green icon that says "book hotels now" on the left side of the site, and another might be green but reads "search hotels" on the right side.

With the testing solution up and running, the first thing Metnick and his team began testing was the various colors of icon buttons. They measured the impact of changing the blue "search hotels" icon to a purple one. The results surprised them. "With changing the button color, we noticed a 25 percent increase in variation in conversions," Metnick says. "To see the impact of changing the button color on a conversion rate was mind blowing."

Another surprising discovery was a 50 percent difference in the number of people clicking on yellow buttons versus blue buttons. Yellow is the worst color, Metnick explains. In addition, he moved the Chicago.com newsletter signup button from the left side to the vertical navigation bar and saw a 300 percent increase in registrations. "If you can get a .75 lift in conversions due to subtle adjustments, you pay for your salaries," he says.

Within the first few weeks of testing the team saw a 25 percent lift in hotel bookings, Metnick says. Next, the site will focus on testing different interface designs in the real estate and ticketing sections of the website. Theater tickets account for the site's second largest revenue source after hotel bookings.

"This type of data testing is everything to us," he says, adding that there are an infinite number of elements his company can test. "This is all we have at the end of the day to present an interface to the millions of users every year and present something that they want to see, as opposed to taking guesses on what may be interesting to them."