Beware the Black Box: A/B Testing, Randomized Controlled Trials, and Experimentation

Beware the Black Box: A/B Testing, Randomized Controlled Trials, and Experimentation

More and more companies are relying on online and offline experiments, in the form of A/B tests and randomized controlled trials, to assess the effectiveness of different treatments (e.g., design choices, marketing promotions, product descriptions). However, the ability to draw causal inferences from these tests hinges on a number of key assumptions that may not always be met. This is particularly true for online platform studies (e.g., Facebook A/B tests, Google AdWords) whose underlying algorithms—often a “black box” to experimental researchers—can result in skewed or “divergent delivery.”  

In this talk, Dr. Isaac will discuss the importance of properly determining whether studies can provide causal versus correlational insights, particularly given the unique value that experiments offer as part of the marketing research “toolbox.” He will also share best practices in experimental research and offer guidance on how to position, design, conduct, and report offline and online studies. The discussion should benefit seasoned researchers who are experienced in running and analyzing experiments, as well as practitioners who are less familiar with this methodology.

Our lunch and meal choices are as follows:

• Cedar Plank Roasted Wild Sockeye Salmon – with seasonal sides
• Fish ‘n Chips – Ivar’s World-Famous dish!
• Blackened True Cod Tacos – Spicy blackened Pacific rockfish, cilantro-cabbage slaw, Cajun remoulade, mango salsa,
  flour tortillas, house-made tortilla chips, pico de gallo, sour cream drizzle
• Cheese Burger – 1/2 pound Newport Pride Beef, Tillamook cheddar, bistro sauce, lettuce, tomato, red onion
• Chicken Saltimboca – Alder grilled chicken, fresh sage, thin sliced prosciutto, smoked mozzarella
  roasted pear, lemon-butter sauce
• Grilled Portobello Stack – Alder grilled Portobello mushroom, roasted red pepper, grilled zucchini and yellow squash,
  sautéed spinach, House specialty rice, balsamic syrup

Professor of Marketing, Chair of the Marketing Department

Mathew Isaac obtained his PhD in Marketing from the Kellogg School of Management at Northwestern University. At the Albers School of Business and Economics at Seattle University, Dr. Isaac teaches Introduction to Marketing, Sales Management, Brand Management, and Marketing Strategy to undergraduates and MBA students. His research focuses primarily on consumer judgment and decision-making, examining how contextual and motivational factors influence product evaluations and purchase intentions. His work has been published in leading scientific journals and featured in business and popular press, including Harvard Business Review, Time, Forbes, Wall Street Journal, The Atlantic, and Fast Company. Prior to entering academia, Dr. Isaac advised media, healthcare, and private equity clients as a Consultant and Manager for Bain & Company and ZS Associates.