A more sustainable alternative to plastic. It’s the CRO Roundtable Roundup!
Thanks to Iqbal Ali, Matt Beischel, Ana Kerkovic, Shiva Manjunath, Rhys Mohun, Craig Sullivan, Marcella Sullivan, David Swinstead, and Dewi Williams for joining us. Want to get in on the action and talk with other cool CRO people? Then you should probably…
One Sentence Takeaway
I’m a busy person; give me the TL:DR
AI and CRO shortcuts promise speed but risk trust, ethics, and long-term value without skilled human judgment.
The Notes
What we discussed this week
- Experimentation platform consolidation
- VWO and AB Tasty merger is a financial scale play
- Revenue per customer differences driving acquisition
- AI positioned as CRO acceleration
- PBX marketed as execution tooling rather than ideation support
- Skepticism toward AI driven front end changes
- Engineers are unlikely to endorse automated overrides
- Feature development is outpacing real user demand
- Competitive pressure shapes aggressive AI messaging
- Prototyping versus live testing
- AI tools enable fast visual proof of concepts
- Slide ready mockups improve stakeholder communication
- Console and CSS hacks achieve similar outcomes
- There should be a clear separation between demonstration and deployment
- Launching AI built variants is viewed as high risk
- AI output regression toward historical averages
- Models optimized on past patterns rather than future discovery
- Training data polluted by SEO driven content
- Baymard research contrasted with low quality internet advice
- Industry obsession with hacks
- Shortcuts framing attention over rigor
- Binary guidance flattens complex decision making
- Trust erosion spreading beyond individual practitioners
- Anti-patterns positioned as safer learning tools
- Best practice language fails across contexts
- Long term credibility gets sacrificed for short term reach
- Ethical pressure inside client work
- Limited choice for agency and in house roles
- Early career practitioners face harder tradeoffs
- Revenue pressure overriding methodological concerns
- Testing reframed as risk exposure
- Trust is a fragile system
- Damage accumulates invisibly over time
- Collapse occurrs suddenly rather than gradually
- Brand trust is harder to rebuild than conversions
- Small ethical cuts compounds toward failure
- AI as ideation support
- Divergent thinking supported through prompt exploration
- Absurd outputs seeding viable human insights
- Discernment capability earned through expertise over time
- Novices unable to judge output quality
- Vibe coding obscures technical debt
- Expertise is required to extract signal from noise
- Consistency demands conflicting with creative strengths
The Quotes
Standout quips from this week
"By using AI, you’re solving for the average of the past"
“Trust degradation is like an inverse hockey stick graph”
“If you’re trying to polish a turd, there it is”
“It removes the nuance from the actual field”
“That’s a bell you cannot unring”
Book Club
Relevant reads recommended this week
Responsible AI
Guides readers step-by-step through the process of establishing robust yet manageable ethical AI initiatives for any size organization, outlining the three core pillars of building a responsible AI strategy: people, process and technology.
Lateral Thinking
In schools we are taught to meet problems head-on: 'vertical thinking'. This works well in simple situations - but we are at a loss when this approach fails. What then?
Hooked
How do successful companies create products people can’t put down? Why do some products capture widespread attention while others flop? What makes us engage with certain products out of sheer habit? Is there a pattern underlying how technologies hook us?
Evil by Design
Learn how companies make us feel good about doing what they want. Approaching persuasive design from the dark side, this book melds psychology, marketing, and design concepts to show why we're susceptible to certain persuasive techniques.
CRO Link Digest
Useful and thought-provoking content shared this week
- Much respect to Optimizely for sending this out – LinkedIn post by David Swinstead regarding Optimizely’s nuanced take on use of AI
- AI Heuristic UX Evaluations with a 95% Accuracy Rate (Human-Level Accuracy) – article for Baymard Institute by Christian Holst
- Sample Size Calculator (Free) – by AB Testguide
- Power & Sample Size Calculator (Free) – by GIGA Calculator
- Sample Size Calculator (Paid) – by Analytics Toolkit
- The Trust Thermocline Explains How Companies “Suddenly” Lose Customers And Employees – Medium article by Adam Fisher
Off-Topic Sidebars
Experimentation isn’t the only thing we talk about at the CRO Roundtable. There’s often a healthy dose of discussion on shared interests, personal passions, and hobbies.
- Questionable wellness products
- Powders promising effort free fitness
- Placebo effects
- Packaging aesthetics substituting for evidence
- Mushroom coffee
- Collagen supplements marketed through fear of aging
- Food culture
- Steamed ginger cabbage soup
- Crock pot techniques for creating natural stock
Sidebar Shareables
Amusing sidebar content shared this week
No Sidebar Shareables this week, sorry!