Winners will receive notification via email. It’s the CRO Roundtable Roundup!
Thanks to Matt Beischel, Iqbal Ali, Shiva Manjunath, and Craig Sullivan for joining us. Want to get in on the action and talk with other cool CRO people? Then you should probably…
One Sentence Takeaway
I’m a busy person; give me the TL:DR
Structured context, smart metrics, and disciplined testing turn noisy data into reliable optimization insight.
The Notes
What we discussed this week
- Context engineering for AI decision-making
- tailor scraped content before model ingestion
- AI needs structured data to make informed decisions
- generic tools often fail to prioritize important information
- context engineering is not solely a technical function
- segment metadata to guide model focus
- Rising AI hallucination rates
- AI hallucination can propagate errors, and chain-of-calls compound errors
- using the analogy of phantom road traffic to illustrate AI halucination propagation
- multiple AI calls increase chances of hallucination
- Price-ending and discount psychology
- ending a price in a non-zero digit is called “charm pricing” and it’s effects are culturally dependent
- round prices signal quality but not strongly supported
- testing different price endings can yield varied results
- Messaging savings effectively impacts customer perception
- the “rule of 100” – when comparing currency versus percent displays, show whichever discount ammount is the larger number
- regional pricing models affect fairness perception
- Experimentation hygiene
- AA tests catch noise and SRM early
- split-URL redirects risk leakage issues
- multiple control groups strengthen data integrity
- SRM kills validity
- The why’s of A/A Testing
- Run A/A tests help diagnose SRM issues
- Validating new metrics and setups
- Identify discrepancies in analytics data
- Understand noise in specific test areas
- The video game industry focuses on the wrong metrics
- loot-box rewards drive addictive play
- metrics chosen to please executives
- cherry-picked metrics misrepresent user satisfaction and ignore churn
- user data is often overlooked in decision making
- Meeting structure & semantic HTML
- good agendas improve AI note-takers
- headings and IDs aid website comprehension
- HTML5 landmarks support accessibility
- Form Design and Errors
- Errors increase form abandonment rates
- Design can mitigate errors through inline correction
- Understanding user input intentions is crucial
The Quotes
Standout quips from this week
- “Everything is okay in the bay.” — Iqbal Ali
- “Bots suck. They’re so obviously bad.” — Shiva
- “It’s like putting a moving sidewalk on top of a moving sidewalk.” — Matt Beischel
Book Club
Relevant reads recommended this week
- No books this week, sorry!
CRO Link Digest
Useful and thought-provoking content shared this week
- A meta-analysis on the effects of just-below versus round prices – research paper by Eve Sarah Troll, Julius Frankenbach, Malte Friese, and David D. Loschelder
- Round, just-below, or precise prices? Cultural differences in the prevalence of price endings in E-commerce – research paper by Alfio Ventura, Eve Sarah Troll, Meikel Soliman, and David D. Loschelder
- Should you run A/A experiments? – Medium article by Iqbal Ali
- What Travels The Wrong Way Down The Motorway At 12mph? – YouTube clip from British panel show QI explaining phantom road traffic
- How to Display Price Discounts on the Product Page: Avoid These 4 Pitfalls – article from the Baymard Institute
- Hofstede’s cultural dimensions theory – a framework for cross-cultural psychology, showing the effects of a society’s culture on the values of its members, and how these values relate to behavior
- Zuko Form Analytics – online form + checkout analytics and optimization tool
Off-Topic Sidebars
Experimentation isn’t the only thing we talk about at the CRO Roundtable. There’s often a healthy dose of discussion on shared interests, personal passions, and hobbies.
- Cycling the London canal network
- wildlife and zoo monkeys spotted
- stopping at cafés and eating fresh fruit
- The video-game industry reliance on bots
- an overabundance of bots inflate engagement, which are used to mask server issues
- Solos mode with bots is not popular among players
- Cheating and server issues are not adequately addressed
- Video-game price hikes
- setting market expectations with $80 titles
- the risk of poor-quality games becoming overpriced
- publishers testing higher tiers
- Quick freelance gigs
- two hours work is better than nothing
- temporarily joining an agency as a hired gun
Sidebar Shareables
Amusing sidebar content shared this week
- Nintendo Switch 2 Games Will Cost $80 For Digital, $90 For Physical – Reddit thread
- After the Switch 2’s Pre-Order Success, I’m Convinced $80 Games Are Here To Stay – op-ed article from ScreenRant