"The focus is often on initial integration and feature deployment; less attention is paid to the continuous evaluation infrastructure that ensures features remain effective, accurate, and reliable over time."
Issue 035TeardownAI for CommerceRef 105
Evaluation harnesses for commerce LLM features: the bit nobody ships
Integrating Large Language Models into commerce platforms is not a 'deploy and forget' task. True value comes from continuous, robust evaluation, a critical phase often overlooked.
Written by
Heddwyn Coombs
Creative and Digital Director
29 years at iWeb
Heddwyn co-founded iWeb and leads creative and digital direction across the agency. Adobe Commerce architect since the Magento 1 era, he writes the strategy notes on platform choice, headless, agentic checkout, AI for commerce and the trade-offs leadership teams hit when modern tooling meets operational reality. Opinionated about data structure, design systems and catalogue size, cautious about unnecessary frontend complexity, and focused on work that stays commercially manageable over time.