When a Successful Sale Becomes a Business Failure
Consider a familiar e-commerce scenario. A customer purchases a refrigerator online after comparing specifications, reviewing product images, and evaluating prices. The transaction completes successfully, generating revenue for the retailer and boosting its conversion metrics. However, when the refrigerator arrives, the customer discovers that it cannot fit through the intended installation space.
The order is canceled or returned. The retailer incurs additional transportation, handling, and customer service expenses. What initially appeared to be a successful transaction ultimately produces a negative business outcome. This hypothetical scenario illustrates a fundamental limitation in conventional e-commerce performance measurement.
Conversion measures whether a transaction occurred. It does not necessarily measure whether that transaction created lasting value for the customer or retailer.
E-commerce product management has traditionally prioritized metrics such as conversion rate, add-to-cart rate, average order value (AOV), and gross merchandise value (GMV). Although these metrics remain important, they primarily capture customer behavior before or during a transaction. They provide limited visibility into outcomes such as product suitability, fulfillment success, preventable returns, and post-purchase satisfaction. This article examines the limitations of conversion-centric measurement and proposes a Purchase Quality Scorecard, a practical framework for connecting purchase decisions to downstream customer and financial outcomes. The objective is not to replace conversion optimization, but to broaden how product managers define and measure success.
1. The Economic Problem With Conversion-Centric Optimization
Product returns represent a substantial economic challenge for online retailers. According to the National Retail Federation’s 2025 Retail Returns Landscape, U.S. retailers projected approximately $849.9 billion in merchandise returns during 2025, with an estimated 19.3% of online sales expected to be returned. This estimate reflects the share of online sales value returned, not the percentage of individual orders. Returns introduce costs across several operational functions, including reverse logistics, product inspection, repackaging, inventory management, and customer support. However, these costs don’t always show up in product performance dashboards. Consider two e-commerce experiences.
Experience A prioritizes a frictionless checkout process designed to maximize transaction completion.
Experience B adds contextual information that helps customers decide whether a product meets their needs before purchasing.
Experience A may generate more immediate orders, whereas Experience B may prevent some unsuitable purchases. If both experiences are evaluated exclusively through checkout conversion, the additional transactions generated by Experience A could make it appear more successful. But if Experience B generates more retained purchases, with lower return costs, it could create greater economic value.
This reveals an important distinction: Maximizing the number of transactions is not necessarily equivalent to maximizing the value of transactions.
Product managers should therefore evaluate both conversion efficiency and the quality of the purchasing decisions their products enable.
2. Understanding Product Fit Uncertainty
One reason customers make unsuitable purchases is the difficulty of evaluating products in digital environments. Unlike physical stores, online shopping limits a customer’s ability to inspect dimensions, assess materials, test functionality, or evaluate compatibility directly. Researchers have studied this problem through the concept of product fit uncertainty.
Hong and Pavlou (2014), in their study Product Fit Uncertainty in Online Markets: Nature, Effects, and Antecedents, distinguish between two dimensions of uncertainty:
- Product quality uncertainty: Whether a product performs as expected or meets quality standards.
- Product fit uncertainty: Whether a product’s attributes align with an individual customer’s requirements or preferences.
Their research found that fit uncertainty was more strongly associated with product returns than quality uncertainty in the market studied. This distinction has important implications for e-commerce product management. A retailer may provide accurate specifications, detailed product descriptions, and high-quality images while customers still struggle to determine whether a product suits their needs.
For example, listing a household appliance’s dimensions does not necessarily show whether it fits a particular installation environment. Similarly, a laptop’s specifications might be accurate, but customers may not understand whether its capabilities meet their intended workload. The issue is not always missing product data. Often, the gap is between the information provided and the customer’s ability to interpret it in their own context.
Evidence From Digital Fit Technologies
Additional research demonstrates the potential business value of reducing this uncertainty. Gallino and Moreno (2018), in The Value of Fit Information in Online Retail: Evidence from a Randomized Field Experiment, investigated virtual fitting technologies through randomized field experiments. Their findings showed that providing virtual fit information increased conversion rates and order value while reducing fulfillment costs associated with returns and customers ordering multiple sizes. This matters because it challenges the assumption that improving purchase quality necessarily reduces immediate conversion.
Under certain conditions, better decision support can improve transaction performance while also improving downstream operational efficiency. Wang, Ramachandran, and Sheng (2021), in Do Fit Opinions Matter? The Impact of Fit Context on Online Product Returns, further demonstrated the importance of contextual fit information. Their research found that customer fit evaluations become more useful when shoppers can interpret those evaluations against relevant reference information.
Together, these findings suggest a broader product management principle: Effective e-commerce experiences should not simply present product information. They should help customers interpret that information to make suitable purchasing decisions.
The cited studies primarily examine product fit in online retail settings such as apparel. Applying their findings to appliances, furniture, or electronics requires category-specific testing rather than assuming identical outcomes.
3. Why Traditional E-Commerce Metrics Are Insufficient
Most e-commerce product teams have established ways to monitor acquisition, engagement, and transaction performance.
|
Measures purchasing efficiency |
Whether purchases are retained |
|
|
Measures initial purchase intent |
||
|
Measures total transaction value |
||
|
Measures checkout effectiveness |
Fulfillment and post-purchase outcomes |
These limitations do not make traditional metrics ineffective. Instead, they reflect the customer journey stages those metrics were designed to evaluate. The challenge arises when product teams use transaction-stage metrics as the primary indicators of overall business success. For instance, an A/B experiment might identify a product page variation that increases checkout conversion by 3%. The team may consider the experiment successful and deploy the variation. However, suppose the variation also increases the frequency of unsuitable purchases, resulting in additional returns.
Without connecting transaction data to post-purchase outcomes, the team may never discover the downstream impact. This suggests that e-commerce performance measurement should move beyond isolated funnel metrics toward a more comprehensive view of the purchase lifecycle.
4. A Proposed Framework: The Purchase Quality Scorecard
To address this measurement gap, I propose the Purchase Quality Scorecard (PQS). The PQS is a conceptual product management framework that combines established business metrics into a coordinated evaluation of purchasing outcomes. Its contribution is not inventing individual metrics. Instead, it structures the use of pre-purchase, post-purchase, financial, and customer experience measures in product experimentation and prioritization. The framework contains four dimensions.
Dimension 1: Retained Conversion Rate
Traditional conversion measures the proportion of eligible shopping sessions that produce orders. Retained conversion measures the proportion of sessions that produce orders not returned within a defined observation period.
For a simplified, single-order-per-session model: Retained Conversion Rate = Orders Retained After the Return Window ÷ Eligible Shopping Sessions × 100
For implementations involving multiple orders, exchanges, or multiple items per order, teams must establish a consistent unit of analysis. Consider the following hypothetical experiment.
Two product page variations receive the same number of eligible shopping sessions.
Version A generates 100 more initial orders. However, Version B produces 110 more retained orders. Traditional conversion measurement favors Version A. Retained conversion favors Version B.
In this hypothetical example, Version B achieves approximately 3.2% higher retained conversion relative to Version A. These calculations are illustrative and do not represent actual experimental findings. They also do not establish that Version B is more profitable. You must also consider differences in product prices, margins, acquisition costs, and fulfillment expenses. Nevertheless, the example shows how post-purchase measurement can change how you interpret product performance.
Dimension 2: Preventable Return Rate
Not every product return results from a poor purchasing decision. A retailer must distinguish among causes such as manufacturing defects, delivery damage, buyer preference changes, and incorrect product selection. The Purchase Quality Scorecard focuses specifically on returns plausibly influenced by pre-purchase information or decision-support experiences.
A proposed measure is: Preventable Return Rate = Returns Attributed to Addressable Pre-Purchase Decision Gaps ÷ Delivered Orders × 100
For example, returns resulting from misunderstood dimensions, compatibility requirements, or product capabilities may be potentially addressable through improvements to digital product experiences. In contrast, transportation damage generally requires operational interventions rather than changes to product pages. Accurate return classification is essential.
Customer-selected return reasons may be incomplete, inconsistent, or ambiguous. Product teams should therefore supplement them with support conversations, delivery feedback, installation reports, and other operational evidence where available. Importantly, classify a return as potentially preventable based on a defined causal hypothesis, not merely because the product team believes it could have been avoided.
Dimension 3: Contribution Margin per Eligible Session
Retained orders alone do not establish financial value. A business may reduce returns while also losing profitable sales. Similarly, two retained orders may generate different contribution margins depending on product costs and fulfillment requirements. I therefore recommend evaluating contribution margin at the shopping-session or experiment-cohort level.
Contribution Margin per Eligible Session = Total Net Contribution From the Cohort ÷ Eligible Shopping Sessions
The numerator should include recognized revenue net of refunds and relevant variable costs, such as product costs, fulfillment, payment processing, and return handling. Finance teams should develop a precise definition to avoid inconsistent accounting treatments or double-counted expenses. This metric helps product managers determine whether better purchase decisions translate into genuine business improvements. For example, a decision-support feature might reduce checkout conversion slightly while increasing retained sales and reducing reverse-logistics costs. Alternatively, the same feature might discourage too many profitable customers from purchasing. Evaluating net contribution alongside retained conversion helps distinguish between these outcomes.
Dimension 4: Post-Purchase Customer Experience
A retained order is not necessarily a satisfactory order. Customers may retain products despite dissatisfaction because returning them is inconvenient, expensive, or time-consuming. Consequently, retained conversion and financial performance should be complemented by measures of post-purchase experience. Relevant indicators include customer satisfaction, product-related complaints, unsuccessful installations, service contacts, and repeat-purchase behavior. Interpret these measures as complementary indicators rather than combine them into an arbitrary numerical score. Use the Purchase Quality Scorecard as a coordinated set of metrics with explicit decision rules, not as a single weighted score whose components can hide important trade-offs.
5. How Product Managers Can Operationalize the Framework
The Purchase Quality Scorecard’s primary value lies in how product teams use it to influence decisions. I propose a five-stage implementation approach.
Stage 1: Identify Addressable Return Problems
Begin by analyzing product returns across major categories and identifying their financial and customer impact.
Return reasons should be classified into groups, such as:
- Product fit or compatibility problems
- Misunderstood features or product expectations
- Fulfillment-related failures
- Manufacturing or quality defects
- Customer preference changes
The objective is to distinguish causes that product experience improvements can influence from those that require other interventions. Product teams can prioritize opportunities based on return frequency, economic impact, customer dissatisfaction, and intervention feasibility.
Stage 2: Define the Purchasing Decision Gap
For each addressable return category, identify the information or decision-support capability customers lacked before checkout.
The central research question becomes: What information would have helped customers make a more suitable purchasing decision before completing the transaction?
Possible interventions include fit calculators, compatibility verification, contextual recommendations, comparison tools, interactive product demonstrations, and clearer specification guidance. Importantly, the intervention should follow problem discovery. Developing sophisticated features without identifying a specific purchasing uncertainty may introduce complexity without delivering measurable value.
Stage 3: Formulate a Testable Hypothesis
A meaningful product hypothesis must connect an intervention with an expected behavioral and business outcome.
For example: Providing contextual compatibility information before checkout will reduce returns caused by product incompatibility without materially reducing contribution margin per shopping session.
This hypothesis is more comprehensive than simply predicting increased engagement with a new compatibility tool. It identifies the expected benefit and establishes an important financial guardrail. The hypothesis also creates a basis for evaluating whether the intervention successfully addresses the underlying problem.
Stage 4: Design Experiments Across the Purchase Lifecycle
Where feasible, randomized controlled experiments provide a rigorous approach to evaluating purchase-quality interventions. Randomly assign customers to treatment and control experiences, allowing product teams to compare outcomes while reducing selection bias. For repeat visitors, assigning experiments at the user level rather than independently for every session may help prevent inconsistent experiences and contamination.
The experiment should include:
Leading indicators: Feature exposure, engagement, add-to-cart behavior, and checkout conversion.
Downstream indicators: Fulfillment success, product-specific returns, retained conversion, and net contribution.
Guardrail metrics: Customer satisfaction, checkout abandonment, service contacts, and relevant operational costs.
An important methodological consideration is the return observation window. A return measured seven days after purchase cannot automatically be compared with one measured after 45 days. Teams should set consistent return-maturity periods, ideally starting at delivery, and exclude or separately account for orders whose observation periods have not yet ended. Another important issue is selection bias. Comparing customers who voluntarily use a compatibility tool with those who do not may produce misleading conclusions. Customers who choose to use decision-support features may already differ in purchase intent, uncertainty, or product knowledge. Randomly assigning users to feature availability, then analyzing all assigned users, provides a more defensible estimate of the intervention’s impact.
Stage 5: Translate Findings Into Product Decisions
The final stage involves interpreting results across the scorecard’s four dimensions. A feature should not automatically be considered successful simply because it reduces returns.
Consider three possible outcomes:
Outcome A: Conversion and retained conversion both increase while return-related costs decline. This represents a potentially strong outcome, subject to profitability and validation of the customer experience.
Outcome B: Checkout conversion declines slightly, but retained conversion and contribution margin improve. The intervention may still create meaningful business value.
Outcome C: Return rates decline because customers abandon purchases, while contribution margin and customer satisfaction deteriorate. This should not be classified as a successful purchase-quality improvement.
These scenarios demonstrate why product teams need a coordinated measurement framework rather than relying on isolated metrics.
6. The Organizational Challenge: Connecting Product and Operations
Implementing purchase-quality measurement is not merely an analytics exercise. It also requires organizational coordination. Product managers typically influence discovery, product information, recommendations, and checkout. Operations teams manage delivery and fulfillment. Customer service teams handle complaints and returns. Finance teams evaluate profitability.
Yet problems originating in the pre-purchase experience may only become visible within post-purchase operations. This creates a structural measurement gap. For example, a recurring return reason may appear in operational reporting without being translated into a prioritized product backlog item. The Purchase Quality Scorecard can help connect these functions through shared definitions, common reporting, and cross-functional ownership.
A practical approach is to hold recurring reviews of preventable returns, identify gaps in purchasing decisions, and prioritize measurable product interventions. This shifts the roadmap conversation from solely asking which features might increase conversion to examining which changes could improve the overall quality and economics of customer purchases.
7. Why Minimizing Returns Should Not Be the Objective
A critical limitation of return-focused optimization is the temptation to treat all returns as undesirable. Returns are essential to online retail. They reduce perceived purchasing risk and provide consumers with confidence when evaluating products they cannot physically inspect.
The National Retail Federation’s 2025 study reported that 82% of surveyed consumers considered free returns important when shopping online. Research by Griffis and colleagues (2012), published in the Journal of Operations Management, also found that the returns management experience can influence subsequent customer purchasing behavior.
These findings highlight a potential trade-off. Restrictive return policies might reduce observed return rates while weakening customer trust and future purchasing behavior. Similarly, interventions that discourage uncertain customers from purchasing may reduce returns without improving total business performance. Therefore, the appropriate objective is not zero returns.
It is reducing avoidable purchasing mistakes while preserving customer confidence, fair return experiences, and sustainable economic outcomes.
8. Limitations and Opportunities for Further Validation
The Purchase Quality Scorecard is a proposed practitioner framework informed by existing research. It has not been independently validated as a unified measurement system. Several limitations require consideration.
First, published evidence about fit information is concentrated in particular retail categories, especially apparel. Results may differ for products with different purchase cycles, prices, fulfillment requirements, or customer behavior.
Second, return-reason data may not fully explain customer motivations. Classifications need consistent definitions and, in some cases, qualitative validation.
Third, post-purchase outcomes take longer to measure than transaction metrics, creating operational challenges for experimentation.
Fourth, customer satisfaction and contribution margin may move in different directions, so teams need explicit business priorities rather than an assumed universal weighting.
Future empirical research could assess whether teams that use purchase-quality measurement make systematically better product decisions than teams that rely primarily on transaction-level optimization. Such research could compare retained conversion, contribution margins, preventable returns, and long-term customer behavior across randomized product interventions. The framework should therefore be viewed as a structured approach for generating and testing better product hypotheses, rather than a proven universal measure of e-commerce success.
Conclusion: Redefining E-Commerce Product Success
E-commerce product management has largely focused on making purchasing easier. Retailers have invested in faster websites, personalized recommendations, optimized product pages, and frictionless checkout experiences. These capabilities remain central to digital commerce. However, increasing transaction efficiency does not automatically ensure customers make suitable purchasing decisions.
Research on product fit uncertainty, digital fitting technologies, and product returns shows the importance of linking pre-purchase information to post-purchase outcomes. The proposed Purchase Quality Scorecard extends this perspective by combining retained conversion, preventable returns, contribution margin, and post-purchase customer experience into a coordinated product evaluation framework.
Its central argument is straightforward: Product managers should optimize not only for purchases completed, but for purchases that create lasting value.
Conversion tells us whether customers bought. Purchase quality helps us evaluate whether those transactions produced suitable products, satisfied customers, and sustainable business outcomes. The next evolution of e-commerce product management should not abandon conversion optimization. It should make conversion part of a more complete definition of success.
References
- National Retail Federation & Happy Returns. (2025). 2025 Retail Returns Landscape. National Retail Federation. https://nrf.com/research/2025-retail-returns-landscape
- Hong, Y., & Pavlou, P. A. (2014). Product fit uncertainty in online markets: Nature, effects, and antecedents. Information Systems Research, 25(2), 328–344. https://doi.org/10.1287/isre.2014.0520
- Gallino, S., & Moreno, A. (2018). The value of fit information in online retail: Evidence from a randomized field experiment. Manufacturing & Service Operations Management, 20(4), 767–787. https://doi.org/10.1287/msom.2017.0686
- Wang, Y., Ramachandran, V., & Sheng, O. R. L. (2021). Do fit opinions matter? The impact of fit context on online product returns. Information Systems Research, 32(1), 268–289. https://doi.org/10.1287/isre.2020.0965
- Griffis, S. E., Rao, S., Goldsby, T. J., & Niranjan, T. T. (2012). The customer consequences of returns in online retailing: An empirical analysis. Journal of Operations Management, 30(4), 282–294. https://doi.org/10.1016/j.jom.2012.02.002
