Litmus recently published the “The 2023 State of ESPs Report“. In an era where data-driven decision-making is not just a trendy phrase but a necessity, industry reports are indispensable tools. These reports often shed light on market trends, consumer behavior, and technological advancements. For those in the email marketing sector, reports such as Litmus’ “The 2023 State of ESPs Report” are a treasure trove of information.
However, like any data source, industry reports have their own set of limitations and caveats. In this blog post, I will critically examine the usefulness and limitations of such reports, using the most recent Litmus report as a case study.
To be clear, this analysis could be applied to any industry report. We are not criticising Litmus; in fact, at emailexpert we applaud them and thoroughly encourage more in the industry to share their findings. Collective analysis of such reports will prove extremely valuable for the industry.
7 Interesting Facts from the Litmus Report
Interesting Fact 1: Salesforce Marketing Cloud is the Most Commonly Used ESP
While this fact provides a snapshot of the current market leader, it doesn’t delve into why Salesforce is the most popular. The Litmus report is based on a survey of over 750 email marketers worldwide, a fraction when compared to Litmus’ 700,000 users and the even larger global market. This raises questions about the report’s generalisability and the potential for bias.
Implications for Decision-Makers: Founders and CMOs should consider this data but also seek additional sources to validate these findings, especially if they are considering a long-term commitment to an ESP like Salesforce, which often has contracts spanning 3 to 5 years.
Interesting Fact 2: 69% of Salesforce Marketing Cloud Users are Satisfied
Satisfaction is a subjective metric and can be influenced by various factors, including lack of experience with other platforms.
Implications for Decision-Makers: Satisfaction rates are useful but should be supplemented with other KPIs like customer retention rates, average revenue per user, and net promoter scores to get a more holistic view.
Interesting Fact 3: Only 9% of Email Marketing Leaders Say Their Email Marketing is Highly Integrated
The report states that only 9% of email marketing leaders feel their email marketing is “very highly integrated” into their other marketing channels. This is a surprisingly low figure, especially in an era where integrated marketing is increasingly essential due to advancements in AI and automation.
However, the report doesn’t delve into several key areas that could provide more context for this statistic. For instance, it doesn’t specify what “highly integrated” means to the respondents. Are they referring to data sharing, coordinated campaigns, or something else? Without a clear definition, the 9% figure lacks context.
Implications for Decision-Makers: Given the low percentage, decision-makers should critically assess their own integration levels and consider whether their current ESP is meeting their needs in this crucial area.
Interesting Fact 4: Salesforce Marketing Cloud is Popular Across Multiple Industries
This could suggest a versatile platform, but without data on the specific needs and satisfaction levels across industries, the finding is incomplete and potentially meaningless.
Implications for Decision-Makers: Before jumping on the Salesforce bandwagon, decision-makers should consider whether the platform meets the unique needs of their industry.
Interesting Fact 5 and 6: 15% of Email Marketing Leaders Have Switched ESPs in the Past Year & 65% of Email Marketing Leaders Have Been With Their Current ESP for at Least One Year
The report reveals that 15% of email marketing leaders have switched ESP providers in the past year, while 65% have stayed with their current ESP for at least one year. While this data provides a snapshot, it lacks depth in several key areas.
Firstly, the report doesn’t offer longitudinal data on ESP switching beyond the one-year mark. Given that many enterprise ESP contracts, like those with Salesforce, can span 3 to 5 years, a one-year snapshot may not capture the full dynamics of ESP switching. Knowing the percentage of switches over 2, 3, or 5 years could offer a more nuanced understanding of customer satisfaction and ESP performance over time.
Implications for Decision-Makers: Decision-makers should consider the length of contracts and the flexibility to switch ESPs when evaluating their options, especially for long-term commitments.
Interesting Fact 7: Email Strategy and Planning is the Most Common Job Responsibility (58%)
The data reveals that 58% of email marketing teams are responsible for “Email strategy and planning,” making it the most common job responsibility. This underscores the strategic importance of email marketing within organisations, suggesting that it’s not just a tactical afterthought but a core focus for many marketing teams.
However, the report leaves several critical questions unanswered. For instance, it doesn’t explore the correlation between the prevalence of strategy and planning roles and the effectiveness of email campaigns. Another missing element is the breakdown of these responsibilities by company size or industry. Do larger organisations with more resources prioritize strategy and planning more than smaller companies? Or is this a universal trend across the board?
Also, the report lists other common responsibilities like Email development (50%), Email operations (49%), Email program management (48%), and Email deliverability (46%), but doesn’t discuss how these roles interact with strategy and planning. Understanding the interplay between these responsibilities could provide a more comprehensive view of email marketing dynamics.
Implications for Decision-Makers: That aside this data should prompt decision-makers to assess whether their organisations are allocating sufficient resources to email strategy and planning, as neglecting this area could result in missed opportunities for ROI.
The Merits of Industry Reports
Comprehensive Overview
Industry reports frequently offer a thorough overview of the market landscape. For example, the Litmus report encompasses everything from the most frequently used Email Service Providers (ESPs) to user satisfaction rates and integration issues. Such a broad perspective is invaluable for founders, CMOs, and other decision-makers who require a panoramic view of the industry.
Data-Driven Insights
These reports are typically underpinned by extensive surveys and data collection efforts. The Litmus report, for instance, is based on a survey of over 750 email marketers worldwide. This lends a degree of empirical rigor to the insights, making them more reliable than anecdotal evidence.
Industry Benchmarks
One of the most valuable facets of industry reports is the provision of benchmarks. Knowing that Salesforce Marketing Cloud has the highest user satisfaction rate, as per the Litmus report, can serve as a useful benchmark for companies evaluating different ESP options.
Limitations and Caveats
While industry reports are valuable, they are not without limitations. It’s crucial to approach these reports with a critical mindset. Here are some common issues, illustrated by the Litmus report:
Sample Size and Generalisability
The Litmus report surveyed around 750 email marketers, a fraction when compared to Litmus’ 700,000 users and the even larger global market. This raises questions about the report’s generalisability and the potential for bias. Given that Litmus itself does not have 100% market penetration, the sample size becomes even more critical in understanding the broader landscape.
Self-Reported Data
Data in these reports is often self-reported through surveys. While this method is practical, it’s subject to various biases like social desirability or recall bias. For instance, user satisfaction rates could be inflated if respondents are more likely to report positive experiences.
Lack of Longitudinal Data
Many industry reports, including the Litmus one, offer a snapshot in time rather than a longitudinal view. While they provide valuable current insights, they may not capture evolving trends or the reasons behind changes in user preferences or market share.
Methodological Transparency
A critical aspect often missing in industry reports is a detailed explanation of the data collection methodology. Without this information, it’s challenging to assess the reliability of the data.
Subjective Metrics
Metrics like “user satisfaction” are inherently subjective and can be influenced by various external factors. For example, a user might report high satisfaction with an ESP simply because they haven’t experienced a better alternative.
The Way Forward
Critical Consumption
The first step in deriving value from industry reports is to consume them critically. Understanding the limitations allows you to interpret the data in a nuanced manner.
Cross-Referencing
Whenever possible, cross-reference the data and insights from one report with other reliable sources. This practice can provide a more rounded view and help mitigate some of the limitations inherent in single-source reports.
Vendor Collaboration
For vendors producing these reports, transparency is key. Providing details about the methodology, sample diversity, and any limitations can enhance the report’s credibility and utility.
Conclusion
Industry reports like the one produced by Litmus offer invaluable insights but come with their own set of limitations. A critical, analytical approach to consuming these reports can help industry professionals make more informed decisions. As the saying goes, “Data doesn’t lie, but it doesn’t tell the whole truth either.”
By understanding both the strengths and weaknesses of these reports, we can get closer to the whole truth.
