Users vs Sessions: Conversion Metric Dilemma

There is a point in every business's life where an agreement must be made about which visit-based metric, sessions or users, should be used to calculate conversion rates at each step of a sales funnel. This metric is the leading barometer of performance and the first to be scrutinized for drops in performance. While it may seem like a schoolyard debate that ultimately has no major impact on business performance, standardizing one metric for reporting and understanding the differences in how they are calculated will assure that data is consistent across all reports and ensure that any performance changes are interpreted without debate.

Users vs Sessions

Users and Sessions, while both representations of a visit to a website, are inherently different and communicate different messages depending on the type of cohort being examined.

Users are a unique measurement of a person visiting a website that is recalculated with each adjustment that is made to a specific cohort.

For example, If we were looking at users within a specific date range and added additional days to the range, then the user metric will recalculate and deduplicate any users who had visited more than once within that new date range, in addition to adding users who have never visited before.

Sessions are an aggregated metric that adds or subtracts the amount of times a user visits a website within a specific cohort.

Using the same example above, if we were to expand our date range then we would either see a flat or rising session count. This is due to the fact that this metric is never recalculated and can only grow or shrink depending on what data was collected within that specific period of time.

How are they different?

Having this understanding in mind, the dilemma of which metric you should use begins as cases can be made for one or the other and produce completely different interpretations of the data being presented. For instance, if we were to look user's and sessions against the same amount of leads for a given range and compared the conversion rates you would clearly see that both would be radically different.

Variables

Users = 875

Sessions = 1,100

Leads = 235

Variances

Users

Conversion Rate - 235/875 = 26.85%

Sessions

Conversion Rate - 235/1,100 = 21.36%

As you can see in the example above, there is a 5.49 percentage point difference between the two measurements, which at specific scales can represent a large shift in performance from our acquisition or sales metrics. This type of change is very alarming and cause incorrect analysis if the two are compared against each other. These issues should force a decision to be made across the organization on which measurement will be used across all reports and a debate should occur with arguments being made similar to the ones below.

Arguments

Sessions

Argument 1: Sessions are static by nature. Sessions are never recalculated and thus provides us with a definitive value that will remain the same regardless if you add up the values in a daily cohort or pull a report with at a monthly cohort level. This type of reporting consistency helps rule out questions of data integrity when co-workers compare two different time cohorts.

Argument 2: Measurement of site effectiveness. When determining the effectiveness of your website, you want to be able to determine the cause for increases or decreases in the amount of traffic and how it is affecting overall business performance. In general, you want to optimize your website to convert users in as little time as possible to eliminate any factors that might persuade them otherwise. Since users removes the visit measurement from analysis, it is difficult to interpret trends in how many sessions are occurring and if a potentially longer cycle of visits is happening that is making a significant impact on your business. This type of occurrence is an immediate red flag and a reason to make an optimization your website.

Users

Argument 1: Shopping by nature is a purchasing process. Unless the item is a necessity in our daily life, we rarely make purchases on impulse as it isn’t a feasible manner of living and tends to lead to a high of probability of dissatisfaction in the decision that is made. Common sense understands these negative aspects and is why we conduct research and weigh our decisions before settling on what is the best fit in our mind. This process creates digital tracks in the form of sessions that build up in a manner that creates inconsistent patterns in visitor behavior, which cause issues when trying to understand the impact each session has on business KPI’s. By using the user metric, the noise contained in the session metric is removed and a better estimate is provided that measures the impact that single user had on the business.

Argument 2: Not every page was created to drive an action. As much as we would love to build a three page website that automatically drives leads or transactions, we all know that this is purely a pipe dream. It is more realistic that users are nurtured through informational pages like a blog or a support page, which are not built with the objective of having a user take an action. It is also common to see a lot of traffic come to the website with the intent to consume this information rather than buy. This type of traffic obscures the truth in the conversion rate and can lead many to make optimizations when changes are not needed. Since the unit volume of the user metric are much smaller than that of sessions, there is much cleaner data to work with and if a few users come visit for only the purpose of consuming information, then they will not have as large an impact on conversion rate as they would when using the session count.

My Thoughts

I have used both measurements for different scenarios and business types in the past and I don’t have a personal commitment to either metric. From a paid channel perspective, I feel more comfortable using the same measurement that the platform is using (typically clicks = sessions) to get an accurate read on my acquisition costs. In other cases involving both organic and paid channels, I tend to feel more comfortable using users because I feel like it is the best interpretation of the value each individual user provides to the business and how much it costs to acquire them.

At the end of the day, there is no bad choice when choosing one over the other. The bottom line is that you should understand what metric will best represent the nature of your business and that it is used throughout all reports across every team. Consistency is key to interpreting and correctly optimizing on specific trends and this should be the main message communicated across the organization.