All publishers know the importance of monitoring their website’s traffic and that traffic’s behavior. Why? Because it’s a very specific key performance indicator (KPI) used to measure the performance of a website.
When you take a look at all the data concerning your website traffic, including where it’s coming from, you can get a better understanding of your audience. From there, you can better strategize to attract even more traffic while keeping your existing traffic on your pages.
Of course, it’s not all as simple as evaluating your data and creating a plan. Unfortunately, publishers like you have to contend with shadow traffic — a non-analytical traffic source that makes up for 20% to 40% of all internet traffic. Even Google Analytics can’t make sense of it.
Shadow traffic has grown over the past decade with the increase in private messaging apps, social media apps, and even the rise of HTTPS. It doesn’t look like it’s going anywhere soon, so your best bet is to learn as much as you can about it — including how to combat it so your traffic numbers come up efficiently and clean.
Keep reading to learn more.
What Exactly Is Shadow Traffic?
Despite the name, shadow traffic isn’t an evil or bad thing, like invalid traffic is. Shadow traffic, also referred to as dark traffic, is essentially the “visits” to your website that your analytics software doesn’t catch.
There are several reasons why your analytics software isn’t catching this traffic. For example, it could be stopped by ad blocking software or the browser, depending on the browser’s configurations. Therefore, shadow traffic is real traffic generated by real users that your analytics software simply misses.
Unfortunately, when analytics events, such as capturing user traffic, fail to reach the analytics servers, your website’s performance and profits are subsequently deflated.
How Shadow Traffic Really Affects Publishers
As mentioned early, between 20% and 40% of user traffic went untracked. Losing out on such a great percentage of visitor data means that publishers like yourself are missing out on pertinent information that would later help you generate more traffic and profits.
Essentially, you’re missing out on the inner workings of your audience, such as their route to your website or particular webpage, their user profile, and the value they bring to you as a visitor on your site. Without this important data, you’re unable to adequately make future decisions for attracting more users. You’re also losing out on potential revenue and possibly missing out on other things you’re doing wrong.
Shadow traffic continues to be a concern as the percentage of it out there increases and affects every type of analytics tool, browser, and publisher. So, if you’re looking to improve your traffic quality and find better ways to measure your website’s performance, you’ll need to first take the steps to address the shadow traffic problem on your website.
You’ll also need to do this while still respecting your audience’s privacy and catering to the user experience.
What’s Causing Shadow Traffic to Increase?
The short answer here is ad blocking software and weak analytics software.
The long answer is the ad blockers, built-in privacy features on most major browsers, and other tools that stop analytics providers from reading events — events as in users coming to your website. The entire purpose of these ad blocking methods is to prevent first-party and third-party data leakage and to avoid over-the-top ad targeting.
Let’s face it, users just aren’t as thrilled about ads as you are. They’re disruptive to the overall user experience because they get in the way and they slow down the page, making it difficult to get to the content they’re looking for. Not to mention, it’s a little strange to feel as if you’re being “watched” or “listened in on” as you constantly get served ads that reflect a recent conversation you’ve had with a friend or internet search.
That’s why upwards of 48% of internet users use some method of ad blocking to prevent these ads from cropping up and to keep their personal information personal — and out of the hands of third-party companies.
Here’s a little bit more of a breakdown:
Ad Blockers
A recent report put out from Adobe and PageFair states that between 2010 and 2020, the use of ad blocking software rose by 21 million to 181 million users. According to this same report, ad blocking is projected to cost publishers a loss of upwards of $40 billion in ad revenue.
While failed analytics data tracking contributes to this, the primary part of the problem is the use of ad blocking software to purposely prevent ads from rendering on the page. Fortunately, there are a few ways to remedy these losses, which we’ll touch on later.
Browser Privacy Features
The built-in browser-based technologies used for blocking ads, unlike ad blockers, don’t actually work by disrupting websites. They do, however, prevent publishers from using data cookies for the purpose of serving up ultra-targeted ads to the users.
When we say browser privacy features, we’re talking about well-known browsers. For example, Safari and its Intelligent Tracking Prevention (ITP) feature, Firefox and its enhanced Tracking Protection feature, and Microsoft Edge’s Tracking Protection feature. All of these features act as “built-in” ad blockers for both first- and third-party analytics software and services.
What Else Contributes to Shadow Traffic?
So far, we know the primary culprits behind shadow traffic are ad blockers and browser privacy features. But those aren’t the only things that contribute to shadow traffic generation.
There are other tools out there, such as the following that factor into shadow traffic percentages:
- VPN-level blocking programs: VPN-level blocking tools work by preventing publishers from tracking internet users so they can’t serve up relevant ads which essentially blocks the ads from popping up. (Think: Nordic VPN)
- Network-level blocking tools: Network-level blocking works to block ads across an entire network. This is different from browser-based blockers in that they block ads in other places like online games and even smart TVs. (Think: Pi-hole)
- Device DNS blocking tools: Device DNS blocking tools block any devices and networks that request access to the user’s information. They also work to block communication with users, which means you can’t pick up any valuable information from them. (Think: AdGuard)
- Device-App-based blocking tools: Device App-based blocking tools block ads from any apps that utilize browsers like Safari to display their web pages. These ad blocking tools make it even more difficult for publishers to reach their users since they not only block ads but block trackers, GDPR notices, and even EU cookies. (Think: Wipr)
How Can You Combat Shadow Traffic?
As previously stated, there are a few ways you can combat shadow traffic to put the pep back into your user traffic (and by pep, we mean data analytics).
Here’s what you should try next:
Using Server Logs
Each time a user visits your site, they automatically connect with your web servers. When this happens, the interactions are tracked and logged on your web server. From there, you can consolidate the logged information and use it to identify the actual traffic hits to distinguish how much traffic you’re realistically bringing to your site each day, week, month, or year.
However, it’s a tricky method. This is because it’s next to impossible to restore and manage logs from different web servers, a Content Delivery Network (CDN), or multiple sites at a time.
If you have a way of managing logs within the server’s log management software, you’ll have a good chance of detecting your missing traffic. However, you won’t get the exact results you’re hoping for.
Leveraging a First-Party Server
Here’s the thing about ad blocking — it usually happens on the user’s web browser, which is why certain access and information is restricted. However, the user still has to interact with the website in question to consume its content. Therefore, the connection between your content server and the user’s browser can’t be restricted.
So, what do you do?
Rather than connect to the server of a third-party analytics provider, you can use your own server to catch the events. Through your server, you can even send out your data that’s related to the traffic’s sources.
Of course, implementing a first-party server can be a little complicated. However, if you do it right, you won’t need third-party analytics to generate an analytics script to paste onto your website — which means less shadow traffic and more valuable traffic.
Using First-Party Analytics Providers
If you can’t figure out how to leverage your own server, you can use first-party analytics vendors — managed services — to access and track your website’s traffic and events through their first-party servers. This approach will allow you to track both your visible and invisible traffic while still maintaining your users’ privacy and the overall user experience.
What it All Comes Down to
By using the right strategy, you can greatly reduce your shadow traffic problem. It may take some trial and error, but once you learn to work around ad blocking software and browser features, you’ll gain more visibility into your user traffic.
It’s all about keeping up with the ever-evolving digital landscape and the knowledge that the average user gains regarding privacy. Essentially, the above methods are a good start for combating shadow traffic — but you’ll need to continue to find ways to work around ad blocking to break through your shadow traffic.