Invalid Traffic Guide

1. IVT overview ​​

This guide covers common invalid traffic (IVT) terms and issues. It will support you in dealing with most IVT cases. If you experience any IVT issues that are not covered here, please contact us for further support (mediaquality@smartclip.tv). 

1.1 Why IVT is a common phenomenon 

Publishers and advertisers alike are often concerned when they receive an IVT notification from their ad verification vendors. Moreover, there are many articles on the internet referring to IVT as a major issue for the digital industry — articles which state that IVT causes advertisers to lose millions of dollars of their marketing budgets each year. This is true in some rare cases in a few markets, but in Europe the situation is different. With market standards established such as ads.txt and sellers.json — to name just a few — European market participants can be sure of a high level of media quality and transparency. These standards are highly effective in reducing IVT issues, but of course, every member of the digital advertising value chain has a role to play in ensuring they are complied with. That is why we at smartclip not only support these standards but also help our business partners to understand and follow them. It is also why we have established robust media quality processes.    

The key learning point here is that notifications about IVT do not need to be a cause for alarm, providing all participants implement market standards and keep a close watch on possible issues. When this is done, IVT can simply be regarded as part of the digital advertising industry – a common phenomenon, but one which must be taken seriously and evaluated carefully in order to avoid any issues.

1.2 IVT vs Fraud

It is important to differentiate between two major terms: IVT and fraud. These terms are often considered to be the same, but that is not 100% accurate — not every impression that is invalid must also be fraudulent. Fraud refers to deliberate — and criminal — acts on the internet that are carried out with the intent to defraud advertisers, publishers, or users. There is a risk that IVT is caused by fraudulent intent, but for the vast majority of invalid impressions, this is not the case. Therefore, it is recommended that the term fraud is used with caution and only when referring to actual criminal acts.

2. IVT deep dive

2.1 Main IVT types

Although IVT is not always fraud, it is still an important topic to understand as it involves an impression that misses its advertising effect. 

First, we must distinguish between the two main IVT types: general invalid traffic (GIVT), which can be seen as the “benign” IVT, and sophisticated invalid traffic (SIVT), which can be seen as the “critical” IVT. 

Second, we need to understand that the source of IVT can be either non-human or human. IVT is mostly non-human traffic. This is traffic generated by bots and crawlers (e.g. programs needed to analyse websites) or server networks (e.g. when a company uses a server behind a firewall). In a few less common situations, a human is the source of the IVT, and these situations are usually where fraud is involved.

In the next section, we look at the two types of IVT (GIVT and SIVT) in more depth, including the sources (non-human or human) of both.

2.2 GIVT

The source of GIVT is always non-human traffic. This is traffic from bots, crawlers, or servers, and most of these sources are so common in the market that every ad verification provider is aware of them and/or they are listed in the IAB/ABC International Spiders & Bots List. In these cases, the bots are described as declared or known bots, and server traffic is described as known data centre traffic. This invalid impression (still GIVT and non-human) supplies details of its source to the ad server and/or ad verification tool — it is simply saying:

“It’s me. You know my name, my address, and the path I travelled.”   

GIVT must be avoided, because it creates requests that cannot be monetised. However, nearly every ad server system and/or ad verification tool can easily identify this type of IVT and its source and can filter out the GIVT before the campaign gets delivered. If this is not the case, it is not a reason to worry — the GIVT source can be quickly identified and the problem can be solved manually. For example, technical integration issues are a common source of GIVT.

2.3 SIVT 

In contrast to GIVT, SIVT is generally more difficult to classify, as both non-human or human traffic can be the source of it. Furthermore, SIVT can be generated unintentionally or on purpose. In our experience, most SIVT is caused unintentionally, but can be easily resolved once identified (see more in section 2.3.1). In some rare cases of SIVT, a fraudulent intent cannot be denied (see more in section 2.3.2).

2.3.1 SIVT requiring bilateral clarification

Some SIVT types are suspicious and need bilateral clarification with the involved parties — including publishers, ad verification vendors, and sales houses — in order to reduce the SIVT rate (side note: names and definitions of the SIVT types may vary slightly between different ad verification providers). 

Non-human SIVT

Ad Clutter: Ad clutter refers to when the number of ads within the user’s viewport is too high — more than four ads in the viewport could count as ad clutter, and the impression might count as invalid. 

Auto-Refresh/Reload: Auto-refreshing or reloading the ad slots is not forbidden. However, every ad needs to be displayed for a certain amount of time before the reload. If an ad slot is reloaded too quickly, it may count as an invalid impression. We recommend a minimum of 30 seconds in between loading of ad slots. Reloading ads only — and not the whole website — is an indicator for a higher-quality website. It shows that the publisher cares about the user experience.

Bots (unknown): These bots are so uncommon that the IAB or ad verification vendors are not aware of them. Most of the time they are the result of as yet unknown analytics systems, contextual targeting providers, etc. In the worst case, these bots create ad impressions on purpose and pretend to be a human user (see section 2.3.2). 

Data Centre Traffic (unknown): In this situation, the source of the traffic is unknown data centres, such as servers, which are unknown because of their limited market spread. Therefore they are not known by the IAB or ad verification vendors. There are forms of unknown data centre traffic that may be fraudulent — see section 2.3.2 for further information.

Geo-Mismatch: This is when most of the website’s traffic does not match the geo-location of the website’s origin. For example, when a website from Germany with German content receives traffic mostly from Russia. Another issue might be that the campaign delivery does not consider the booked geo-targeting of the client.

Hidden Ads: Hidden ads are displayed in a way that they overlay each other almost completely or are overlaid by components of the website such as pictures, articles, and graphics. This is often an unintentional issue that needs to be fixed by the publisher. If the ads are hidden on purpose, the reason could be fraudulent — see section 2.3.2 for further information.

Human SIVT 

There is only one human SIVT type that needs bilateral clarification.

Cash for Surfing: This is when users create an account and simply get paid for watching ads. The involvement of the user is low, and some ad verification vendors might count such an impression as SIVT.  

2.3.2 SIVT that can be considered as fraud

In a few less common situations, there are types of SIVT that need to be treated as fraud. It should be highlighted that, fortunately, these types of SIVT are uncommon – especially in Europe – and there are solutions to prevent almost all of them (side note: names and definitions of the SIVT types may vary slightly between different ad verification providers). 

Non-human SIVT

Creative Hijacking: Creative hijacking refers to when ad calls (requests to the ad server) or ad tags (HTML that contains and delivers the actual ad) on a website are copied and, without the consent of the publisher or the advertisers, placed elsewhere in untrustworthy environments. 

Data Centre Traffic (unknown): This type of SIVT is considered fraudulent when traffic is sent from unknown data centres, such as servers, that are deliberately avoiding identification. This can be the case when the publisher buys traffic from untrustworthy providers.

Domain/App Spoofing: Domain/app spoofing refers to when buyers are deceived with fake traffic sources and the actual impression is generated on an inferior website of low quality with non-brand-safe content. It also refers to when impressions are supposedly delivered on a high-quality website, but it is actually a faked domain. This is especially relevant for programmatic deals.

Hidden Ads/Ad Stacking: Hidden ads are displayed in a way that they overlay each other almost completely or are overlaid by components of the website such as pictures, articles, and graphics. If the ads are hidden on purpose — for example, to simply place as many ads on a website as possible — this type of SIVT needs to be treated as fraud. 

Hijacked Devices: When devices are hijacked, websites are loaded on the user’s computer in the background without that user noticing or initiating the loading.

Malicious Bots: Malicious bots are non-human programs and scripts that generate impressions while pretending to be human. 

Malicious Click Bots: Malicious click bots are similar to malicious bots, but they create clicks instead or on top of ad impressions.

Surfbars: Surfbars are programs that automatically load additional sites when the user visits the actual website. Therefore the user does not actually view or engage with these additional websites. 

Human SIVT

Surf and Click Farms: Surf and click farms are where people are paid or forced to visit websites and create ad impressions and clicks. These are often large-scale operations, and when they are discovered – and subsequently closed down – it is sometimes reported in the news.

3. How to prevent IVT

Transparency and communication are central to preventing IVT, and for publishers, a few key rules apply:

3.1 Know your audience and business partners 

Analyse your website(s), app(s), and users. Gather as much data as possible and — while ensuring compliance with the general data protection regulation (GDPR) — share this information with third parties such as demand-side platforms (DSPs) and sales houses. If you can provide data such as unique users, page impressions per user, and bounce rates, it will increase trust in the business relationship. Do not work with untrustworthy traffic providers, and do not buy traffic from them. 

3.2 Guarantee transparency 

Easy methods — some mandatory — already exist to guarantee more transparency among buyers. Ensure complete ads.txt and app-ads.txt files are in place, and update both files regularly so that the descriptions, such as DIRECT or RESELLER, are accurate. Transfer the referrer domain and/or Bundle ID so everyone can retrace where your traffic comes from. For in-app inventory, ensure you send the App Store URL. Also ensure authorised sellers are validated via the sellers.json.

3.3 Fulfil market standards

Market standards reduce complexity and gain trust. In the last couple of years, the Open Measurement Software Development Kit (OM SDK) and the Transparency & Consent Framework 2.0 (TCF 2.0) have been the two most important standards introduced. The OM SDK offers a simple approach to measuring the viewability of ads — which is key information for clients. TCF 2.0 covers all GDPR topics and gives the power back to the user on what kind of information they want to share. By using the TCF 2.0 framework including the consent string signal, you share this information with your supply-side platform (SSP) and/or ad server so they know what kind of ad they are allowed to show to the user and what kind of data might get collected. Important: Please note that implementing TCF 2.0 and GDPR is not a choice, it is a law.  

3.4 Follow defined rules of ad implementation

The Coalition for Better Ads and the IAB have published comprehensive documentation on how ads on your website(s) and app(s) should work. Also, many DSPs have rule sets that publishers should follow to guarantee high-quality ad placement and implementation. More information on quality criteria can be found in the smartclip resource hub:

• Quality Criteria: In-Stream
• Quality Criteria: Out-Stream

3.5 Check the implementation of the ad call

Check the implementation of the ad call to ensure all the information needed by the SSP and/or ad server is transferred in the correct way — for example, the website(s) or app(s) general information or the user agent information. If this is not done correctly, it might lead to high IVT rates — for example, if you declare your traffic as web traffic and we expect app traffic.    

4. Conclusion and recommendations

4.1 Conclusion 

Every potential IVT issue must be taken seriously and needs to be evaluated with care. Nevertheless it is not a cause for alarm as IVT will always be part of the digital advertising industry. In order to resolve IVT issues, it is important to understand the two main types of IVT — GIVT and SIVT — as well as the difference between SIVT types that need bilateral clarification and SIVT types that can be considered as fraudulent. The term fraud must be used carefully — most SIVT issues are caused unintentionally without fraudulent intent.     

4.2 Recommendations 

There are many ways to ensure fraudsters are left with as little room to operate as possible, and all revolve around the two key requirements of delivering successful digital advertising campaigns – transparency and communication. Be sure to adhere to the following:

• Know your audience and traffic sources
• Know your clients who buy your inventory and the vendors who resell your inventory
• Guarantee a complete and transparent supply chain 
• Fulfil given market standards
• Check the implementation of the ad call regarding defined rules and required information