Spotting the spoof: User agent spoofing unmasked
Spotting the spoof: User agent spoofing unmasked
The dark web didn’t get its name by chance – the guise of secrecy is often the preferred route for fraudsters to carry out malicious activities online. On the open web, however, some fraudsters find successful cover via user agent spoofing, also known as browser spoofing.
This article will focus on user agent spoofing and how it can be a particular threat to individuals and businesses in the hands of fraudsters. We’ll dive into what user agents are, how spoofing works, the common ways in which this type of fraud is carried out, and how to prevent user agent spoofing using the latest fraud and authentication technology.
Understanding user agent spoofing
At its core, user agent spoofing involves altering the user agent string to disguise a browser’s identity. In non-fraudulent online transactions, the user string is a set of characters that lets servers and network peers identify the application, operating system, vendor, and/or version of the requesting user agent. For the purposes of this article, we’ll focus on the more nefarious purposes like ad fraud, click farms, web scraping, and bypassing content restrictions.
What is a user agent string?
A user agent is like a digital passport that provides websites with essential information about the visitor’s browser, application, and operating system. When a browser sends a request to a server, it includes a user agent string in the user agent header of the HTTP request, which helps the server present the correct version of the website.
The user agent string contains details such as the browser type and version, the operating system, and sometimes the device type. User-Agent strings can be complex due to historical reasons, but they generally follow this pattern of providing browser, operating system, and rendering engine details. Here’s an example with each component explained:
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.5845.111 Safari/537.36- Mozilla/5.0: This part of the string is a legacy identifier. Many browsers, including Chrome, include this for compatibility reasons, but it doesn't mean the browser is Mozilla.
- Windows NT 10.0; Win64; x64: This indicates the operating system (Windows 10), the architecture (64-bit), and the platform (x64).
- AppleWebKit/537.36: This specifies the rendering engine being used, which in this case is Apple’s WebKit engine (commonly used by Chrome and Safari). The version number (537.36) indicates the WebKit version.
- (KHTML, like Gecko): KHTML is the engine that WebKit is based on, and "like Gecko" means the browser behaves similarly to Firefox (which uses the Gecko engine).
- Chrome/116.0.5845.111: This is the version of Chrome (v116.0.5845.111).
- Safari/537.36: Even though this is Chrome, it includes "Safari" in the string for compatibility purposes, showing the version of WebKit, which is shared with Safari.
This information, also known as the browser’s user agent, enables websites to tailor their content and functionality to provide the best user experience. It can also be used for device fingerprinting to help identify devices by various attributes unique to them for the purposes of fraud prevention, or to ensure compatibility.
By understanding user agents and strings, developers can optimize their websites for various devices, operating systems and browsers.
User agent spoofing as a useful tool
User agent spoofing or browser spoofing works by presenting a false user agent string using various tools and techniques. Developers use user agent spoofing to test websites across different devices, altering the user agent string through browser settings or through browser extensions, like User-Agent Switcher and Manager. This lets them test how websites behave on different devices without actually changing the physical device.
Commonly altered signals in developer user agent spoofing include the user agent strings, JavaScript capabilities, screen resolution, language preferences, and time zone settings.
Malicious user agent spoofing
While user agent spoofing can be a versatile tool with a wide range of positive applications such as web development and developer testing, it’s also useful for certain forms of fraud such as ad fraud, click farms, website scraping, and bypassing authentication for fraudulent purposes. The negative implications of these activities should interest any developer team or business seeking to shore up its security posture.
Ad fraud and click farms
In ad fraud, or advertising fraud, fraudsters present a custom user agent string to make a hidden or outdated browser appear as a legitimate user, with the malicious intent of generating fake traffic on paid ads. This fake traffic can trick advertising platforms into believing they are reaching real, unique visitors.
Click farms, often located in regions with low-cost labor, use outdated smartphones and other devices to generate ad clicks. By employing user agent spoofing, these operations can mask their true identity and make their devices appear modern and located within desirable locations. This practice has successfully deceived ad platforms like Google and Facebook, leading to significant financial losses for advertisers.
For example, the Terracotta click farm was a sophisticated ad fraud operation discovered by authorities in 2020, notable for its scale and the advanced methods it used to evade detection. Terracotta used emulators to mimic a wide range of devices and user behaviors, making it difficult for ad platforms like Google and Facebook to detect the fraud.
Bypassing content restrictions
User agent spoofing is commonly used for bypassing content restrictions imposed by geo-blocks, i.e., accessing content that is otherwise unavailable in a specific region. These can range from streaming services, news websites, and sporting events (Premier League football fans understand this desperation!).
While bypassing geo-blocks might seem harmless at the hands of the casual content-consumer, it can have broader implications for many businesses. Unfortunately, in many cases, it undermines the efforts of content providers to control the distribution of their material based on regional licensing agreements. Additionally, it can expose users to legal risks if they violate the terms of service of the platforms they are accessing.Web scraping and web exploits
By altering the user-agent string, fraudsters can evade primitive bans based on IP and user-agent combinations. This tactic is particularly effective against systems without a Device Fingerprinting (DFP) solution (more on how this can help in the next section), which are unable to accurately identify and block malicious activity.
Fraudsters often pair user-agent spoofing with proxies or VPNs to mask the true origin of their traffic, making it appear as though requests are coming from different devices or locations. This method is used for various malicious purposes, including large-scale web scraping, bypassing rate limits or bans, and exploiting the soft limits of free-tier services that allocate credits based on weak factors like IP or user-agent. By continually changing their user-agent, fraudsters can fly under the radar, maintain persistent access, and maximize the benefits they extract from these services.
Spotting the spoof with Stytch Device Fingerprinting
So how might businesses unmask these attempts to prevent user and organizational harm? Given the multitude of ways in which a user agent can be spoofed, a best-in-class solution should go beyond a free traffic audit to look at many more subtle indicators and signs of foul-play.
Stytch Device Fingerprinting (DFP) deploys one of the industry's strongest user-agent deception detection engines that looks at a wide range of signals to ultimately determine if a user's request was legitimately made by the claimed browser or if data from the user agent has been spoofed and is operating from different browsers. Even under challenging scenarios such as incognito browsing and VPN use, Stytch’s Device Fingerprinting solution merges industry-standard and proprietary signals to guarantee stable and unique identifiers.
The core of Stytch’s Device Fingerprinting product is a JavaScript library that gathers various device attributes and sends them to Stytch’s backend for processing. This approach creates a telemetry ID that can be used to identify and block fraudulent actors with precision.
To learn more, chat with an authentication expert today or get started to see how Stytch can help you spot and stave off digital deception before any serious harm is done.
Share
