This is a message from our vendor on how they are improving web tracking. Posting here for transparency. You shouldn't notice any impacts.
We are committed to improving the accuracy of your metrics and are excited to share some upcoming enhancements to our click tracking processes that will go live in our release tomorrow night on March 11th, 2025.
Current Bot-Click Tracking Methodology
Our current bot-click tracking process relies on the 'honeypot' method, which is an industry-standard technique for identifying bot clicks. This involves adding a hidden link to every email that is sent out of our system, which is undetectable to the recipients of your messages. Any click on these 'honeypot' links are recognized as non-human, and we cross-reference this data with other incoming message activity data to distinguish between human and bot interactions.
However, as bots become more sophisticated by scrambling their IPs, masking their user agent signatures, and mimicking human behavior, filtering their activity out of our product's reporting has become more challenging in recent times. These bots are typically security systems implemented by end-user email clients or IT services to scan links for potential threats.
Current Detection Methods
To address this issue in the past, we have already implemented the following detection methods:
- Known Bot User Agents: We cross-reference our bot click tracking data with known bot user agents (e.g., 'python', 'Googlebot') and remove this activity from your message's metrics.
- Rapid Multiple Clicks: We remove activity from your message's metrics when we see clicks from the same recipient (email address) and IP address on multiple links within an extremely short time frame (less than 1.5 seconds).
- Bot Click User Agent and IP Matches: We cross-reference bot click tracking data for matches on bot click user agent and IP for recipient click/open events and remove these results.
Upcoming Enhancements
To further improve our detection and filtering of non-human interactions, we will be implementing the following improvements to our detection methods in our March 11th release:
- Timestamp Cross-Referencing: We will cross-reference bot click tracking data for timestamps. If a bot clicks the 'honeypot' link, any click or open around that same time will be considered a bot click. The time differential will be based on the number of links in the message and the estimated time for the bot to crawl through all the links.
- Excluding Rapid Mass Click Events: For incoming batches of clicks/opens, we will analyze if a certain link was clicked by 90% of subscribers within two minutes of the message being sent. If so, we will remove all clicks for that hyperlink. This pattern is commonly exhibited by security systems checking links that lead to forms in the footer of messages (e.g., unsubscribes, manage preference forms). We recognize that legit human interactions may get filtered out based on these criteria. We attempt to recognize humans who also clicked the link in the two minutes after send by isolating subscribers with unique user agents as compared to the majority of the mass click event.
It is important to note that all activities identified as non-human will be removed at the point of ingestion. These removed clicks and opens will be stored in removal tracking tables for the same duration as our click/opens history and subscriber data (see information on our Data Retention Policy for more details).
If you observe that legitimate activity may be filtered out by mistake, our teams can analyze these tables to determine which methodologies led to the removal of any message activity event and potentially restore them to actual tracking statistics if necessary.
We appreciate your patience and understanding as we work to enhance our tracking processes. Our goal is to provide you with the most accurate and reliable metrics possible.
------------------------------
Tirza Austin
Senior Manager, Online Community
American Society of Civil Engineers
1801 Alexander Bell Drive
Reston, VA 20191
------------------------------