[XB] Search Spaminator: Search Bot Protection, AI Scraper Blocking & Honeypot 1.0.0

Protect XenForo search from automated bots, AI crawlers, content scrapers, and excessive search requests without challenging visitors on every page.

Search Spaminator adds a lightweight verification step before protected searches are executed. Visitors agree to the search rules once per session, while automated scrapers that ignore the form cannot access search results. Hidden honeypot fields, submission checks, optional CAPTCHA, and configurable bot exclusions provide additional protection.

Reduce unnecessary search activity, protect member content from automated collection, and monitor suspicious requests through detailed logs and statistics.

---

Features:​

Search Bot & Scraper Protection​

• Search Bot Protection: Require visitors to complete a verification form before accessing protected XenForo search results.
• AI Scraper Protection: Limit automated crawlers attempting to enumerate forum posts, threads, and member content through search.
• Search Rules Confirmation: Ask visitors to accept the configured search rules before their search is processed.
• Once-Per-Session Verification: Remember successful verification for the current session, with an optional configurable lifetime.
• Seamless Search Continuation: Automatically continue to the originally requested results after successful verification.
• Honeypot Fields: Detect automated form submissions using hidden checkbox and selection fields.
• Automated Submission Detection: Apply timing and form verification checks to identify suspicious scripted submissions.
• Fake Search Results: Show trapped bots a normal-looking empty search result instead of revealing that their request was detected.
• Second-Chance Verification: Optionally allow another attempt when only one honeypot field is triggered, reducing false positives from browser autofill.
• Optional CAPTCHA: Use XenForo's configured CAPTCHA provider for every challenge or only after suspicious verification attempts.
• Lightweight Protection: Operate without mandatory external anti-bot services, third-party blocklists, or CAPTCHA providers.

Protected Search Areas​

• Keyword Search: Protect standard XenForo search requests.
• Member Content Search: Protect Find all content by member and Find all threads by member searches.
• What's New & New Posts: Optionally protect new-content result pages, including compatible pages provided by other add-ons.
• Tag Search Pages: Optionally require verification before visitors access tag results.
• Find Threads: Protect supported thread discovery pages.
• Profile Postings: Optionally protect the postings tab on member profiles.
• Independent Area Controls: Enable or disable verification separately for each supported search area.

Permissions & Bot Exclusions​

• Bypass Permission: Allow selected user groups to access protected searches without verification.
• Guest Protection: Challenge guests by default while allowing registered members to bypass verification.
• Recognized Robot Exclusions: Optionally exempt robots recognized by XenForo, including search engine crawlers.
• Robot Verification: Optionally verify supported recognized search engines through DNS checks before allowing them to bypass the challenge.
• Custom User-Agent Exclusions: Exempt additional browsers, crawlers, or clients using configurable User-Agent matching.
• IPv6 Network Grouping: Optionally group IPv6 addresses by network for challenge limits, CAPTCHA escalation, and statistics.

Bot Detection Logs​

• Honeypot Log: Record suspicious requests with IP addresses, user agents, requested URLs, and triggered detection checks.
• IP Address Filtering: Search logs using individual IP addresses, wildcard ranges, or CIDR networks.
• Search Challenge Counter Log: Review verification displays, successful passes, and trapped requests by IP address and date.
• Protected Area Filtering: Filter suspicious requests by the search area they attempted to access.
• Trap Severity: Distinguish between softer verification failures and stronger honeypot detections.
• IP Ban Integration: View existing XenForo IP ban status and open the native IP ban form from administrator logs.
• Permission-Based Access: Independently control who can view and clear supported logs.
• Automatic Log Cleanup: Configure retention periods for recorded requests and detection data.

Search Protection Statistics​

• Challenge Statistics: Monitor verification forms shown, successful passes, trapped requests, and unique IP addresses.
• Daily Activity Charts: Review challenge and detection activity over time.
• Flexible Reporting Periods: Analyze today or the previous 7, 30, or 365 days.
• Yearly Activity Heatmap: Visualize search challenge activity across the year.
• Verified IP Rate: See the percentage of recorded IP addresses that successfully completed verification.
• Top IP Addresses: Identify addresses generating the most verification or trap activity.
• IP Ban Status: Distinguish currently banned addresses in reports and optionally include them in top-IP rankings.
• Protected Area Statistics: Compare suspicious activity across different search areas.
• Trap Breakdown: Review detection counts and proportions by protected area and trap severity.
• Configurable Statistics Accuracy: Choose approximate challenge counting to reduce database writes or exact counting for more detailed statistics.

Administration & Testing​

• Centralized Configuration: Manage protected areas, verification behavior, bot exclusions, CAPTCHA, logging, and statistics from XenForo options.
• Testing Mode: Temporarily display verification challenges regardless of bypass settings to test the configured protection.
• Visible Honeypot Testing: Reveal enabled trap fields during testing to verify their behavior.
• Automatic Test Reset: Automatically disable testing mode after five minutes.
• Custom Search Rules: Customize the verification message using a XenForo phrase.
• Guest Cache Protection: Optionally bypass XenForo guest page caching for protected routes so cached search results do not bypass verification.

---

Useful for:​

• AI Scraper Defense: Reduce automated collection of forum content through member searches, keyword searches, and other protected discovery pages.
• Content Protection: Make bulk enumeration of member posts and threads more difficult for automated clients.
• Server Load Reduction: Avoid executing protected search queries for requests that stop at verification.
• Visitor-Friendly Protection: Challenge search activity instead of interrupting normal browsing on every page.
• Scraper Investigation: Identify suspicious search patterns and frequently challenged IP addresses.
• Bot Traffic Analysis: Understand which search areas attract automated requests and how often visitors complete verification.

---

Setup:​

• Configuration: Open Options > [XB] Search Spaminator to configure protected areas, verification behavior, and exclusions.
• Search Rules: Edit the pb_search_spaminator_search_rules_text phrase to customize the rules displayed on the verification form.
• Permissions: Configure the bypass permission and access to logs for the appropriate user groups.
• CAPTCHA: Optionally enable the CAPTCHA provider already configured in XenForo.
• Testing: Enable testing mode to verify challenges and honeypot behavior before using the settings on live traffic.
• Public Log: Available at search-spaminator/ to members with the Can view log permission.
• Administrator Statistics: Available under Logs > Search Spaminator > Statistics.
• Challenge Counters: Available under Logs > Search Spaminator > Search challenge counters.
• Trap Log: Available under Logs > Search Spaminator > Search trap log.

---

FAQ:​

• Does the search run before the visitor checks the box?
  No. For requests that require verification, the form loads instead of running the protected search. Submitting it without agreeing to the rules shows the form again. The original search is preserved and continues only after all required checks pass.
• What happens to bots that never submit the form?
  Their protected searches do not run. Leaving the form untouched is not a successful verification or a triggered trap. They may keep requesting the form, but those requests do not execute the protected search.
• Can a bot check the visible box and pass?
  A capable bot can check the box, so the checkbox alone does not prove that a visitor is human. The submission must also pass the form security and timing checks, the script check when enabled, and any required captcha, without triggering a blocking trap. A bot that successfully completes verification is counted as a pass and can continue searching, just like a human visitor.
• Does this reduce server load with the default settings?
  It avoids running protected searches for requests that stop at verification, reducing unnecessary search work. Serving and validating the form still uses server resources. The actual saving depends on traffic and configuration; the statistics do not measure CPU time or database load saved. By default, only guests are challenged, keyword and member content searches are protected, and recognized robots are excluded. Visitors with bypass permission or a valid previous verification also continue without a new challenge.
• How should I read the statistics?
  Total Challenges and the period counters show recorded displays of the verification form, subject to the counting mode below. They are not completed searches or necessarily different visitors. Passed counts successful verifications, including repeat verifications and bots that pass. The percentage shown beside it is the share of recorded unique IP addresses with at least one successful verification, not Passed divided by Total Challenges. Traps triggered counts recorded trap events, not every request stopped at the form. Unique IP addresses counts recorded addresses, not people; IPv6 grouping can combine addresses into one network. Average per day divides the last 30 days of recorded displays by 30, even for a new installation.
• Does a low verified IP rate prove that the remaining traffic is bots?
  No. Visitors may abandon the form or fail a check, and several people can share an IP address. These figures do not distinguish abandoned forms from every type of failed verification, prove that every unsuccessful visitor is a bot, or count every blocked search. Successful verification is remembered for the session, so later searches can run without adding another pass.

---

Notes:​

• Challenge Counting: Approximate counting is enabled by default to reduce database writes. Exact statistics can be enabled when more detailed request counts are needed.
• IPv6 Grouping: Optional IPv6 network grouping affects challenge limits, statistics, and CAPTCHA escalation. It is disabled by default.
• Reverse Proxies: Configure trusted proxies correctly in XenForo when using Cloudflare or another reverse proxy to ensure accurate IP logging.
• Guest Page Caching: The Bypass guest page cache option is enabled by default to prevent cached protected results from bypassing verification.
• Recognized Robots: Recognized robots are excluded by default. Optional DNS verification can provide additional checks for supported search engine crawlers.
• Search Engine Indexing: Search engine crawlers can bypass verification when recognized and excluded. Disabling robot exclusions may prevent crawlers from accessing protected search and tag pages.

---

Recommended Extensions:​

Extend your XenForo spam protection to other forms with these complementary add-ons:

• [XB] Registration Spaminator: Add spam protection to member registration.
• [XB] Login Spaminator: Add spam protection to the login form.
• ✉ [XB] Contact Us Spaminator: Add spam protection to the contact form.

---

Screenshots:​

Search Verification & Public Log​

Administrator Statistics & Logs​

Configuration​