AI caused our systems to crash – and what we did about it - AlphaSys
Summary
Over the last few weeks, some of our websites began experiencing short, sharp bursts of 502 Bad Gateway errors. Users would click a link and instantly hit an error – no loading delay, no pattern – just brief outages that typically resolved themselves within 60–90 seconds. • When cross-referencing server access logs, we noticed a high frequency of requests from AI crawlers – specifically user agents like: While these bots are typically benign, the volume and timing of their visits lined up with the symptoms we observed: high load, server response delays, and 502s triggered at the network level. • Bot Management: We began blocking specific AI user agents from resource-heavy paths using web rules. • Monitoring Enhancements: We’re testing tools like Dark Visitor to gain deeper visibility into bot activity.