IBM architect explains how to scale AI pipelines for peak traffic

When a winter storm knocked network equipment offline, the operations team had to sort through a surge of trouble tickets. The AI tool built to summarize those tickets stopped producing summaries just as engineers needed them most.Mundakkapatta Dileep Kainary, an application architect at IBM who has spent more than two decades building enterprise systems, described the failure and the redesign in a DZone article on an LLM pipeline that survives traffic spikes. His takeaway: Counting requests is not enough if each request can suddenly become much more demanding.The stakes can be high. IT service management vendor Virima cites an estimate of about $2 million an hour in median downtime costs for high-impact outages at large enterprises. For teams using AI to help respond to incidents, the question is whether the tool can keep working when the systems it monitors are under the most strain.Demand and failure can arrive togetherIn Mundakkapatta’s account, the winter storm in early 2026 took