Local LLM Deployment for SOC: How Many Incidents Can One NVIDIA RTX PRO 6000 Handle?

In the second part of their experiment, analysts from R-Vision evaluated how a locally deployed large language model performs under realistic SOC workloads rather than isolated laboratory benchmarks. The configuration remained consistent with Part 1: the Qwen3.5-122B-A10B-GPTQ model served by vLLM on an NVIDIA RTX PRO 6000 Blackwell Max-Q GPU equipped with 96 GB of video memory.

The team shifted focus from measuring raw concurrency and generation speed to simulating actual Security Operations Center operations. Real anonymized incidents from their internal SOC were used instead of synthetic test cases. The model handled five core tasks: ranking open incidents by priority, summarizing key events and assets, searching for similar recent incidents, performing retrospective searches across deeper history, and generating preliminary verdicts based on prior steps.

Two distinct load profiles were tested. In a calm shift scenario, 3–5 L1/L2 analysts managed a steady flow of 10–15 incidents per hour. Background SOAR orchestration typically processed one or two incidents concurrently, generating up to six parallel requests during the data-collection phase. Interactive analyst chats added another two to three concurrent sessions, keeping overall concurrency between 8 and 12 requests.

Under peak conditions, such as a mass phishing campaign, 5–7 analysts faced 50–100 incidents arriving rapidly. When all 16 sequences were allocated to the background pipeline, up to four incidents could be processed simultaneously on the first phase (12 requests). With five sequences reserved for interactive work, three incidents could run in parallel. A full enrichment cycle of approximately 60 seconds allowed the queue of 50 incidents to be cleared in roughly 15–20 minutes.

To prevent interactive sessions from starving background tasks, an AI Gateway layer was introduced. It separated traffic into two pools: a high-priority “Interactive” pool limited to five concurrent requests with context up to 120k tokens, and a lower-priority “Background SOAR” pool supporting up to 11 concurrent requests with shorter contexts. The gateway also enforced context-length limits and dynamic prioritization to protect KV-cache capacity.

The experiment confirmed that one RTX PRO 6000 can sustain a background throughput of five incidents per minute, equating to 300 incidents per hour or 7,200 per day, when processing typical 10k–20k token incidents. The Mixture-of-Experts architecture of Qwen3.5-122B-A10B delivered effective intelligence close to a 122B model while maintaining generation speeds comparable to a 10B model, and disabling the thinking mode further improved responsiveness without sacrificing verdict quality for SOC tasks.