Incidents and security
How to respond to failures, overheating, attacks and data breaches.
Failures, overheating, attacks: your organization makes the difference between a minor incident and a costly crisis.
How an incident unfolds
Failures are no longer windows that interrupt you: they are tickets in a queue that you handle at your own pace.
- A ticket appears (a number like INC-0042, priority, subject, target, description). A small, discreet notification announces it; an alarm only sounds for a P1.
- Open the Tickets panel (J key or the “Tickets (J)” button) to prioritize, assign a team (or your character) and track progress.
- The intervention has three phases: travel, diagnosis, repair. A specialized team always meets the SLA if it leaves right away.
- Ticket resolved: bonus and XP, plus one more article in the knowledge base.
Priorities and SLAs
Deadlines are counted in game hours (at x1 speed, 1 real second = 1 game minute; x5 and x24 speed everything up proportionally):
| Priority | Level | SLA |
|---|---|---|
| P1 | Critical | 6 h |
| P2 | High | 12 h |
| P3 | Medium | 24 h |
| P4 | Low | 72 h |
If the SLA is exceeded, you pay a penalty only once (compensation and reputation), but the repair is never lost: it continues and the bonus is reduced. A ticket ignored for 3 times its SLA is escalated: a second penalty, and your cloud tenants' SLA drops. There are no more per-second penalties.
Where failures come from
They depend on the actual load: more clients, loaded racks, heat, power draw and a high PUE mean more tickets. An idle datacenter has none.
The knowledge base
Each type of failure has its article. Every resolution fills it in (mastery from 0 to 5 at 1, 3, 6, 10 and 15 resolutions): symptoms, diagnostic command (useful in the KVM console, K), procedure and tips, with up to 55% less diagnosis time. Complex failures go faster once the related basic article is mastered.
Incident types
The higher your level, the more varied the incidents. Here are the game's scenarios:
| Incident type | Category | Severity | From level | Priority (SLA) | Bonus | Action |
|---|---|---|---|---|---|---|
| Critical Overheating on Rack | Thermal & Cooling | Warning | 1 | P4 (72 h) | $220 | Force PWM Ventilation & Balance the Load |
| Chilled Water Leak Under the Floor (X) | Hydraulics & Chilled Water | Critical | 2 | P1 (6 h) | $380 | Isolate the Valve & Switch to the N+1 Loop |
| Inverter Compressor Failure (X) | CRAC Cooling | Critical | 3 | P2 (12 h) | $420 | Recalibrate the EEV & Restart the Compressor |
| 400V PDU Phase Imbalance (X) | 400V Power Distribution | Warning | 1 | P4 (72 h) | $290 | Switch the ATS & Balance the 3 Phases |
| LFP Cell Thermal Drift (BESS Room) | UPS & BESS Room | Critical | 3 | P1 (6 h) | $520 | Isolate BESS String #02 & Balance the BMS |
| Harmonics & Partial Discharge at the 20 kV Substation | 20 kV High-Voltage Substation | Critical | 4 | P1 (6 h) | $640 | Engage the AHF Harmonic Filter & Stabilize the 20 kV |
| Water/Sludge Contamination in the V16 Generator Fuel Tank | V16 Generators | Warning | 2 | P3 (24 h) | $350 | Purge the Settling Tank & Loop-Filter the Fuel |
| Degraded RAID-Z2 Array | 42U SAN & NVMe Storage | Warning | 1 | P3 (24 h) | $310 | Engage the NVMe Hot-Spares & Rebuild the RAID |
| 400G QSFP-DD Optical Signal Loss (X) | 400G Optics & Interconnect | Critical | 2 | P2 (12 h) | $390 | Fail Over the ECMP Fiber Uplink & Isolate the Laser |
| Kernel Panic & Multi-Bit ECC Error (X) | Hypervisors & GPU Clusters | Warning | 2 | P4 (72 h) | $340 | Isolate the ECC RAM Pages & Restart the Hypervisor |
| BGP Route Leak & AS Hijack | Core BGP Routing & IXP Peering | Critical | 4 | P1 (6 h) | $580 | Enforce RPKI ROA Validation & Take Back the Prefixes |
| VESDA Laser Alarm: Particles | VESDA Fire Detection | Critical | 3 | P1 (6 h) | $490 | Cut the Targeted eFuse & Abort the FM-200 Dump |
| Intrusion Attempt / Cloned RFID Badge (IT Dock) | Physical Security & Access Control | Critical | 3 | P1 (6 h) | $460 | Lock the Mantrap & Revoke the Cloned Badge |
Major data breaches
At certain milestones (every 50 levels), a major data breach occurs: thousands of client records are exfiltrated, with heavy compensation costs and reputation loss. It stays a P1 ticket. The SecOps team and security upgrades limit the cost.
Never leave a P1 ticket without an assigned team while you handle something else: the 6-game-hour SLA goes by fast, especially at high speed.