Event on Jan 3, 2024: A malicious actor used leaked credentials to log into Orange Spain's RIPE account, creating invalid RPKI ROAs that switched IP space, causing a 3 - 4 hour connectivity disruption for 9 million+ customers.
- Timeline: Orange Spain employee's computer was compromised, password leaked. Malicious actor found credentials, logged in with no 2FA. First changes at 09:38, actor signed ROAs at 13:50 pointing 2M+ IPs to non-Orange ASN, and traffic was greatly impacted at 14:30. Malicious ROA removed at 17:30 and reachability mostly restored by 19:00.
- Lessons Learned: RIPE NCC didn't enforce 2FA, had no way to force all account members to use it. Reaction took a long time. Account shouldn't have been compromised by malware with "ripeadmin" password.
- Network Response: During the outage, impacted prefixes were almost unreachable except for some upstreams or peerings. Networks with direct Orange/OpenTransit connectivity or not doing ROV were not affected. Route origin validation deployment has been accelerating, still resulting in a large part of the internet unable to reach Orange Spain.
- Visibility Over Time: Using bgp.tools data, it took until 14:11 for networks to start dropping the prefix from their routing tables. Visibility began to be restored at 17:47 and it took about an hour for most observing BGP feeds to see the prefix again. The drop of visibility was in stages due to different network refresh times.
- AWS RPKI ROV Quirk: Amazon Web Services could still send data during the outage despite implementing ROV. It's believed they have a special behavior not to mark a prefix invalid if it would cause a large traffic shift.
- Mitigation Tactics: If a network has a bad ROA, RPKI software has a tool called SLURM to filter out or inject data.
- New bgp.tools Tooling: A new historical RPKI search tool was made using data from RIPE RPKI Archive and RPKIViews, accessible via IP prefix on bgp.tools or directly at https://bgp.tools/rpki-history?cidr=91.198.241.0%2f24&asn=206924.
- Stay Updated: Use RSS feed or follow on Fediverse @mailto:benjojo@benjojo.co.uk.
**粗体** _斜体_ [链接](http://example.com) `代码` - 列表 > 引用
。你还可以使用@
来通知其他用户。