Latency values for programmers in 2024
Latency values are important for modern software. A program usually waits for I/O, another thread to complete a task, or a network request to complete. In a system, a database lock will create contention for other database connections. A global interpreter lock will force other threads to wait. These are important and common problems to consider. But hardware has certainly become faster over the decades. Here is what I learned in 2024.
| Operation | Time in nanosecond | Time in microsecond | Time in millisecond | Comparasion | Reference |
|---|---|---|---|---|---|
| L1 cache reference | 0.7 | Zen 5 numbers | |||
| L2 cache reference | 2.5 | ||||
| Branch mispredict | 8 | ||||
| L3 cache reference | 8 | ||||
| Mutex lock/unlock | 25 | ||||
| Main memory reference | 70 | 100x L1 | |||
| Send 2 kB over 10 Gbps network | 1600 | 1.6 | |||
| Compress 1K bytes with Snappy | 2000 | 2 | |||
| Read 1 MB sequentially from memory | 10000 | 10 | ~50 GB/s DDR5 | ||
| Read 4K randomly from SSD | 20000 | 20 | ~10 GB/s NVMe | ||
| Read 1 MB sequentially from NVMe | 100000 | 100 | |||
| Round trip within same datacenter | 100000 | 100 | ~0.5GB/sec SSD, 100x memory, 20x NVMe | ||
| SSD seek time | 160,000 | 160 | 2,000x main memory | ~150MB/sec | |
| Read 1 MB sequentially from SSD | 2,000,000 | 2000 | 2 | ||
| AWS region round trip | 2,000,000 | 2000 | 2 | ||
| Read 1 MB sequentially from HDD | 6,000,000 | 6000 | 6 | ||
| Send 1 MB over 1 Gbps network | 10,000,000 | 10000 | 10 | ||
| HDD random seek time | 10,000,000 | 10000 | 10 | Often times, data are located close by. The typical seek time should be lower. | |
| Send packet CA->Netherlands->CA | 150,000,000 | 150000 | 150 |
Storage device and compute device innovations are most relevant to software applications. Apart from latency numbers, DDR5 runs at 6,000 MT/s, so this is 48,000 MB/s (random access). SSDs perform surprisingly well in terms of sequential read/write nowadays.
| Sources |
|---|
| https://static.googleusercontent.com/media/sre.google/en//static/pdf/rule-of-thumb-latency-numbers-letter.pdf |
| http://web.archive.org/web/20240815121029/https://gist.github.com/BlackHC/2d0a3a21542b524a7cf2f8eac977481e |
| https://colin-scott.github.io/personal_website/research/interactive_latency.html |
| https://cloud.ibm.com/docs/vpc?topic=vpc-network-latency-dashboard |
| https://en.wikipedia.org/w/index.php?title=Hard_disk_drive_performance_characteristics&oldid=1221433550 |
Leave a Comment