Latency values for programmers in 2024

Latency values are important for modern software. A program usually waits for I/O, another thread to complete a task, or a network request to complete. In a system, a database lock will create contention for other database connections. A global interpreter lock will force other threads to wait. These are important and common problems to consider. But hardware has certainly become faster over the decades. Here is what I learned in 2024.

Operation Time in nanosecond Time in microsecond Time in millisecond Comparasion Reference
L1 cache reference 0.7       Zen 5 numbers
L2 cache reference 2.5        
Branch mispredict 8        
L3 cache reference 8        
Mutex lock/unlock 25        
Main memory reference 70     100x L1  
Send 2 kB over 10 Gbps network 1600 1.6      
Compress 1K bytes with Snappy 2000 2      
Read 1 MB sequentially from memory 10000 10     ~50 GB/s DDR5
Read 4K randomly from SSD 20000 20     ~10 GB/s NVMe
Read 1 MB sequentially from NVMe 100000 100      
Round trip within same datacenter 100000 100     ~0.5GB/sec SSD, 100x memory, 20x NVMe
SSD seek time 160,000 160   2,000x main memory ~150MB/sec
Read 1 MB sequentially from SSD 2,000,000 2000 2    
AWS region round trip 2,000,000 2000 2    
Read 1 MB sequentially from HDD 6,000,000 6000 6    
Send 1 MB over 1 Gbps network 10,000,000 10000 10    
HDD random seek time 10,000,000 10000 10   Often times, data are located close by. The typical seek time should be lower.
Send packet CA->Netherlands->CA 150,000,000 150000 150    

Storage device and compute device innovations are most relevant to software applications. Apart from latency numbers, DDR5 runs at 6,000 MT/s, so this is 48,000 MB/s (random access). SSDs perform surprisingly well in terms of sequential read/write nowadays.

Sources
https://static.googleusercontent.com/media/sre.google/en//static/pdf/rule-of-thumb-latency-numbers-letter.pdf
http://web.archive.org/web/20240815121029/https://gist.github.com/BlackHC/2d0a3a21542b524a7cf2f8eac977481e
https://colin-scott.github.io/personal_website/research/interactive_latency.html
https://cloud.ibm.com/docs/vpc?topic=vpc-network-latency-dashboard
https://en.wikipedia.org/w/index.php?title=Hard_disk_drive_performance_characteristics&oldid=1221433550

Tags:

Updated:

Leave a Comment