Latency values for programmers in 2024

Latency values are important for modern software. A program usually waits for I/O, another thread to complete a task, or a network request to complete. In a system, a database lock will create contention for other database connections. A global interpreter lock will force other threads to wait. These are important and common problems to consider. But hardware has certainly become faster over the decades. Here is what I learned in 2024.

Operation	Time in nanosecond	Time in microsecond	Time in millisecond	Comparasion	Reference
L1 cache reference	0.7				Zen 5 numbers
L2 cache reference	2.5
Branch mispredict	8
L3 cache reference	8
Mutex lock/unlock	25
Main memory reference	70			100x L1
Send 2 kB over 10 Gbps network	1600	1.6
Compress 1K bytes with Snappy	2000	2
Read 1 MB sequentially from memory	10000	10			~50 GB/s DDR5
Read 4K randomly from SSD	20000	20			~10 GB/s NVMe
Read 1 MB sequentially from NVMe	100000	100
Round trip within same datacenter	100000	100			~0.5GB/sec SSD, 100x memory, 20x NVMe
SSD seek time	160,000	160		2,000x main memory	~150MB/sec
Read 1 MB sequentially from SSD	2,000,000	2000	2
AWS region round trip	2,000,000	2000	2
Read 1 MB sequentially from HDD	6,000,000	6000	6
Send 1 MB over 1 Gbps network	10,000,000	10000	10
HDD random seek time	10,000,000	10000	10		Often times, data are located close by. The typical seek time should be lower.
Send packet CA->Netherlands->CA	150,000,000	150000	150

Storage device and compute device innovations are most relevant to software applications. Apart from latency numbers, DDR5 runs at 6,000 MT/s, so this is 48,000 MB/s (random access). SSDs perform surprisingly well in terms of sequential read/write nowadays.

Sources
https://static.googleusercontent.com/media/sre.google/en//static/pdf/rule-of-thumb-latency-numbers-letter.pdf
http://web.archive.org/web/20240815121029/https://gist.github.com/BlackHC/2d0a3a21542b524a7cf2f8eac977481e
https://colin-scott.github.io/personal_website/research/interactive_latency.html
https://cloud.ibm.com/docs/vpc?topic=vpc-network-latency-dashboard
https://en.wikipedia.org/w/index.php?title=Hard_disk_drive_performance_characteristics&oldid=1221433550