Latency Limits
Summary
- Latency vs Throughput
- Components of RPC Latency
- Sources of overhead in RPC
- Control Transfer
- Protocol Processing
Latency vs Throughput
Latency := Elapsed time
Throughput := Events per unit time
RPC Performance:
- Hardware overhead
- How network is itnerfaced to computer
- Software overhead
Components of RPC Latency
- Client Call
- Controller Latency
- Time on Wire
- Interrrupt Handling
- Server setup to execute call
- Server execution and reply
- Client setup to receive results and restart
Sources of overhead in RPC
- Marshaling
- Data copying
- Control transfer
- Protocol processing
Marshaling and Data Copying
Copying is biggest source of latency
Three copies of data:
- Client Stub
- Kernel buffer
- DMA to controller
You can reduce copies by marshaling into kernel buffer directly, or by using shared descriptors between client stub and kernel
Control Transfer
Potentially 4 context switches, but at least two are required
- Call in
- Result in
- Result overlap with network communication
Can even reduce down to one context switch if we spin
- Server still needs to ctx switch tho
Protocol Processing
What transport for RPC?
- If the LAN is reliable, we can focus on reducing latency
Choices for reducing latency in transport:
- No low-level ACKs
- Hardware checksum for packet integrity
- No client side buffering since client blocked
- Overlap server side buffering with result transmission