The cost of a syscall
Part of the Building the Log Package series How I thought through the log package design before writing any code
Syscalls are not free they involve a context switch from user space to kernel space. Your Go program runs in user space think of it as a regular employee working at their desk. The kernel is the IT department they control all the actual hardware: disk, network, memory.
The rule is: you cannot touch hardware yourself. Every time you need something from hardware, you have to get up from your desk, walk to IT, make a request, wait for them to handle it, and walk back.
That walk to IT and back that's a context switch. The price you pay:
Time: a syscall costs roughly 1,000-10,000 nanoseconds depending on the system. That sounds tiny. But if you're writing 1 byte at a time and calling write() for each byte, on a million records that's seconds of pure overhead doing nothing useful.
CPU state flush: when you switch from user space to kernel space, the CPU has to save everything it was doing registers, instruction pointer, stack pointer. Then restore it all when returning. It's like being interrupted mid-sentence, writing down every word you were about to say, handling something else, then reading your notes to continue. Every time.
Cache disruption: your program's data was warm in the CPU cache. The kernel runs its own code, potentially evicting your data from cache. When you return to user space, cache misses are more likely. Memory accesses get slower.
This is exactly why bufio.Writer exists. Instead of:
write "h" → syscall
write "e" → syscall
write "l" → syscall
write "l" → syscall
write "o" → syscall
= 5 syscallsIt buffers in memory and does:
accumulate "hello" in memory buffer
when buffer is full → one syscall writing 4096 bytes at once
= 1 syscallThe compounding effect matters at scale. One extra syscall per record is invisible at 100 records and at 10 million records per second which is normal for Kafka-scale systems. Unnecessary syscalls are the difference between a system you trust at 2am and one that pages you at 2am.
That's the price being paid: time, CPU efficiency, and cache performance, all compounding under load.