Using bufio doesn't seem to change the numbers.
os.Open and os.Stdout may already be buffered.
Anyway, 6prof shows:
1142 samples (avg 1 threads)
 34.94%compress/flate.*decompressor·huffmanBlock
 33.27%compress/flate.*decompressor·huffSym
 15.94%hash/crc32.update
  8.49%compress/flate.*decompressor·moreBits
  5.34%bufio.*Reader·ReadByte
  0.70%runtime.memmove

Which clearly says that 77% of the time is spent in flate, and
16% is spent in crc32 (total 92.6%). ReadByte doesn't seem like
much of the time in that scenario. Also, I think zlib itself
uses big buffers rather than byte-at-a-time. IIRC you pass it
a variable length buffer, and it tells you when it has consumed
that and needs more data to process.