Golang io.TeeReader() example.

README.md

Golang io.TeeReader() example

• Opening a source words file.
• Create a io.TeeReader(), writing read file chunks also to a gzip writer.
• Output source chunks as read and total bytes.
• Finally, output compressed bytes and total compressed size.

Output

$ go run main.go
64: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vivamus
64:  eleifend, est vel malesuada fermentum, ligula magna mollis mass
64: a, vel tristique odio neque non ligula. Vestibulum lacinia erat
64: dignissim neque molestie, in tempus nisl suscipit. Duis sit amet
64:  eros egestas, dignissim diam vel, pharetra nisl. Proin placerat
64:  urna viverra, tristique nisi quis, eleifend mi. Sed in mauris e
64: t lectus facilisis porta. Quisque bibendum euismod convallis. Pr
64: oin eget ligula ac odio vehicula efficitur a pretium leo. Maecen
64: as sed augue nunc. Praesent eget mi nulla. Sed at tellus et diam
64:  maximus egestas. Aenean dui orci, aliquet vitae ante sit amet,
64: vestibulum mollis est. Vestibulum consectetur, lacus eu bibendum
64:  molestie, erat turpis egestas diam, ac lacinia orci nulla nec m
64: agna. Maecenas rhoncus felis et ipsum finibus, in blandit odio c
64: ursus. Suspendisse pretium, ligula quis gravida scelerisque, orc
64: i orci efficitur nisl, sed euismod nibh odio eu orci. Maecenas c
50: ursus ultrices lorem, at consequat leo viverra a.
ci. Maecenas c
Bytes read: 1010

Compressed: [31 139 8 0 0 0 0 0 2 255 84 147 95 170 252 54 12 133 223 187 138 179 0 147 61 20 250 216 66 203 15 238 187 198 214 100 4 182 156 107 73 161 203 47 118 238 252 233 203 48 132 88 58 254 190 147 63 251 224 6 57 44 26 74 175 125 192 196 65 141 61 33 119 53 206 206 30 3 84 228 16 203 162 59 184 138 111 248 146 147 90 24 184 178 220 89 75 2 155 227 228 138 70 149 45 168 16 238 60 26 171 71 75 168 178 71 37 52 218 149 208 122 173 98 104 100 70 105 157 241 33 230 242 29 140 94 164 67 121 254 213 174 63 231 54 124 177 185 220 162 70 67 165 44 42 4 30 228 40 178 171 152 73 251 57 211 122 157 111 114 130 40 156 219 17 6 21 171 176 176 44 199 76 254 71 136 189 110 9 30 221 192 59 155 147 165 143 113 69 168 205 104 9 199 131 6 251 160 53 103 195 223 163 139 226 168 148 87 128 24 74 56 229 228 49 40 125 220 67 197 4 223 33 150 94 136 208 100 195 47 46 51 90 163 24 98 96 71 229 236 97 184 83 150 42 38 134 163 15 167 13 255 132 216 28 116 147 27 107 137 6 14 177 214 203 212 114 210 4 248 204 194 251 28 115 1 166 124 17 60 249 33 121 62 224 251 93 178 44 133 56 6 187 76 130 220 55 252 69 156 89 201 96 92 64 177 207 204 161 121 14 37 54 86 191 230 54 129 70 157 10 102 114 114 56 215 26 43 249 66 212 232 95 89 61 184 16 110 248 157 149 73 81 66 208 71 150 4 170 19 136 227 20 39 6 169 243 71 199 206 183 215 159 86 176 249 255 116 127 180 48 77 247 115 87 188 169 188 133 47 29 30 227 144 87 154 21 49 77 40 207 210 204 72 215 133 160 156 175 62 126 176 24 143 174 115 195 157 235 101 231 250 50 238 162 114 11 91 165 186 85 210 34 126 97 206 49 44 108 195 175 176 131 181 136 25 63 41 191 58 63 75 128 125 208 41 133 96 153 43 143 101 54 93 97 214 207 219 210 44 89 90 78 158 190 85 110 143 107 27 199 122 251 35 239 181 31 81 125 72 102 67 157 159 115 154 154 22 182 239 160 89 176 254 44 40 104 251 237 191 0 0 0 255 255 77 50 188 104 242 3 0 0]
Compressed size: 523

main.go

package main

import (
	"bytes"
	"compress/gzip"
	"fmt"
	"io"
	"os"
)

const readBufSize = 64

func main() {
	// open source file
	fh, err := os.Open("words.txt")
	if err != nil {
		fmt.Println("Error:", err)
		return
	}

	defer fh.Close()

	// create gzip compressor `shrink`, writing to `result` and passing through `tee`
	result := new(bytes.Buffer)
	shrink, _ := gzip.NewWriterLevel(result, gzip.BestCompression)
	tee := io.TeeReader(fh, shrink)

	// read through file
	buf := make([]byte, readBufSize)
	bytesRead := 0
	for {
		len, err := tee.Read(buf)
		if err == io.EOF {
			break // end of file
		}

		// bytes read and buffer
		fmt.Printf("%d: %s\n", len, string(buf))
		bytesRead += len
	}

	fmt.Println("Bytes read:", bytesRead)

	// end compressing
	shrink.Close()

	compressedSize := result.Len()
	fmt.Println("\nCompressed:", result.Bytes())
	fmt.Println("Compressed size:", compressedSize)
}

words.txt

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vivamus eleifend, est vel malesuada fermentum, ligula magna mollis massa, vel tristique odio neque non ligula. Vestibulum lacinia erat dignissim neque molestie, in tempus nisl suscipit. Duis sit amet eros egestas, dignissim diam vel, pharetra nisl. Proin placerat urna viverra, tristique nisi quis, eleifend mi. Sed in mauris et lectus facilisis porta. Quisque bibendum euismod convallis. Proin eget ligula ac odio vehicula efficitur a pretium leo. Maecenas sed augue nunc. Praesent eget mi nulla. Sed at tellus et diam maximus egestas. Aenean dui orci, aliquet vitae ante sit amet, vestibulum mollis est. Vestibulum consectetur, lacus eu bibendum molestie, erat turpis egestas diam, ac lacinia orci nulla nec magna. Maecenas rhoncus felis et ipsum finibus, in blandit odio cursus. Suspendisse pretium, ligula quis gravida scelerisque, orci orci efficitur nisl, sed euismod nibh odio eu orci. Maecenas cursus ultrices lorem, at consequat leo viverra a.