Example of writing a gzip file in go (golang). This appends data to the same file every time it is run. It is important to Flush() the data before closing the file.

gzipWriter.go

/*
  This is an example of a golang gzip writer program,
  which appends data to a file.
  


*/

package main

import (
  "bufio"
	"compress/gzip"
	"log"
	//	"fmt"
	"os"
)

type F struct {
	f  *os.File
	gf *gzip.Writer
	fw *bufio.Writer
}

func CreateGZ(s string) (f F) {

	fi, err := os.OpenFile(s, os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0660)
	if err != nil {
		log.Printf("Error in Create\n")
		panic(err)
	}
	gf := gzip.NewWriter(fi)
	fw := bufio.NewWriter(gf)
	f = F{fi, gf, fw}
	return
}

func WriteGZ(f F, s string) {
	(f.fw).WriteString(s)
}

func CloseGZ(f F) {
	f.fw.Flush()
	// Close the gzip first.
	f.gf.Close()
	f.f.Close()
}

func main() {
	f := CreateGZ("Append.gz")
	WriteGZ(f, "gzipWriter.go created this file.\n")
	CloseGZ(f)

}

066 is rw-rw---- i.e. group can write to the file. It may be better to change to 0644 rw-r--r--
Also, except of embedded devices, CPUs time is not so big limit than IO so it makes sense to use a better compression level:

gf, _ := gzip.NewWriterLevel(f, flate.BestCompression)

Another possible improvement is to increase buffer size. gzip has a window up to 32 Kb but by default bufio has 4Kb.

// Gzip anyway have a 32Kb window so smaller buffer useless
bf:= bufio.NewWriterSize(gf, 32768)

But that also means that you can't make zcat/zgrep untill the buffer is filled. Also when process is killed you must flush and close the file otherwise all the data will be lost. You have to listen for SIGINT and SIGTERM