CODEWITHSUNDEEP
gogzip
stian
stian

Reputation: 1987

How can I use gzip on a string?

I want to use Go to read out a chunk from a file, treat it as a string and gzip this chunk. I know how to read from the file and treat it as a string, but when it comes to compress/gzip I am lost.

Should I create an io.writer, which writes to a buf (byte slice), use gzip.NewWriter(io.writer) to get a w *gzip.Writer and then use w.Write(chunk_of_file) to write the chunk of file to buf? Then I would need to treat the string as a byte slice.

Upvotes: 18

Views: 33312

Answers (2)

Zombo
Zombo

Reputation: 1

If the result is not going back into a file, then you could just use Flate directly. You save a bit of overhead from Gzip. Another option is Brotli. Examples:

package main

import (
   "bytes"
   "compress/flate"
   "github.com/andybalholm/brotli"
)

func compressFlate(data []byte) ([]byte, error) {
   var b bytes.Buffer
   w, err := flate.NewWriter(&b, 9)
   if err != nil {
      return nil, err
   }
   w.Write(data)
   w.Close()
   return b.Bytes(), nil
}

func compressBrotli(data []byte) []byte {
   var b bytes.Buffer
   w := brotli.NewWriterLevel(&b, brotli.BestCompression)
   w.Write(data)
   w.Close()
   return b.Bytes()
}

Result:

package main

import (
   "bytes"
   "fmt"
)

func main() {
   data := bytes.Repeat([]byte("hello world"), 999_999)
   f, err := compressFlate(data)
   if err != nil {
      panic(err)
   }
   b := compressBrotli(data)
   fmt.Println(len(f) == 21379, len(b) == 40)
}

Upvotes: 1

Intermernet
Intermernet

Reputation: 19378

You can just write using gzip.Writer as it implements io.Writer.

Example:

package main

import (
    "bytes"
    "compress/gzip"
    "fmt"
    "log"
)

func main() {
    var b bytes.Buffer
    gz := gzip.NewWriter(&b)
    if _, err := gz.Write([]byte("YourDataHere")); err != nil {
        log.Fatal(err)
    }
    if err := gz.Close(); err != nil {
        log.Fatal(err)
    }
    fmt.Println(b.Bytes())
}

Go Playground

If you want to set the compression level (Default is -1 from compress/flate) you can use gzip.NewWriterLevel.

Upvotes: 38

Related Questions