CODEWITHSUNDEEP
go
LeMoussel
LeMoussel

Reputation: 5767

Most efficient way to convert a [][]byte to []string in golang

To convert [][]byte to []string, I do this 

data, err := ioutil.ReadFile("test.txt")
if err != nil {
    return nil, err
}

db := bytes.Split(data, []uint8("\n"))

// Convert [][]byte to []string
s := make([]string, len(db))
for i, val := range db {
    s[i] = string(val)
}
fmt.Printf("%v", s)

I am new to golang, I'm not sure is most efficient way to do this.

Upvotes: 0

Views: 1774

Answers (2)

cd1
cd1

Reputation: 16534

If you actually want to convert a file content to a []string, you can use bufio.Scanner which is cleaner (IMO) and more efficient than the code you posted:

func readFile(filename string) ([]string, error) {
    file, err := os.Open(filename)
    if err != nil {
        return nil, err
    }   
    defer file.Close()

    scanner := bufio.NewScanner(file)

    var data []string

    for scanner.Scan() {
        line := scanner.Text()
        data = append(data, line)
    }   
    if err = scanner.Err(); err != nil {
        return nil, err
    }   

    return data, nil 
}

Here's a benchmark* comparing the original function (readFile1) and my function (readFile2):

BenchmarkReadFile1-8         300       4632189 ns/op     3035552 B/op      10570 allocs/op
BenchmarkReadFile2-8        1000       1695820 ns/op     2169655 B/op      10587 allocs/op

*the benchmark read a sample file of 1.2 MiB and ~10K lines

The new code runs in 36% of the time and 71% of the memory used by the original function.

Upvotes: 1

syntagma
syntagma

Reputation: 24334

The most effective way would be to remove this step: db := bytes.Split(data, []uint8("\n")) and instead iterate over data like that:

func main() {
    data, _ := ioutil.ReadFile("test.txt")

    s := make([]string, 0)
    start := 0
    for i := range data {
        if data[i] == '\n' {
            elem := string(data[start : i-1])
            s = append(s, elem)
            start = i
        }
    }
    fmt.Printf("%v", s)
}

Or if you want to convert [][]byte to []string:

func convert(data [][]byte) []string {
    s := make([]string, len(data))
    for row := range data {
        s[row] = string(data[row])
    }

    return s
}

Upvotes: 2

Related Questions