CODEWITHSUNDEEP
go
Dan Tanner
Dan Tanner

Reputation: 2444

Go - How increase maximum Stdin input length?

What's the recommended way to allow more than 1024 characters when reading from standard input in Go?

For example, this code using bufio.Scanner has a maximum input length of 1024.

package main

import (
    "bufio"
    "fmt"
    "os"
)

func main() {
    scanner := bufio.NewScanner(os.Stdin)
    scanner.Scan()
    input := scanner.Text()
    fmt.Println(input)
}

Update after some suggested answers... Sorry guys - I must still be doing something wrong, or misstated the question. I tried both suggestions and still see the same issue. Below is an updated version of the code. The symptom is that the scanner won't accept input after the 1024th character. e.g. Try to run this then paste in a string that's 1025 characters long, and it'll stop accepting input after character 1024. 

package main

import (
    "bufio"
    "bytes"
    "fmt"
    "log"
    "os"
)

func main() {
    var buffer bytes.Buffer
    scanner := bufio.NewScanner(os.Stdin)

    for {
        done := scanner.Scan()
        buffer.WriteString(scanner.Text())
        if done == true {
            if scanner.Err() != nil {
                log.Fatal("Error scanning input")
            }
            break
        }
    }
    fmt.Println(buffer.String())
}

Upvotes: 4

Views: 2766

Answers (3)

Danver Braganza
Danver Braganza

Reputation: 1355

The main problem with this code is that the return value of scanner.Scan is not true if the scanner is done, but true if the scanner is not done.

While we're fixing that, let's tidy up another thing, since you don't need to use 

if done == true

but can simply do

if done

Also, remember that done = Scanner.Text(), and serves as the terminating condition, and we can tidy up the code really nicely with:

for scanner.Scan() { // This does what you want, looping until scanner
                     // is no longer true.
    buffer.WriteString(scanner.Text())
}
if err := scanner.Err(); err != nil {           // Let's hold on to err...
    log.Fatalf("Error scanning input: %s", err) // and use it here.
}

You get to discard a lot of lines, and you only check Err() once at the end of the loop. No need to use break.

Hope that works for you, and send me a line if you need something else.

Edit: Two more things to be aware of.

1) bufio.Scanner.Scan() will be eating all your separator tokens. Maybe that's okay, or maybe you want to keep adding them back in.

2) bufio.Scanner.Scan() may panic if your SplitFunc is wrong, or someone is crafting poor input for your SplitFunc, and bytes.Buffer.WriteString() may panic if the buffer gets too large. If you're using these in a production system, make sure that you handle these panics in whatever way makes sense.

Upvotes: 1

FogleBird
FogleBird

Reputation: 76772

You can do this instead of using Scanner...

reader := bufio.NewReaderSize(os.Stdin, 65536)
line, isPrefix, err := reader.ReadLine()

Write a helper function to keep reading until you've read a complete line...

func ReadLine(reader *bufio.Reader) (string, error) {
    result := ""
    for {
        line, isPrefix, err := reader.ReadLine()
        if err != nil {
            return result, err
        }
        result += string(line)
        if !isPrefix {
            return result, nil
        }
    }
}

Upvotes: 0

sberry
sberry

Reputation: 132018

You are ignoring the return value of scanner.Scan() which is a bool indicating whether you have reached the end.

Specifically:

It returns false when the scan stops, either by reaching the end of the input or an error. After Scan returns false, the Err method will return any error that occurred during scanning, except that if it was io.EOF, Err will return nil.

So you need to continue running scanner.Scan() in a loop, until it returns false, then check .Err() to ensure you didn't stop scanning because of a non-EOF error.

Upvotes: 5

Related Questions