How safe are Golang maps for concurrent Read/Write operations?

Question

According to the Go blog,

Maps are not safe for concurrent use: it's not defined what happens when you read and write to them simultaneously. If you need to read from and write to a map from concurrently executing goroutines, the accesses must be mediated by some kind of synchronization mechanism. (source: https://blog.golang.org/go-maps-in-action)

Can anyone elaborate on this? Concurrent read operations seem permissible across routines, but concurrent read/write operations may generate a race condition if one attempts to read from and write to the same key.

Can this last risk be reduced in some cases? For example:

• Function A generates k and sets m[k]=0. This is the only time A writes to map m. k is known to not be in m.
• A passes k to function B running concurrently
• A then reads m[k]. If m[k]==0, it waits, continuing only when m[k]!=0
• B looks for k in the map. If it finds it, B sets m[k] to some positive integer. If it doesn't it waits until k is in m.

This isn't code (obviously) but I think it shows the outlines of a case where even if A and B both try to access m there won't be a race condition, or if there is it won't matter because of the additional constraints.

Answer 1

I am going to share one package that internally handles all these things, so first, I will tell you what I was doing and getting issues while iterating the map and also updating, at the same time, issues in concurrent writing and reading.

I have one map and update it with every tick of the WebSocket (I was getting LTP of 20 scripts via socket), so for 20 scripts, a lot of ticks are coming continuously and in the same map; I have to do some different operations one operation in every 2 seconds in a different go routine and another operation in every 1 minute in different go routine so the program was crashing continuously after some time. Then I used concurrent-map and removed all the mutex I was using. Internally, the concurrent-map package developer handled everything on their side, and ran program smoothly.

Package Github Repository : Link
To know more about package : Link

Special Thanks to concurrent-map developers

Answer 2

Before Golang 1.6, concurrent read is OK, concurrent write is not OK, but write and concurrent read is OK concurrent read and write is not OK but the compiler won't complain. Since Golang 1.6, map cannot be read when it's being written. So After Golang 1.6, concurrent access map should be like:

package main

import (
    "sync"
    "time"
)

var m = map[string]int{"a": 1}
var lock = sync.RWMutex{}

func main() {
    go Read()
    time.Sleep(1 * time.Second)
    go Write()
    time.Sleep(1 * time.Minute)
}

func Read() {
    for {
        read()
    }
}

func Write() {
    for {
        write()
    }
}

func read() {
    lock.RLock()
    defer lock.RUnlock()
    _ = m["a"]
}

func write() {
    lock.Lock()
    defer lock.Unlock()
    m["b"] = 2
}

Or you will get the error below:

ADDED:

You can detect the race by using go run -race race.go

Change the read function:

func read() {
    // lock.RLock()
    // defer lock.RUnlock()
    _ = m["a"]
}

Another choise:

As we known, map was implemented by buckets and sync.RWMutex will lock all the buckets. concurrent-map use fnv32 to shard the key and every bucket use one sync.RWMutex.

Answer 3

Map is concurrent safe for read only in Golang. Let's say, your map is written first and never be written again then you don't need any mutex type of thing to make sure that only one go routine is accessing your map. I have given an example below about map concurrent safe reading.

package main

import (
    "fmt"
    "sync"
)

var freq map[int]int

// An example of concurrent read from a map
func main()  {
    // Map is written before accessing from go routines
    freq = make(map[int]int)
    freq[1] = 1
    freq[2] = 2

    wg := sync.WaitGroup{}
    wg.Add(10)

    for i:=1;i<=10;i++ {
        // In go routine we are only reading val from map
        go func(id int, loop int) {
            defer wg.Done()
            fmt.Println("In loop ", loop)
            fmt.Println("Freq of 1: ", freq[id])
        }(1, i)
    }

    wg.Wait()
}

Answer 4

You can use sync.Map which is safe for concurrent use. The only caveat is that you are gonna give up on type safety and change all the reads and writes to your map to use the methods defined for this type

Answer 5

After long discussion it was decided that the typical use of maps did not require safe access from multiple goroutines, and in those cases where it did, the map was probably part of some larger data structure or computation that was already synchronized. Therefore requiring that all map operations grab a mutex would slow down most programs and add safety to few. This was not an easy decision, however, since it means uncontrolled map access can crash the program.

The language does not preclude atomic map updates. When required, such as when hosting an untrusted program, the implementation could interlock map access.

Map access is unsafe only when updates are occurring. As long as all goroutines are only reading—looking up elements in the map, including iterating through it using a for range loop—and not changing the map by assigning to elements or doing deletions, it is safe for them to access the map concurrently without synchronization.

As an aid to correct map use, some implementations of the language contain a special check that automatically reports at run time when a map is modified unsafely by concurrent execution.

Answer 6

As the other answers here stated, the native map type is not goroutine-safe. A couple of notes after reading the current answers:

1. Do not use defer to unlock, it has some overhead that affects performance (see this nice post). Call unlock directly.
2. You can achieve better performance by reducing time spent between locks. For example, by sharding the map.
3. There is a common package (approaching 400 stars on GitHub) used to solve this called concurrent-map here which has performance and usability in mind. You could use it to handle the concurrency issues for you.

Answer 7

You can store a pointer to an int in the map, and have multiple goroutines read the int being pointed to while another writes a new value to the int. The map is not being updated in this case.

This wouldn't be idiomatic for Go and not what you were asking.

Or instead of passing a key to a map, you could pass the index to an array, and have that updated by one goroutine while others read the location.

But you're probably just wondering why a map's value can't be updated with a new value when the key is already in the map. Presumably nothing about the map's hashing scheme is being changed - at least not given their current implementation. It would seem the Go authors don't want to make allowances for such special cases. Generally they want code to be easy to read and understand, and a rule like not allowing map writes when other goroutines could be reading keeps things simple and now in 1.6 they can even start to catch misuse during normal runtimes - saving many people many hours of debugging.

Answer 8

Concurrent read (read only) is ok. Concurrent write and/or read is not ok.

Multiple goroutines can only write and/or read the same map if access is synchronized, e.g. via the sync package, with channels or via other means.

Your example:

1. Function A generates k and sets m[k]=0. This is the only time A writes to map m. k is known to not be in m.
2. A passes k to function B running concurrently
3. A then reads m[k]. If m[k]==0, it waits, continuing only when m[k]!=0
4. B looks for k in the map. If it finds it, B sets m[k] to some positive integer. If it doesn't it waits until k is in m.

Your example has 2 goroutines: A and B, and A tries to read m (in step 3) and B tries to write it (in step 4) concurrently. There is no synchronization (you didn't mention any), so this alone is not permitted / not determined.

What does it mean? Not determined means even though B writes m, A may never observe the change. Or A may observe a change that didn't even happen. Or a panic may occur. Or the Earth may explode due to this non-synchronized concurrent access (although the chance of this latter case is extremely small, maybe even less than 1e-40).

Related questions:

Map with concurrent access

what does not being thread safe means about maps in Go?

What is the danger of neglecting goroutine/thread-safety when using a map in Go?

Answer 9

Go 1.6 Release Notes

The runtime has added lightweight, best-effort detection of concurrent misuse of maps. As always, if one goroutine is writing to a map, no other goroutine should be reading or writing the map concurrently. If the runtime detects this condition, it prints a diagnosis and crashes the program. The best way to find out more about the problem is to run the program under the race detector, which will more reliably identify the race and give more detail.

Maps are complex, self-reorganizing data structures. Concurrent read and write access is undefined.

Without code, there's not much else to say.