Reputation: 2886
I'm making multiples http request:
type item struct{
me []byte
}
items := getItems()
for _, me := range items {
me.save()
}
In order to do it efficiently I'm doing it with go rutines, my first approach was to make it like a pool of go rutines:
items := getItems()
var wg sync.WaitGroup
wg.Add(len(items))
for _, me := range items {
go func(me item) {
me.save()
wg.Done()
}(me)
}
wg.Wait()
But they all try to make http request at the same time and some of them fail because of my bandwidth can't handle them all.
So I try channels along with select
instead :
channel1 := make(chan item)
channel2 := make(chan item)
channel3 := make(chan item)
var wg sync.WaitGroup
items := getItems()
wg.Add(len(items))
go func() {
for me := range channel1 {
me.save()
wg.Done()
}
}()
go func() {
for me := range channel2 {
me.save()
wg.Done()
}
}()
go func() {
for me := range channel3 {
me.save()
wg.Done()
}
}()
for _, me := range items {
select {
case channel1 <- me:
case channel2 <- me:
case channel3 <- me:
}
}
But adding more go rutines to find the max go rutines my bandwidht can handle, my code gets larger and larger, and I try to do this:
max:=7
var channels []chan item
for i:=0;i<max;i++{
channel=make(chan item)
channels=append(channels,channel)
}
for _, me := range items {
select {
//???????????????
}
}
but I'm not prety sure how to do it as the last approach
Also take in mind that "Select from slice of channels" was a question already asked, but their answers were only when select
is listening which channel arrive first, in my case I want Select
to send any free channel, so it's different
Upvotes: 1
Views: 789
Reputation: 2886
I had a wrong approach and the answer was easy with worker pools
type item struct {
me []byte
}
func worker(canalFiles <-chan item, wg *sync.WaitGroup) {
for file := range canalFiles {
file.save()
wg.Done()
}
}
func main() {
var wg sync.WaitGroup
items := getItems()
wg.Add(len(items))
canalFiles := make(chan item)
for i := 0; i < 8; i++ {
go worker(canalFiles, &wg)
}
for _, file := range items {
canalFiles <- file
}
fmt.Printf("waiting.....")
wg.Wait()
}
Upvotes: 0
Reputation: 1363
You can use reflect.Select, creating a slice of SelectCase
structs with Dir=SelectSend
, like this:
max:=7
var channels []chan item
for i:=0;i<max;i++{
channel=make(chan item)
channels=append(channels,channel)
}
for _, me := range items {
cases := make([]reflect.SelectCase, max)
for j := 0; j < max; j++ {
cases[j] = reflect.SelectCase{
Dir: reflect.SelectSend,
Chan: reflect.ValueOf(channels[j]),
Send: reflect.ValueOf(me)
}
}
reflect.Select(cases)
}
Upvotes: 3