CODEWITHSUNDEEP
c#multithreadinglist
Alexander
Alexander

Reputation: 20244

Thread-safe list or concat multiple lists?

I have two lists of dictionaries. I will query one user per Thread, up to a total of 50 users/threads.

The first list, "users", contains only one entry per thread. The second list, "items", may contain thousands of entries per thread (300 is a good average).

At the moment, I query for users sequentially in a single thread: First, the data for one user is collected (this is implemented synchronously and includes ~100ms of plain waiting), then user will be added to "users", and then every item will be processed, one at a time, and added to "items" (processing takes around a dozen milliseconds for 1000 items).

While the lists are ordered, I don't have to preserve any order at all.

I now thought as follows:

But before I proceed like this, I'd like to ask you whether this is the best approach, or whether you can think of better solutions. In the to-be-parallelized part, I will do nothing on either list but Add items.

Upvotes: 1

Views: 453

Answers (1)

Troy Gizzi
Troy Gizzi

Reputation: 2519

Per this Thread-Safe Collections article, any collection in the System.Collections.Concurrent namespace (which includes ConcurrentBag) should be fine for your multi-threading needs.

I can say from experience that using lock around a simple Dictionary (in the System.Collections.Generic namespace) worked fine for a huge in-memory, add-only collection. In production, it regularly held millions of items with no issues, even with a periodic process to spin through and remove "expired" items.

So bottom line, you shouldn't need to resort to building 50 separate lists for the items and then concatenate them afterward. But of course, that approach would probably work just fine too. It may just come down to whatever you personally find to be the most convenient.

Upvotes: 2

Related Questions