CODEWITHSUNDEEP
c#.netlinqdeferred-execution
tgabb
tgabb

Reputation: 379

How to generate elements of IEnumerable One by One

Suppose I have an IEnumerable<Thing> enumerable object and that each Thing in this list is expensive to generate.

When I go to execute the following code there is considerable overhead when the code reaches line 1, because, as I understand it, the entire list is generated right there before beginning iteration.

foreach (Thing t in enumerable)   // line 1
{                                 // line 2
    DoStuffWith(t);               // line 3
}

Is there a way to generate each Thing only as needed? Like this

while(enumerable.HasMoreThings())
{
    var e = enumerable.GenerateNextThing();
    DoStuffWith(e);
}

Original code:

using (var db = new PolyPrintEntities())
{
    var sw = new Stopwatch();
    sw.Start();

    IEnumerable<NotificationDto> notifications = db.EmailServiceNotifications
        .Select(not => new NotificationDto
        {
            Notification_ID = not.Notification_ID,
            StoredProcedure = not.StoredProcedure,
            SubjectLine = not.SubjectLine,
            BodyPreface = not.BodyPreface,
            Frequency = not.Frequency,
            TakesDateRange = not.TakesDateRange,
            DateRangeIntervalHours = not.DateRangeIntervalHours,
            TakesAssociateId = not.TakesAssociateId,
            NotificationType = not.NotificationType,
            Associates = not.SubscriptionEvent.SubscriptionSubscribers
                .Select(ss => new AssociateDto
                {
                    Record_Number = ss.Associate.Record_Number,
                    Name = ss.Associate.Name,
                    Email = ss.Associate.EMail
                })
        });

    sw.Stop();
    Console.WriteLine($"Enumeration:{sw.ElapsedMilliseconds}"); //about .5 seconds
    sw.Reset();

    sw.Start();

    // List is generated here, this is where lag happens. 
    int i = 0;

    foreach (var n in notifications)
    {
        if (i == 0)
        {
            sw.Stop();
            Console.WriteLine($"Generation:{sw.ElapsedMilliseconds}"); //about 10 seconds
            i++;
        }

        var ret = n.GetEmails();
        db.EmailQueues.AddRange(ret);
    }

    db.SaveChanges();
}

Upvotes: 2

Views: 303

Answers (3)

tgabb
tgabb

Reputation: 379

I figured it out.

It appears that after further refining mycode I have obtained some empirical data which disagrees with what i thought was going on.

I used the sw to measure time in a different way:

        using (var db = new PolyPrintEntities())
        {
            var sw = new Stopwatch();
            sw.Start();
            IEnumerable<NotificationDto> notifications = db.EmailServiceNotifications
                .Select(not => new NotificationDto
                {
                    Notification_ID = not.Notification_ID,
                    StoredProcedure = not.StoredProcedure,
                    SubjectLine = not.SubjectLine,
                    BodyPreface = not.BodyPreface,
                    Frequency = not.Frequency,
                    TakesDateRange = not.TakesDateRange,
                    DateRangeIntervalHours = not.DateRangeIntervalHours,
                    TakesAssociateId = not.TakesAssociateId,
                    NotificationType = not.NotificationType,
                    Associates = not.SubscriptionEvent.SubscriptionSubscribers
                        .Select(ss => new AssociateDto
                        {
                            Record_Number = ss.Associate.Record_Number,
                            Name = ss.Associate.Name,
                            Email = ss.Associate.EMail
                        })

                });



            sw.Stop();
            Console.WriteLine($"Enumeration:{sw.ElapsedMilliseconds}"); //about .5 seconds
            sw.Reset();
            sw.Start();

            //Each item appears to be generated One by One 
            int i = 0;
            foreach (var n in notifications)
            {
                sw.Stop();
                Console.WriteLine($"Generation_{i}:{sw.ElapsedMilliseconds}"); //about 1 second
                var ret = n.GetEmails();
                db.EmailQueues.AddRange(ret);
                sw.Start();
                i++;
            }
            db.SaveChanges();
        }

and the output:

Enumeration:404
Generation_0:985
Generation_1:986
Generation_2:986

So it appears that what I thought I needed to do, I was in fact already doing.

I took care to make the data coming back from the database as non uniform as possible, as to avoid obtaining erroneous data.

Upvotes: 1

Dave
Dave

Reputation: 3017

Foreach is a c# language construct that gets turned into something very similar to your second code section. IEnumerable was designed to do exactly what want, it will give 1 item at a time only when you need it

Upvotes: 2

kuskmen
kuskmen

Reputation: 3775

Simply yield them.

while(enumerable.HasMoreThings()) 
{
    yield return enumerable.GenerateNextThing();
}

Each time someone want next item (caller of this function perhaps) this function will create and yield items lazily.

Upvotes: 3

Related Questions