CODEWITHSUNDEEP
performanceperl
slitomonous
slitomonous

Reputation: 83

Creating an array of hashes is showing slow performance, how can this be improved?

Running my script through Devel::NYTProf showed that the following portion of code took up the vast majority of running time. The function creates a hash that's easier to work with, and that hash gets pushed into an array. I'm wondering how this can be done without taking as long as it does.

From Devel::NYTProf:

# Statements | Time on line | Calls | Time in sub
21092   16.4s   21092   273s     push( @events, { create_events_hash($1, $2, $j, @eventHash) } );
# spent 273s making 21092 calls to Parser::create_events_hash, avg 12.9ms/call

# ...                   

# spent 273s (268+4.95) within Parser::create_events_hash which was called 21092 times, 
    avg 12.9ms/call: # 21092 times (268s+4.95s) by 
    Parser::findNewMessages at line 86, avg 12.9ms/call  
# Statements | Time on line 

sub create_events_hash {
    21092   159s            my ( $dateIndex, $msgIDIndex, $eventHashIndex, @eventHash ) = @_;                       

    21092   81.8ms          my %holder;
    21092   137ms           $holder{ID} = $eventHashIndex;
    21092   190ms           $holder{msgDate} = $dateIndex;
    21092   243ms           $holder{ReceivedAt} = $eventHash[$eventHashIndex]{ReceivedAt};
    21092   181ms           $holder{msgID} = $msgIDIndex;
    21092   193ms           $holder{FromHost} = $eventHash[$eventHashIndex]{FromHost};
    21092   187ms           $holder{Priority} = $eventHash[$eventHashIndex]{Priority};

    21092   97.6s           return %holder;
                    }

Upvotes: 1

Views: 324

Answers (2)

Hameed
Hameed

Reputation: 2277

For starters, I'd send @eventhash as reference.

push(@events, {create_events_hash($1, $2, $j, \@eventHash)});

It is more effecient this way as it won't make a copy of the array.

Upvotes: 6

Len Jaffe
Len Jaffe

Reputation: 3484

It might be that you can't really squeeze very much time out of a bunch of assignment statements.

You may have to examine your algorithm, why are you calling that sub 21k times?

You already have an array of hashes. Maybe you're iterating in a really inefficient way.

Plus you're passing a hash out to the function, instead of a hash ref. So try passing a reference in, as @Hameed suggested, and try passing a reference out as well, and see what that does.

Passing the reference out will probably have the least impact on your code, and then you can see what kind of impact passing refs vs objects has on the runtime. But I will agree with Hameed - the large array you're passing in will take more time to copy onto the stack then the small hash that you are returning.

Upvotes: 1

Related Questions