CODEWITHSUNDEEP
pythonreferencenetworkxram
d3scmps
d3scmps

Reputation: 1

networkx merging graph ram issue

I'm quite a newbie with networkx and it seems that i'm having RAM issues when running a function that merges two different graphs. This function adds up the weight of edges that are common to both graphs. I have a list of 8~9 graphs each containing about 1000-2000 nodes which I merge in this loop:

FinalGraph = nx.Graph()
while len(graphs_list)!=0:
    FinalGraph  = merge_graphs(FinalGraph,graphs_list.pop())

using this function

def merge_graphs(graph1, graph2):
    edges1 = graph1.edges
    edges2 = graph2.edges 
    diff1 = edges1 - edges2
    if diff1==edges1:
        return nx.compose(graph1,graph2)
    else:
        common_edges = list(edges1 - diff1)
        for edges in common_edges:
            graph1[edges[0]][edges[1]]['weight'] += graph2[edges[0]][edges[1]]['weight']
        return nx.compose(graph2, graph1)

When running my script, my computer will always freeze when reaching this loop. Am i doing some kind of bad reference cycle or something ? Am i missing something in the networkx doc more effective that could help me not use this function for my purpose ? Thanks for reading me, I hope i'm making sense

Upvotes: 0

Views: 107

Answers (1)

Joel
Joel

Reputation: 23887

There seems to be a lot of extra work going on here caused by you trying to check if the conditions allow you to use compose. This may be contributing to the trouble. I think it might work better to just iterate through the edges and nodes of the graph. The following looks like a more direct way to do it (and doesn't require creating as many variables, which might be contributing to the memory issues)

final_graph = nx.Graph()
for graph in graphs_list:
    final_graph.add_nodes_from(graph.nodes())
    for u, v, w in graph.edges(data=True):
        if final_graph.has_edge(u,v):
            final_graph[u][v]['weight'] += w
        else:
            final_graph.add_edge(u,v,weight = w)

Upvotes: 0

Related Questions