CODEWITHSUNDEEP
pythonnetworkx
user13056355
user13056355

Reputation:

Memory error creating large graphs in NetworkX

I have decided to work with Python for processing graphs. I use NetworkX for loading the edge list to construct the graph. When I try to load edges with nx.read_edgelist I receive MemoryError.

The graph has 4 million nodes and 34 million edges and my PC has 12GB of RAM. Is it possible to load that graph into 12GB? Is there any optimal solution?

Upvotes: 4

Views: 2010

Answers (1)

Christopher Peisert
Christopher Peisert

Reputation: 24154

NetworkX uses dictionaries to implement graph data structures. Each edge uses at least 100 bytes of memory. Hence, with 34 million edges, you would would likely need more than 3.4 GB.

One solution would be to use cloud computing and scale up memory as needed.

An alternative to NetworkX is graph-tool, which is a Python module that implements core data structures and algorithms in C++.

See also

Upvotes: 3

Related Questions