CODEWITHSUNDEEP
pythonpython-3.xpython-3.4networkx
geofurb
geofurb

Reputation: 489

Python NetworkX gives MemoryError with plenty of unused RAM?

I've got a file full of "Events", each of which contains a few edges. The file's about 32 GB, but only contains a few edges per Event. Around the 2GB mark in memory, Python34 gives me a MemoryError when adding an edge in NetworkX. At this point the graph has ~3.5M edges and ~2M nodes.

for line in fopen :
    line = line.strip()
    if line :
        myevent = json.loads(line)
        sink, sources = EvParser.getEdges(myevent)
        for source in sources :
                    graph.add_edge(source, sink)

My system's 64-bit Win7, 16 GB of RAM, developing in LiClipse. What the heck's going on here?

Upvotes: 0

Views: 236

Answers (1)

geofurb
geofurb

Reputation: 489

Resolved. Was using a 32-bit Python installation. For some reason this caused it to give out far short of 4 GB.

Upvotes: 1

Related Questions