CODEWITHSUNDEEP
python
NovaPoi
NovaPoi

Reputation: 73

Python ignore and skip error line, and continue with the next line

What I have tried:

import simplejson
import pandas as pd
with open('/tmp/test.json') as f:
    try:
        data = pd.DataFrame(simplejson.loads(line) for line in f)
    except Exception as e:
        pass
data.to_csv('/tmp/data.csv')

Then I got the error message:

NameError: name 'data' is not defined.

What should I do to fix this issue?

Upvotes: 1

Views: 3631

Answers (3)

user14665310
user14665310

Reputation: 545

Put data.to_csv("/tmp/data.csv") inside the try statement, or add code in the except statement to create the data object some other way if an exception is thrown on the data line (and it does not create the data object).

Upvotes: 1

PaxPrz
PaxPrz

Reputation: 1928

For a better explaination, here how your program is running internally,

Inside try block:

  1. First the pd.Dataframe gets instantiated

    some_memory_loc STORES pd.DataFrame(simplejson.loads(line) for line in f)

  2. Then the data variable saves the memory address and type being DataFrame:

    data = some_memory_loc AND TYPE(data) = pd.DataFrame

  3. Then Program moves out of try catch block

Now, What actually happening in your case is, during the first step, program generates an exception. Thus runs directly to except block. So the 2nd step was never executed. Thus, your python program doesn't know what data is.

I hope you have understood and thus can solve on your own.

Upvotes: 2

NotAName
NotAName

Reputation: 4347

This means that you are successfully catching an error and going to the condition in except statement and as a result your data is never loaded and the variable data is never defined. And then when you try to save data to csv, this triggers an error.

What you should do instead is to put your to_csv statement inside try-except as well:

import simplejson
import pandas as pd
with open('/tmp/test.json') as f:
    try:
        data = pd.DataFrame(simplejson.loads(line) for line in f)
        data.to_csv('/tmp/data.csv')
    except Exception as e:
        pass

Also, it is a horrible anti-pattern in Python to do except Exception as e: pass. This will result in the code that is nigh impossible to debug because you are catching every single error that might happen and not specifically the error you expect to happen. When you do this you should either log the full error stack or print it to console to be aware of what exact error you are catching:

    try:
        data = pd.DataFrame(simplejson.loads(line) for line in f)
    except Exception as e:
        err = str(e)
        print(err)

Or you should be catching a specific error that you expect to happen here:

    try:
        data = pd.DataFrame(simplejson.loads(line) for line in f)
    except FileNotFoundError:
        pass

Upvotes: 2

Related Questions