Identifying and removing extra tabs in a tab delimited text file

Question

I have a text files with 120 columns and thousands of rows where the delimiter is a tab. In some rows, there is an extra tab present making it seem, in that row, like there are 121 columns. The location of this extra tab is not known to be the same for all the text files.

I am wondering if anyone has any thoughts on efficiently locating the extra tab and removing it programmatically.

Answer 1

You can use a regex as separator in read_csv.

Use '\t+' (one or more tabulations):

df = pd.read_csv('your_file.csv', sep='\t+', engine='python')