Reindexing only valid with uniquely valued index objects

Question

There are a couple of articles about this issue already, but none of them solve my issue.

I have two sets of Python dataframes (df_A1, df_A2 and df_B1, df_B2) and I want to combine the A's together and the B's together. I can concat the A's just fine, but the B's give me the error 'Reindexing only valid with uniquely valued index objects'. There are a couple of causes that I found online but that won't work for me.

• Duplicate column names. The B dataframes have indeed columns with the same name, but the A dataframes have this as well and it works just fine with that. So this shouldn't be an issue in the same code. Also, I used df.index.is_unique which returns True for all dataframes, but of course df.columns.is_unique returns False.

• Empty columnnames. I had that as well and made sure all empty columnnames are now filled using [item or '_empty_' for item in <list_columns>]

• Using ignore_index=True. Did that for the A's and B's, the A's are still working with concat and the B's still don't.

I have the traceback here:

Traceback (most recent call last):

  File ~\AppData\Local\anaconda3\Lib\site-packages\spyder_kernels\customize\utils.py:209 in exec_encapsulate_locals
    exec_fun(compile(code_ast, filename, "exec"), globals)

  File c:\<path>\python_script.py:168
    df_B = pd.concat([df_B1 , df_B2], axis=0, ignore_index=True)

  File ~\AppData\Local\anaconda3\Lib\site-packages\pandas\core\reshape\concat.py:395 in concat
    return op.get_result()

  File ~\AppData\Local\anaconda3\Lib\site-packages\pandas\core\reshape\concat.py:680 in get_result
    indexers[ax] = obj_labels.get_indexer(new_labels)

  File ~\AppData\Local\anaconda3\Lib\site-packages\pandas\core\indexes\base.py:3885 in get_indexer
    raise InvalidIndexError(self._requires_unique_msg)

InvalidIndexError: Reindexing only valid with uniquely valued Index objects

Does anyone has an idea on how to fix this?