CODEWITHSUNDEEP
pythonpysparkpalantir-foundryfoundry-code-workbooks
Turpan
Turpan

Reputation: 537

UserWarning: DataFrame constructor is internal. Do not directly use it. in Foundry

I'm running a Code Workbook in palantir foundry, where one of the steps is below code that takes data from a dataset:

import pyspark.sql.functions as F
def df_kyc (JOHNNY):
JOHNNY = JOHNNY.select(*[
F.col(c).cast("string").alias(c) if t == "timestamp" else F.col(c)
for c, t in JOHNNY.dtypes])
JOHNNY = JOHNNY.toPandas()
return JOHNNY

however, after running the preview, I receive following log error:

" /opt/spark/work-dir/environment/lib/python3.8/site-packages/pyspark/sql/dataframe.py:148: UserWarning: DataFrame constructor is internal. Do not directly use it. warnings.warn("DataFrame constructor is internal. Do not directly use it.") "

Do you know what's wrong with that please? Thanks

Upvotes: 0

Views: 103

Answers (0)

Related Questions