huy
huy

Reputation: 1914

How to add column description with transform_df in Foundry?

Normally, I can add a column description with transform like this:

from transforms.api import Input, Output, transform
from utils import COLUMN_DESCRIPTIONS


@transform(
        output=Output("/Shared/output"),
        raw=Input("/Shared/raw_input")
)
def clean_table(raw, output):
    raw = raw.dataframe()
    output.write_dataframe(
        raw,
        column_descriptions=COLUMN_DESCRIPTIONS
    )

Can I do a similar thing with transform_df?

Upvotes: 2

Views: 497

Answers (1)

domdomegg
domdomegg

Reputation: 1816

Unfortunately it's not currently possible to output column descriptions with transform_df, you have to use the transform decorator for this. This is because transform_df returns a native Spark DataFrame, which doesn't support column descriptions.

More information about column descriptions is available in the Palantir docs on column metadata.

Upvotes: 1

Related Questions