CODEWITHSUNDEEP
scalaapache-sparkapache-kafkaavro
B. Bal
B. Bal

Reputation: 121

to_avro function for schema registry not found in spark-avro

I'm trying to write to a kafka sink using this documentation: https://docs.databricks.com/spark/latest/structured-streaming/avro-dataframe.html#example-with-schema-registry

However, I can't find to_avro function with more than one parameter in spark-avro(https://mvnrepository.com/artifact/org.apache.spark/spark-avro_2.11). Does anybody know if it has been moved/renamed or I should use another library? Thank you!

Upvotes: 0

Views: 659

Answers (1)

Som
Som

Reputation: 6323

In spark 3.0, these functions has been moved to org.apache.spark.sql.avro.functions.scala object

check this - spark-repo jira- SPARK-26870

Also the databricks doc you are referring is based on spark 3

Upvotes: 0

Related Questions