StackOverflow Questions for Tag: apache-spark-dataset

Sanjit Jha

Reputation: 1

Spark map() operation fails with NotSerializableException

apache-sparkserializationapache-spark-mllibapache-spark-datasetapache-spark-ml

Score: 0

Answers: 0

tsar2512

Reputation: 2994

Encoder for Row Type Spark Datasets

javaapache-sparkapache-spark-sqlapache-spark-datasetapache-spark-encoders

Score: 38

Answers: 3

Georg Heiler

Reputation: 17724

Why do columns change to nullable in Apache Spark SQL?

apache-sparkapache-spark-sqlapache-spark-dataset

Score: 11

Answers: 2

Trango

Reputation: 21

Error when import VectorAssembler in Jupyter lab - for Pyspark

apache-sparkpysparkjupyter-labapache-spark-datasetapache-spark-ml

Score: 0

Answers: 2

Amir Bashir

Reputation: 1

Spark Using Wrong Catalog in Hive Metastore: How to Use a Specific Catalog Instead of Default 'hive'?

apache-sparkapache-spark-sqlapache-spark-dataset

Score: 0

Answers: 0

Trango

Reputation: 21

PySpark error when "from pyspark.ml.feature import VectorAssembler"

apache-sparkpysparkjupyter-labapache-spark-datasetapache-spark-ml

Score: 0

Answers: 0

Lara

Reputation: 333

Why is Spark explode function much slower than a flat map function to split array?

apache-sparkapache-spark-sqlapache-spark-dataset

Score: 8

Answers: 0

Asif Hussain Shahid

Reputation: 1

Question regarding the behaviour of UserDefinedType

apache-spark-sqlapache-spark-dataset

Score: 0

Answers: 0

Mork

Reputation: 23

Spark read from MongoDB and filter by objectId indexed field

mongodbapache-sparkapache-spark-dataset

Score: 0

Answers: 2

kk jj

Reputation: 17

ClassCastException in Spark 3.4.1 during Dataset reduce operation

scalaapache-sparkapache-spark-datasetapache-spark-encoders

Score: 0

Answers: 0

marios

Reputation: 8996

Differences between Spark's Row and InternalRow types

apache-sparkapache-spark-sqlapache-spark-dataset

Score: 12

Answers: 2

Joy Cheng

Reputation: 1

RowEncoder.apply(schema).resolveAndBind() and Row/InternalRow serializer/deserializer equivalent in Spark 3.5

scalaapache-sparkapache-spark-sqlapache-spark-datasetapache-spark-encoders

Score: 0

Answers: 1

Antonio Ye

Reputation: 51

Spark Dataframe na.fill for nested columns

scalaapache-sparkapache-spark-dataset

Score: 0

Answers: 2

Antonio Ye

Reputation: 51

Question about Dataset/Dataframe mapPartitions iterator

scalaapache-sparkapache-spark-dataset

Score: 0

Answers: 1

Yudovin Artsiom

Reputation: 109

Spark DataFrame: find and set the main root for child

apache-sparkapache-spark-sqlapache-spark-datasetapache-spark-2.0

Score: 2

Answers: 1

MSS

Reputation: 3633

Joining multiple Spark datasets in Java makes it very slow

apache-sparkjoinlazy-evaluationapache-spark-dataset

Score: 0

Answers: 0

Capacytron

Reputation: 3739

deltalake scala api for unit-testing

scalaapache-sparkdelta-lakeapache-spark-dataset

Score: 0

Answers: 1

optimal substructure

Reputation: 125

Dataframes and Datasets in Spark

apache-sparkapache-spark-sqlapache-spark-dataset

Score: 1

Answers: 1

Capacytron

Reputation: 3739

Splitting Spark dataset / rdd into X smaller datasets, like randomSplit but w/o random

apache-sparkrddapache-spark-dataset

Score: 0

Answers: 0

David Regan

Reputation: 319

Spark UDF doesn't get decoded dataset class using org.typelevel.frameless encoder injection

scalaapache-sparkinheritanceapache-spark-datasetframeless

Score: 1

Answers: 1

PreviousPage 1Next

StackOverflow Questions for Tag: apache-spark-dataset

Spark map() operation fails with NotSerializableException

Encoder for Row Type Spark Datasets

Why do columns change to nullable in Apache Spark SQL?

Error when import VectorAssembler in Jupyter lab - for Pyspark

Spark Using Wrong Catalog in Hive Metastore: How to Use a Specific Catalog Instead of Default &#39;hive&#39;?

PySpark error when &quot;from pyspark.ml.feature import VectorAssembler&quot;

Why is Spark explode function much slower than a flat map function to split array?

Question regarding the behaviour of UserDefinedType

Spark read from MongoDB and filter by objectId indexed field

ClassCastException in Spark 3.4.1 during Dataset reduce operation

Differences between Spark&#39;s Row and InternalRow types

RowEncoder.apply(schema).resolveAndBind() and Row/InternalRow serializer/deserializer equivalent in Spark 3.5

Spark Dataframe na.fill for nested columns

Question about Dataset/Dataframe mapPartitions iterator

Spark DataFrame: find and set the main root for child

Joining multiple Spark datasets in Java makes it very slow

deltalake scala api for unit-testing

Dataframes and Datasets in Spark

Splitting Spark dataset / rdd into X smaller datasets, like randomSplit but w/o random

Spark UDF doesn&#39;t get decoded dataset class using org.typelevel.frameless encoder injection

Spark Using Wrong Catalog in Hive Metastore: How to Use a Specific Catalog Instead of Default 'hive'?

PySpark error when "from pyspark.ml.feature import VectorAssembler"

Differences between Spark's Row and InternalRow types

Spark UDF doesn't get decoded dataset class using org.typelevel.frameless encoder injection