Tronald Dump
Reputation: 1350
Reading Multiple SequnceFiles in PySpark?
Is there a way to read Multiple Sequence Files in a single go?
sc.SequnceFile(['filepath_1.seq','filepath_2.seq','filepath_3.seq'])
Upvotes: 0
Views: 70
Answers (1)
Alper t. Turker
Reputation: 35249
Use glob pattern
sc.SequnceFile('filepath_*.seq')
or comma-separated strings:
sc.SequnceFile(",".join(['filepath_1.seq', 'filepath_2.seq', 'filepath_3.seq']))
Upvotes: 1
Related Questions
- pyspark read multiple csv files at once
- PySpark Reading Multiple Files in Parallel
- Reading in Spark data frame from multiple files
- read multiple parquet file at once in pyspark
- How to Read multiple files in different Pyspark Dataframes using spark.read.jdbc
- reading multiple csv files using pyspark
- Spark multiple CSV reads?
- Spark: reading many files with read.csv
- reading a files which has several lines in pyspark
- Reading Multiple Files in Spark and Processing it before Appending