CODEWITHSUNDEEP
azureazure-data-lakeu-sql
Tiam
Tiam

Reputation: 73

Does U-Sql support cursors for iterating through data sets and extracting more data based on row values?

Does Azure Data Lake Analytics and U-SQL support the notion of Cursors in scripts?

I have a data set that contains paths to further data sets I would like to extract and I want to output the results to separate files.

At the moment I can't seem to find a solution for dynamically extracting and outputting data based on values inside data sets.

Upvotes: 1

Views: 275

Answers (2)

Michael Rys
Michael Rys

Reputation: 6684

U-SQL currently expect that files are known at compile time. Thus, you cannot do extraction or outputting based on locations provided inside a file.

You can specify filesets in the EXTRACT statement that will be somewhat data driven. We are currently working on adding the ability to use filesets on OUTPUT as well.

You can file feature requests at http://aka.ms/adlfeedback.

Cheers Michael

Upvotes: 2

minghan
minghan

Reputation: 1113

You might be able to write a Processor to iterate over the rows in the primary dataset. However, you might not be able to access the additional datasets in the Processor.

Another work around might be to concatenate all the additional datasets and perform a join with the primary dataset.

Upvotes: 0

Related Questions