CODEWITHSUNDEEP
ssis
Brian Bates
Brian Bates

Reputation: 186

In SSIS how to segment a Data Source into multiple RAW files

I have a stored procedure that returns around 10 million rows for a month ordered by date. I would like to create a RAW file that includes the records for each day.

Is there a way to stream the records into a Raw file until the date changes and then close that RAW file and create a new RAW file for the next day and begin inserting records into the new RAW file?

Upvotes: 0

Views: 41

Answers (1)

billinkc
billinkc

Reputation: 61211

Assuming your source query looks like SELECT * FROM MySource WHERE ActionDate > '2019-11-01' AND ActionDate < '2019-12-01'; I'd likely structure my data flow as OLE DB Source -> Conditional Split (on month's day) -> 31 Raw File Destinations. Every day, we'd rewrite the raw files with either blanks or actual data.

If I misread the question and you'd like to just fill a file for the current day's data, I'd filter my source data to only pull today's data. You can have the package create the raw file, just specify that it's "Create always"

You can then rotate files in based on day number in case you need to keep previous raw file data.

Upvotes: 1

Related Questions