CODEWITHSUNDEEP
datastage
Shrav1
Shrav1

Reputation: 1

lookuptable file is not getting removed automatically in datastage

I came across an issue in datastage where the temp files created during the file load named: lookuptable* is not getting removed automatically. For example the lookup file for previous day resides in the datasets folder which consumes around 20 GB and again a new lookup file is created in the same path for the current day and when the size gets increased to a certain limit the job fails with an error : no space left on device. So we had to remove these large lookup files and then run the job again which completes successful. As per my understanding these lookup files should get removed automatically after the file load is complete. Is there a reason for this file to reside in the path? What shall I do to get these files removed automatically after the file load is successful?? Please suggest a solution for this issue.

Thanks in advance for your help..!!

Upvotes: 0

Views: 379

Answers (2)

Atul Singh
Atul Singh

Reputation: 59

If the reference file is so huge to fit in your memory space why to use lookup, try to use join stage.

Upvotes: 0

disaster
disaster

Reputation: 1

Yes. Lookup files are to be removed or moved to archives once the loading is successful. And they will be done so if your job design is such :-)

place your in a sequncer and then once your job is done then remove teh file ( rm < file name > for ascii file and orchadmin rm < dataset name with extn>

Upvotes: 0

Related Questions