CODEWITHSUNDEEP
csvimportsas
Alvin SIU
Alvin SIU

Reputation: 1042

Use SAS to import CSV file with 3070 fields

I am using AIX 5.3 and SAS 9.1.3

I have a 2.7G CSV file.
The file has about 110k lines.
Each line has 3070 fields.

I use filename myCSVfile LRECL=32768 ;
Then, I use Proc import ... delimiter = "," ...

The SAS program seems hang up.
It still there even after running 1 hour.

I head the file to have only 40 lines and try again.
This time, also seems hang up.
Also still there after running 1 hour.
NO improvement with this small file.

The log file is 0 byte.
Also, when I ps -ef , the process has eat 10% CPU.
At last I have to kill it before leaving the office.

Any hint to solve this problem ?

Alvin SIU

Upvotes: 0

Views: 1206

Answers (2)

Robert Penridge
Robert Penridge

Reputation: 8513

Also try adding the truncover option? Check to make sure there aren't any lines > 32767 chars as that will cause issues. Do this check using an external program (ie. not SAS).

Cheers Rob

Upvotes: 0

itzy
itzy

Reputation: 11755

Do you need to use proc import? This might work better:

data tmp;
  infile './your_data_file' dlm=',' lrecl=as_big_as_you_need;
  input var1 var2 ...;
run;

Running it this way will also allow you to inspect what errors crop up in the log file as they happen.

Upvotes: 2

Related Questions