CODEWITHSUNDEEP
sas
Tony
Tony

Reputation: 2929

How to import large dataset into SAS?

I have a rather large dataset in text file with about 25 million rows and 200 columns (all of them are numeric). I would like to run some summary statistics and data analysis (survival analysis) on them.

  1. What is the fastest way to import the data into SAS?

  2. How much memory do I need for my PC in order to run such a large dataset?

Upvotes: 3

Views: 4088

Answers (1)

Matt Parker
Matt Parker

Reputation: 27339

  1. I'm not sure that anything is going to be much faster than just reading your dataset in using PROC IMPORT. Specifying your informats and formats in advance might help speed things up a bit, but PROC IMPORT infers these from only the first 20 records by default, so it's not like it's going to read your whole dataset to figure out what data types to use. The fact that your columns are all numeric will probably help. The most important thing is to be sure to save the results to a permanent dataset (i.e., specify a library for it) - if you only have to import the data once, it doesn't really matter if it takes a long time.

  2. One of the nice things about SAS is that it keeps data on disk rather than in memory by default, so the size of your RAM doesn't really limit the size of your dataset. It might limit what you can do with that dataset, but I don't know enough about SAS's internal operations to be able to predict what you'll have trouble with.

Upvotes: 4

Related Questions