How can I store data permanently in SAS using a Libname?

[WWGD]

Hi All,
Trying to get basics of inputting data into SAS ( University Version, i.e. in remote Virtual Directory in SAS Studio . I hope my layout , nor my general writing are too confusing. Feel free to suggest a different format):
Please critique:
We want to read file 'FileName' into SAS for some processing
1) To input/import data when 'FileName' is _not_ in SAS in non-permanent form , i.e., to be stored in 'Work' directory and will disappear after session is over.:

--------------------------------------------
1.1) DATA [FileName1] ;( Original File name [FileName] modified nto , e.g. FileName1)
1.2) INFILE [Pathname for FileName];
1.3) INPUT [Field1] [Field2]...[Fieldn]; : These are the fields/columns from [FileName] we want to import
1.4) RUN;
Data will be read into temporary file within 'Works' folder in Virtual Drive. File will disappear once session is closed.
----------------------------------------------

2) To store data file [FileName] permanently in SAS in Folder [Folder]. We use a 'LibName which is an alias for the name for the folder where we want to store things together with the pathname for the folder.:

2.1) LIBNAME [Alias for FileName] [ Pointer for or Pathname for [Folder] ];
2.2) SET [FileName, from step 1.1];
2.3) RUN ;

Is this correct?

 

[Ibix]

1 - More or less. The syntax for INPUT is quite complex and let's you specify field widths and things like that. I'm not familiar with the details - just Google if what you try doesn't quite work.

2 - No. Your libname statement is about right, but then you need to create a copy of the dataset by creating a new dataset in the target libname. You do that by qualifying the dataset name with a libname. So having read in a dataset called myfile, copy it to permanent store with

* Make perm an alias for your permanent storage;
libname perm "/path/to/directory";
* Create a dataset called myfile in perm, containing a copy of myfile;
data perm.myfile;
    set myfile;
run;

If the dataset has many columns, it can be worth using SAS' builtin compression. This is completely transparent - SAS reads its compressed files without any extra effort on your part. But it compresses rows separately, so it's only worth doing if your dataset is wide.

* Make perm an alias for your permanent storage;
libname perm "/path/to/directory";
* Create a compressed dataset called myfile in perm, containing a copy of myfile;
data perm.myfile(compress=binary);
    set myfile;
run;

Unless you are doing a lot of writing to the permanent libname I recommend immediately de-assigning it and reassigning it read only.

libname perm clear;
libname perm "/path/to/directory" access=readonly;

This way it's harder to accidentally overwrite something in permanent storage.

 

[WWGD]

Thanks for the detailed comments.i s it at least easy to work with the data once you have it in the system?

 

[Ibix]

Fairly straightforward, yes, although it's fairly obvious that the language just grew rather than being properly designed. Basically a data step reads in a data set line by line, executes the instructions you wrote, outputs the line, and repeats until it runs out of data.

Between data steps and the sort, summary and transpose procs you can usually massage the data into the form you want and then there are loads of analytical procs that do statistical tests and model fitting and stuff like that in a fairly simple way. You can also write SQL if you want with proc sql.

If you need a many-to-many join, use proc sql. Otherwise datasteps are usually faster. Always check the log file carefully (automated tools are you friend!) because SAS has some very odd ideas about what should be an error or warning and what only counts as a note.

 

[FactChecker]

Thanks for the detailed comments.i s it at least easy to work with the data once you have it in the system?

There is a learning curve to it, but it is a very good language for manipulating the data and generating reports.

 

[WWGD]

Does anyone know a good "centralized" reference? I have watched videos and read pages, but it is a bit here, a bit there. A problem too is that it is often too informal. I would like to know, if possible, e.g., when importing from the virtual (SAS Studio) drive, use this syntax.
When creating a file from scratch use... Anyone know any such source ( Including books, of course)?

EDIT: Also, sorry if this is a dumb question, but, does above suggestion on Libname apply to files in the virtual drive, one's PC drive, or does this not matter?

 

[Ibix]

I'm afraid I learned on the job, updating code from a recently-departed SAS wizard with some input from my boss.

EDIT: Also, sorry if this is a dumb question, but, does above suggestion on Libname apply to files in the virtual drive, one's PC drive, or does this not matter?

I use my permanent storage for things that take ages to generate, or the originals of some things. That is, things that are expensive (or impossible without sacrifices to the Gods of the IT Department) to recover if I accidentally PROC SORT NODUPKEY the wrong thing (which I have seen someone do). So I create permanent storage libnames with access=readonly to force a conscious choice to write to those folders. It's risk management. Don't bother if there's nothing you care about in a folder.