Multi Threaded Processing in SAS
Two heads think better than one. This is often true in the real world. However, in SAS it is more true to say that several heads think faster than one. This post shows you how to split processes to several threads to decrease wall clock time and increse performance of your processes in SAS. Also, we will look at a simple example of how using multiple threads can vastly increase PROC SORT performance.
Multi threaded processing has been enabled in SAS since version 9.0. With multi threaded processing, the SAS Procedures that are threads enabled can take advantage of multiple CPUs by splitting processes and I/O operations on to several threads. This generally reduces wall clock time at the possible cost of additional CPU resources.
An Multithreaded PROC SORT Example
Not all procedures or processes are thread enabled. So far, the Base SAS Procedures enabled for threaded processing are the SORT, MEANS, REPORT, TABULATE, SQL and SUMMARY Procedures. In SAS/STAT the GLM, LOESS, REG and ROBUSTREG Procedures are supported. Furthermore in SAS 9.4 even more statistical procedures were thread enabled.
Here, we will look at an example of using threaded processing with PROC SORT. When we perform a threaded sort, we split up the process. We pass data to several processors. They work on sorting the chunk of data passed top them. Finally the sorted data is merged back together. See the figure to the left for a two threaded sort. First, let us create a large data set with 100 Mio observations of the variable x, which is a random number between 1 and 10.
data RandData; do i=1 to 10e7; x=ceil(10*rand('uniform')); output; end; run;
Now, let us do a simple experiment and perform four distinct PROC SORT runs with 1, 2, 4 and 8 CPUs available for threaded sorting. We use the THREADS Global Option along with the CPUCOUNT= System Option to specify how many processors PROC SORT has available for concurrent processing.
options threads cpucount=1; proc sort data=RandData; by x; run;
First, I set the number of CPUs to 1. This procedure run takes about 53.3 seconds in real time. I perform the same process for 2, 4 and 8 available processors. These took 32.2, 25.8 and 24.5 seconds respectively . That means less than half the wall clock time compared to the single threaded start run. And all it took was changing a simple system option.
If you are not sure how many CPUs you have available for multithreaded processing by default, use PROC OPTIONS like this
proc options group=performance; run;
Now, check the log and find the CPUCOUNT Option to see the available number of CPUs.
This post demonstrated how to utilize multiple threads to increase performance of procedure runs in SAS. Multi threaded processing can greatly reduce wall clock time. Naturally this costs CPU recourses (There are no free lunches). However, usually the time saved is worth it. I like to control the available number of CPUs in my System Option part of my Autoexec File.
You can download the entire code from this post here.