{"id":126,"date":"2018-04-23T05:29:10","date_gmt":"2017-04-23T06:27:57","guid":{"rendered":""},"modified":"2022-03-22T09:01:48","modified_gmt":"2022-03-22T07:01:48","slug":"spss-batch-process-files-with-python","status":"publish","type":"post","link":"https:\/\/www.spss-tutorials.com\/spss-batch-process-files-with-python\/","title":{"rendered":"SPSS – Batch Process Files with Python"},"content":{"rendered":"\n\n

Running syntax over several SPSS<\/a> data files in one go is fairly easy. If we use SPSS with Python<\/a> we don't even have to type in the file names. The Python os (for o<\/strong>perating s<\/strong>ystem) module will do it for us.
\nTry it for yourself by downloading spssfiles.zip<\/a>. Unzip these files into d:\\spssfiles<\/code> as shown below and you're good to go.<\/p>\n\n \n \"SPSS \n<\/span> \n \n \n \n

Find All Files and Folders in Root Directory<\/h2>\n\n

The syntax below creates a Python list<\/a> of files and folders in rDir<\/code>, our root directory. Prefixing it with an r as in r'D:\\spssfiles'<\/code> ensures that the backslash<\/a> doesn't do anything weird.<\/p>\n\n

*Find all files and folders in root directory.
<\/strong>
begin program.
import os
rDir = r'D:\\spssfiles'
print os.listdir(rDir)
end program.<\/div>\n\n

Result<\/h2>\n\n \n \"Python \n<\/span> \n \n\n \n

Filter Out All .Sav Files<\/h2>\n \n

As we see, os.listdir()<\/code> creates a list of all files and folders in rDir<\/code> but we only want SPSS data files. For filtering them out, we first create and empty list with savs = []<\/code>. Next, we'll add each file to this list if it endswith(\".sav\")<\/code>.<\/p>\n \n

*Add all .sav (SPSS data) files to Python list.
<\/strong>
begin program.
import os
rDir = r'D:\\spssfiles'
savs = []
for fil in os.listdir(rDir):
    if fil.endswith(".sav"):
        savs.append(fil)
print savs
end program.<\/div> \n \n

Using Full Paths for SPSS Files<\/h2>\n\n

For doing anything whatsoever with our data files, we probably want to open them. For doing so, SPSS needs to know in which folder they are located. We could simply set a default directory in SPSS with CD as in\nCD \"d:\\spssfiles\".<\/span>\nHowever, having Python create full paths to our files with os.path.join()<\/code> is a more fool proof approach for this.\n<\/p>\n\n

*Create full paths to all .sav files.
<\/strong>
begin program.
import os
rDir = r'D:\\spssfiles'
savs = []
for fil in os.listdir(rDir):
    if fil.endswith(".sav"):
        savs.append(os.path.join(rDir,fil))
for sav in savs:
    print sav
end program.<\/div>\n

Result<\/h2>\n \n \n \"SPSS \n<\/span> \n \n\n

Have SPSS Open Each Data File<\/h2>\n\n

Generally, we open a data file in SPSS with something like\nGET FILE \"d:\\spssfiles\\mydata.sav\".<\/span>\nIf we replace the file name with each of the paths in our Python list, we'll open each data file, one by one. We could then add some syntax we'd like to run on each file. Finally, we could save our edits with\nSAVE OUTFILE \"...\".<\/span>\nand that'll batch process multiple files. In this example, however, we'll simply look up which variables each file contains with spssaux.GetVariableNamesList()<\/code>.<\/p>\n\n

*Open all SPSS data files and print the variables they contain.
<\/strong>
begin program.
import os,spss,spssaux
rDir = r'D:\\spssfiles'
savs = []
for fil in os.listdir(rDir):
    if fil.endswith(".sav"):
        savs.append(os.path.join(rDir,fil))
for sav in savs:
    spss.Submit("GET FILE '%s'."%sav)
    print sav,spssaux.GetVariableNamesList()
end program.<\/div> \n\n

Result<\/h2>\n \n \n \"SPSS \n<\/span> \n \n \n \n

Inspect which Files Contain “Salary”<\/h2>\n\n

Now suppose we'd like to know which of our files contain some variable “salary”. We'll simply check if it's present in our variable names list and -if so- print back the name of the data file.<\/p>\n \n

*Report all .sav files that contain a variable "salary" (case sensitive).
<\/strong>
begin program.
import os,spss,spssaux
rDir = r'D:\\spssfiles'
findVar = 'salary'
savs = []
for fil in os.listdir(rDir):
    if fil.endswith(".sav"):
        savs.append(os.path.join(rDir,fil))
for sav in savs:
    spss.Submit("get file '%s'."%sav)
    if findVar in spssaux.GetVariableNamesList():
        print sav
end program.<\/div>\n\n\n

Result<\/h2>\n \n \n \"SPSS \n<\/span> \n \n \n

Circumvent Python’s Case Sensitivity<\/h2>\n\n

There's one more point I'd like to cover: since we search for “salary”, Python won't detect “Salary” or “SALARY” because it's fully case sensitive. I you don't like that, the simple solution is to convert all variable names for all files to lower()<\/code>case.
\nA basic way to change all items in a Python list<\/strong> is\n[i... for i in list]<\/span>\nwhere i...<\/code> is a modified version of i<\/code>, in our case i.lower()<\/code>. This technique is known as a Python list comprehension and the syntax below uses it to lowercase all variable names (line 13).<\/p>\n\n\n\n

*Report all .sav files that contain a variable "salary" (case insensitive).
<\/strong>
begin program.
import os,spss,spssaux
rDir = r'D:\\spssfiles'
findVar = 'salary'
savs = []
for fil in os.listdir(rDir):
    if fil.endswith(".sav"):
        savs.append(os.path.join(rDir,fil))
for sav in savs:
    spss.Submit("get file '%s'."%sav)
    if findVar.lower() in [varNam.lower() for varNam in spssaux.GetVariableNamesList()]:
        print sav
end program.<\/div>\n

Note: since I usually avoid all uppercasing in SPSS variable names, the result is identical to our case sensitive search.

\nThanks for reading.<\/p>\n\n\n\n","protected":false},"excerpt":{"rendered":"

Running syntax on several SPSS data files in one go is fairly easy with Python. This tutorial walks you through.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[290],"tags":[],"class_list":["post-126","post","type-post","status-publish","format-standard","hentry","category-spss-python-ii-examples"],"_links":{"self":[{"href":"https:\/\/www.spss-tutorials.com\/wp-json\/wp\/v2\/posts\/126","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.spss-tutorials.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.spss-tutorials.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.spss-tutorials.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.spss-tutorials.com\/wp-json\/wp\/v2\/comments?post=126"}],"version-history":[{"count":0,"href":"https:\/\/www.spss-tutorials.com\/wp-json\/wp\/v2\/posts\/126\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.spss-tutorials.com\/wp-json\/wp\/v2\/media?parent=126"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.spss-tutorials.com\/wp-json\/wp\/v2\/categories?post=126"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.spss-tutorials.com\/wp-json\/wp\/v2\/tags?post=126"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}