I’m having a pretty simple issue. I have a dataset (small sample shown below)

22 85 203 174 9 0 362 40 0
21 87 186 165 5 0 379 32 0
30 107 405 306 25 0 756 99 0
6 5 19 6 2 0 160 9 0
21 47 168 148 7 0 352 29 0
28 38 161 114 10 3 375 40 0
27 218 1522 1328 114 0 1026 310 0
21 78 156 135 5 0 300 27 0

The first issue I needed to cover was replacing each space with a comma I did that with the following code

import fileinput

with open('Data_Sorted.txt', 'w') as f:
    for line in fileinput.input('DATA.dat'):
        line = line.split(None,8)
        f.write(','.join(line))

The result was the following

22,85,203,174,9,0,362,40,0
21,87,186,165,5,0,379,32,0
30,107,405,306,25,0,756,99,0
6,5,19,6,2,0,160,9,0
21,47,168,148,7,0,352,29,0
28,38,161,114,10,3,375,40,0
27,218,1522,1328,114,0,1026,310,0
21,78,156,135,5,0,300,27,0

My next step is to grab the values from the last column, check if they are less than 2 and replace it with the string ‘nfp’.

I’m able to seperate the last column with the following

for line in open("Data_Sorted.txt"):
    columns = line.split(',')

    print columns[8]

My issue is implementing the conditional to replace the value with the string and then I’m not sure how to put the modified column back into the original dataset.

Solution:

There’s no need to do this in two loops through the file. Also, you can use -1 to index the last element in the line.

import fileinput

with open('Data_Sorted.txt', 'w') as f:
    for line in fileinput.input('DATA.dat'):
        # strip newline character and split on whitespace
        line = line.strip().split()

        # check condition for last element (assuming you're using ints)
        if int(line[-1]) < 2:
            line[-1] = 'nfp'

        # write out the line, but you have to add the newline back in
        f.write(','.join(line) + "\n")

Further Reading:

• Negative list index?
• Understanding Python's slice notation

I create a list as follows:

['v0' if x%4==0 else 'v1' if x%4==1 else 'v2' if x%4==2 else 'v3' for x in list_1]

How to generalize the creation of such a list, so that it can be easily expanded by a larger number of variables and subsequent conditions?

Solution:

String formatting

Why not use a modulo operation here, and do string formatting, like:

['v{}'.format(x%4) for x in list_1]

Here we thus calculate x%4, and append this to 'v‘ in the string. The nice thing is that we can easily change 4, to another number.

Tuple or list indexing

In case the output string do not follow such structure, we can construct a list or tuple to hold the values. Like:

# in case the values do not follow a certain structure
vals = ('v0', 'v1', 'v2', 'v3')
[vals[x%4] for x in list_1]

By indexing it that way, it is clear what values will map on what index. This works good, given the result of the operation – here x%4 – maps to a ℕ_n (with a reasonable small n).

(Default)dictionary

In case the operation does not map on a ℕ_n, but still on a finite number of hashable items, we can use a dictionary. For instance:

d = {0: 'v0', 1: 'v1', 2: 'v2', 3: 'v3'}

or in case we want a “fallback” value, a value that is used given the lookup fails:

from collections import defaultdict

d = defaultdict(lambda: 1234, {0: 'v0', 1: 'v1', 2: 'v2', 3: 'v3'})

where here 1234 is used as a fallback value, and then we can use:

[d[x%4] for x in list_1]

Using d[x%4] over d.get(x%4) given d is a dictionary can be more useful if we want to prevent that lookups that fail pass unnoticed. It will in that case error. Although errors are typically not a good sign, it can be better to raise an error in case the lookup fails than add a default value, since it can be a symptom that something is not working correctly.