Category: python

numpy array indicator operation

I want to modify an empty bitmap by given indicators (x and y axis).
For every coordinate given by the indicators the value should be raised by one.

So far so good everything seems to work. But if I have some similar indicators in my array of indicators it will only raise the value once.

>>> img
array([[0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0]])

>>> inds
array([[0, 0],
       [3, 4],
       [3, 4]])

Operation:

>>> img[inds[:,1], inds[:,0]] += 1

Result:

>>> img
    array([[1, 0, 0, 0, 0],
           [0, 0, 0, 0, 0],
           [0, 0, 0, 0, 0],
           [0, 0, 0, 0, 0],
           [0, 0, 0, 1, 0]])

Expected result:

>>> img
    array([[1, 0, 0, 0, 0],
           [0, 0, 0, 0, 0],
           [0, 0, 0, 0, 0],
           [0, 0, 0, 0, 0],
           [0, 0, 0, 2, 0]])

Does someone have an idea how to solve this? Preferably a fast approach without the use of loops.

Solution:

This is one way. Counting algorithm courtesy of @AlexRiley.

For performance implications of relative sizes of img and inds, see @PaulPanzer’s answer.

# count occurrences of each row and return array
counts = (inds[:, None] == inds).all(axis=2).sum(axis=1)

# apply indices and counts
img[inds[:,1], inds[:,0]] += counts

print(img)

array([[1, 0, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [0, 0, 0, 2, 0]])

Difference between get and dunder getitem

I am reading Fluent Python and trying to get a deeper understanding of dictionaries.

So when I run the below, the results are easy to understand in that both get() and dunder getitem() return the same result

sample = {'a':1, 'b':2}
print(sample.__getitem__('a')) # 1
print(sample.get('a')) # 1

When I subclass dict with get(), I get a working instance

class MyDict(dict):
    def __missing__(self, key):
        return 0

    def get(self, key):
        return self[key]

d = MyDict(sample)
print(d['a']) # 1
print(d['c']) # 0

Now if I replace get() with dunder getitem() I get an error and I am unsure why.

class MyDict2(dict):
    def __missing__(self, key):
        return 0

    def __getitem__(self, key):
        return self[key]

d = MyDict2(sample)
print(d['a'])
print(d['c'])

error

RecursionError: maximum recursion depth exceeded while calling a Python object

So the question is, what is the difference between get and dunder getitem in this situation and why does this cause a recursion error?

Solution:

That is because self[key] in MyDict2.__getitem__(key) is equivalent to (i.e., calls) self.__getitem__(key) => infinite recursion.

Generate all possible lists by replacing elements with 0

I would like to create from a list all the different list were 0,1,2,3…all element are replaced by an other
For example, if the replacement item is 0:

L=[1,2,3]
->[1,2,3],[0,2,3],[1,0,3],[1,2,0],[0,0,3],[0,2,0],[1,0,0],[0,0,0]

So far, I’ve tried I managed to do what I whant using Itertools but only in the case where 1 value is replaced by 0
Does anyone know how to do this ?

Solution:

This is one way using itertools. The benefit of this method is that it is lazy.

A new list is produced on every __next__ call of the generator transformer.

Alternatively, as below, you can output all combinations by calling list on the generator function.

from itertools import combinations, chain

A = [1, 2, 3]

def transformer(x):
    idx = chain.from_iterable(combinations(range(len(x)), i) for i in range(len(x)+1))
    for indices in idx:
        y = x.copy()
        for j in indices:
            y[j] = 0
        yield y

res = list(transformer(A))

print(res)

[[1, 2, 3], [0, 2, 3], [1, 0, 3], [1, 2, 0], [0, 0, 3], [0, 2, 0], [1, 0, 0], [0, 0, 0]]

Splitting a string from right at intervals in python

I’m trying to split a string from the right. Following is the code.

string = "abcde" 
n = len(string)
slices = [string[i-3:i] for i in range(n,0,-3)]
print (slices)

I get the output as ['cde', '']. I’m trying to get ['cde', 'ab']

But when I split it from left it gives the proper output, i.e..,

string = "abcde" 
slices = [string[i:i+3] for i in range(0,n,3)]
print (slices)

output: ['abc', 'de']

Can anyone point out where am I going wrong?

Solution:

You are close. You need to floor the first indexing argument at 0:

x = "abcde" 
n = len(x)
slices = [x[max(0,i-3):i] for i in range(n,0,-3)]

['cde', 'ab']

The reason your code does not work is because with positive indices, falling off the end means going as far as you can.

While negative indices means starting from the end, rather than going to the start and no further.

Running Python code in Vim without saving

Is there a way to run my current python code in vim without making any changes to the file? Normally, when I want to test my code from within vim, I would execute this:

:w !python

However, this overrides the current file I am editing. Often, I add print statements or comment stuff out to see why my code isn’t working. I do not want such changes to overwrite a previous version of whatever .py file I’m currently working on. Is there a way to do so? Perhaps a combination of saving to a temporary file and deleting it afterwards?

Solution:

You have already answered your own question:

:w !python

will run the file in python without saving it. Seriously, test it out yourself! make some changes, run :w !python and then after it runs, run :e!. It will revert all of your changes.

The reason this works is because :w does not mean save. It means write, and by default, it chooses to write the file to the currently selected file, which is equivalent to saving. In bash speak, it’s like 

cat myfile > myfile

But if you give an argument, it will write the file to that stream rather than saving. In this case, your writing it to python, so the file is not saved.

I wrote a much longer answer on this topic here.

Convert string list to list in python

I have a string as below , 

val = '["10249/54","10249/147","10249/187","10249/252","10249/336"]'

I need to parse it and take the values after / and put into list as below

['54','147','187','252','336']

My code: [a[a.index('/')+1:] for a in val[1:-1].split(',')]

Output : ['54"', '147"', '187"', '252"', '336"']

It has double quotes also ” which is wrong.
After i tried as below

c = []
for a in val[1:-1].split(','):
    tmp = a[1:-1]
    c.append(tmp[tmp.index('/')+1:])

Output :

['54', '147', '187', '252', '336']

Is there any better way to do this?

Solution:

You can do it in one line pretty easily:

from ast import literal_eval
a = [i.split('/')[-1] for i in literal_eval(val)]
a
>>>['54', '147', '187', '252', '336']

literal_eval() converts your string into a literal list.

Removing an item from a list of lists based on each of the lists first element

Given: 

a = [[1,2],[3,4],[5,6],[7,8]]
b = 3

I would like to remove an item of a that has b as it’s first item. So in this case we would remove [3,4] to give:

a = [[1,2],[5,6],[7,8]]

My current code is:

if b in [i[0] for i in a]:
    pos = [i[0] for i in a].index(b)
       del a[pos]

This works but it is slow. What would a better way to do this be?

EDIT:
I’ve not tested performance before so I may be doing this wrong but I get this:

def fun1():
    lst = [[x, 2*x] for x in range(1000000)]
    lst = [x for x in lst if x[0] != 500]
    return lst

def fun2():
    lst = [[x, 2*x] for x in range(1000000)]
    for i in reversed(range(len(lst))):
        if lst[i][0] == 500:
            del lst[i]
    return lst

cProfile.runctx('fun1()', None, locals())
        6 function calls in 0.460 seconds

cProfile.runctx('fun2()', None, locals())
        6 function calls in 0.502 seconds

Solution:

Reverse delete a, modifying it in-place:

for i in reversed(range(len(a))):
    if a[i][0] == 3:
        del a[i]

An in-place modification means that this is more efficient, since it does not create a new list (as a list comprehension would).

Since OP requests a performant solution, here’s a timeit comparison between the two top voted answers here.

Setup –

a = np.random.choice(4, (100000, 2)).tolist()

print(a[:5])
[[2, 1], [2, 2], [3, 2], [3, 3], [3, 1]]

List comprehension –

%timeit [x for x in a if x[0] != b]
11.1 ms ± 685 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Reverse delete –

%%timeit
for i in reversed(range(len(a))):
    if a[i][0] == 3:
        del a[i]

10.1 ms ± 146 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

They’re really close, but reverse delete has a 1UP on performance because it doesn’t have to generate a new list in memory, as the list comprehension would.

Find the a 4 digit number who's square is 8 digits AND last 4 digits are the original number

From the comments on my answer here, the question was asked (paraphrase):

Write a Python program to find a 4 digit whole number, that when multiplied to itself, you get an 8 digit whole number who’s last 4 digits are equal to the original number.

I will post my answer, but am interested in a more elegant solutions concise but easily readable solution! (Would someone new-ish to python be able to understand it?)

Solution:

Here is a 1-liner solution without any modules:

>>> next((x for x in range(1000, 10000) if str(x*x)[-4:] == str(x)), None)
9376

If you consider numbers from 1000 to 3162, their square gives you a 7 digit number. So iterating from 3163 would be a more optimized because the square should be a 8 digit one. Thanks to @adrin for such a good point.

>>> next((x for x in range(3163, 10000) if str(x*x)[-4:] == str(x)), None)
9376

Find String Between Two Substrings in Python When There is A Space After First Substring

While there are several posts on StackOverflow that are similar to this, none of them involve a situation when the target string is one space after one of the substrings.

I have the following string (example_string):
<insert_randomletters>[?] I want this string.Reduced<insert_randomletters>

I want to extract “I want this string.” from the string above. The randomletters will always change, however the quote “I want this string.” will always be between [?] (with a space after the last square bracket) and Reduced.

Right now, I can do the following to extract “I want this string”.

target_quote_object = re.search('[?](.*?)Reduced', example_string)
target_quote_text = target_quote_object.group(1)
print(target_quote_text[2:])

This eliminates the ] and that always appear at the start of my extracted string, thus only printing “I want this string.” However, this solution seems ugly, and I’d rather make re.search() return the current target string without any modification. How can I do this?

Solution:

Your '[?](.*?)Reduced' pattern matches a literal ?, then captures any 0+ chars other than line break chars, as few as possible up to the first Reduced substring. That [?] is a character class formed with unescaped brackets, and the ? inside a character class is a literal ? char. That is why your Group 1 contains the ] and a space.

To make your regex match [?] you need to escape [ and ? and they will be matched as literal chars. Besides, you need to add a space after ] to actually make sure it does not land into Group 1. A better idea is to use \s* (0 or more whitespaces) or \s+ (1 or more occurrences).

Use

re.search(r'\[\?]\s*(.*?)Reduced', example_string)

See the regex demo.

import re
rx = r"\[\?]\s*(.*?)Reduced"
s = "<insert_randomletters>[?] I want this string.Reduced<insert_randomletters>"
m = re.search(r'\[\?]\s*(.*?)Reduced', s)
if m:
    print(m.group(1))
# => I want this string.

See the Python demo.

Return list of primes up to n using for loop

I have just picked up learing python and I am trying to create a simple function which accepts an integer and returns a list of all primes from 2 to that integer.

I have created the function but code doesn’t seem to work. I have found solutions only for more efficient (and complex) methodes (like this one Finding prime numbers using list comprehention) for this problem which don’t really help me in finding my mistake. 

def list_of_primes(n):
    primes = []
    for y in range (2, n):
        for z in range(2, y):
            if y % x == 0:
                continue
            else:
                primes.append(y)
        primes.sort()
        return primes

What is wrong with the code?

Solution:

There are several errors in your code. Below is a working implementation of your algorithm.

def list_of_primes(n):
    primes = []
    for y in range (2, n):
        for z in range(2, y):
            if y % z == 0:
                break
        else:
            primes.append(y)
    primes.sort()
    return primes

list_of_primes(20)

# [2, 3, 5, 7, 11, 13, 17, 19]

Explanation

  • Indentation is crucial in Python.
  • You need to test if y is divisible by z, not by a variable x which has not been defined.
  • Sort your list and return at the very end, both outside your outer for loop.
  • Use break to skip a number when it is found to be non-prime.
  • Apply your else statement on the inner for loop, not as part of the if / else clause.