Category: python

Python: pass "not" as a lambda function

I need to pass a function as a parameter that works as the boolean “not”. I tried something like this but it didn’t work because not isn’t a function.

theFunction(callback=not) # Doesn't work :(

I need to do the following, but I wonder if there exists any predefined function that does this simple job, so that I don’t have to redefine it like this:

theFunction(callback=lambda b: not b, anotherCallback=lambda b: not b)

Note: I can’t change the fact that I have to pass a function like this because it’s an API call.

Solution:

Yes, there is the operator module: https://docs.python.org/3.6/library/operator.html

import operator
theFunction(callback=operator.not_)

python static overloading of member function

I’m new to python and have a simple question: Is it possible to statically overload a member fuction? Like for example so:

class A:
    text=""
    def __init__(self, text):
        self.text = text

    def print(self):
        print (self.text)

    @staticmethod
    def print():
        print ("static")

a = A("test") 
a.print()

In this case “static” is printed out. Is it impossible to call the member function if a static function with the same name exists? And if not, who do I call it?

Solution:

You aren’t overloading the method; you are replacing it with the static method. You’ll have to pick a different name for the static method.

Turn iterable of tuple into iterables of each element (Python)

I have an iterable delta that generates tuple of two numbers (dx, dy), and I want to compute the sum of each. The following doesn’t work since delta is disposed after the first iteration.

x = sum(dx for dx, dy in delta)
y = sum(dy for dx, dy in delta)

Any idea? I’m thinking in the direction of somehow turning delta into two iterables of dx and dy, but have reached nothing so far.

Solution:

Use zip() and map() functions to apply the sum() on each column:

x, y = map(sum, zip(*delta))

Splitting a List Based on a Substring

I have the following list:

['1(Reg)', '100', '103', '102', '100', '2(Reg)', '98', '101', '100', '3(Reg)', '96', '99', '98', '4(Reg)', '100', '100', '100', '100', '5(Reg)', '98', '99', '99', '100', '6(Reg)', '99.47', '99.86', '99.67', '100']

I want to split this list into multiple lists so that each sublist will have the substring “(Reg)” appear once:

[['1(Reg)', '100', '103', '102', '100'],
['2(Reg)', '98', '101', '100'],
['3(Reg)', '96', '99', '98'],
['4(Reg)', '100', '100', '100', '100'],
['5(Reg)', '98', '99', '99', '100'],
['6(Reg)', '99.47', '99.86', '99.67', '100']]

I’ve tried joining the list with a delimiter and splitting it by (Reg), but that didn’t work. How can I split the list into a nested list like above?

Solution:

Here is one way, though not necessarily optimal:

from itertools import zip_longest

lst = ['1(Reg)', '100', '103', '102', '100', '2(Reg)', '98', '101', '100',
       '3(Reg)', '96', '99', '98', '4(Reg)', '100', '100', '100', '100',
       '5(Reg)', '98', '99', '99', '100', '6(Reg)', '99.47', '99.86', '99.67', '100']

indices = [i for i, j in enumerate(lst) if '(Reg)' in j]
lst_new = [lst[i:j] for i, j in zip_longest(indices, indices[1:])]

# [['1(Reg)', '100', '103', '102', '100'],
#  ['2(Reg)', '98', '101', '100'],
#  ['3(Reg)', '96', '99', '98'],
#  ['4(Reg)', '100', '100', '100', '100'],
#  ['5(Reg)', '98', '99', '99', '100'],
#  ['6(Reg)', '99.47', '99.86', '99.67', '100']]

String without quotes within a string (python)

I’m trying to write text to file, but I have other text I need to include besides the targeted string. When I’m looping over the targeted strings it is printed with quotes as the quotes are needed for other text.
How to remove quotes from a string that I’m inserting each loop?

list=['random', 'stuff', 1]
with open(textfile, 'a') as txtfile:
    for item in list:
        print("""Need to have stuff before %a and after each loop string"""  
        %item, file=txtfile)

Output: Need to have stuff before ‘random’ and after each loop string;
Wanted output: Need to have stuff before random and after each loop string

Solution:

You can use str.format:

>>> li=['random', 'stuff', 1]
>>> for item in li:
...    print("before {} after".format(item))
... 
before random after
before stuff after
before 1 after

Or you can use %s with the % operator:

>>> for item in li:
...    print("before %s after" % item)
... 
before random after
before stuff after
before 1 after

(And don’t call a list list or you will overwrite the Python function of the same name…)

How to flatten/concatenate multiple columns with similar information based on one index column in pandas?

I have a question about flattening or collapsing a dataframe from several columns in one row with information about a key to several rows each with the same key column and the appropriate data. Suppose a dataframe is something like this:

df = pd.DataFrame({'CODE': ['AA', 'BB', 'CC'],
              'START_1': ['1990-01-01', '2000-01-01', '2005-01-01'],
              'END_1': ['1990-02-14', '2000-03-01', '2005-12-31'],
              'MEANING_1': ['SOMETHING', 'OR', 'OTHER'],
              'START_2': ['1990-02-15', None, '2006-01-01'],
              'END_2': ['1990-06-14', None, '2006-12-31'],
              'MEANING_2': ['ELSE', None, 'ANOTHER']})
  CODE     START_1       END_1  MEANING_1     START_2       END_2 MEANING_2
0   AA  1990-01-01  1990-02-14  SOMETHING  1990-02-15  1990-06-14      ELSE
1   BB  2000-01-01  2000-03-01         OR        None        None      None
2   CC  2005-01-01  2005-12-31      OTHER  2006-01-01  2006-12-31   ANOTHER

and I need to get it into a form somewhat like this:

  CODE       START         END    MEANING
0   AA  1990-01-01  1990-02-14  SOMETHING
1   AA  1990-02-15  1990-06-14       ELSE
2   BB  2000-01-01  2000-03-01         OR
3   CC  2005-01-01  2005-12-31      OTHER
4   CC  2006-01-01  2006-12-31    ANOTHER

I have a solution as follows:

df_a = df[['CODE', 'START_1', 'END_1', 'MEANING_1']]
df_b = df[['CODE', 'START_2', 'END_2', 'MEANING_2']]
df_a = df_a.rename(index=str, columns={'CODE': 'CODE',
                                'START_1': 'START',
                                'END_1': 'END',
                                'MEANING_1': 'MEANING'})
df_b = df_b.rename(index=str, columns={'CODE': 'CODE',
                                'START_2': 'START',
                                'END_2': 'END',
                                'MEANING_2': 'MEANING'})
df = pd.concat([df_a, df_b], ignore_index=True)
df = df.dropna(axis=0, how='any')

Which yields the desired result. Of course this doesn’t seem very pythonic and is clearly not ideal if you have more than 2 column groups which need to be collapsed (I actually have 6 in my real code). I’ve examined the groupby(), melt() and stack() methods but haven’t really found them to be very useful yet. Any suggestions would be appreciated.

Solution:

Use pd.wide_to_long:

pd.wide_to_long(df, stubnames=['END', 'MEANING', 'START'],
                i='CODE', j='Number', sep='_', suffix='*')

Output:

                    END    MEANING       START
CODE Number                                   
AA   1       1990-02-14  SOMETHING  1990-01-01
BB   1       2000-03-01         OR  2000-01-01
CC   1       2005-12-31      OTHER  2005-01-01
AA   2       1990-06-14       ELSE  1990-02-15
BB   2             None       None        None
CC   2       2006-12-31    ANOTHER  2006-01-01

Then, we can drop Number column/index and dropna’s if you wish, e.g. df.reset_index().drop('Number', 1).

Turn iterable of tuple into iterables of each element (Python)

I have an iterable delta that generates tuple of two numbers (dx, dy), and I want to compute the sum of each. The following doesn’t work since delta is disposed after the first iteration.

x = sum(dx for dx, dy in delta)
y = sum(dy for dx, dy in delta)

Any idea? I’m thinking in the direction of somehow turning delta into two iterables of dx and dy, but have reached nothing so far.

Solution:

Use zip() and map() functions to apply the sum() on each column:

x, y = map(sum, zip(*delta))

Rolling max with pandas on large datasets is very slow

I have a pandas dataframe df which has a DatatimeIndex spanning about 2 years, 2 columns and over 30 million rows of float64 data. I quickly noticed that there is a stark difference in performance between df.rolling('1d').mean() and df.rolling('1d').max()

n=100000; import timeit; r=df[:n].rolling('1d'); timeit.timeit(lambda: r.max(), number=1)
2.5886592870228924
n=100000; import timeit; r=df[:n].rolling('1d'); timeit.timeit(lambda: r.mean(), number=1)
0.011829487979412079
n=1000000; import timeit; r=df[:n].rolling('1d'); timeit.timeit(lambda: r.max(), number=1)
53.8340517100296
n=1000000; import timeit; r=df[:n].rolling('1d'); timeit.timeit(lambda: r.mean(), number=1)
0.06093513499945402

As you can see, df.rolling('1d').mean() is several hundred times faster than df.rolling('1d').max(). I would expect it to be somewhat faster since to calculate the maximum pandas presumably has to keep track the order of all values in the rolling window at each step. However, it’s easy to see how to implement this by adding at most a log factor, so I would expect a smaller difference. If this is the best one can do, using df.rolling('1d').max is going to be a pain on the full dataset, since it looks like it will take hours each time.

Having run into efficiency issues with pandas before (Series.iloc indexing), I am curious as to whether this is a pandas issue or if there is a faster way to solve this problem.

Solution:

Pandas is using naive implementation of running max, with linear scan over the window for every sample. Thus, it is linear complexity times size of the window, i.e. for few hundred+ samples per day it will be hundred times slower than the mean.

Possible workaround: take by minutes, then over this max by hours, then days – it should result in a kind of log effect, but constants might eat up all asymptotic advantage.

Better solution: contribute heap min_max implementation to pandas

Python 3-x: Nested list in a for loop – how to get a "horizontal" output

for i, j in cards: 
# cards = a list containing list of cards - RANDOM OUTPUTS, more pairs can be added to the list depending on the number that the user puts in
        print(i)
        print(j)
        print("\t")

How do I make it so that the output becomes:

TS  6S  JS
AH  5S  AS

Instead of:

TS
AH

6S
5S

JS
AS

I posted a question similar to this but I wasn’t being specific enough, and I edited too late. Apologies in advance

EDIT – ‘cards’ code: 

deck = deck_create()
def deal_cards(deck, num_players):

    cards= []
    for i in range(num_players):

        hands.append([deck.pop(), deck.pop()])

    return hands

Solution:

I think you need,

for i in range(len(cards[0])):
    for items in cards:
        if items[i] in cards[-1]:
            print(items[i],end="\n")
        else:
             print(items[i],end="\t")

python static overloading of member function

I’m new to python and have a simple question: Is it possible to statically overload a member fuction? Like for example so:

class A:
    text=""
    def __init__(self, text):
        self.text = text

    def print(self):
        print (self.text)

    @staticmethod
    def print():
        print ("static")

a = A("test") 
a.print()

In this case “static” is printed out. Is it impossible to call the member function if a static function with the same name exists? And if not, who do I call it?

Solution:

You aren’t overloading the method; you are replacing it with the static method. You’ll have to pick a different name for the static method.