Tag: python-3.6

json.loads() returns a string

Why is json.loads() returning a string? Here’s is my code:

import json

d = """{
    "reference": "123432",
    "business_date": "2019-06-18",
    "final_price": 40,
    "products": [
        {
            "quantity": 4,
            "original_price": 10,
            "final_price": 40,
        }
    ]
}"""

j = json.loads(json.dumps(d))
print(type(j))

Output:

<class 'str'>

Shouldn’t it returning a json object? What change is required here?

Solution:

Two points:

  1. You have a typo in your products key : "final_price": 40, should be "final_price": 40 (without comma)
  2. j should be json.loads(d)

Output

dict

EDIT

Reasons why you can not have a trailing comma in a json objects are explained in this post Can you use a trailing comma in a JSON object?

Unfortunately the JSON specification does not allow a trailing comma. There are a few browsers that will allow it, but generally you need to worry about all browsers.

Python, Pandas: A Better way to get the first None position in list which give maximum consecutive None count

I have lists that contain None like the following lists.

l1 = [None, 1, None, None, 2, None, None]
l2 = [None, 1, 1, None, None, None, 2, None, None]

I want to get the first None position in this list which gives the maximum consecutive None count.

get_start_None_pos(l1) # should return 2
get_start_None_pos(l2) # should return 3

My current approach with Pandas which works fine but it too slow when I have so many lists to deal with.

def get_start_None_pos(l: list) -> int:
    s = pd.Series(l)
    s = s.isna()
    s = s.cumsum() - s.cumsum().where(~s).ffill().fillna(0)
    return int(s.idxmax() - s.max() + 1)

I would like to know, is there any better way to solve something like this?

Solution:

Here’s one with NumPy –

def maxconsecNone_start(l):
    a = np.isnan(np.asarray(l, dtype=np.float64))
    a1 = np.r_[False,a,False]
    idx = np.flatnonzero(a1[:-1] != a1[1:])
    return idx[2*(idx[1::2]-idx[::2]).argmax()]

Sample runs –

In [49]: l1
Out[49]: [None, 1, None, None, 2, None, None]

In [50]: l2
Out[50]: [None, 1, 1, None, None, None, 2, None, None]

In [51]: maxconsecNone_start(l1)
Out[51]: 2

In [52]: maxconsecNone_start(l2)
Out[52]: 3

How to Write an AWS Python3 Lambda Function using a zip file on Windows OS

I have looked all over for a tutorial or help on creating a python3 lambda function from a zip file using the Lambda Management Console on Windows OS. I have, unfortunately, been unlucky. Here is where I am at…

Following the instructions on the AWS website here: https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html

  • I create a folder on my desktop called ‘APP’. In that folder I save a file with my python code called ‘twilio_test.py’ at the root level of ‘App’.

My Python code:

import twilio

def lambda_handler(event, context):
    account_sid = '##########################'
    auth_token = '###########################'

    client = Client(account_sid, auth_token)

    message = client.messages.create(
            to = '###########',
            from_ = '###########',
            body = "Test")
    return("success")
  • Since I am using the twilio library I pip install it in the root of my ‘APP’ folder based on the instructions found in the above link. The instructions say specifically, “Install any libraries using pip. Again, you install these libraries at the root level of the directory.”:

pip install twilio -t \path\to\directory

  • I then zip the contents of ‘APP’ based on the quoted instruction, “Zip the content of the project-dir directory, which is your deployment package. Zip the directory content, not the directory.” This creates a zip file called ‘twilio_test’.

  • I then go to the AWS lambda management console, upload the zip file ‘twilio_test’.

Here is where I am getting confused. What should be the handler?

Have I correctly done everything so far up until this point? If not, what is the best way to go about installing twilio, zipping a file and then using it in AWS lambda?

Although it is inappropriate to say that AWS lambdas are inherently difficult to use, I can say that I am inherently confused.

Solution:

You should set the handler to python_file_name.function_name. So in your case it should be twilio_test.lambda_handler.

From the documentation:

… You specify the function name in the Python code to be used as the handler when you create a Lambda function. For instructions to create a Lambda function using the console, see Create a Simple Lambda Function. In this example, the handler is hello_python.my_handler (file-name.function-name) …

Why does python behave this way with variables?

I have been trying to understand why python behaves this way, in the block of code below. I have done my research but couldn’t find a good answer so I came here to see if anyone can point me in the right direction or provide a good clarification.
I understand that it has to do with some old ALGOL principle, but I don’t fully understand it. 

var = 5

def func1():
    print(var)
func1()

def func2():
    var = 8
    print(var)
func2()

def func3():
    print(var)
    var = 8
func3()

The output of this code is as follows:

5
8
UnboundLocalError: local variable ‘var’ referenced before assignment

I understand why we get the outputs ‘5’ and ‘8’. But with ‘func3()’, I was expecting an output of ‘5’. As it seems, the interpreter thinks that I want to print the local ‘var’ in the function instead of the global ‘var’. So it throws this error.

Or maybe if a variable is defined somewhere inside of the function, then the function will default to the local variable, instead of a global one with the same name.

But why exactly does python behave this way ? I am not complaining, I am just trying to understand something…

How could I use a predefined global variable in a function, then define a local variable with the same name inside of the same function, without changing the value of the global variable ? ( in python of course )

Thanks in advance to everyone here. You are amazing people ! 🙂

Edit_1: Thanks every one for the great answers. I totally understand that it is a bad and unpractical idea to use a predefined global variable in a function, then define a local variable with the same name inside of the same function. I was just thinking about it from a theoretical perspective, because I saw it in a college lecture. XD
I can’t find a single use case, in which it would be optimal to do that either !

Edit_2: I already read the PEP8 and I know that being explicit is better than being implicit. 🙂
It’s true. Otherwise the code will be confusing and lead to bugs.
That question was just about some useless and impractical college theory that I was trying to understand.

Edit_3:
Now I fully understand why it happens and what is going on here. Thanks to Randall Valenciano for providing this link to a blog that explains it very well.

What happens is that the function is interpreted as a whole, and not line by line. So when the function is being interpreted, the variable declarations of any defined variables, are moved to the top of the function. So when we are printing ‘var’, the function is using the locally declared variable that doesn’t have any value assigned to it yet, and then the interpreter complains about it and throws and error.

Thanks to all of you again ! 🙂
You have been of great help to me ! Now I finally understand what is going on there under the hood.

Solution:

Your var is defined as a global variable. In each function when you’re only reading the var you’re accessing the global var, but the moment there’s an assigned value to var somewhere in the function, python treats every var within the function as a local variable. Thus why your last function failed, because print(var) (the local varaible) was called before var = 8 was assigned.

You can read up a bit about more in these threads:
How bad is shadowing names defined in outer scopes?
Python nonlocal statement

The best thing to do is be explicit about your code so it’s no longer confusing if you’re trying to reference a local, nonlocal or global variable.

In this case, assuming your intention is to keep using the global var, do this:

var = 5

def func1():
    print(var)
func1()

def func2():
    global var
    var = 8
    print(var)
func2()

def func3():
    global var
    print(var)
    var = 8  # technically this is not necessary any more var = 8 was already assigned when func2() is called
func3()

The output is thus:

5
8
8

Edit: Thanks to juanpa.arrivillaga‘s comment – I missed your original question.

How could I use a predefined global variable in a function, then
define a local variable with the same name inside of the same
function, without changing the value of the global variable ? ( in
python of course )

The short answer is – define the local var first like you did in func2() and you’re good. The longer answer though is – why would you want to do that? It creates confusion and becomes a headache to track which is when you have variables of the same name in different scope. A better approach would be name your local var to be local_var or something so it’s distinctly different and easily traced.

get_job_info() giving item not found error : python-jenkins

I am using python-jenkins and python 3.6

Trying to get the information regarding the jobs in jenkins but facing an error.

File “/usr/local/lib/python3.6/dist-packages/jenkins/init.py”,
line 359, in get_job_info
self._build_url(JOB_INFO, locals()) File “/usr/local/lib/python3.6/dist-packages/jenkins/init.py”, line
451, in jenkins_open
raise NotFoundException(‘Requested item could not be found’) jenkins.NotFoundException: Requested item could not be found

jobs = server.get_all_jobs()
for j in jobs:
    jobName = j['name'] # get job name
    print(jobName)
    lastJobId = getLastJobId(session, jobName) # get last locally stored job of this name
    print(lastJobId)
    lastBuildNumber = server.get_job_info(jobName)

get_all_jobs() is working fine but get_job_info() is not working.

Any help will be appreciated.

Solution:

You might need to use the fullname key instead of the name key.

jobName = j['fullname'] # get job name

There could be multiple jobs with the same job name, but in different folders/views. And fullname will fully qualify a specific job.