json.decoder.JSONDecodeError on csv (res) content #1570
Description
Environment details
- OS type and version: macOS 10.15.7 (19H1419)
- Python version: 3.8.5
- pipenv version: 2020.8.13
- pip version: 20.2.3
google-api-python-clientversion: 2.26.1
Steps to reproduce
Trying to use generateCSV api from adsense.accounts.reports.saved.generateCsv, but it might be a more general issue with non json format content.
Code example
...
saved_report_id = 'accounts/pub-xxx/reports/yyy_zzz'
def main(argv):
# Authenticate and construct service.
credentials = adsense_util.get_adsense_credentials()
with discovery.build('adsense', 'v2', credentials = credentials) as service:
try:
# Let the user pick account if more than one.
account_id = adsense_util.get_account_id(service)
# Retrieve report.
# if saved_report_id:
result = service.accounts().reports().saved().generateCsv(name=saved_report_id, dateRange='LAST_30_DAYS') .execute()
print(result)
...Stack trace
Traceback (most recent call last):
File "src/generate_report.py", line 88, in <module>
main(sys.argv)
File "src/generate_report.py", line 63, in main
print(result.execute())
File "[...]/lib/python3.8/site-packages/googleapiclient/_helpers.py", line 131, in positional_wrapper
return wrapped(*args, **kwargs)
File "[...]/lib/python3.8/site-packages/googleapiclient/http.py", line 939, in execute
return self.postproc(resp, content)
File "[...]/lib/python3.8/site-packages/googleapiclient/model.py", line 218, in response
return self.deserialize(content)
File "[...]/lib/python3.8/site-packages/googleapiclient/model.py", line 281, in deserialize
body = json.loads(content)
File "[...]/.pyenv/versions/3.8.5/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "[...]/.pyenv/versions/3.8.5/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/[...]/.pyenv/versions/3.8.5/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Possible cause
The code involved is in googleapiclient/model.py and the class is JsonModel(BaseModel) specifically this part of code (275-283):
def deserialize(self, content):
try:
content = content.decode("utf-8")
except AttributeError:
pass
body = json.loads(content)
if self._data_wrapper and isinstance(body, dict) and "data" in body:
body = body["data"]
return bodyThe problem appears when the content of response is not in json format (with generateCsv api the content is csv text) so it produces the JSONDecodeError.
Possible patch
One of the following could work:
- In case the request asks for csv generation, the response should be routed to a correct new callable postprocessor method that deals with csv format
- change the above lines of code in this way:
def deserialize(self, content):
try:
content = content.decode("utf-8")
except AttributeError:
pass
try:
body = json.loads(content)
except json.decoder.JSONDecodeError:
body = content
if self._data_wrapper and isinstance(body, dict) and "data" in body:
body = body["data"]
return body