Skip to content

Datapackage should provide a cache mechanism #61

@femtotrader

Description

@femtotrader

Hello,

I think datapackage should provide a cache mechanism.

For this (if user want to have this cache mechanism 2 optional dependencies could be requests and requests-cache)

I don't suggest to monkey-patch requests to use CachedSession by default but to pass a CachedSession instead of a requests.Session

One possible use could be :

import datapackage
import requests_cache
import datetime
session = requests_cache.CachedSession(cache_name='cache', backend='sqlite', expire_after=datetime.timedelta(days=60))
datapkg = datapackage.DataPackage('http://data.okfn.org/data/cpi/', session=session)

Default value of parameter session should beNone`.
This session should be stored as a member of DataPackage.

When session is not None request will be performed using

self.session.get(url)

Kind regards

PS : a similar approach was used in https://github.com/femtotrader/pandas_datareaders_unofficial

edit: and is now (oct 2015) used in official "pandas-datareader" https://github.com/pydata/pandas-datareader/

see also pydata/pandas-datareader#48

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions