Skip to content

Improved performance of cf.read "select" parameter for PP/UM files #201

@davidhassell

Description

@davidhassell

When selecting fields with the "select" parameter of cf.read with PP/UM file inputs, the currently selection occurs after all of the input files have been processed. This can be made much more efficient by avoiding processing the non-selected fields.

E.g.

import re
import cf
f = cf.read('~/aaadpa#pg000003385c1+.pp', select=["stash_code=3237", "stash_code=1"])
g = cf.read('~/aaadpa#pg000003385c1+.pp', select=re.compile('^stash_code=\d\d\d\d$'))

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestperformanceRelating to speed and memory performanceum/ppRelating to UM or PP format files

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions