Skip to content

feature request: multiprocessing the validation of a single file  #1721

@pierrecamilleri

Description

@pierrecamilleri

Using the frictionless validate --parallel flag at the command line or validate(parallel = True) in the python code does not seem to trigger any parallel treatment.

Performance of the validation on a moderately large csv file (~30 mb) does not change with or without this option. In addition,
monitoring cpu shows that only 1 core seems to be solicited.

In the case of a datapackage, the command does not even run (see #1644)

To reproduce

time frictionless validate --schema schema.json --parallel data.csv vs time frictionless validate --schema schema.json data.csv

Metadata

Metadata

Assignees

No one assigned

    Labels

    commsDocumentation related issuesfeatureNew functionality

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions