Skip to content

XYZ CSV skipinitialspace#15832

Merged
AUTOMATIC1111 merged 1 commit intodevfrom
xyz-csv-skipinitialspace
Jun 8, 2024
Merged

XYZ CSV skipinitialspace#15832
AUTOMATIC1111 merged 1 commit intodevfrom
xyz-csv-skipinitialspace

Conversation

@w-e-w
Copy link
Copy Markdown
Collaborator

@w-e-w w-e-w commented May 18, 2024 •
edited
Loading

Description

if there's a special character in side a CSV value. the value needs to be quoted
the issue is without skipinitialspace "quoting is finicky"

csv.Dialect.skipinitialspace

spaces immediately following the delimiter are ignored

skipinitialspace makes parseing CSV more consistent when quotes are involved

demo
without skipinitialspace (current)

aaa, "b,b",ccc
    ^
# notice that there is a space

will be parsed as 4 item

aaa
"b
b"
ccc

only the input without space is parsed as expected

aaa,"b,b", ccc

to

aaa
b,b
ccc

with skipinitialspace will be parsed as 3 it will consistently parsed as aaa,"b,b", ccc aaa, "b,b",ccc regardless of the extra Spaces

aaa
b,b
ccc

demo test script

from csv import reader
from itertools import chain
from io import StringIO


def csv_without_skipinitialspace(data_str):
    return list(map(str.strip, chain.from_iterable(reader(StringIO(data_str)))))


def csv_with_skipinitialspace(data_str):
    return list(map(str.strip, chain.from_iterable(reader(StringIO(data_str), skipinitialspace=True))))


def print_list(csv_list):
    for item in csv_list:
        print(item)
        print('-'*10)


if __name__ == '__main__':
    # input_string = '''aaa,"b,b", ccc'''
    input_string = '''aaa, "b,b", ccc'''
    print_list(csv_without_skipinitialspace(input_string))
    print('='*10)
    print_list(csv_with_skipinitialspace(input_string))
Details

output of aaa, "b,b", ccc'

aaa
----------
"b
----------
b"
----------
ccc
----------
==========
aaa
----------
b,b
----------
ccc
----------

aaa,"b,b", ccc'

aaa
----------
b,b
----------
ccc
----------
==========
aaa
----------
b,b
----------
ccc
----------

Checklist:

@w-e-w w-e-w requested a review from AUTOMATIC1111 as a code owner May 18, 2024 10:27
@w-e-w w-e-w mentioned this pull request Jun 1, 2024
Open
1 task
@AUTOMATIC1111 AUTOMATIC1111 merged commit 5977cb0 into dev Jun 8, 2024
@AUTOMATIC1111 AUTOMATIC1111 deleted the xyz-csv-skipinitialspace branch June 8, 2024 07:46
@w-e-w w-e-w mentioned this pull request Jun 9, 2024
Closed
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AUTOMATIC1111 AUTOMATIC1111 AUTOMATIC1111 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@w-e-w @AUTOMATIC1111