Skip to content

Enable the client to specify a custom csv delimiter for parsing documents#2803

Closed
MixusMinimax wants to merge 2 commits intomeilisearch:mainfrom
MixusMinimax:feature/custom-csv-delimiter
Closed

Enable the client to specify a custom csv delimiter for parsing documents#2803
MixusMinimax wants to merge 2 commits intomeilisearch:mainfrom
MixusMinimax:feature/custom-csv-delimiter

Conversation

@MixusMinimax
Copy link
Copy Markdown
Contributor

@MixusMinimax MixusMinimax commented Sep 27, 2022 •
edited
Loading

Pull Request

What does this PR do?

Fixes #2806

This PR adds the ability to specify custom csv delimiters, which I think is a very common use case, especially with large amounts of pre-existing data.

Description

In meilisearch-lib, I added a u8 parameter to the DocumentAdditionFormat::Csv document type.

In meilisearch-http, I added the field pub csv_delimiter: Option<char>, to UpdateDocumentsQuery (the actual query parameter is in camelCase thanks to serde: csvDelimiter)

The default delimiter is the comma (44), just like before. For example, the query for data delimited with semicolons:

POST localhost:7700/indexes/example/documents?csvDelimiter=%3B

cargo test succeeded on my system, and I was able to add documents using different delimiters using postman.

PR checklist

Please check if your PR fulfills the following requirements:

  • Does this PR fix an existing issue?
  • Have you read the contributing guidelines?
  • Have you made sure that the title is accurate and descriptive of the changes?

@curquiza
Copy link
Copy Markdown
Member

curquiza commented Sep 28, 2022

Hello @MixusMinimax thanks for your PR, since it impacts the product we need to discuss with the team first, and we will review it as soon as possible 😄

@curquiza
Copy link
Copy Markdown
Member

curquiza commented Sep 29, 2022 •
edited
Loading

For traceability, more information about the state of this PR here: #2806 (comment) 😇

@curquiza curquiza marked this pull request as draft September 29, 2022 11:45
@curquiza curquiza mentioned this pull request Jan 31, 2023
Closed
2 tasks
@irevoire irevoire mentioned this pull request Feb 16, 2023
Merged
@MixusMinimax
Copy link
Copy Markdown
Contributor Author

MixusMinimax commented Feb 16, 2023 •
edited
Loading

Hi!
Should I update this now to be up to date with main? I think it could work like this: have a struct of options, implement FromRequest for it, and pass it to add_documents. This could include the csv delimiter, and what quotation marks to use (csv fields that include the delimiter themselves are put in quotation marks in the csv standard)

Update: Ignore this, this PR is replaced by #3505

@bors bors Bot closed this in 1e9ac00 Feb 20, 2023
@curquiza
Copy link
Copy Markdown
Member

curquiza commented Feb 20, 2023

Hello @MixusMinimax
Sorry for the late answer! Thanks for your previous work, it helped us implement #3505, you will be put as a contributor in the next release changelog of course 😇

@MixusMinimax
Copy link
Copy Markdown
Contributor Author

MixusMinimax commented Feb 20, 2023

No worries, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Enable the client to specify a custom csv delimiter for parsing documents

2 participants

@MixusMinimax @curquiza