Skip to content

Import proposal answers#8271

Merged
leio10 merged 81 commits intodecidim:developfrom
mainio:feature/import_proposal_answers
Sep 30, 2021
Merged

Import proposal answers#8271
leio10 merged 81 commits intodecidim:developfrom
mainio:feature/import_proposal_answers

Conversation

@lahdeero
Copy link
Copy Markdown
Contributor

@lahdeero lahdeero commented Aug 31, 2021 •
edited
Loading

🎩 What? Why?

Imports proposal answers from file (excel, csv, json).

📌 Related Issues

https://meta.decidim.org/processes/roadmap/f/122/proposals/14922

Testing

  1. Create .xlsx file, colums: id, state, answer/en (depends on organization locales)
  2. Write rows to excel. Example row: 1, accepted, This is proposal is very nice.
  3. Go to admin panel -> components -> Proposals (component must match proposal's component!) -> Import (dropdown) -> Import proposal answers from a file
  4. Upload file and select "Import"
  5. Check proposal answers

📋 Checklist

🚨 Please review the guidelines for contributing to this repository.

  • ❓ CONSIDER adding a unit test if your PR resolves an issue.
  • ✔️ DO check open PR's to avoid duplicates.
  • ✔️ DO keep pull requests small so they can be easily reviewed.
  • ✔️ DO build locally before pushing.
  • ✔️ DO make sure tests pass.
  • ✔️ DO make sure any new changes are documented in docs/.
  • ✔️ DO add and modify seeds if necessary.
  • ✔️ DO add CHANGELOG upgrade notes if required.
  • ✔️ DO add to GraphQL API if there are new public fields.
  • ✔️ DO add link to MetaDecidim if it's a new feature.
  • ❌AVOID breaking the continuous integration build.
  • ❌AVOID making significant changes to the overall architecture.

📷 Screenshots

image

image

♥️ Thank you!

@andreslucena andreslucena changed the title Feature import proposal answers Import proposal answers Aug 31, 2021
ahukkanen
ahukkanen reviewed Aug 31, 2021
View reviewed changes
ahukkanen
ahukkanen reviewed Aug 31, 2021
View reviewed changes
ahukkanen
ahukkanen reviewed Aug 31, 2021
View reviewed changes
ahukkanen
ahukkanen reviewed Aug 31, 2021
View reviewed changes
ahukkanen
ahukkanen reviewed Aug 31, 2021
View reviewed changes
ahukkanen
ahukkanen requested changes Aug 31, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@ahukkanen ahukkanen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some review comments when going through.

lahdeero and others added 18 commits August 31, 2021 16:26
@lahdeero
Fix import notifications
7cea85c
@lahdeero
Suggested improvements and fixing tests again
1f73b5b
@lahdeero
Normalize locales
7830a71
@lahdeero
Add system spec
d5ee1d2
@lahdeero
Remove tempfile and fix tests vol. 100
1a5de65
@lahdeero
Make method private
49988fa
@lahdeero
Add dynamic asset to gitignore
b813f2c
@lahdeero
Remove lock file
0d6bdf0
@ahukkanen
Merge branch 'develop' into feature/import_proposal_answers
86831b7
@lahdeero
Fix answer notification
3090eb0
@lahdeero
Merge branch 'feature/import_proposal_answers' of github.com:mainio/d…
3b654af 
…ecidim into feature/import_proposal_answers
@ahukkanen
Create admin log records for the answers
630644e
@ahukkanen
Do not answer emendations
cbda3d1
@ahukkanen
Allow empty columns in-between of columns with values
ad794b4
@ahukkanen
Remove the creator dropdown and improve importer message configuration
05e1466 
- Refactors the import view to only consider a single importer as
  this is the desired way it works
- Adds message configurations to the importer manifests that define
  different messages during the import process for each importer
- Improve some of the messages, make them more understandable
@ahukkanen
Consider empty rows in the XLSX import reader
55f6370
@ahukkanen
Add the test excel file to the dev assets
29a1428
@ahukkanen
Fix resource validity check for proposals import
968e959
@ahukkanen
Fix broken specs
2282d9e
@ahukkanen
Add specs for the import readers example file generation
5df68d2
@ahukkanen
Fix comment
afecc21
@ahukkanen
Fix terminology in the CSV reader spec
2f278ff
@ahukkanen
Add system specs for testing the proposal answer example downloads
18a4f4b
@ahukkanen
Refactor the creator static methods
bc0c3da
@ahukkanen
Implement import verifier to improve the data verification logic
36c7790
@ahukkanen
Update the specs after refactoring and add new specs
b3f0c7b
@ahukkanen
Fix issue with i18n "unused" translations
f0af4ea
@ahukkanen
Fix a bug in the importer when calling only verify
aa51e5e 
When the `prepare` method has not been called before `verify`, the
data headers were not available yet.
@ahukkanen
Reflect the refactoring to the importer spec in proposals
56c68cb
@ahukkanen
Move admin import specs to correct file paths
45121c1
@ahukkanen
Move the proposals creators under the import namespace
2b864da
@ahukkanen
Correct namespace for proposal and answer creator specs
92b69bf
@ahukkanen
Add comment for import verifier initialize method
b83f77b
@ahukkanen
Fix reference to the ProposalCreator class
c15a623
@ahukkanen
Support nested JSON with the JSON reader + make example files nested
6afb123
@ahukkanen
Document how to define the import example data
7b532d1
@ahukkanen
Copy link
Copy Markdown
Contributor

ahukkanen commented Sep 10, 2021

Just to note here that the bugs/issues are already fixed and we have also added better errors in these situations:

  • Missing column headers in the import file
  • Duplicate column headers in the import file
  • Invalid import file (e.g. incorrectly formatted JSON)

Feel free to test in the QA environment linked above.

@andreslucena
Copy link
Copy Markdown
Member

andreslucena commented Sep 14, 2021

Hi! I tried to import some proposals answers but I get an error:

 There was a problem during the import  
 Found error in the import file on lines 3 and 5. Please check that these lines are formatted correctly and contain valid records.

I have the same error with CSV and XLSX.

This is the file that I'm using: proposals-answers-example.csv, in this process: https://pr8271.decidimqa.mainiotech.fi/admin/participatory_processes/pb-2022/components/3/imports?name=answers

Also regarding the wording of the error, Grammarly says that it should be "Found an error in the import file on lines 3 and 5. Please check that these lines are formatted correctly and contain valid records."

@ahukkanen
Copy link
Copy Markdown
Contributor

ahukkanen commented Sep 14, 2021

@andreslucena Your import document contains amendment IDs on lines 3 and 5. You cannot import answers for amendments, so these lines are incorrect.

We'll fix the wording with the error message.

@ahukkanen
Copy link
Copy Markdown
Contributor

ahukkanen commented Sep 14, 2021

@andreslucena The wording is now fixed for the error message.

@andreslucena
Copy link
Copy Markdown
Member

andreslucena commented Sep 15, 2021

You cannot import answers for amendments, so these lines are incorrect.

You're right, after removing those lines everything works as expected.

I thought that somehow I got to import answers to Amendments because that's what I've seen in the admin panel:

imatge

But I think it's actually an unrelated bug (from the seed contents), so we can skip that for now.

Ready to be reviewed by you @decidim/maintainers

leio10
leio10 approved these changes Sep 30, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@leio10 leio10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huge and outstanding job! 👏🔝

@leio10 leio10 merged commit 6af00df into decidim:develop Sep 30, 2021
@ahukkanen ahukkanen deleted the feature/import_proposal_answers branch September 30, 2021 08:20
@alecslupu alecslupu added this to the 0.26.0 milestone Jul 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ahukkanen ahukkanen ahukkanen requested changes

+1 more reviewer

@leio10 leio10 leio10 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

in-review module: proposals

Projects

None yet

Milestone

0.26.0

Development

Successfully merging this pull request may close these issues.

5 participants

@lahdeero @ahukkanen @andreslucena @leio10 @alecslupu