Skip to content

Add participatory spaces to open data files#13313

Merged
alecslupu merged 26 commits intodevelopfrom
feature/open-data-spaces
Sep 17, 2024
Merged

Add participatory spaces to open data files#13313
alecslupu merged 26 commits intodevelopfrom
feature/open-data-spaces

Conversation

@andreslucena
Copy link
Copy Markdown
Member

@andreslucena andreslucena commented Aug 27, 2024

🎩 What? Why?

Until now, the "Download Open Data files" link from the footer only returned some components:

  • meetings (and its comments)
  • projects
  • proposals (and its comments)
  • results (and its comments)

With this change, we're also adding ParticpatorySpaces:

  • assemblies
  • conferences
  • participatory processes
  • initiatives

One important thing to notice is that as some spaces can have different publication scopes (as in "not published" or "private" or "private and transparent"), then we need to take that into account when downloading them.

Another thing to mention is that as the Serializer is used for the import/export features, I created a new method for this in the ParticipatorySpaceManifest called open_data_serializer. With this, we can differentiate between the attributes that need to be exported in each case.

📌 Related Issues

Testing

  1. Click on the "Download Open Data files" in the footer.
  2. See the flash message that is being generated.
  3. Click again, download the zip file
  4. Open it
  5. See that you now have the spaces
  6. Check that the spaces are not published/not private/if they are private they are also transparent. See acceptance criteria Add spaces to the CSV open data file  #13248.

📷 Screenshots

Footer with the link highlighted

Before

Files in the zip

After

Files in the zip

Modal window importing a participatory process CSV in LibreOffice Calc

♥️ Thank you!

github-actions[bot]
github-actions bot previously approved these changes Aug 27, 2024
github-actions[bot]
github-actions bot previously approved these changes Sep 3, 2024
github-actions[bot]
github-actions bot previously approved these changes Sep 4, 2024
github-actions[bot]
github-actions bot previously approved these changes Sep 4, 2024
github-actions[bot]
github-actions bot previously approved these changes Sep 13, 2024
@andreslucena
Copy link
Copy Markdown
Member Author

To be honest, i would go for the inheritance way of doing things because is very easy to forget adding attributes in additional files, and we may end up having an attribute in regular export side, but not in open data, vice versa, and may cause issues.

Ok, I agree with what you're saying.

I've done it in 2d6688b

This is ready for another round @alecslupu

@andreslucena andreslucena dismissed alecslupu’s stale review September 13, 2024 09:35

Changes were applied

Copy link
Copy Markdown
Contributor

@alecslupu alecslupu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andreslucena , can you take a look on the category and attachments fields, for some reason, the CSV export for those 2 fields are plain hashses ...

alecslupu.go.ro-open-data-assemblies.csv

github-actions[bot]
github-actions bot previously approved these changes Sep 16, 2024
github-actions[bot]
github-actions bot previously approved these changes Sep 16, 2024
@andreslucena
Copy link
Copy Markdown
Member Author

@andreslucena , can you take a look on the category and attachments fields, for some reason, the CSV export for those 2 fields are plain hashses ...

@alecslupu I decided to just remove them from the OpenData CSV as I don't like adding them on a column. I'm leaning in adding them to another file (i.e. alecslupu.go.ro-open-data-attachmnets.csv) but I'm not 100% convinced on that neither.

For the moment as I don't have a solution that I'm happy with, I think it's better to just remove them from the current PR. That was done in c02910e (I moved them to the other serializers that are used for the Import/Export feature in the admin panel, where this is not an issue as they're using JSON).

Can you check this PR again please? Thanks!

@andreslucena andreslucena dismissed alecslupu’s stale review September 16, 2024 12:54

Ready for another round

Copy link
Copy Markdown
Contributor

@alecslupu alecslupu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Assemblies, conferences and initiatives are ok. There is an issue with participatory_processes

github-actions[bot]
github-actions bot previously approved these changes Sep 16, 2024
Copy link
Copy Markdown
Contributor

@alecslupu alecslupu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👀

  • Assemblies ✔️
  • Conferences ✔️
  • Initiatives ✔️
  • Processes ✔️

Merging with failing pipeline on codecov. 2 Check Spelling pipelined skipped.

@alecslupu alecslupu added this to the 0.30.0 milestone Sep 17, 2024
@alecslupu alecslupu merged commit af37d31 into develop Sep 17, 2024
@alecslupu alecslupu deleted the feature/open-data-spaces branch September 17, 2024 09:52
greenwoodt pushed a commit to greenwoodt/decidim that referenced this pull request Sep 17, 2024
* Extract 'Download Open Data files' example to its own spec

* Add missing resources to spec

* Extract shared context file with common methods

* Add open data file for participatory processes

* Extract shared example file with includes/does not include examples

* Add open data file for assemblies

* Add open data file for conferences

* Fix exporting of the open data file

* Add ExportManifest#open_data_serializer to have more control in the serialized fields

* Fix rubocop offenses

* Workaround Bullet exceptions in CI for these examples

* Remove unecessary include in Conferences

* Add OpenDataAssemblySerializer

* Add OpenDataConferenceSerializer

* Add OpenDataInitiativeSerializer

* Fix comment for OpenDataParticipatoryProcessSerializer

* Add attachments to the open data serializers

* Add IDs to initiatives export

Co-authored-by: Alexandru Emil Lupu <contact@alecslupu.ro>

* Fix typo

Co-authored-by: Alexandru Emil Lupu <contact@alecslupu.ro>

* Fix fields in initiatives

- Replace id with reference (for consistency with other spaces' exports)
- Add missing fields from the export

* Introduce ParticipatorySpaceSerializer

Also remove some not needed attributes (such as
decidim_organization_id), and move others out of the OpenData
serializers (as weight).

* Add assembly type to Assembly serializers

* Fix typo

* Remove categories and attachments from OpenData serializers

* Move steps serialization from OpenDataParticipatoryProcessSerializer to ParticipatoryProcessSerializer

---------

Co-authored-by: Alexandru Emil Lupu <contact@alecslupu.ro>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

No open projects
Archived in project

Development

Successfully merging this pull request may close these issues.

Add spaces to the CSV open data file

3 participants