CSL4LibreOffice - C [GSoC '24] by subhramit · Pull Request #11577 · JabRef/jabref

subhramit · 2024-08-04T23:42:47Z

New features for CSL-OO/LO integration

[PR - C of the GSoC '24 CSL4LibreOffice Project]
Follow-up to #11521

Summary:

Implemented "reference marks" to fix resetting index issue for numeric CSL styles
Added feature to auto-generate and sync a bibliography section (list of references) in the LibreOffice document based on existing citations
Added support for in-text citations and empty citations for CSL styles
Added support for underlined CSL styles
Added support for alphanumeric CSL styles
Implemented "smart spaces" feature for CSL citation insertion
Miscellaneous refinement and additions (e.g. new icon for "add bibliography", fixed extra newlines in case of numeric styles, disabled irrelevant buttons when CSL style is selected, upgraded to StAX parsing for information from .csl files etc.)

Background:

Problem 1: The behavior of in-text citations was independent of bibliographic citations, even if they corresponded to the same entry. For example, they could have different numbering (for numeric styles).
Problem 2: As mentioned by @Siedlerchr in CSL4LibreOffice - B [GSoC '24] #11521 (comment), a major inconvenience while citing entries independently was that the numbering index (for numeric citation styles) would get reset to 1 every time.
Problem 3: We first tried ways to persist consistent numbering within an OO session, the numbering would get reset again on restarting JabRef or reconnecting/restarting the LibreOffice document instance.
Problem 4: There was no availability of a "make bibliography" feature that would automatically read the in-text citations in the document and generate a bibliography consisting of corresponding bibliographic citations.

This PR addresses these problems by introducing two new features to JabRef's OO/LO integration - (A) Reference Marks and (B) Auto-generation of bibliography.

Demos:

We can now seamlessly insert citation entries (both independently or in a grouped fashion) without having to worry about maintaining the numbering across citations of any type. The numbers are maintained consistently using Reference Marks attached to citation entries.
We can now auto-insert bibliography into the document based on our in-text citations by pressing the loop arrow ("Make bibliography") button in the OpenOffice panel.

testVideo.mp4

The citation numbers are now maintained consistently across JabRef and OO/LO sessions by reading existing Reference Marks in the document.

test2.mp4

The implementation of the internal format of Reference Marks is very similar to that of Zotero, which consists of "JABREF_{citationKey} CID_{citationNumebr} {UID}".

Special thanks to @antalk2, who had worked on similar problem statements for JStyles for JabRef's OO/LO integration, for his help in guiding and making this PR possible.

Follow-up: #11636, #11712

Mandatory checks

~~Change in CHANGELOG.md described in a way that is understandable for the average user (if applicable)~~
~~Tests created for changes (if applicable)~~
Manually tested changed features in running JabRef (always required)
Screenshots added in PR description (for UI changes)
Checked developer's documentation: Is the information available and up to date? If not, I outlined it in this pull request.
Checked documentation: Is the information available and up to date? If not, I created an issue at https://github.com/JabRef/user-documentation/issues or, even better, I submitted a pull request to the documentation repository.

…liography

koppor · 2024-08-12T20:49:50Z

Next time, please do look into the small red crosses on GitHub!

Fixed by @Siedlerchr #11616 (needed to hurry up, because our tests on main branch were red after the merge)

subhramit · 2024-08-12T21:01:16Z

Next time, please do look into the small red crosses on GitHub!

Fixed by @Siedlerchr #11616 (needed to hurry up, because our tests on main branch were red after the merge)

For some reason:

Initially, for the last 30 commits at least, only fetcher tests were failing
First time the merge failed, only fetcher and checkstyle were red.
Second time the merge failed, only fetcher test was red (even though architecture and checkstyle error still existed).
At no point previously the unit tests were red. Only fetcher and JDK deployment tests started failing the last time.
Finally unit tests panicked after merge.

subhramit · 2024-08-12T21:08:42Z

Look at this:

(from cc36acc or any commit above).

subhramit · 2024-08-12T22:31:34Z

Look at this:

(from cc36acc or any commit above).

Delayed panic of tests fixed in #11615

koppor · 2024-08-13T06:50:34Z

At no point previously the unit tests were red. Only fetcher and JDK deployment tests started failing the last time.

My screenshot is from this PR and shows the red cross.

It was the last merge queue attempt.

Fortunately, triggered by this, we fixed our CI and itwill.be more easy to see the errors. 🎉

Finally unit tests panicked after merge.

The PR was merged without using the merge queue. Thus, the (failing) tests by the merge queue were skipped.

koppor · 2024-08-13T06:54:04Z

Now, we all learned about the difference of "pull_request" and "pull_request_target" the long way 😅

subhramit · 2024-09-09T11:14:35Z

Major follow-up PR (apart from PR-D): #11712
[Adds number re-distribution for citations].

koppor · 2025-03-25T15:32:21Z

Decision to leave the bibliography title format

@subhramit I hope that when a user changes the text / formatting manually, this change is NOT overwritten, but kept?

Typically, paper templates provide formatting guidelines and names to headings.

subhramit · 2025-03-25T15:45:30Z

I hope that when a user changes the text / formatting manually, this change is NOT overwritten, but kept?

If by manually, you mean without JabRef, then unfortunately the entire section is detected and re-generated, and that I don't know if it is possible to detect and avoid.

If you are referring to PR #12784:
This gives the user flexibility to set the text (any) and formatting (as per popular formats) if it coincides with the templates you are talking about. If the formatting is not in the list, they can always overwrite it manually at the end of their work - which is very doable. Point of the PR was to enable JabRef reduce the user's effort a bit more - earlier it was more disadvantageous as both the text and the format were hardcoded, and the user would have to manually change both of them at the end anyway (if other text/formats were needed).

koppor · 2025-03-25T15:50:41Z

If by manually, you mean without JabRef, then unfortunately the entire section is detected and re-generated, and that I don't know if it is possible to detect and avoid.

Yes.

Before regeneration: Search for heading, store attributes (style, text)
Regenerate
Restore style from step 1.

But the solution of #12784 also seems to be OK.

subhramit · 2025-03-25T15:59:04Z

Before regeneration: Search for heading, store attributes (style, text)

Regenerate

Restore style from step 1.

Difficult task development-wise using UNO (and would add more lag in auto-refresh mode), compared to tiny effort by user if they just change at the end...
Can of worms:
If in future we have unified style-switching across style families, this will get even more complicated (e.g. CSL -> JStyle: Respect what's specified in the .jstyle file or prioritize user's manual changes and conditionally disable those properties in the jstyle, and so on - style-dependent implementations would be needed for a tiny thing).

subhramit · 2025-03-25T16:09:11Z

If in future we have unified type-switching across style families, this will get even more complicated (e.g. CSL -> JStyle: Respect what's specified in the .jstyle file or prioritize user's manual changes and conditionally disable those properties in the jstyle, and so on - style-dependent implementations would be needed for a tiny thing).

I think I just invited an additional ADR for whoever works on this. 😄

subhramit added 30 commits August 3, 2024 05:01

Handle numbers for one instance

42d2a8d

Use reference marks

caedc3a

Rehauled implementation

bf9d39e

Remove comment

fa39637

Revert removals

bb441b4

Revert removals

649bcc6

Revert removals

9c47086

Cover edge cases of numeric citation types

929cf1d

Cover edge cases of numeric citation types

89211c3

Better handling of numeric styles

1e4d5a9

Format doc

10bf9e0

Move isNumericStyle method to CitationStyle core

3d6c313

Log

2a24cfc

Gosh, this finally works.

1475130

First Implementation - Make bibliography

c9fe7a2

Better implementation

6643bfd

Remove duplicate notification

05a50fc

Penultimate - finishing reference section, fix formatting

c1f5c6a

Fix a-tag link text

93b153e

Rename functions: less confusing names

0750f56

Fixing in-text groups - phase 1

38c4731

in-text citations fix phase 2

a04d826

Broken implementation for future reference

622fda6

Working model

f723bf8

Merge resolve

1802ccb

Rename: CSL Reference Mark Model & Handler files

d105eaa

Merge branch 'sync-numbers' into make-bibliography

47b5bda

Merge, order of imports

e8def54

Fix: Local storage of last citation number

38447c7

Remove unused initializer

439e2d4

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 12, 2024

Merge branch 'main' into make-bibliography

30a19b2

subhramit enabled auto-merge August 12, 2024 20:10

Siedlerchr previously approved these changes Aug 12, 2024

View reviewed changes

subhramit added 2 commits August 13, 2024 01:46

Fix checkstyle

72e39b7

Merge remote-tracking branch 'origin/make-bibliography' into make-bib…

cc36acc

…liography

subhramit dismissed Siedlerchr’s stale review via cc36acc August 12, 2024 20:17

Siedlerchr approved these changes Aug 12, 2024

View reviewed changes

Siedlerchr disabled auto-merge August 12, 2024 20:29

Siedlerchr merged commit 8bf6f05 into JabRef:main Aug 12, 2024

Siedlerchr deleted the make-bibliography branch August 12, 2024 20:29

This was referenced Aug 14, 2024

Update JabRef material icons pack #11622

Merged

CSL4LibreOffice - D [GSoC '24] #11636

Merged

This was referenced Aug 24, 2024

fix tests #11616

Merged

Auto-sync CSL Bibliography feature #11683

Merged

Fix CSL bibliography ordering for non-numeric styles #11685

Merged

subhramit mentioned this pull request Sep 9, 2024

CSL4LibreOffice - E [Add re-distribution of numeric CSL citations] #11712

Merged

7 tasks

subhramit mentioned this pull request Mar 25, 2025

Functionality to modify CSL bibliography title and format #12784

Merged

7 tasks

Uh oh!

Conversation

subhramit commented Aug 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

New features for CSL-OO/LO integration

Summary:

Background:

Demos:

Mandatory checks

Uh oh!

Uh oh!

koppor commented Aug 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

subhramit commented Aug 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

subhramit commented Aug 12, 2024

Uh oh!

subhramit commented Aug 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

koppor commented Aug 13, 2024

Uh oh!

koppor commented Aug 13, 2024

Uh oh!

subhramit commented Sep 9, 2024

Uh oh!

koppor commented Mar 25, 2025

Uh oh!

subhramit commented Mar 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

koppor commented Mar 25, 2025

Uh oh!

subhramit commented Mar 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

subhramit commented Mar 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

subhramit commented Aug 4, 2024 •

edited

Loading

koppor commented Aug 12, 2024 •

edited

Loading

subhramit commented Aug 12, 2024 •

edited

Loading

subhramit commented Aug 12, 2024 •

edited

Loading

subhramit commented Mar 25, 2025 •

edited

Loading

subhramit commented Mar 25, 2025 •

edited

Loading

subhramit commented Mar 25, 2025 •

edited

Loading