Skip to content

CSL4LibreOffice - B [GSoC '24]#11521

Merged
Siedlerchr merged 111 commits into
JabRef:mainfrom
subhramit:oo-architecture-2
Jul 25, 2024
Merged

CSL4LibreOffice - B [GSoC '24]#11521
Siedlerchr merged 111 commits into
JabRef:mainfrom
subhramit:oo-architecture-2

Conversation

@subhramit

@subhramit subhramit commented Jul 21, 2024

Copy link
Copy Markdown
Member

Major updates for CSL-OO/LO integration

[PR - B of the GSoC '24 CSL4LibreOffice Project]
Follow-up to #11477

Summary:

  • Refactored existing code, revamped and unified the backend architecture for style type selection
  • Implemented preference storage for selected style (JStyle/CSL style) to make it persist across JabRef sessions
  • Implemented "auto-scroll to and highlight" feature for currently selected CSL style when opening/reopening the "Select style" dialog
  • Implemented a label in the dialog to keep the user informed about the currently set JStyle/CSL style
  • Miscellaneous bug fixes and added fail-safety for edge cases (such as missing styles, empty selections, etc.)

Additions/Improvements:

Follow-up: #11577

Mandatory checks

  • Change in CHANGELOG.md described in a way that is understandable for the average user (if applicable)
  • Tests created for changes (if applicable)
  • Manually tested changed features in running JabRef (always required)
  • Screenshots added in PR description (for UI changes)
  • Checked developer's documentation: Is the information available and up to date? If not, I outlined it in this pull request.
  • Checked documentation: Is the information available and up to date? If not, I created an issue at https://github.com/JabRef/user-documentation/issues or, even better, I submitted a pull request to the documentation repository.

subhramit added 30 commits May 31, 2024 20:24
@Siedlerchr Siedlerchr added this pull request to the merge queue Jul 25, 2024
Merged via the queue into JabRef:main with commit 771c4cd Jul 25, 2024
@Siedlerchr Siedlerchr deleted the oo-architecture-2 branch July 25, 2024 18:30
@subhramit

subhramit commented Jul 25, 2024

Copy link
Copy Markdown
Member Author

Follow-up: #11535

@LoayGhreeb

Copy link
Copy Markdown
Member

@Siedlerchr, @subhramit please fix this exception, JabRef is not starting 😓

ERROR: Unexpected exception: java.lang.NullPointerException: Cannot invoke "java.io.InputStream.readAllBytes()" because "source" is null
	at org.jabref@100.0.0/org.jabref.logic.citationstyle.CitationStyle.createCitationStyleFromSource(CitationStyle.java:67)
	at org.jabref@100.0.0/org.jabref.logic.citationstyle.CitationStyle.createCitationStyleFromFile(CitationStyle.java:121)
	at org.jabref@100.0.0/org.jabref.logic.citationstyle.CitationStyle.getDefault(CitationStyle.java:136)
	at org.jabref@100.0.0/org.jabref.preferences.JabRefPreferences.<init>(JabRefPreferences.java:728)
	at org.jabref@100.0.0/org.jabref.preferences.JabRefPreferences.getInstance(JabRefPreferences.java:875)
	at org.jabref@100.0.0/org.jabref.Launcher.main(Launcher.java:64)

IntelliJ:
image

@LoayGhreeb

Copy link
Copy Markdown
Member

Maybe this is because I didn't include the submodules in my clone?

@Siedlerchr

Copy link
Copy Markdown
Member

Seems like, you need to run git submodule init

But we should nonetheless catch this

@subhramit

Copy link
Copy Markdown
Member Author

@Siedlerchr, @subhramit please fix this exception, JabRef is not starting 😓

ERROR: Unexpected exception: java.lang.NullPointerException: Cannot invoke "java.io.InputStream.readAllBytes()" because "source" is null
	at org.jabref@100.0.0/org.jabref.logic.citationstyle.CitationStyle.createCitationStyleFromSource(CitationStyle.java:67)
	at org.jabref@100.0.0/org.jabref.logic.citationstyle.CitationStyle.createCitationStyleFromFile(CitationStyle.java:121)
	at org.jabref@100.0.0/org.jabref.logic.citationstyle.CitationStyle.getDefault(CitationStyle.java:136)
	at org.jabref@100.0.0/org.jabref.preferences.JabRefPreferences.<init>(JabRefPreferences.java:728)
	at org.jabref@100.0.0/org.jabref.preferences.JabRefPreferences.getInstance(JabRefPreferences.java:875)
	at org.jabref@100.0.0/org.jabref.Launcher.main(Launcher.java:64)

Fixed in #11536, thanks for pointing out!

@subhramit

Copy link
Copy Markdown
Member Author

@antalk2 Hey, are you still on GitHub by any chance?

@antalk2

antalk2 commented Aug 3, 2024

Copy link
Copy Markdown
Contributor

yes

@subhramit

subhramit commented Aug 3, 2024

Copy link
Copy Markdown
Member Author

yes

@antalk2 Hey, hey! It's so good to see your reply.
I am an undergraduate student currently contributing for GSoC to JabRef. My project involves work with the OO/LO integration so that CSL styles are now supported.
As you can see, the CSL Styles work now, but I am stuck in a major roadblock trying to get the reference numbers correct for numeric type citations. I tried to do it using reference marks (I know you have already done it for JStyles but I had difficulty adapting them so tried to do them independently from scratch the way Zotero has been doing it): https://github.com/zotero/zotero-libreoffice-integration/blob/main/build/source/org/zotero/integration/ooo/comp/ReferenceMark.java
The current problems that I face are (all pertaining to CSL):

  1. The reference marks don't wrap around the citation text, but are inserted after the text.
  2. [Major] On restarting JabRef, the numbering starts from 1 again for the same document. It is as if I am not reading the reference marks correctly when a connection is made, and getting the respective citation numbers. I tried to do lazy instantiation and everything, just isn't working. Could there be a problem with how I wrote readExistingMarks() in MarkManager?

If you get even a bit of time, could you take a look at subhramit#12 and let me know what I'm doing wrong?
CC: @Siedlerchr

@antalk2

antalk2 commented Aug 3, 2024

Copy link
Copy Markdown
Contributor

Question 1: reference marks don't wrap around the citation text

  1. The reference marks don't wrap around the citation text, but are
    inserted after the text.

In short: in src/main/java/org/jabref/logic/openoffice/oocsltext/CSLCitationOOAdapter.java

   void insertCitation(XTextDocument doc, XTextCursor cursor, BibEntry entry, String citation)

seems to create an empty reference mark, then insert text into it.
The opposite order: insert text to cursor and create a reference mark around the inserted text is probably easier.

The call ReferenceMark mark = markManager.createReferenceMark(entry, "ReferenceMark");
seems to ignore the XTextCursor cursor passed in to insertCitation.

Background

Both zotero (https://github.com/zotero/zotero-libreoffice-integration/blob/main/build/source/org/zotero/integration/ooo/comp/ReferenceMark.java)
and https://github.com/JabRef/jabref/blob/main/src/main/java/org/jabref/logic/openoffice/backend/NamedRangeReferenceMark.java does some "dancing"
with the cursors, inserting and removing paragraph marks and some other characters.

It has two reasons.

  1. The UNO API presents the text and markup as a flat list of
    locations, not a tree of XML-ish nodes. In the flat
    representation an empty cursor points to between two characters.
    An empty text range also points to between two characters.
    The order of a cursor, range-start, range-end is not clear:
    the cursor could be before, inside or after the empty text range.
    Avoiding empty text ranges helps in controlling this order:
    if there is at least one character between to points their order is well defined.

    Attempt to do this seems to be missing from
    src/main/java/org/jabref/logic/openoffice/oocsltext/ReferenceMark.java

  2. Controlling where text formatting is "inherited" from when inserting text.
    Text insertion usually inherits formatting from the left (previous character),
    except when inserting at the start of a paragraph (then from the right), and
    when inserting into an empty paragraph (probably from the paragraph mark on the right)

My recollection of what did I try and why.

I was first trying to control (1.) by keeping REFERENCE_MARK_LEFT_BRACKET and REFERENCE_MARK_RIGHT_BRACKET within the reference marks textrange.
Moving over these ZERO_WIDTH_SPACE characters with pushing the cursor arrow keys shows looks like the cursor stopping there, so I tried to minimize the use of these brackets by removing them if
there were enough characters between them.
I was trying to stabilize the reference marks (not deleting and recreating them). The user can
still remove these brackets and the text inside, leaving us an empty reference mark that needs handling.
Later I found that (2.) still needs handling (it probably happened in recreating or refilling the
References) and I started to inject another ZERO_WIDTH_SPACE with the intent to hold (keep) the text formatting applied.
This would allow the user to apply some (probably "direct") formatting to the
range (in particular including the initial ZERO_WIDTH_SPACE). When
we refresh the visible content of the reference mark, the new
content would inherit the formatting from this
ZERO_WIDTH_SPACE. Unfortunately this ZERO_WIDTH_SPACE is not
normally visible to the user, thus hard to reliably include in the range to be formatted "by hand".

Question 2: readExistingMarks

  1. [Major] On restarting JabRef, the numbering starts from 1 again
    for the same document. It is as if I am not reading the
    reference marks correctly when a connection is made, and getting
    the respective citation numbers. I tried to do lazy
    instantiation and everything, just isn't working. Could there be
    a problem with how I wrote readExistingMarks() in MarkManager?

On restarting JabRef, the numbering starts from 1 again for the same document.
https://github.com/subhramit/jabref/pull/12/files#

I do not remember having a difficulty in getting and parsing reference mark names.

(Probably not relevant her: If I remember correctly, attempt to create a bookmark with a name already in use results
in a bookmark with a name different from what was requested (some numbering suffix added).
An attempt to create a reference mark with a name already in use results in two reference marks
with the same name, but name-based lookup will only yield one of these.)

If I understand correctly, you encode citationNumber in

public ReferenceMark createReferenceMark(BibEntry entry, String fieldType) 

like this

 String name = CSLCitationOOAdapter.PREFIXES[0] + citationKey + " RND" + citationNumber;

This would look like

"JABREF_{citationKey} RND{citationNumber}"

Parsing:

 private void updateCitationInfo(String name) {
    String[] parts = name.split(" ");
    if (parts.length >= 3) {

I do not see the 3 parts in the

"JABREF_{citationKey} RND{citationNumber}"

format. Does citationKey contain a space? Or a space is missing after "RND"?

@subhramit

subhramit commented Aug 3, 2024

Copy link
Copy Markdown
Member Author

Thank you for your invaluable guidance. I will try to tweak the implementation as per your comments.

Does citationKey contain a space? Or a space is missing after "RND"?

Good catch, I was missing appending a random string as the third part for uniqueness.

@antalk2

antalk2 commented Aug 3, 2024

Copy link
Copy Markdown
Contributor

The code in

 src/main/java/org/jabref/logic/openoffice/backend/NamedRangeManagerReferenceMark.java
 src/main/java/org/jabref/logic/openoffice/backend/NamedRangeReferenceMark.java

seems not to depend on details of what is encoded in the reference mark name and the format of the text in the reference mark.

public List<String> getUsedNames(XTextDocument doc)

does not even filter the reference mark names (for jabref-specific pefixes)

public XTextCursor getFillCursor(XTextDocument doc)

getFillCursor should provide an XTextCursor strictly within the reference mark, (surrounded by REFERENCE_MARK_LEFT_BRACKET and REFERENCE_MARK_RIGHT_BRACKET), and

public void cleanFillCursor(XTextDocument doc)

public void cleanFillCursor(XTextDocument doc) should clean up the brackets after modifying the content.

@subhramit

Copy link
Copy Markdown
Member Author

The code in

 src/main/java/org/jabref/logic/openoffice/backend/NamedRangeManagerReferenceMark.java
 src/main/java/org/jabref/logic/openoffice/backend/NamedRangeReferenceMark.java

seems not to depend on details of what is encoded in the reference mark name and the format of the text in the reference mark.

public List<String> getUsedNames(XTextDocument doc)

does not even filter the reference mark names (for jabref-specific pefixes)

public XTextCursor getFillCursor(XTextDocument doc)

getFillCursor should provide an XTextCursor strictly within the reference mark, (surrounded by REFERENCE_MARK_LEFT_BRACKET and REFERENCE_MARK_RIGHT_BRACKET), and

public void cleanFillCursor(XTextDocument doc)

public void cleanFillCursor(XTextDocument doc) should clean up the brackets after modifying the content.

Okay, will look into these.
Update: I finally got the parsing correct, thanks to your hints. The marks are being read on restart now :)
The text wrap seems to be tricky, working on that currently.

@koppor

koppor commented Aug 3, 2024

Copy link
Copy Markdown
Member

Maybe this is because I didn't include the submodules in my clone?

Sub modules are "just" in the file .gitmodules (OK and some git internal object storage: https://stackoverflow.com/a/5033973/873282). One cannot omit a single file in a clone. As Chris pointed out: The initialization was missing 😅

@subhramit subhramit mentioned this pull request Aug 4, 2024
6 tasks
@subhramit

Copy link
Copy Markdown
Member Author

Follow-up: #11535

Follow up: #11577

@subhramit

subhramit commented Sep 9, 2024

Copy link
Copy Markdown
Member Author

Follow-up: #11712

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants