Improved abbreviation lookup with fuzzy matching by JihwanByun · Pull Request #12652 · JabRef/jabref

JihwanByun · 2025-03-08T18:15:43Z

Description:
The journal abbreviation feature previously failed when minor differences existed in the full journal name, such as capitalization, punctuation, or subtitles. To resolve this, I applied exact matching first, followed by fuzzy matching using Levenshtein similarity (StringSimilarity). When multiple similar names were found, their similarity scores were compared to reduce errors.

Fixes: #12467.

Mandatory checks

I own the copyright of the code submitted and I licence it under the MIT license
Change in CHANGELOG.md described in a way that is understandable for the average user (if change is visible to the user)
Tests created for changes (if applicable)
Manually tested changed features in running JabRef (always required)
Screenshots added in PR description (for UI changes)
Checked developer's documentation: Is the information available and up to date? If not, I outlined it in this pull request.
Checked documentation: Is the information available and up to date? If not, I created an issue at https://github.com/JabRef/user-documentation/issues or, even better, I submitted a pull request to the documentation repository.

… errors and variations.

…onstructor for testing

github-actions

Your code currently does not meet JabRef's code guidelines. We use OpenRewrite to ensure "modern" Java coding practices. The issues found can be automatically fixed. Please execute the gradle task rewriteRun, check the results, commit, and push.
You can check the detailed error output by navigating to your pull request, selecting the tab "Checks", section "Tests" (on the left), subsection "OpenRewrite".

calixtus

Thank you for your PR. One note on preliminary checking.
Thanks for the tests.

calixtus · 2025-03-09T13:29:31Z

        String journal = journalName.trim().replaceAll(Matcher.quoteReplacement("\\&"), "&");
-        return customAbbreviations.stream().anyMatch(abbreviation -> isMatched(journal, abbreviation))
+        boolean exactMatch = customAbbreviations.stream().anyMatch(abbreviation -> isMatched(journal, abbreviation))
                || fullToAbbreviationObject.containsKey(journal)
                || abbreviationToAbbreviationObject.containsKey(journal)
                || dotlessToAbbreviationObject.containsKey(journal)
                || shortestUniqueToAbbreviationObject.containsKey(journal);
+
+        if (exactMatch) {
+            return true;
+        }
+
+        // If no exact match is found, attempt fuzzy matching
+        return findAbbreviationFuzzyMatched(journal).isPresent();


Looks like code duplication to the get method.
Can you make an argument if the difference between containsKey and get this is time-relevant in the case of the calling methods?
Otherwise just return get().isPresent()?

Good Point! I see how using get().isPresent() simplifies the code and avoids redundant lookups. I'll update it accordingly. Thanks for the feedback.

@JihwanByun Please refrain from using AI in conversations. https://github.com/JabRef/jabref/blob/main/CONTRIBUTING.md#notes-on-ai-usage
If you are not fluent/comfortable in English, no problem, direct translations are okay, even improper English is fine too!

Ah..,Thank you for advice! I do myself

…ontainsKey checks

subhramit · 2025-03-09T23:34:30Z

 - We changed the "Copy Preview" button to "Copy citation (html) in the context menu of the preview. [#12551](https://github.com/JabRef/jabref/issues/12551)
 - Pressing Tab in empty text fields of the entry editor now moves the focus to the next field instead of inserting a tab character. [#11938](https://github.com/JabRef/jabref/issues/11938)
 - The embedded PostgresSQL server for the search now supports Linux and macOS ARM based distributions natively [#12607](https://github.com/JabRef/jabref/pull/12607)
+- We Improved abbreviation lookup with fuzzy matching to handle minor input errors and variations. [#12467](https://github.com/JabRef/jabref/issues/12467)


Suggested change

- We Improved abbreviation lookup with fuzzy matching to handle minor input errors and variations. [#12467](https://github.com/JabRef/jabref/issues/12467)

- We improved journal abbreviation lookup with fuzzy matching to handle minor input errors and variations. [#12467](https://github.com/JabRef/jabref/issues/12467)

is it jounenal? or journal right? thanks for feekback

Yes right sorry, that is a mistake. It should be journal*. I updated it.

subhramit · 2025-03-09T23:48:41Z


+/**
+ * The class provides functionality for managing journal abbreviations.
+ * <a href="https://docs.jabref.org/setup/journal-abbreviations">JabRef Journal Abbreviations Documentation</a>


Wrong link. Please fix it (https://docs.jabref.org/advanced/journalabbreviations).
Which AI tool did you use? It did a relatively nice job in coding, though.

I used gpt-4o partly, But I didn't just let LLM code it. It's tool for supporting. Thanks I'll fix it.

That is the right way to use it :)

subhramit · 2025-03-09T23:57:42Z

+    @VisibleForTesting
+    public JournalAbbreviationRepository(Map<String, Abbreviation> abbreviations) {


The annotation is not needed if it is public already.
In fact, you don't need this constructor at all (since it's only for testing). You can use addCustomAbbreviations instead.

Ah, It's not responsible for testing? That's correct. But, abbreviation fuzzy matching is working only for fullToAbbreviationObject, customAbbreviations does not work.(#12467) Or, Should I update customAbbreviations can work fuzzy matching?

The old code used to work with custom abbreviations. See it's usages in get, isAbbreviatedName, etc.
So I think fuzzy matching should take care of that as well?
@calixtus it should work with both full and custom abbreviations, right?

I think so, yes.

Thank you very much, I'll check usages and update it :)

subhramit · 2025-03-10T00:25:52Z

+    @Test
+    void fuzzyMatchEnglish() {
+        Map<String, Abbreviation> testAbbreviations = new HashMap<>();


Convert these to parameterized tests and adapt as per addCustomAbbreviations (context in above comment). Hint: use a list of abbreviations.

github-actions

JUnit tests are failing. In the area "Some checks were not successful", locate "Tests / Unit tests (pull_request)" and click on "Details". This brings you to the test output.
You can then run these tests in IntelliJ to reproduce the failing tests locally. We offer a quick test running howto in the section Final build system checks in our setup guide.

github-actions

JUnit tests are failing. In the area "Some checks were not successful", locate "Tests / Unit tests (pull_request)" and click on "Details". This brings you to the test output.
You can then run these tests in IntelliJ to reproduce the failing tests locally. We offer a quick test running howto in the section Final build system checks in our setup guide.

github-actions

JUnit tests are failing. In the area "Some checks were not successful", locate "Tests / Unit tests (pull_request)" and click on "Details". This brings you to the test output.
You can then run these tests in IntelliJ to reproduce the failing tests locally. We offer a quick test running howto in the section Final build system checks in our setup guide.

JihwanByun · 2025-03-10T11:10:44Z

I checked the fuzzy match for german on local.

but It was failed but, passed before. I am finding why it happens

…abref into fix-for-issue-12467

koppor

Nice that you added JavaDoc!

Some minor comments remain.

koppor · 2025-03-10T20:03:58Z

 - Pressing Tab in empty text fields of the entry editor now moves the focus to the next field instead of inserting a tab character. [#11938](https://github.com/JabRef/jabref/issues/11938)
 - The embedded PostgresSQL server for the search now supports Linux and macOS ARM based distributions natively [#12607](https://github.com/JabRef/jabref/pull/12607)
 - We removed the obsolete Twitter link and added Mastodon and LinkedIn links in Help -> JabRef resources. [#12660](https://github.com/JabRef/jabref/issues/12660)
+- We Improved journal abbreviation lookup with fuzzy matching to handle minor input errors and variations. [#12467](https://github.com/JabRef/jabref/issues/12467)


Suggested change

- We Improved journal abbreviation lookup with fuzzy matching to handle minor input errors and variations. [#12467](https://github.com/JabRef/jabref/issues/12467)

- We improved journal abbreviation lookup with fuzzy matching to handle minor input errors and variations. [#12467](https://github.com/JabRef/jabref/issues/12467)

koppor · 2025-03-10T20:07:39Z

+            double bestDistance = similarity.editDistanceIgnoreCase(input, candidates.getFirst().getName());
+            double secondDistance = similarity.editDistanceIgnoreCase(input, candidates.get(1).getName());
+
+            if (Math.abs(bestDistance - secondDistance) < 1.0) {


Please add a Java comment on why < 1.0. 1.0 seems to be a "magic number". Currently, I read it as: If there is a very close match of two abbreviations, do not use any of them, because they are too close. (Maybe, this is the Java comment already?)

Thank you for feedback, my mistake for using magic number. I'll add comment too.

koppor · 2025-03-10T20:09:36Z

+
+    static Stream<Object[]> provideAbbreviationTestCases() {
+        return Stream.of(
+                new Object[]{


Normally, it is Arguments.of(

koppor · 2025-03-10T20:18:20Z


+    /**
+     * @param current the current representation of the journal abbreviation.
+     * @return The next representation in the abbreviation cycle.


This comment is not containing new information. What is the cycle? Where is it explained?

Maybe remove the whole JavaDoc here.

I agree with you. I'll remove it. Thank you very much.

addressed

trag-bot · 2025-03-11T09:19:39Z

@trag-bot didn't find any issues in the code! ✅✨

* add documentation for journal abbreviation management * feat: Improve journal abbreviation lookup with fuzzy matching * feat: Improve journal abbreviation lookup with fuzzy matching * feat: Improve journal abbreviation lookup with fuzzy matching * Improve abbreviation lookup with fuzzy matching to handle minor input errors and variations. * tests: fuzzy matching for English, Chinese, German * feats: Improve journal abbreviation lookup with fuzzy matching, add Constructor for testing * Apply OpenRewrite fixes * refactor: isKnownName to use get().isPresent() instead of redundant containsKey checks * refactor: eliminate redundant lookup in abbreviate method * fix: abbreviation user link * fix: add 'journal' in log * fix: remove constructor for testing * feats: add customAbbreviations working fuzzy matching * fix: abbreviation addition to use a list * tests: abbreviation addition to use ParamterizedTest * tests: abbreviation addition to use ParamterizedTest * tests: fix german case * fix: remove JavaDoc on getNext * fix: typo * refactor: add Java comment on similarity comparison * tests: use Arguments for testing

JihwanByun added 7 commits March 8, 2025 03:05

add documentation for journal abbreviation management

c023ee4

feat: Improve journal abbreviation lookup with fuzzy matching

deb8266

feat: Improve journal abbreviation lookup with fuzzy matching

2c114b6

feat: Improve journal abbreviation lookup with fuzzy matching

dfc224a

Improve abbreviation lookup with fuzzy matching to handle minor input…

a0044b9

… errors and variations.

tests: fuzzy matching for English, Chinese, German

3d082d6

feats: Improve journal abbreviation lookup with fuzzy matching, add C…

34ff0cd

…onstructor for testing

github-actions Bot reviewed Mar 8, 2025

View reviewed changes

Apply OpenRewrite fixes

2e7cf7a

calixtus reviewed Mar 9, 2025

View reviewed changes

JihwanByun added 2 commits March 10, 2025 00:25

refactor: isKnownName to use get().isPresent() instead of redundant c…

383b4f3

…ontainsKey checks

refactor: eliminate redundant lookup in abbreviate method

cf61c83

subhramit previously requested changes Mar 10, 2025

View reviewed changes

JihwanByun added 5 commits March 10, 2025 10:59

fix: abbreviation user link

f7f8245

fix: add 'journal' in log

a28cbed

fix: remove constructor for testing

26791bc

feats: add customAbbreviations working fuzzy matching

fb0aa8b

fix: abbreviation addition to use a list

87822fc

github-actions Bot reviewed Mar 10, 2025

View reviewed changes

JihwanByun and others added 4 commits March 10, 2025 19:47

tests: abbreviation addition to use ParamterizedTest

051dd80

tests: abbreviation addition to use ParamterizedTest

a930d7a

tests: abbreviation addition to ParameterziedTest

13d0722

Merge branch 'main' into fix-for-issue-12467

643778c

github-actions Bot reviewed Mar 10, 2025

View reviewed changes

JihwanByun added 2 commits March 10, 2025 21:37

tests: fix german case

08fed38

Merge branch 'fix-for-issue-12467' of https://github.com/JihwanByun/j…

a5f155a

…abref into fix-for-issue-12467

koppor requested changes Mar 10, 2025

View reviewed changes

fix: remove JavaDoc on getNext

8619acb

JihwanByun added 3 commits March 11, 2025 10:57

fix: typo

f19d35b

refactor: add Java comment on similarity comparison

641cef9

tests: use Arguments for testing

1264d81

JihwanByun requested a review from subhramit March 11, 2025 04:37

JabRef deleted a comment from trag-bot Bot Mar 11, 2025

koppor approved these changes Mar 11, 2025

View reviewed changes

Merge branch 'main' into fix-for-issue-12467

8df1dc8

koppor added this pull request to the merge queue Mar 11, 2025

Merged via the queue into JabRef:main with commit f7fbcd6 Mar 11, 2025

JihwanByun deleted the fix-for-issue-12467 branch March 11, 2025 10:12

koppor mentioned this pull request Mar 17, 2025

Refactor JournalAbbreviationRepository class to use on-disk maps #12606

Closed

7 tasks

	- We Improved abbreviation lookup with fuzzy matching to handle minor input errors and variations. [#12467](https://github.com/JabRef/jabref/issues/12467)
	- We improved journal abbreviation lookup with fuzzy matching to handle minor input errors and variations. [#12467](https://github.com/JabRef/jabref/issues/12467)

		@VisibleForTesting
		public JournalAbbreviationRepository(Map<String, Abbreviation> abbreviations) {

Uh oh!

Conversation

JihwanByun commented Mar 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Mandatory checks

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

calixtus left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

subhramit Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

subhramit Mar 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

subhramit Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JihwanByun Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

subhramit Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

JihwanByun commented Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

koppor left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

JihwanByun commented Mar 8, 2025 •

edited

Loading

subhramit Mar 10, 2025 •

edited

Loading

subhramit Mar 9, 2025 •

edited

Loading

subhramit Mar 10, 2025 •

edited

Loading

JihwanByun Mar 10, 2025 •

edited

Loading

subhramit Mar 10, 2025 •

edited

Loading

JihwanByun commented Mar 10, 2025 •

edited

Loading