[WIP] Fix change on disk detection by koppor · Pull Request #5669 · JabRef/jabref

koppor · 2019-11-24T11:11:21Z

This will solve the whole change-on-disk-dection things similar as conflicts are solved for a shared-database. This PR builds on #5640 and is an alternative to #5665.

Implementation motivation

In 2016, I designed the database synchronization feature (https://docs.jabref.org/collaborative-work/sqldatabase), which I briefly described at #2866 (comment):

Multiple users should be supported "seamlessly"
If user A and B work on the bibfile, a modification by user A on a bibtex entry B1 should not trigger any "changes" dialog if user B did not do any changes.
A "Merge Entries" dialog should popup if users A and B modified the same entry.

How this works can be tried out using the Shared SQL Database Feature of JabRef, where @obraliar and me worked hard to have a very nice user experience.

Technically, this is not that easy to achieve. However, users will be happy, if it "just works".

At each instance of JabRef:

Local state L is known
Remote state R is known
In case an update is made remotely, the diff can be applied to R.
So R' is known. Based on R and R', the entries E where a change was made, are known.
Based on E, changes can be applied to L.
If in L the entry was not modified: just apply the update.
If in L the entry was modified: Start the merge entries dialog.

Implementation idea

Current implementation state

TODO:

Add tests (multi-threading: update file in parallel, ... !)
saved flag local in the update monitor
Work on work on BibdatabaseDiff only
Trigger the merge entries popup

Optional:

Instead of the merge entries dialog, the changes should be displayed
Add a button to the change display panel to open the merge entries dialog, because the merge entries dialog has a visual diff instead of a text-only description.

DONE:

After a synchronization is triggered: the referenceFile is updated with the current database
Synchronize change detection and file writing: should not happen in parallel
Use !Thread.currentThread().isInterrupted() instead of true in endless loop in thread

Change in CHANGELOG.md described (if applicable)
Tests created for changes (if applicable)
Manually tested changed features in running JabRef (always required)
Screenshots added in PR description (for bigger UI changes)
Checked documentation: Is the information available and up to date? If not: Issue created at https://github.com/JabRef/user-documentation/issues.

- Add tests - Guard AtomicFileWriter with a try-with-resources during save - use Objects.requireNonNull in DefaultFileUpdateMontitor - add comment - some auto-formattings - Fix typo in comment

TODO: - Add test - saved flag local in the update monitor DONE: - After a synchronization is triggered: the referenceFile is updated with the current database - Synchronize change detection and file writing: should not happen in parallel - Use !Thread.currentThread().isInterrupted() instead of true in endless loop in thread

Siedlerchr · 2019-11-24T11:25:17Z

        try (WatchService watcher = FileSystems.getDefault().newWatchService()) {
            this.watcher = watcher;
-            while (true) {
+            while (!Thread.currentThread().isInterrupted()) {


Use Thread.onSpinWait in addition to give an optimization hint for the cpu
https://jaxenter.de/java-9-jep-285-spin-wait-loops-65884

tobiasdiez · 2019-11-24T12:30:20Z

The biggest problem I see is to have a robust 3-way merge, that takes all the changes, applies some automatically and only reports conflicts. The problem is that the bibtexkey can change, so that you don't have a definitive way of telling which entries in the 3 databases correspond to each other (e.g. the user renames a key, and creates a new entry with the old key). For shared libraries, you solved this by introducing a unique identifier for each entry...

Please don't forget that there are more than entries that can change (groups, strings, meta data, ...).

So how should we proceed? I agree that this PR is a better alternative to #5665 but there is still a lot of work required to make it work. Should we for now merge #5665 for now and then revert it after the 5.0 release?

koppor · 2019-12-01T22:28:46Z

To prevent issues such as #4877 i added synchronized at some places. Maybe, this could be a quick fix to solve that issue?

Maybe, having a working with maximum job count of 1 for saving could help. Refs #5680 (review)

koppor · 2020-07-27T22:05:37Z

Will take longer to work on this. Therefore moved to JabRef#442.

12dbba5 Create tyndale-bulletin (#5673) b0746db Create Engineered Regeneration (#5682) e38b953 wikipedia citation template (#5662) 5e7f731 Create early-music-history.csl (#5679) 86443f3 Create zeitschrift-fur-politik.csl (#5676) 68f1996 Create annals-of-work-exposures-and-health.csl (#5666) 1ba9dc6 Create brazilian-journal-of-psychiatry.csl (#5672) 438f92c fix error for speech in ama styles (#5693) 7a0c2d3 set initialize-with-hyphen to false (#5689) 3bd2765 Update emu-austral-ornithology.csl (#5671) 31492b2 fix various errors in natura-croatica.csl (#5687) 94d6b23 Update iso690-author-date-cs.csl (#5677) 5d017da minor update on the "Haute école de gestion de Genève - ISO 690" style (#5665) 2cad8f6 add ibid/subsequent to comparative-politics.csl (#5669) de0b116 Create taylor-and-francis-vancouver-national-library-of-medicine.csl (#5650) ed87f99 Update bulletin-de-correspondance-hellenique.csl (#5663) git-subtree-dir: buildres/csl/csl-styles git-subtree-split: 12dbba5

0654e16 Create scandinavian-journal-of-information-systems.csl (#5716) ce2d537 Update journal-of-computer-applications-in-archaeology.csl (#5715) 755d3d3 Create human-rights-law-review.csl (#5626) 0feda94 Create journal-of-intercultural-studies.csl (#5709) ae4756d Update acta-universitatis-agriculturae-sueciae.csl (#5713) 323d9ac Update mohr-siebeck-recht.csl (#5559) 15530a8 Bch corr (#5712) 094a1af Create forschungsjournal-soziale-bewegungen-fjsb.csl (#5699) cb91566 initialize authors and editors (#5714) 2d5cfff Create cancer-biomarkers.csl (#5703) 5e264d5 Update multidisciplinary-digital-publishing-institute.csl (#5708) 46e961f Create klinische-padiatrie.csl (#5711) e81e877 Create bulletin-archeologique-des-ecoles-francaises-a-l-etranger.csl (#5704) 0029c5a Create polar-research.csl 🧊 (#5702) 7db1361 Update vancouver-imperial-college-london.csl (#5641) b953e9f Update iso690-author-date-fr-no-abstract.csl (#5706) 91eda8c Update thieme-german.csl (#5710) ebe0787 Update harvard-imperial-college-london.csl (#5643) 2d4db76 Fix UNESCO IIEP in text 436cbf4 Create revue-archeologique-de-narbonnaise.csl (#5688) 5150bcf Create journal-of-computer-assisted-tomography.csl (#5690) dd6f050 Create anti-trafficking-review.csl (#5658) 08e622f Create the-angle-orthodontist.csl (#5685) c6a1907 journal-of-palm-oil-research.csl fix several errors (#5686) 6cbe29d Create bern-university-of-applied-sciences-school-of-agricultural-for… (#5684) f590dc1 Update biomed-central.csl (#5701) 1efce81 Update turabian-author-date.csl (#5695) 12dbba5 Create tyndale-bulletin (#5673) b0746db Create Engineered Regeneration (#5682) e38b953 wikipedia citation template (#5662) 5e7f731 Create early-music-history.csl (#5679) 86443f3 Create zeitschrift-fur-politik.csl (#5676) 68f1996 Create annals-of-work-exposures-and-health.csl (#5666) 1ba9dc6 Create brazilian-journal-of-psychiatry.csl (#5672) 438f92c fix error for speech in ama styles (#5693) 7a0c2d3 set initialize-with-hyphen to false (#5689) 3bd2765 Update emu-austral-ornithology.csl (#5671) 31492b2 fix various errors in natura-croatica.csl (#5687) 94d6b23 Update iso690-author-date-cs.csl (#5677) 5d017da minor update on the "Haute école de gestion de Genève - ISO 690" style (#5665) 2cad8f6 add ibid/subsequent to comparative-politics.csl (#5669) de0b116 Create taylor-and-francis-vancouver-national-library-of-medicine.csl (#5650) ed87f99 Update bulletin-de-correspondance-hellenique.csl (#5663) git-subtree-dir: buildres/csl/csl-styles git-subtree-split: 0654e16

* Squashed 'buildres/csl/csl-styles/' changes from 3a6a0a7..0654e16 0654e16 Create scandinavian-journal-of-information-systems.csl (#5716) ce2d537 Update journal-of-computer-applications-in-archaeology.csl (#5715) 755d3d3 Create human-rights-law-review.csl (#5626) 0feda94 Create journal-of-intercultural-studies.csl (#5709) ae4756d Update acta-universitatis-agriculturae-sueciae.csl (#5713) 323d9ac Update mohr-siebeck-recht.csl (#5559) 15530a8 Bch corr (#5712) 094a1af Create forschungsjournal-soziale-bewegungen-fjsb.csl (#5699) cb91566 initialize authors and editors (#5714) 2d5cfff Create cancer-biomarkers.csl (#5703) 5e264d5 Update multidisciplinary-digital-publishing-institute.csl (#5708) 46e961f Create klinische-padiatrie.csl (#5711) e81e877 Create bulletin-archeologique-des-ecoles-francaises-a-l-etranger.csl (#5704) 0029c5a Create polar-research.csl 🧊 (#5702) 7db1361 Update vancouver-imperial-college-london.csl (#5641) b953e9f Update iso690-author-date-fr-no-abstract.csl (#5706) 91eda8c Update thieme-german.csl (#5710) ebe0787 Update harvard-imperial-college-london.csl (#5643) 2d4db76 Fix UNESCO IIEP in text 436cbf4 Create revue-archeologique-de-narbonnaise.csl (#5688) 5150bcf Create journal-of-computer-assisted-tomography.csl (#5690) dd6f050 Create anti-trafficking-review.csl (#5658) 08e622f Create the-angle-orthodontist.csl (#5685) c6a1907 journal-of-palm-oil-research.csl fix several errors (#5686) 6cbe29d Create bern-university-of-applied-sciences-school-of-agricultural-for… (#5684) f590dc1 Update biomed-central.csl (#5701) 1efce81 Update turabian-author-date.csl (#5695) 12dbba5 Create tyndale-bulletin (#5673) b0746db Create Engineered Regeneration (#5682) e38b953 wikipedia citation template (#5662) 5e7f731 Create early-music-history.csl (#5679) 86443f3 Create zeitschrift-fur-politik.csl (#5676) 68f1996 Create annals-of-work-exposures-and-health.csl (#5666) 1ba9dc6 Create brazilian-journal-of-psychiatry.csl (#5672) 438f92c fix error for speech in ama styles (#5693) 7a0c2d3 set initialize-with-hyphen to false (#5689) 3bd2765 Update emu-austral-ornithology.csl (#5671) 31492b2 fix various errors in natura-croatica.csl (#5687) 94d6b23 Update iso690-author-date-cs.csl (#5677) 5d017da minor update on the "Haute école de gestion de Genève - ISO 690" style (#5665) 2cad8f6 add ibid/subsequent to comparative-politics.csl (#5669) de0b116 Create taylor-and-francis-vancouver-national-library-of-medicine.csl (#5650) ed87f99 Update bulletin-de-correspondance-hellenique.csl (#5663) git-subtree-dir: buildres/csl/csl-styles git-subtree-split: 0654e16 * Squashed 'buildres/csl/csl-locales/' changes from 0cc3885f61..d5ee85de8e d5ee85de8e Period after Übers. added (#241) git-subtree-dir: buildres/csl/csl-locales git-subtree-split: d5ee85de8e74d4109509014758b6f496a968ff03 * fix merge error Co-authored-by: github actions <jabrefmail+webfeedback@gmail.com> Co-authored-by: Siedlerchr <siedlerkiller@gmail.com>

3bb4b5f infoclio.ch styles for German: remove non-breaking space delimiters (#5754) adf28db Create journal-of-health-care-for-the-poor-and-underserved.csl (#5752) 0713a8e Update chinese-gb7714-2005-numeric.csl (#5737) 1cd3754 Update china-national-standard-gb-t-7714-2015-author-date.csl (#5746) c2536b7 Update china-national-standard-gb-t-7714-2015-numeric.csl (#5745) f8c1392 Create steel-research-international.csl (#5720) 21fe1f5 Create asian-myrmecology.csl (#5718) 91e9e2b Update harvard-university-of-the-west-of-england.csl (#5734) dd453d1 fix minor erros in polar-research.csl (#5730) 038a8f5 Remove group around no-date cluster (#5731) 0710b51 remove et-al from bibtex.csl (#5728) bbd703d Add editorial-director to universite-laval-departement-des-sciences-historiques.csl (#5727) 58ea430 Create german-journal-of-agricultural-economics.csl (#5717) 0654e16 Create scandinavian-journal-of-information-systems.csl (#5716) ce2d537 Update journal-of-computer-applications-in-archaeology.csl (#5715) 755d3d3 Create human-rights-law-review.csl (#5626) 0feda94 Create journal-of-intercultural-studies.csl (#5709) ae4756d Update acta-universitatis-agriculturae-sueciae.csl (#5713) 323d9ac Update mohr-siebeck-recht.csl (#5559) 15530a8 Bch corr (#5712) 094a1af Create forschungsjournal-soziale-bewegungen-fjsb.csl (#5699) cb91566 initialize authors and editors (#5714) 2d5cfff Create cancer-biomarkers.csl (#5703) 5e264d5 Update multidisciplinary-digital-publishing-institute.csl (#5708) 46e961f Create klinische-padiatrie.csl (#5711) e81e877 Create bulletin-archeologique-des-ecoles-francaises-a-l-etranger.csl (#5704) 0029c5a Create polar-research.csl 🧊 (#5702) 7db1361 Update vancouver-imperial-college-london.csl (#5641) b953e9f Update iso690-author-date-fr-no-abstract.csl (#5706) 91eda8c Update thieme-german.csl (#5710) ebe0787 Update harvard-imperial-college-london.csl (#5643) 2d4db76 Fix UNESCO IIEP in text 436cbf4 Create revue-archeologique-de-narbonnaise.csl (#5688) 5150bcf Create journal-of-computer-assisted-tomography.csl (#5690) dd6f050 Create anti-trafficking-review.csl (#5658) 08e622f Create the-angle-orthodontist.csl (#5685) c6a1907 journal-of-palm-oil-research.csl fix several errors (#5686) 6cbe29d Create bern-university-of-applied-sciences-school-of-agricultural-for… (#5684) f590dc1 Update biomed-central.csl (#5701) 1efce81 Update turabian-author-date.csl (#5695) 12dbba5 Create tyndale-bulletin (#5673) b0746db Create Engineered Regeneration (#5682) e38b953 wikipedia citation template (#5662) 5e7f731 Create early-music-history.csl (#5679) 86443f3 Create zeitschrift-fur-politik.csl (#5676) 68f1996 Create annals-of-work-exposures-and-health.csl (#5666) 1ba9dc6 Create brazilian-journal-of-psychiatry.csl (#5672) 438f92c fix error for speech in ama styles (#5693) 7a0c2d3 set initialize-with-hyphen to false (#5689) 3bd2765 Update emu-austral-ornithology.csl (#5671) 31492b2 fix various errors in natura-croatica.csl (#5687) 94d6b23 Update iso690-author-date-cs.csl (#5677) 5d017da minor update on the "Haute école de gestion de Genève - ISO 690" style (#5665) 2cad8f6 add ibid/subsequent to comparative-politics.csl (#5669) de0b116 Create taylor-and-francis-vancouver-national-library-of-medicine.csl (#5650) ed87f99 Update bulletin-de-correspondance-hellenique.csl (#5663) git-subtree-dir: buildres/csl/csl-styles git-subtree-split: 3bb4b5f

koppor added 4 commits November 21, 2019 22:35

Add tests for changed

b5ac16a

- Add tests - Guard AtomicFileWriter with a try-with-resources during save - use Objects.requireNonNull in DefaultFileUpdateMontitor - add comment - some auto-formattings - Fix typo in comment

Fix missing explicit dependency

0d52fe0

Begin to work on BibdatabaseDiff only

097790e

Siedlerchr reviewed Nov 24, 2019

View reviewed changes

koppor mentioned this pull request Nov 24, 2019

Improve change scanner #5665

Merged

5 tasks

koppor mentioned this pull request Dec 20, 2019

Improve things arround change detection #5770

Merged

5 tasks

koppor mentioned this pull request Jan 28, 2020

Integrate BibSonomy #5888

Open

krisgun mentioned this pull request Feb 24, 2020

[duplicate] Integrate ShortSicence and BibSonomy stevensdavid/jabref#1

Open

koppor force-pushed the master branch 5 times, most recently from b8ef7b7 to 21c6e5e Compare March 4, 2020 17:02

koppor self-assigned this Apr 14, 2020

tobiasdiez marked this pull request as draft April 17, 2020 15:58

koppor mentioned this pull request Jul 27, 2020

Fix exception when saving and autosave trigger at the same time #6694

Closed

5 tasks

koppor closed this Jul 27, 2020

koppor deleted the fix-change-on-disk-detection branch July 27, 2020 22:05

koppor mentioned this pull request Nov 12, 2020

Synchronization with Overleaf #156

Open

koppor mentioned this pull request Jan 25, 2021

Sharelatex Integration JabRef/jabref-koppor#445

Draft

koppor added the status: freeze Issues posponed to a (much) later future label Nov 13, 2021

koppor mentioned this pull request Nov 13, 2021

Add "Accept external changes" button for "The library has been modified by another program" #8235

Open

koppor mentioned this pull request May 21, 2023

Enhancements to handling of externally modified files #8431

Open

5 tasks

hasibagen mentioned this pull request May 3, 2026

Add (simple) git support #12350

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[WIP] Fix change on disk detection#5669

[WIP] Fix change on disk detection#5669
koppor wants to merge 4 commits into
masterfrom
fix-change-on-disk-detection

koppor commented Nov 24, 2019

Uh oh!

Siedlerchr Nov 24, 2019

Uh oh!

tobiasdiez commented Nov 24, 2019

Uh oh!

koppor commented Dec 1, 2019

Uh oh!

koppor commented Jul 27, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

koppor commented Nov 24, 2019

Implementation motivation

Implementation idea

Current implementation state

Uh oh!

Siedlerchr Nov 24, 2019

Choose a reason for hiding this comment

Uh oh!

tobiasdiez commented Nov 24, 2019

Uh oh!

koppor commented Dec 1, 2019

Uh oh!

koppor commented Jul 27, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants