Skip to content

Add OPDS sync feature#13946

Merged
Frenzie merged 43 commits into
koreader:masterfrom
Volterxien:master
Jul 4, 2025
Merged

Add OPDS sync feature#13946
Frenzie merged 43 commits into
koreader:masterfrom
Volterxien:master

Conversation

@Volterxien

@Volterxien Volterxien commented Jun 13, 2025

Copy link
Copy Markdown
Contributor

Aims to fill #8494.
Expects new OPDS entries to be added to the front of the list (sorted by new). -- I've added a patch to calibre-web to support OPDS browsing shelves by new.
Allows up to 1 duplicate title in the event you want 2 copies that have different URL.
Will not currently restore deleted files. You need to either grab them manually or do a force sync which will overwrite everything.
To enable, check the "Sync catalog" button on a given catalog.

Potential future work:
Optionally do a full sync and restore missing files without overwriting.
Configure sync to be done optionally done in background


This change is Reviewable

Comment thread plugins/opds.koplugin/opdsbrowser.lua Outdated
Comment thread plugins/opds.koplugin/opdsbrowser.lua Outdated
Comment thread plugins/opds.koplugin/main.lua Outdated
@Frenzie

Frenzie commented Jun 13, 2025

Copy link
Copy Markdown
Member

Occasionally I wondered if some of those functions would be clearer split it up into two functions, but effectively just a few nits.

Screenshots for others.

Now:

This PR:

Comment thread plugins/opds.koplugin/main.lua Outdated
@Volterxien

Copy link
Copy Markdown
Contributor Author

I'm working on refactoring because as it stands it won't ask to connect to wifi if not already connect. It just fails. So I'm working on that and will submit a fix for these issues with that

@Frenzie

Frenzie commented Jun 13, 2025

Copy link
Copy Markdown
Member

Alright. 👍

@hius07

hius07 commented Jun 14, 2025

Copy link
Copy Markdown
Member

Can it be used with other servers than calibre-web?

What will happen if a user tries to sync a public OPDS catalog?
Will it start downloading thousands of books, unable to stop?

I'm not sure doing checkDownloadFile() in a loop is a good idea (because of delay and infomessage for every file).
Please look at downloadDownloadList().

@Frenzie

Frenzie commented Jun 14, 2025

Copy link
Copy Markdown
Member

Good point, I was tired and hadn't thought of that.

@Volterxien

Copy link
Copy Markdown
Contributor Author

Thanks again for all the help and input. Very appreciated! And I'll look to address the future work in near future

@hius07

hius07 commented Jul 4, 2025

Copy link
Copy Markdown
Member

Thank you. Awaiting for users' feedback and unavoidable bugs to be fixed.

@Volterxien

Copy link
Copy Markdown
Contributor Author

Going to ask a potentially silly question: Do I need to make a new fork to make new PRs? Or can I just update my fork to master and make new changes and open new PR? I don't want to accidentally re-open this one with new commits

@Frenzie

Frenzie commented Jul 4, 2025

Copy link
Copy Markdown
Member

The way most of us work is to leave the master branch alone and to make changes in a feature branch. But it doesn't much matter what you do as long as it works for you. Deleting the fork and starting a new one can be perfectly valid, but it might be difficult to work on more than one thing at a time.

@Volterxien

Copy link
Copy Markdown
Contributor Author

Great. Yeah, it hadn't occured to me to make a branch on my forked branch. I'll do that. Thanks!

@JohannesBe

Copy link
Copy Markdown

Woop woop, looking forward to the nightly build 🙂

@Volterxien

Copy link
Copy Markdown
Contributor Author

@JohannesBe Let me know if you want the calibre-web patch to support the per-page sorting by new. Otherwise, there is an alternative work-around but I think it's kind of annoying lol

@Volterxien

Copy link
Copy Markdown
Contributor Author

I've also added sync content to the opds support wiki page if you guys want to provide any feedback. If we like it, I can suggest this or similar be included in the user guide

@hius07

hius07 commented Jul 5, 2025

Copy link
Copy Markdown
Member

Actually there is no detailed info about plugins in the Guide, I think it would be enough if @offset-torque just add a link to the OPDS wiki page.

@Volterxien

Copy link
Copy Markdown
Contributor Author

I just realized (in using this to do a full sync of my library) that the list gets imported in inverse order sorting by modification time. My thought should be newest at the front but not sure if you agree. If so I'll make a new PR to implement this.

@hius07

hius07 commented Jul 6, 2025

Copy link
Copy Markdown
Member

By the way, if there are 10 new books to sync, and the max number is set to 5, which five books are downloaded: 1...5 or 6...10?

@offset-torque

Copy link
Copy Markdown

Linked OPDS wiki page is clear and informative Volterxien. I will add an OPDS section to the library management chapter of the guide and add a link to this wiki page.

I see that wiki is edited 47 times. If it needs frequent updates like this, it is better to leave it as a wiki page for quicker changes since our guide has a slower update cycle.

@Volterxien

Copy link
Copy Markdown
Contributor Author

By the way, if there are 10 new books to sync, and the max number is set to 5, which five books are downloaded: 1...5 or 6...10?

The newest ones only. Hence I think the list should be front filled. I'll fix this today

@Volterxien

Copy link
Copy Markdown
Contributor Author

Linked OPDS wiki page is clear and informative Volterxien. I will add an OPDS section to the library management chapter of the guide and add a link to this wiki page.

I see that wiki is edited 47 times. If it needs frequent updates like this, it is better to leave it as a wiki page for quicker changes since our guide has a slower update cycle.

@offset-torque The last update to the wiki page before me was a year ago. That said, I have no issue with it just being linked!

@Volterxien

Copy link
Copy Markdown
Contributor Author

I'm thinking about this more and I think maybe it should just grab the newest ones?
Here's my thinking:

  • You can adjust sync size to accommodate a larger list
  • If you downloaded the wrong size, you can do a force sync -- maybe add option to do a resync and a force sync?
  • Under the hood, to do this correctly, you would need to possibly fetch a really long list if you haven't updated in awhile and the list is very active

That said, I think the change to have the newest stuff appear at the front of the list is very important for doing a larger sync so still a worthwhile change

@hius07

hius07 commented Jul 6, 2025

Copy link
Copy Markdown
Member

All this is up to you.
Don't you want to wait a bit for the feedback and discuss further development with interested users?

@Volterxien

Copy link
Copy Markdown
Contributor Author

Sure. Let's wait before doing any more changes then!

@offset-torque

Copy link
Copy Markdown

The last update to the wiki page before me was a year ago

I see, if it is stable surely I can add it to the guide. We have more formatting options in the guide, so it is more user friendly to read a topic there.

I see that discussion is still ongoing here. Please update the wiki accordingly after all the changes are done and ping me so I can write the guide section based on that.

@komadorirobin

Copy link
Copy Markdown

The new OPDS sync works great – thank you @Volterxien! I do have a another feature request, though. Is it possible to add the ability to set up multiple sync folders, so that you may add a sync folder for each catalog? For example, I have Vinland Saga (manga) set up as a OPDS catalog, and it would be neat to have it automatically add new volumes directly to that subfolder in /manga/Vinland Saga file structure within KOReader.

@Volterxien

Volterxien commented Aug 13, 2025

Copy link
Copy Markdown
Contributor Author

I'll see what we can do. The current catalog settings page is pretty cramped. We discussed adding a per catalog file extension setting as well. Thanks for the feedback!

Wondering if this opens up a reason to add a "sync settings" tab in the catalog settings widget to add the per catalog file type and move the sync checkbox.

@IlyaBizyaev

Copy link
Copy Markdown

Is there a user manual somewhere for this feature? https://github.com/koreader/koreader/wiki/OPDS-support does not go into detail.

I've set up a Kavita server and added its “Recently Updated” catalog to KOReader, but when I press “Sync” or “Force sync”, KOReader just says “Up to date” — and the configured sync folder remains empty.

I can navigate into the catalog manually, and it shows all of my books and lets me download them one by one, but the suggested download location is not the configured “sync folder”.

I've also tried restarting the app in case the sync happens on startup only, but no luck.

@Volterxien

Copy link
Copy Markdown
Contributor Author

@IlyaBizyaev thanks for pointing out the lack of clarity. You must edit the catalog and check the sync box to enable the syncing features! I'll make sure to update the wiki page to better reflect that.
Please feel free to provide any other feedback/issues you have!

@IlyaBizyaev

IlyaBizyaev commented Sep 29, 2025

Copy link
Copy Markdown

I have checked “Sync catalog”, yes. I currently have two OPDS catalogs configured, following the Wiki page:

  • "Kavita" (added by the server URL, sync disabled)
  • "Kavita - Recently Updated" (added through the OPDS browser to get the “sorted by new” behavior, sync enabled)

Also note:

  • I have not set up any file extension filters
  • I have not changed the download limit, it's at the default 50.

But I guess I'm missing something else?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants