Skip to content
This repository was archived by the owner on Sep 10, 2025. It is now read-only.

82 boltdb#85

Merged
skx merged 14 commits intomasterfrom
82-boltdb
Mar 16, 2022
Merged

82 boltdb#85
skx merged 14 commits intomasterfrom
82-boltdb

Conversation

@skx
Copy link
Copy Markdown
Owner

@skx skx commented Mar 13, 2022

Use BoltDB to store feed-state

This pull-request largely neuters the use of ~/.rss2email/state, instead storing state of the feed-items inside a BoltDB database.

The migration here is seamless:

  • When a feed is fetched we get a list of items.
  • We look to see if the item is new, in the legacy database OR the new one.
    • If the item is new we record it ONLY in boltdb.
    • Then we remove the legacy state-file.

The end result is that legacy state-files will be cleaned up, and removed and all state will end up in BoltDB.

The BoltDB file will continue to grow, as that is not yet pruned, but that will be an easy change to make in the future.

As part of this we've also disabled CGO, and generated binaries for all platforms.

This would close #82, when merged.

Currently this is undergoing private testing, but feedback from casual observers would be most welcome!

skx added 2 commits March 13, 2022 18:35
This pull-request largely neuters the use of ~/.rss2email/state,
instead storing state of the feed-items inside a BoltDB database.

The migration here is seamless:

* When a feed is fetched we get a list of items.
* We look to see if the item is new, in the legacy database OR the new
  one.
  * If the item is new we record it ONLY in boltdb.
  * THen we remove the legacy state-file.

The end result is that legacy state-files will be cleaned up,
and removed and all state will end up in BoltDB.

The BoltDB file will continue to grow, as that is not yet pruned,
but that will be an easy change to make in the future.

As part of this we've also disabled CGO, and generated binaries
for all platforms.

This would close #82, when merged.
@skx skx self-assigned this Mar 13, 2022
skx added 12 commits March 13, 2022 19:07
This avoids duplication in finding the appropriate directory
in all the placse we need to refer to it.
In the past we'd expire feed-items from feeds, but if an actual
feed was removed from the config-file we'd maintain the complete
history fo that feed.

Now we remove feeds that have been orphaned from the BoltDB.

To confirm this:

* Run `rss2email` with some feeds.
* Comment out a feed.
* Run `rss2email` to do the prunining.
* Confirm it is gone with `rss2email seen`
@skx skx merged commit 7f45598 into master Mar 16, 2022
@skx skx deleted the 82-boltdb branch March 16, 2022 15:01
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Our cache should refer to the feed from which it came.

1 participant