wallabag-logseq

wallabag-logseq is a cli tool to sync annotations of wallabag articles to a logseq-styled markdown file.

Table of Contents

• Demo
• Opinionated wallabag article exporter to markdown format for Logseq
• Usage
  • First Usage
  • Adjusting the configuration
    • Example
    • Paths and default values
    • Wallabag API Credentials
  • Adjusting the templates
    • Page Template
    • Article Template
  • Options
• TODOs
• Inspiration
• Migration instructions
  • From version 2.x.y to 3.x.y
  • From version 1.x.y to 2.x.y

Demo

An example output markdown file of the software with the default templates looks like this:

tags:: wallabag, export

- ### Windows 10 now lets you install WSL with a single command
  collapsed:: false
  url:: https://www.bleepingcomputer.com/news/microsoft/windows-10-now-lets-you-install-wsl-with-a-single-command/
  wallabag-url:: https://my-wallabag-url.com/view/15
  tags:: [[linux]], [[2min]]
  site:: [[www.bleepingcomputer.com]]
  author:: [[Sergiu Gatlan]]
  date-published:: [[2021-07-30]] 15:08:24
  date-saved:: [[2021-08-10]] 14:35:31
  date-archived:: [[2021-10-26]] 17:38:39
        - WSL commands
                - wsl.exe --install
                - wsl --update
                - wsl --status
- ### "The Best Programming Advice I Ever Got" with Andrew Binstock
  collapsed:: false
  url:: http://www.informit.com/articles/printerfriendly/1966027
  wallabag-url:: https://my-wallabag-url.com/view/33
  tags:: [[5min]]
  site:: [[www.informit.com]]
  date-saved:: [[2016-04-12]] 21:53:02
  date-archived:: [[2017-06-27]] 12:57:29
        - keep classes to no more than 50-60 lines of code

In the first example, the text WSL commands is the comment i annotated to the following three bullet points. The second example keep classes to no more… shows an annotation without a custom text, thus being only the highlighted text from the article without any comment.

Opinionated wallabag article exporter to markdown format for Logseq

The following requirements were made.

• all exported articles go into one markdown file
• only annotated, archived wallabag articles are exported
• this means articles having annotations, but are not archived, are not exported
• dates are automatically linked
• in the article markdown template empty newlines are removed to fulfill Logseq's markdown
• cache all exported articles and when syncing again use the wallabag's API support for since queries
• new articles, which are not yet cached, are only appended at the end of the markdown file
• ATM, changed articles are newly appended to the file
  • thus, creating duplicates
  • this was done to preserve articles being changed in Logseq only
  • of course a manual merge is needed if you changed that article in Logseq only and it was changed afterwards in wallabag
  • if it was not changed in Logseq, the old entry can of course be deleted (manually only)

Usage

First Usage

1. Download the latest release for your machine. E.g. version 4.1.0 for Linux 64 Bit: wget https://codeberg.org/strubbl/wallabag-logseq/releases/download/4.1.0/wallabag-logseq_4.1.0_Linux_x86_64.zip.
2. Unzip the file: unzip wallabag-logseq_4.1.0_Linux_x86_64.zip
3. Run the binary once: ./wallabag-logseq. Now, you'll get a default config.json file and some templates in the same folder.
4. Adjust the configuration values in your config.json.
5. Run the binary again ./wallabag-logseq.

Adjusting the configuration

Example

This is the default config created by wallabag-logseq 4.1.0:

{
  "ArticleTemplateFilePath": "t-article.template",
  "AutoTagsFilePath": "auto-tags.txt",
  "CachingFilePath": "caching.json",
  "LogseqMarkdownFilePath": "wallabag-articles.md",
  "PageTemplateFilePath": "t-page.template",
  "WbgoConfig": {
    "WallabagURL": "https://your-wallabag-url",
    "ClientID": "your-client-id",
    "ClientSecret": "your-client-secret",
    "UserName": "your-user-name",
    "UserPassword": "your-user-password"
  }
}

Paths and default values

Paths need to be relative or absolute. wallabag-logseq does not understand your shell or environment variables.

• ArticleTemplateFilePath: This is the path to the article template file. For more information see the following paragraph about adjusting the templates.
• AutoTagsFilePath: This is the path to the file for the auto tags. That file shall be a simple text file with one Logseq tag in each line, sorted from longest tag to shortest.
• CachingFilePath: This is the path to the cache file, which gets created during a successful wallabag-logseq run. It caches articles which were fetched via the wallabag API. It is used to see which articles were already processed, so they do not get appended to the output file again.
• LogseqMarkdownFilePath: This is the path to the markdown file, which shall be read by Logseq. Ideally this points to a path in your Logseq graph, e.g. /home/username/my-logseq-graph/pages/wallabag-articles.md.
• PageTemplateFilePath: This is the path to the page template file. For more information see the following paragraph about adjusting the templates.
• WbgoConfig: See the following paragraph about credentials

Wallabag API Credentials

The default config has a map called WbgoConfig. All configuration options in there relate to your wallabag server instance. The WbgoConfig corresponds to the wallabag config of the library wallabago. Yes, you need to define all these parameters due to a wrong implementation of OAuth 2 in wallabag server. For further information why this is necessary, please read the corresonding issue wallabag/wallabag#2800 in the server repository.

Adjusting the templates

When wallabag-logseq is run for the first time, two default templates are created in your file system: the page template and the article template. These templates are written in the Go template format.

Page Template

By default the page template is saved to the file t-page.template. In the config you can adjust the path to this file. Of course you can adjust the template itself, too. This page template is only used, when the Logseq markdown file does not exist, so that a new file needs to be created. So, during normal usage of this tool, it is only used once in the first run with a valid configuration. At the moment, the default page template only defines the header of the page.

Article Template

By default the article template is saved to the file t-article.template. In the config you can adjust the path. Of course you can adjust the template itself, too. It is used for every new archived, annotated article. If you change it later in the course of using this tool and you want it to be applied to the existing articles too, you should backup the logseq markdown file to a different location and delete the file in the path for the Logseq markdown file. This way it gets newly created using your adjusted article template.

Options

The binary has some optional parameters:

$ ./wallabag-logseq -h
Usage of ./wallabag-logseq:
  -config string
        path to config JSON file (default "config.json")
  -d    get debug output (includes verbose mode)
  -dd
        get even more debug output (includes debug mode)
  -learn-auto-tags
        prints the tags from the Logseq's wallabag markdown file combined with already existing tags from your config, result is printed ordered by length and then alphabetically
  -v    print version
  -verbose
        verbose mode

With the -config parameter, you can adjust the path to your location of the config.json. Without this parameter, a file named config.json in the current path is assumed. If at the given path for the config file no file exists, a new default config file gets created at that path.

TODOs

• auto-tagging of articles is implemented. There's room for optimization, e.g. tagging only the first occurance in a paragraph incl. its subparagraphs, but that kind of implementation is hard. At the moment the first occurance in every new line is tagged, also in all subparagraphs.
• sort articles chronologically with oldest first and newest at the end, not clear by which of the dates (PublishedAt, UpdatedAt, CreatedAt or ArchivedAt) see #3

Inspiration

The start of this project was inspired by logseq-omnivore. I tested Omnivore and Logseq with that plugin and needed to have something similar for wallabag.

Migration instructions

From version 2.x.y to 3.x.y

Starting with version 3.0.0 the list of auto tags was moved from the configuration file to a separate text file. That text file shall be a simple text file with one Logseq tag in each line, sorted from the longest to the shortest tag. There is no automatic migration of the config to this new structure. Manual steps are needed for a semi-automatic transition.

0. Make a backup of your whole wallabag-logseq files.
1. Still with version 2, be sure to apply all auto tags to your wallabag-articles.md. This means, that every auto tag configured in the configuration's array AutoTags is at least used once in the your wallabag-articles.md. If you have added tags to your configuration's AutoTags array, which are not used in the wallabag-articles.md, these tags need to be migrated manually. Note them down.
2. Update to wallabag-logseq version 3.x.y.
3. Change your configuration manually:
   1. Add the variable AutoTagsFilePath with a file path to your future auto tags text file, e.g. auto-tags.txt.
   2. Create this file and add your noted tags, which should be migrated manually.
   3. Remove the variable AutoTags from your configuration.
4. Use the flag -learn-auto-tags to update the list of auto tags, e.g. ./wallabag-logseq -learn-auto-tags. This command prints all tags, which can be found in your wallabag-articles.md - of course besides the ones already in your auto tags text file.
5. Migration done.

From version 1.x.y to 2.x.y

The configuration parameter OutputFilePath has been renamed to LogseqMarkdownFilePath. You need to update this option in your existing config manually.