[Feature] JavaScript Addons Support by krmanik · Pull Request #7958 · ankidroid/Anki-Android

krmanik · 2020-12-26T04:34:44Z

Pull Request template

Purpose / Description

Using AnkiDroid JS API a deck can be extended to implement features like progress bar or custom card layouts. But it requires user to manually add the html/css/js to card templates. This is implementation for a way to inject code to all deck templates when addons get activated.

Fixes

Fixes #7959

Approach

Download addons from https://www.npmjs.com/search?q=keywords:ankidroid-js-addon inside the app
In app there are implementation to check for valid addons for AnkiDroid
It will check following string in latest package.json

{
    "name": "ankidroid-js-addon-progress-bar",
    "version": "1.0.0",
    "author": "https://github.com/infinyte7",
    "homepage": "https://github.com/infinyte7/Anki-Custom-Card-Layout",
    "ankidroid_js_api": "0.0.1",
    "addon_type": "reviewer",
    "keywords": ["ankidroid-js-addon"]
}

There isValidPackageJson boolean function will check for above values if present then the addons is valid.

The enabled addons' index.js content wrapped in <script> tag will be added to fillFlashCard() in AbstractFlashcardViewer.java

There are two types of addon reviewer and note_editor. In this PR, reviewer addon is implemented.

Idea for note_editor addon:
The javascript function can be called using https://github.com/evgenyneu/js-evaluator-for-android/#call-a-javascript-function . So, in note editor multiple buttons will get added to toolbar with tag containing addons' name. When the toolbar button pressed, it will take selected text from FieldEditText and pass to jsEvaluate java function. The jsEvaluate java function will get tag of clicked toolbar button and open js file with name equal to tag of button and pass the selected text to addon's function implementation. The addon's js function will return the manipulated string to FieldEditText in note editor. Even if other implementation can be done without selected text as defined in js file in addon's folder. This will implemented in separated PR.

How Has This Been Tested?

Tested on emulator

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration (SDK version(s), emulator or physical, etc)

Learning (optional, can help others)

https://developer.android.com/guide/topics/ui/layout/recyclerview

Checklist

Please, go through these checks before submitting the PR.

[ x ] You have not changed whitespace unnecessarily (it makes diffs hard to read)
[ x ] You have a descriptive commit message with a short title (first line, max 50 chars).
[ x ] Your code follows the style of the project (e.g. never omit braces in if statements)
[ x ] You have commented your code, particularly in hard-to-understand areas
[ x ] You have performed a self-review of your own code
[ x ] UI changes: include screenshots of all affected screens (in particular showing any new or changed strings)

krmanik · 2020-12-26T04:35:39Z

This is first commit and need help in implementing way to store the activated addons status.

AnkiDroid/src/main/java/com/ichi2/anki/AddonsAdapter.java

codecov · 2020-12-26T04:47:39Z

Codecov Report

Merging #7958 (c6b1ac9) into master (3cca3b1) will decrease coverage by 0.41%.
The diff coverage is 6.97%.

❗ Current head c6b1ac9 differs from pull request most recent head f1bb756. Consider uploading reports for the commit f1bb756 to get more accurate results

@@             Coverage Diff              @@
##             master    #7958      +/-   ##
============================================
- Coverage     37.11%   36.70%   -0.42%     
+ Complexity     3854     3791      -63     
============================================
  Files           361      358       -3     
  Lines         36326    36209     -117     
  Branches       4798     4778      -20     
============================================
- Hits          13484    13291     -193     
- Misses        21310    21409      +99     
+ Partials       1532     1509      -23

Impacted Files	Coverage Δ	Complexity Δ
...Droid/src/main/java/com/ichi2/anki/AddonModel.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (?)`
...id/src/main/java/com/ichi2/anki/AddonsAdapter.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (?)`
.../java/com/ichi2/anki/NavigationDrawerActivity.java	`30.28% <0.00%> (-0.71%)`	`13.00 <0.00> (ø)`
...id/src/main/java/com/ichi2/anki/AddonsBrowser.java	`3.21% <3.21%> (ø)`	`2.00 <2.00> (?)`
...n/java/com/ichi2/anki/AbstractFlashcardViewer.java	`38.87% <41.46%> (-0.72%)`	`168.00 <2.00> (-1.00)`
...kiDroid/src/main/java/com/ichi2/utils/MapUtil.java	`0.00% <0.00%> (-50.00%)`	`0.00% <0.00%> (-2.00%)`
.../src/main/java/com/ichi2/anki/web/HttpFetcher.java	`0.00% <0.00%> (-16.42%)`	`0.00% <0.00%> (-2.00%)`
.../java/com/ichi2/utils/DatabaseChangeDecorator.java	`42.68% <0.00%> (-10.98%)`	`20.00% <0.00%> (-6.00%)`
...roid/src/main/java/com/ichi2/ui/FixedEditText.java	`44.82% <0.00%> (-9.46%)`	`5.00% <0.00%> (-2.00%)`
...d/src/main/java/com/ichi2/utils/ClipboardUtil.java	`0.00% <0.00%> (-8.34%)`	`0.00% <0.00%> (-3.00%)`
... and 94 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3cca3b1...f1bb756. Read the comment docs.

Arthur-Milchior · 2020-12-26T07:57:00Z

I really like the general idea.
Given that there is no monkey patching, I hope that this will ensure that add-ons live together better in ankidroid than in anki (even if it became better in anki the two last years)
I hope and fear it should be cool to have a place to list all add-ons, maybe vouch them, and a way to update easily. But I guess that's not necessary to consider all of this immediately.

A small note about screen capture. I believe you could easily improve it by turning on android's preference option to display where you click/touch. This way, it'll become easier to follow what you are doing.

krmanik · 2020-12-26T08:50:12Z

This will greatly help in introducing some features of Anki addons that can be implemented using JavaScript for AnkiDroid using JS API. For example Card info during review. But that need to added to all decks manully. Using feature in this PR, the content that needs to be added to reviewer will have done once.

I will implement the feature if file with .jsaddon extension valid for addons or not using their manifest.json file.

Also there will be some place where user can search and find AnkiDroid JS addons like https://addon-docs.ankiweb.net/

krmanik · 2020-12-28T18:42:12Z

krmanik · 2020-12-28T18:46:03Z

This is second commit to this PR.
The addons containing {{Deck}} or {{Subdeck}} needs to converted to respective deckname. And similar transformation to be done if addons contains {{anything}}

AnkiDroid/src/main/java/com/ichi2/anki/AbstractFlashcardViewer.java

krmanik · 2020-12-29T13:28:55Z

krmanik · 2020-12-29T13:36:03Z

For importing addon, addon must contain two files.

content.html
manifest.json

content.html is file which get added to webview and manifest.json used for getting id.

The id in manifest.json and folder name of addon must be same.

The structure of js_addon_progress_bar_1234578

js_addon_progress_bar_1234578
↳ content.css
↳ content.html
↳ content.js
↳ manifest.json

In this case addon folder name and id both are js_addon_progress_bar_1234578.

The folder js_addon_progress_bar_1234578 and files zipped with .jsaddon extension.

For testing I attached two jsaddons

js_addons_progress_bar_12345678.zip

js_addons_top_bar_12345678.zip

Change the extension from zip to jsaddon.

mikehardy · 2020-12-29T13:49:45Z

I'm sorry I haven't given this a real scan yet, but I wonder if it we could use the standard package.json format and thus rely on npmjs.com for distribution etc 🤔 ?

krmanik · 2020-12-29T14:05:23Z

I haven't implemented any packages for npmjs so I will study the distribution. The format can be updated to package.json but for distribution I will read the documentation and then try to implement.

Distribution from npmjs will be better because there will no need for Anki Ecosystem to view and manage those files. That totally rely on developer. I will view the documentation of npmjs for better implementation. :)

mikehardy · 2020-12-29T14:10:03Z

the package.json format is definitely the most well known. It defines an entry point for the package, naming (and npmjs.com will enforce unique names :-)) plus I think you can tag on npmjs.com with keywords so we could do a search or something for "ankidroid-js-addon" or something if we make a convention for that. It can also define homepages and version and dependencies (regular ones, devDependencies, peerDependencies) etc.

For our purposes we likely won't be using anything but the most basic keywords in the package.json format but it keeps us using standard things which is nice

krmanik · 2020-12-29T14:29:57Z

I understood it. I will change the manifest.json to package.json and will use useful and needed keywords from package.json.

krmanik · 2020-12-30T12:40:07Z

Implementation

Published addon as npm package to npmjs
e.g. https://www.npmjs.com/package/ankidroid-js-addon-progress-bar
In AnkiDroid downloading latest package info as json file from
http://registry.npmjs.org/{ file name from edit text }/latest
e.g. http://registry.npmjs.org/ankidroid-js-addon-progress-bar/latest

I haven't found way to download npm package directly.
https://stackoverflow.com/questions/50933603/download-npm-dist-package-without-having-to-install-npm
Reading json file and getting tarball link to download the package
The npm package file downloded with .tgz
e.g. ankidroid-js-addon-progress-bar.tgz
Decompressing the tar file to addons folder using https://github.com/thrau/jarchivelib
In AbstractFlashcardViewer.java, the index.js read from addons/ankidroid-js.../package/index.js and added to reviewer

Demo

krmanik · 2020-12-30T12:41:04Z

This is outdated now.

For importing addon, addon must contain two files.
content.html
manifest.json
content.html is file which get added to webview and manifest.json used for getting id.

The id in manifest.json and folder name of addon must be same.

The structure of js_addon_progress_bar_1234578
js_addon_progress_bar_1234578
↳ content.css
↳ content.html
↳ content.js
↳ manifest.json
In this case addon folder name and id both are js_addon_progress_bar_1234578.

The folder js_addon_progress_bar_1234578 and files zipped with .jsaddon extension.

For testing I attached two jsaddons

js_addons_progress_bar_12345678.zip

js_addons_top_bar_12345678.zip

Change the extension from zip to jsaddon.

mikehardy · 2020-12-30T14:50:45Z

I have still been booked up completely on some contract work but every time I see one of your animated gifs it is 🤤 ! this looks very powerful

krmanik · 2020-12-30T15:35:08Z

The information need for contract is stored in package.json file. Email, ankidroid_api, author
If ankidroid_api does not match api in AnkiDroid then .tgz file will not downloaded. So checking email, ankidroid_api, keywords in package.json will stop from downloading other npmjs package.

I think, gifs is better than written description for explaining workflow of the PR. As it changes ui element also. :)

krmanik · 2020-12-30T15:47:29Z

Also I think that one file in wiki should list all the addons that has been added to npmjs by a developer like wiki/Third-Party-Apps. New addons can be listed by the addon developer.

mikehardy · 2020-12-30T16:49:00Z

Is there way to do "discovery by convention" with a search powered by someone else?
What I mean concretely is: what if we had a convention of specific tags/keywords you should use when you upload to npmjs.com, then for JS addons we had a wiki page that had the npmjs.com search for those keywords in it, and we used the same search in the app?

That way all the search/discovery would be offloaded to npmjs.com and add on authors? zero maintenance burden unless npmjs.com changed their URL query interface...

krmanik · 2020-12-30T17:02:48Z

I found this url useful.
https://www.npmjs.com/search?q=keywords:ankidroid-js-addon

In app we may add option for Get more addons.

It will prompt for opening link in browser. (In wiki a line should be mentioned for addon developer to add keyword for discoverability, like ankidroid-js-addon)
https://www.npmjs.com/search?q=keywords:ankidroid-js-addon
Users view the listed addons on npmjs and select any addon
Copy their npm i ankidroid-js-addon-... and paste it in AnkiDroid->Addons

krmanik · 2020-12-31T13:04:07Z

Implementation

It will download latest info of addon file
Check for following info if present in .json file

name
version 
author
ankidroid_js_api
homepage
keywords

If in latest json file, ankidroid_js_api is equal to 0.0.1 (current js api version) and keywords is equals to ankidroid-js-addon then get tarball link for downloading addon.tgz file and extract then copy it to AnkiDroid/addons folder.

https://github.com/ankidroid/Anki-Android/pull/7958/files#diff-e71d85656c47c1c4a8f0135d9b62eb6db857881841574c79522848f29725386dR348-R375

Also added option to Get more addons. It will open url in browser with url:
https://www.npmjs.com/search?q=keywords:ankidroid-js-addon

krmanik · 2021-01-02T16:05:59Z

Implementation

Added addon type, reviewer and note_editor
The content of addon with reviewer type will get added to reviewer from js file wrapped in <script> tag
The addon with note_editor type to be implemented.

Idea:
The javascript function can be called using https://github.com/evgenyneu/js-evaluator-for-android/#call-a-javascript-function . So, in note editor multiple buttons will get added to toolbar with tag containing addons' name. When the toolbar button pressed, it will take selected text from FieldEditText and pass to jsEvaluate java function. The jsEvaluate java function will get tag of clicked toolbar button and open js file with name equal to tag of button and pass the selected text to addon's function implementation. The addon's js function will return the manipulated string to FieldEditText in note editor. Even if other implementation can be done without selected text as defined in js file in addon's folder. This will implemented in separated PR.

…to addons

Update addon info popup layout and remove prefs when addon delete

krmanik · 2021-03-14T10:51:15Z

Update 1: Change addon info popup layout

Update 2: Remove prefs when addon delete

Update 3: Added a new addon for button and card counts to npmjs

…to addons

- added volley library for downloading json from npmjs - code cleanup

- Options in advance settings - Layout for showing message when no addons added - Open npmjs.org url in AnkiDroid - comments added

…to addons

krmanik · 2021-04-01T14:05:07Z

JS Addons in Advance settings
Layout for showing message to download addons when no addons added
Open npmjs.org in app

…to addons

Arthur-Milchior

Are 22 commits really useful ? Hard to review from scratch right now. For example, I'd like "first commit" to explain a little bit more what I'm reading. I don't know NPM at all, I don't really know JS, so I need commit message to guide me through what's going on to be able to review this

Arthur-Milchior · 2021-04-02T20:38:23Z

AnkiDroid/src/test/java/com/ichi2/anki/AddonsBrowserTest.java

@@ -0,0 +1,7 @@
+package com.ichi2.anki;
+
+public class AddonsBrowserTest extends RobolectricTest {


What's the point of an empty class?

To pass build of project.

krmanik · 2021-04-02T20:49:44Z

I think it is not good idea to review from First commit because there are lot of changes done.
Should I squash all commit in to one?

krmanik · 2021-04-02T21:04:30Z

I am creating new PR for this with detailed commit message.

Arthur-Milchior · 2021-04-02T22:16:28Z

Squashing everything will not really help me. I can not really read 1157 new lines of code and check that I'm okay with all of them. It's particularly hard because I don't know anything about JS and NPM and other tool you use.

It is possible that you actually can not split this into smaller commits that all makes sens by themselves; it sometime happens and makes things hard. However, ideally, what I would love to see is a story. You make a first commit that compiles, pass tests, with a commit message that explains to me and future reader why you do it, why it is useful. Then you add another part of the code etc...
Using git rebase -i (for interactive) you usually can change history, modify a commit to correct what was added in it.
Of course, it works best if you do it progressively through the history of the PR; rewritting all of this as a consistent story later requires more thought process.

krmanik · 2021-04-02T22:30:46Z

I have created separate PR because creating is easy and better than rewriting history. :)
Also I have tried to explain the working of js addons idea.

First Commit

b480174

krmanik marked this pull request as draft December 26, 2020 04:35

krmanik commented Dec 26, 2020

View reviewed changes

AnkiDroid/src/main/java/com/ichi2/anki/AddonsAdapter.java Outdated Show resolved Hide resolved

krmanik mentioned this pull request Dec 26, 2020

[Feature] JavaScript Addons support for AnkiDroid #7959

Open

Enabled/Disabled Status, Info Popup

cde2b9c

krmanik commented Dec 28, 2020

View reviewed changes

AnkiDroid/src/main/java/com/ichi2/anki/AbstractFlashcardViewer.java Show resolved Hide resolved

krmanik added 2 commits December 29, 2020 21:08

Import and Delete option

babdd00

if addonsContent null

eb6169e

krmanik marked this pull request as ready for review December 29, 2020 13:31

Import from npmjs

f23d73e

Updated for getting valid addon package file

a11d93d

krmanik added 2 commits January 1, 2021 01:56

Added update addon option

a0a2365

Addons type reviewer and note editor

604187b

krmanik marked this pull request as draft January 2, 2021 16:13

krmanik mentioned this pull request Jan 11, 2021

Note editor addon #8011

Closed

krmanik added 4 commits February 15, 2021 21:46

Merge remote-tracking branch 'origin/master' into addons

a8370a5

Merge branch 'master' of https://github.com/ankidroid/Anki-Android in…

7a4c0e8

…to addons

Update build.gradle

972981e

popup layout and prefs

9e02a1d

Update addon info popup layout and remove prefs when addon delete

krmanik added 6 commits March 14, 2021 18:53

Delete Template.java

cce00a7

Merge branch 'master' of https://github.com/ankidroid/Anki-Android in…

da4d497

…to addons

added volley library

e8923d9

- added volley library for downloading json from npmjs - code cleanup

jsAddonContent in onCreate

1176b01

Cleanup and Added Test class

a0e4d94

reimplemented OkHttp, it works now

c6b1ac9

krmanik marked this pull request as ready for review March 19, 2021 09:27

krmanik added 2 commits April 1, 2021 21:49

Options in advance settings and open url in app

38a4869

- Options in advance settings - Layout for showing message when no addons added - Open npmjs.org url in AnkiDroid - comments added

Merge branch 'master' of https://github.com/ankidroid/Anki-Android in…

f1bb756

…to addons

krmanik added 2 commits April 3, 2021 04:16

Merge branch 'master' of https://github.com/ankidroid/Anki-Android in…

cafa807

…to addons

Update build.gradle

7c50ab7

Arthur-Milchior reviewed Apr 2, 2021

View reviewed changes

krmanik closed this Apr 2, 2021

		@@ -0,0 +1,7 @@
		package com.ichi2.anki;

		public class AddonsBrowserTest extends RobolectricTest {

Uh oh!

Conversation

krmanik commented Dec 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request template

Purpose / Description

Fixes

Approach

How Has This Been Tested?

Learning (optional, can help others)

Checklist

Uh oh!

krmanik commented Dec 26, 2020

Uh oh!

Uh oh!

codecov bot commented Dec 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Arthur-Milchior commented Dec 26, 2020

Uh oh!

krmanik commented Dec 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

krmanik commented Dec 28, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

krmanik commented Dec 28, 2020

Uh oh!

Uh oh!

krmanik commented Dec 29, 2020

Uh oh!

krmanik commented Dec 29, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mikehardy commented Dec 29, 2020

Uh oh!

krmanik commented Dec 29, 2020

Uh oh!

mikehardy commented Dec 29, 2020

Uh oh!

krmanik commented Dec 29, 2020

Uh oh!

krmanik commented Dec 30, 2020

Uh oh!

krmanik commented Dec 30, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

This is outdated now.

Uh oh!

mikehardy commented Dec 30, 2020

Uh oh!

krmanik commented Dec 30, 2020

Uh oh!

krmanik commented Dec 30, 2020

Uh oh!

mikehardy commented Dec 30, 2020

Uh oh!

krmanik commented Dec 30, 2020

Uh oh!

krmanik commented Dec 31, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

krmanik commented Jan 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

krmanik commented Mar 14, 2021

Uh oh!

krmanik commented Apr 1, 2021

Uh oh!

Arthur-Milchior left a comment

Choose a reason for hiding this comment

Uh oh!

Arthur-Milchior Apr 2, 2021

Choose a reason for hiding this comment

Uh oh!

krmanik Apr 2, 2021

Choose a reason for hiding this comment

Uh oh!

krmanik commented Apr 2, 2021

Uh oh!

krmanik commented Dec 26, 2020 •

edited

Loading

codecov bot commented Dec 26, 2020 •

edited

Loading

krmanik commented Dec 26, 2020 •

edited

Loading

krmanik commented Dec 28, 2020 •

edited

Loading

krmanik commented Dec 29, 2020 •

edited

Loading

krmanik commented Dec 30, 2020 •

edited

Loading

krmanik commented Dec 31, 2020 •

edited

Loading

krmanik commented Jan 2, 2021 •

edited

Loading