Skip to content

Add LuaData and Dictionary Lookup History#3161

Merged
Frenzie merged 11 commits intokoreader:masterfrom
Frenzie:dict_hist
Oct 7, 2017
Merged

Add LuaData and Dictionary Lookup History#3161
Frenzie merged 11 commits intokoreader:masterfrom
Frenzie:dict_hist

Conversation

@Frenzie
Copy link
Copy Markdown
Member

@Frenzie Frenzie commented Sep 2, 2017

Fixes #2033, fixes #2998.

screenshot_2017-09-02_17-30-26-fs8
screenshot_2017-09-02_17-30-34-fs8
screenshot_2017-09-02_17-35-33-fs8

@poire-z
Copy link
Copy Markdown
Contributor

poire-z commented Sep 2, 2017

Any reason not to save them in .sdr/metadata.epub.lua ?
With this new file lookup_history.lua, we need to decide what to do with them on "Purge .sdr" (and as I use that to cleanup .sdr directories, my preference would be to delete them :)

(Also, vocabulary builder folks would may be like to have it as a global file, with may be references to the book aside the date.)

@Frenzie
Copy link
Copy Markdown
Member Author

Frenzie commented Sep 2, 2017

See #2789, but I guess I mostly just wanted to do something fast and fluffy. The data storage format may well require more thought indeed.

(Also, vocabulary builder folks would may be like to have it as a global file, with may be references to the book aside the date.)

Is that so? My personal reading log includes a vocabulary builder feature of sorts and I quite like it on a per-book basis. I consider the occasional intrusions (e.g., HardWare.fr in the bottom left of the picture) more of a bug than a feature. But yeah, this is definitely #2998 and I didn't bother to research Kindle gobbledygook. 😇

2017-09-02 18 10 15

@poire-z
Copy link
Copy Markdown
Contributor

poire-z commented Sep 2, 2017

OK :) but we need to decide about what to do on Purge .sdr.
And on Delete, the lookup_history would currently be deleted. Thinking about people who reads, have their lookup words saved, delete the book from their device, and lost the per-book vocabulary built list (which is ok for me, but may be not for people who want this feature).
You don't use a new notebook for each book you start reading, and burn it when you finished the book :) You have a nice old and used single notebook that carries your lifelong history of words :)

@Frenzie
Copy link
Copy Markdown
Member Author

Frenzie commented Sep 2, 2017

I think that when things like bookmarks are separated out (or perhaps even before) it would make sense to have a few simple checkmarks. To me "Purse .sdr" mostly means "reset display settings," which isn't really the same thing anyway.

So if you click on Purge .sdr (possibly renamed something like Reset (document)), you'll be presented with a few checkmark options like:

  • Bookmarks
  • Highlights
  • Display settings
  • (Dictionary lookup history?)
  • All

@Frenzie Frenzie changed the title Add dictionary history [WIP] Add dictionary history Sep 5, 2017
@Eduardomb22
Copy link
Copy Markdown

Is it possible to add a history for "full text search"too? or just the dictonary seach history?

@Frenzie
Copy link
Copy Markdown
Member Author

Frenzie commented Sep 14, 2017

I'm hoping to get some more feedback on the storage method (and possibly for someone else to implement it :-P but probably no such luck).

But yeah, once that is out of the way it should be no trouble saving all kinds of things.

@poire-z Any opinion on that JSON Lines thing I mentioned in #2789?

@Frenzie
Copy link
Copy Markdown
Member Author

Frenzie commented Sep 26, 2017

Adding cross-reference to #2566.

@Frenzie Frenzie force-pushed the dict_hist branch 2 times, most recently from c6039ef to 6ebe093 Compare October 3, 2017 07:05
@Frenzie
Copy link
Copy Markdown
Member Author

Frenzie commented Oct 3, 2017

@poire-z I wanted to do this about two weeks ago but Android got in the way. I've pushed my append-mostly data design, append-only in its current initial implementation.

Data files look like this (depending a little on what you stick in them of course):

LookupHistoryEntry{
    ["data"] = {
        [1] = {
            ["word"] = "the",
            ["time"] = 1507013495
        }
    },
    ["index"] = "lookup_history"
}
LookupHistoryEntry{
    ["data"] = {
        [2] = {
            ["word"] = "dare",
            ["time"] = 1507013497
        }
    },
    ["index"] = "lookup_history"
}
LookupHistoryEntry{
    ["data"] = {
        [3] = {
            ["word"] = "the",
            ["time"] = 1507013831
        }
    },
    ["index"] = "lookup_history"
}

@Frenzie Frenzie force-pushed the dict_hist branch 2 times, most recently from 37c3393 to 82d1ac1 Compare October 3, 2017 07:18
@Frenzie Frenzie changed the title [WIP] Add dictionary history [WIP] Add LuaData and Dictionary Lookup History Oct 3, 2017
Copy link
Copy Markdown
Contributor

@poire-z poire-z left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok, I trust your magic :)

return self
end

--- Adds item to the front of a table.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's no more to the front, isn't it?

-- some magic to allow for self-describing function names
local _local = {}
_local.__index = _local
setmetatable(_G, _local)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that polluting the global namespace with LookupHistoryEntry and such? If yes, any way to revert it when no more needed?
Not that I really care :) we're making sure everywhere things are local, so it shouldn't hurt.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's local _local so it doesn't. ;-)

-- bla.lua
do
	local _local = {}
	_local.__index = _local
	setmetatable(_G, _local)
	_local["Entry"] = "whatever"
	print(Entry)
end

_local["AnotherEntry"] = "bla"
> dofile("bla.lua")
whatever
bla.lua:10: attempt to index global '_local' (a nil value)
stack traceback:
	bla.lua:10: in main chunk
	[C]: in function 'dofile'
	stdin:1: in main chunk
	[C]: in ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, so that was indeed more magical that I thought :)


-- none of this for LuaData
LuaData.reset = nil
LuaData.flush = nil
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be for later: we could accumulate things to actually do the open/write/close once in this flush()

if f_out ~= nil then
os.setlocale('C', 'numeric')
f_out:write(self.name.."Entry")
f_out:write(dump(data))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got me pondering, till I realized LookupHistoryEntry{..} is actually a valid function call in lua, the ( ) are not needed.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, paraphrasing Roberto Ierusalimschy somewhat freely, Lua is just a fancy form of BibTeX.

https://www.lua.org/pil/12.html

If this format reminds you of BibTeX, it is not a coincidence. BibTeX was one of the inspirations for the constructor syntax in Lua.

@Frenzie Frenzie force-pushed the dict_hist branch 3 times, most recently from f0501cf to 6aa75bb Compare October 6, 2017 19:15
@Frenzie
Copy link
Copy Markdown
Member Author

Frenzie commented Oct 7, 2017

@poire-z Should be good for another look. The first part of the spec is a wholesale copy from LuaSettings and the second part was strongly inspired by DocSettings. (I don't quite understand the separation of DocSettings besides the legacy settings file handling but that aside.)

The idea is that:

  1. Under everyday use we may very well only ever append.
  2. Instead of trying to be too fancy in doing nil overrides and such, we just handle more difficult cases (i.e., indexed tables) by rewriting the whole file.
  3. For safety we keep a number of backups. I've included a max_backups so that more demanding stored data (bookmarks?) could opt to set it to a lower number. I haven't yet decided whether it can be 0 or if it should be a forced minimum of 1.

--- Opens a LuaData file.
function LuaData:open(file_path, name)
if name then self.name = name end
local new = {file=file_path, data={}}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that name, stored as self.name and not in new (the instance), prevent having multiple LuaData instances?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not quite sure what you mean?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lookup_history = LuaData:open(self.ui.doc_settings.sidecar .. "/lookup_history.lua", "LookupHistory")
-- LuaData.name = "LookupHistory"
bookmarks= LuaData:open(self.ui.doc_settings.sidecar .. "/bookmarks.lua", "Bookmarks")
-- LuaData.name = "Bookmarks"
lookup_history:saveSetting("toto", "tutu")
lookup_history:append() -- or :flush()

Wouldn't these last ones use LuaData.name = "Bookmarks" in their f_out:write(self.name.."Entry") ?

(Haven't tested, just asking, cause I don't know enough about the magical power of metatable and co:)

Copy link
Copy Markdown
Member Author

@Frenzie Frenzie Oct 7, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you do that you will indeed run into trouble because you're reusing the same instance. You'd have the same problem in LuaSettings.

The correct way to do that would be something like this:

local lookup_history_luadata = LuaData:new{name="bla"} -- our own separate LuaData instance
local lookup_history = lookup_history_luadata:open(self.ui.doc_settings.sidecar .. "/lookup_history.lua")

Perhaps I should remove the name argument from open to encourage proper instances? It's probably more of a leftover from my earliest tests than a well thought out feature anyway.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course I could write it so that :open does what :new does but that doesn't seem like a good idea with regard to internal consistency with other parts of the program (and Lua practices in general).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I got the feeling the setmetatable stuff was to make the new table a full instance of LuaData, and so that open() was indeed returning a LuaData instance (just like a :new() would).
I thought it would just need name to be part or new for it to be fully standalone.

No favorite way of doing it. It's just that, contrary to our single LuaSetting (G_reader_settings), you will probably have more LuaData, so better be ready for concurrent uses.

@poire-z
Copy link
Copy Markdown
Contributor

poire-z commented Oct 7, 2017

Looking ok.
Would you care (in this one or a futur PR) making that dict lookup history something that can be disabled, in Dictionary settings > Keep an history of looked up words ? And another one to reset this history?

@Frenzie Frenzie force-pushed the dict_hist branch 3 times, most recently from 1d7f5e8 to e2d100b Compare October 7, 2017 19:37
@Frenzie Frenzie changed the title [WIP] Add LuaData and Dictionary Lookup History Add LuaData and Dictionary Lookup History Oct 7, 2017
@Frenzie Frenzie merged commit d23d016 into koreader:master Oct 7, 2017
@Frenzie Frenzie deleted the dict_hist branch October 7, 2017 20:14
mwoz123 added a commit to mwoz123/koreader that referenced this pull request Nov 4, 2017
* Timer Reader with normal time format (koreader#3244)

* [fix, UX] Remove rounded corners from fullscreen History (koreader#3256)

Prevent visibility of readers's footer edges.

* [UX] Smaller bottom menu icons (koreader#3257)

And unrelated small optimisation in imagewidget

* Adapt ND to changes in code where 0 means no limit (koreader#3258)

+ reorganize config, improved comments

* Bump base for smoother image scaling (koreader#3259)

* README: fix typo

* ImageWidget: use MuPDF for scaling images (koreader#3260)

* Cache scaled images with scale_for_dpi

* Don't update original self.scale_for_dpi

A same ImageWidget object can be free()'d and re-render()ed,
and we would then lose scale_for_dpi on next renderings

* [fix] Some Android issues (koreader#3262)

Bump base for versioned Android libs.

Fixes koreader#3214. Fixes koreader#3112. Fixes koreader#3091. Fixes koreader#3039. Fixes koreader#2976. Fixes koreader#2960. Fixes koreader#2947.

* [UX] Add home button (koreader#3263)

Tap to go HOME. Hold to set current folder as HOME. When no HOME is set yet tap also asks to set current folder as HOME.

See koreader#2957 (comment)

Closes koreader#3200.

* [fix, Android] restore patch.lua and fix odd anonymous function issue

See koreader#3214 (comment) and koreader#3118 (comment)

* [Android] Enable CoverBrowser (koreader#3264)

I was reminded by @KenMaltby that this is disabled on Android.

https://www.mobileread.com/forums/showthread.php?p=3586162#post3586162

* Add back button to statistics (koreader#3267)

Closes: koreader#3236 and koreader#3235
Details:

> Also, when looking in days, we see info how many hours we read. It would be great if we could tap a day, it opens info on book(s) we read that day with info pertaining to that day, and if we tap on book, we receive book info for that book.

>  Statistics plugin is really great, I started to look in my reading statistics now much more, but all the time I can go only in one way. And I need to exit at lowest level of info. It would be great if I can use statistics plugin like part of menu, to go up and down within statistics data and exit at my wish, not when I have to.

Also added:

 * new statistics book by week
 * new statistics book by month
 * last year by week now we can see days in selected week

* [UX] Smaller borders (koreader#3266)

For years they've been smaller on higher DPI devices and likely very few people realized it was technically a bug. These values round up on lower DPI and smaller screen devices.

References koreader#3265

* README: update Android NDK/SDK info

* [fix] Avoid crash when set home (koreader#3269)

* README: various style, grammar, clarifications & updates

* [fix] Add hold_callback to IconButton (koreader#3271)

Also fix path display update.

Fixes koreader#3270.

* CoverBrowser: use MuPDF for down-scaling images (koreader#3272)

* README: more grammar

* README: update info about screen size/DPI

* README: typo and grammar

* CoverBrowser: use a cache for crengine books extraction (koreader#3277)

We previously disabled cache, but that may cause excessive
memory usage with big books. We now use a temporary
cache directory, that we clean when no more needed.

* Rename bookmark (koreader#3275)

* Allow for colored rendering (koreader#3276)

* Allow for colored rendering

Available with all engines (CRE, PDF, Images).
Needs to manually add setting: "color_rendering" = true

* Disable color for djvudocument

* Use Screen:isColorEnabled()

* Bump base

* [lang] OPDSCatalog ConfirmBox ok_text/cancel_text (koreader#3281)

* Fix refresh in Time range (koreader#3278)

* kodev: run with catchsegv by default (koreader#3283)

See koreader#2878 (comment)

Also fix `./kodev run -h` as alias for `--help` as it's always overwritten by `-h` in the sense of the much more important `--screen-height`.

* Time reader with normal time format v2 (koreader#3279)

* Allow for toggling color rendering

New menu item in Screen submenu.
hasColorScreen enabled for SDL device.

* Enable color rendering on Android

* [Fix] Show 'Follow Link' even when no dict installed

* [Fix] Full refresh when showing ImageViewer

* [fix] KOSync plugin server location (koreader#3288)

This is only step one. It remains to be determined why the connection is downgraded to sslv3, which will result in a handshake failure.

* Inform once about color rendering on supported devices (koreader#3289)

* [fix] KOSync server TLS connection (koreader#3291)

Bumps base for updated luasec.

The forum reminded me Sync was broken https://www.mobileread.com/forums/showthread.php?p=3587993 (as I don't use it myself). This finishes 23e2183.

Thanks to @houqp for helping with debugging Cloudflare. See https://support.cloudflare.com/hc/en-us/articles/203135314-How-can-I-enable-SSLv3-

Fixes koreader#2738. Fixes koreader#2178. (Duplicate bug reports by @Hzj-jie. :-P) Fixes koreader#3159. Fixes koreader#3158. Fixes koreader#2877.

* README: Change gcc requirement to 4.8

Technically 4.7 is probably fine for all but debugging symbols, for which a simple version check and workaround could be added. However, since all of the CI tests run 4.8 there is no regression prevention and it would be more future maintenance trouble than it's worth.

* credocument reader optimisation (avoid multiple TOC builds) (koreader#3292)

Avoid unnecessary work in ReaderView:onSetViewMode() and
ReaderRolling:updatePos() that would result in TOC being reset
and rebuilt (which can take time on books with huge TOC) during
reader setup.

* CoverBrowser: avoid crash when indexing some cre documents (koreader#3293)

* [feat] Pocketbook840 enable frontlight (koreader#3294)

* Detect PocketBook840 by GetSoftwareVersion()

* implement Frontlight-support for 840 via inkview

* [fix] Android frontlight control

* Revert 2 commits that caused crengine scroll mode side effects (koreader#3295)

394be8a (koreader#2855) and 2b3b310 (koreader#3183) introduced side effects (scroll mode
crashing and TOC being reset and rebuild). This reverts parts of them
not yet reverted.

* README: small typo

* kodev: add run android convenience shortcut (koreader#3297)

* [fix] kodev: default NDKABI=14 if not set for NDK 15 standalone toolkit

* ISSUE_TEMPLATE: add request to share crash.log (koreader#3296)

* Migrate Goodreads to https (koreader#3298)

* Bump base (koreader#3301)

Includes:

* [fix] Android viewport (fixes koreader#3148)
* Makefile: add luajit-clean target and auto-call it (koreader/koreader-base#531)

* [fix] Avoid multiple refreshes when opening credocuments (koreader#3300)

Only noticeable on Kindle (which uses REAGL as partial refresh).

* [fix] Android screen blackout on first light change (koreader#3303)

* Fix footer, stats, TOC position with cre in scroll mode (koreader#3304)

This gives the page position to these modules even in scroll mode.
Also, in readerrolling: don't query battery/charging status
when crengine does not need it (used only when it shows its top
progress bar).

* [fix] Crash on highlight in some situations (koreader#3306)

* Fix crash with keyboard navigation of onHold buttontables (koreader#3307)

* Fix crash with keyboard navigation of onHold buttontables

Would crash when encountering a separator or when the number
of buttons in a row changes.

* Reset previous selected item on new buttontable

* refresh footer (koreader#3313)

* Bump base to prevent Travis LuaJIT rebuild (koreader#3314)

* Avoid some full refreshes on Kindle (koreader#3315)

"partial" refresh causes a full (without black flash) refresh on
Kindle (which uses REAGL mode for partial refresh). This causes a
full redraw of widgets, which is a bit distracting with some of them:
- dictquicklookup: when showing next definition
- infomessage: when displaying a new one (Wikipedia Save as epub)

Also fix bottom menu, that even when closed, would still register
bottom area as dirty: this would cause top menu navigation to
cause a full partial refresh, only noticable on Kindle.

* [ReadTimer] Time from now (koreader#3311)

* Add LuaData and Dictionary Lookup History (koreader#3161)

* Add dictionary history

Fixes koreader#2033, fixes koreader#2998.

* Add LuaData

* table handling in base settings

* Add LuaData spec

* Option to disable show bottom menu on top menu activation (koreader#3316)

* Bump base (koreader#3318)

Includes koreader/koreader-base#539.

Changes to koptcontext.lua. Look here if anything weird changes in PDFs.

* [fix, Android] Don't steal frontlight control on start (koreader#3319)

I believe this should be `if isKobo()`, or better yet that the entire
block should be moved to `KoboPowerD:init()` because afaik that is the
only platform where the system doesn't provide trustworthy frontlight
information. But to be absolutely sure that I don't break anything (and I
don't want to spend any time on this atm) I'm temporarily excluding only
Android where this behavior is known to be problematic.

See discussion in koreader#3118 (comment)

References koreader#3118 (using keyword "references" because phrases like "possibly fixes"
result in GH autoclose).

* Add CircleCI (koreader#3321)

* CircleCI fixes

* shellcheck 0.4.5 fix `LC_ALL: en_US.UTF8` (can be removed for shellcheck 0.4.6)

* hush updated luacheck on `reader.lua`; we're not assigning any variables but `= nil` is redundant

* Better vertical centering of text in its box

Decide baseline vertical position according to font metrics, instead
of the hardcoded 0.7 (in textwidget, which made the text a little
bit up in its box), and 0.75 (in toggleswitch, which made the text a
little bit down in its box). This usually gives a value around 0.72
or 0.73 with our ui fonts, which looks about right.
ReaderFooter: add bottom padding, now that our text goes a few pixels lower

* Normalize some widgets appearance (those using ButtonTable)

This makes button heights similar in all uses of ButtonTable.
It depended on how the ButtonTable was used in each widget
(previously, first and last row may have different sizes than
the others).

buttontable.lua: more even buttons height whether zero_sep or not
framecontainer.lua: added padding_top/bottom/left/right (similar to
what was done for iconbutton)

The following widgets have been adapted for this, with some
additional fixes:

buttondialog.lua
buttondialogtitle.lua: wider title with adequate padding
confirmbox.lua + multiconfirmbox.lua: dismissable via tap outside
inputdialog.lua + multiinputdialog.lua: more even vertical padding between elements
imageviewer.lua
textviewer.lua
datewidget.lua
timewidget.lua

Additionaly: frontlightwidget.lua: fixed width of progress bar that
was exceeding window width since the Size scaling adjustements

* Flash KeyValuePage item when callback (koreader#3322)

* Fix readerfooter_spec (koreader#3326)

My bad, leftover from koreader#3323 but not caught due to some Travis → CircleCI migration birth pains.

* CircleCI docs (koreader#3327)

* Environment variable key replaced by CircleCI GitHub push key.
  See <https://circleci.com/docs/1.0/adding-read-write-deployment-key/>.

* Temporarily added `|| true` to luacov so it won't cause a fail.

* Fix stats in cre scroll mode (koreader#3331)

* CircleCI cache fix (koreader#3329)

* Update luarocks/shellcheck/shfmt check based on `deps-{{ arch }}-{{ checksum ".ci/install.sh" }}`
* We generate a git-rev-base to check whether we can trust the cache
  with `build-{{ arch }}-{{ checksum "git-rev-base" }}`

Binary dependencies require `{{ arch }}` because there are different CPUs used on the servers.
More information here: https://discuss.circleci.com/t/use-the-arch-cache-template-key-if-you-rely-on-cached-compiled-binary-dependencies/16129

* [fix] Statistics: onPosUpdate and save stats when closing document (koreader#3332)

* Fix cre scroll page update and allow jumping to page (koreader#3333)

This makes Go to, Skim to and TOC page selection work in
scroll mode, and page given to other module more accurate
(previously, we were one action lagging).

* CircleCI parallelization

* Enable parallelism in .circleci/config.yml
* Add BUSTED_SPEC_FILE to Makefile testfront
* Use it in .ci/test.sh with xargs for test parallelization

NB This is the dumb method of improving test time.
Ideally we want a workflow fan-in/fan-out approach.

* [fix] MockTime spec

* [fix] #nocov on broken scroll mode tests

* CircleCI: run docs & coverage in first container only (koreader#3335)

I wrote this whole complicated wofkflow-based config file, but except
in the case of a base rebuild it wouldn't really be any faster than
this simple tweak.

```
defaults: &defaults
    docker:
      - image: houqp/kobase:0.0.5
        environment:
          EMULATE_READER: 1
          # this is for shellcheck 0.4.5 and lower; can be removed for 0.4.6
          LC_ALL: en_US.UTF8

version: 2
jobs:
  install-and-build:
    <<: *defaults
    steps:
      - checkout
      - restore_cache:
          keys:
            # binary dependencies require {{ arch }} because there are different CPUs in use on the servers
            - deps-{{ arch }}-{{ checksum ".ci/install.sh" }}
      # need to init some stuff first or git will complain when sticking in base cache
      - run: git submodule init base && git submodule update base && pushd base && git submodule init && git submodule update && popd
      # we can't use command output directly for cache check so we write it to git-rev-base
      - run: pushd base && git_rev_base=$(git describe HEAD) && popd && echo $git_rev_base && echo $git_rev_base >git-rev-base
      - restore_cache:
          keys:
            - build-{{ arch }}-{{ checksum "git-rev-base" }}
      - run: echo 'export PATH=${HOME}/bin:${PATH}' >> $BASH_ENV
      - run:
          name: setup
          command: .ci/before_install.sh
      - run:
          name: install
          command: .ci/install.sh
      - save_cache:
          key: deps-{{ arch }}-{{ checksum ".ci/install.sh" }}
          paths:
            - "/home/ko/bin"
            - "/home/ko/.luarocks"
            # compiled luarocks binaries
            - "install"
      - run:
          name: fetch
          command: .ci/fetch.sh
      - run:
          name: check
          command: .ci/check.sh
      - run:
          name: build
          command: .ci/build.sh
      - save_cache:
          key: build-{{ checksum "base/git-rev" }}
          paths:
            - "/home/ko/.ccache"
            - "base"
      - persist_to_workspace:
          # Must be an absolute path, or relative path from working_directory
          root: "./"
          # Must be relative path from root
          paths:
            # front build
            - "koreader-emulator-x86_64-linux-gnu/koreader"

  test:
    <<: *defaults
    parallelism: 4
    steps:
      - checkout
      - restore_cache:
          keys:
            # binary dependencies require {{ arch }} because there are different CPUs in use on the servers
            - deps-{{ arch }}-{{ checksum ".ci/install.sh" }}
      # need to init some stuff first or git will complain when sticking in base cache
      - run: git submodule init base && git submodule update base && pushd base && git submodule init && git submodule update && popd
      # we can't use command output directly for cache check so we write it to git-rev-base
      - run: pushd base && git_rev_base=$(git describe HEAD) && popd && echo $git_rev_base && echo $git_rev_base >git-rev-base
      - restore_cache:
          keys:
            - build-{{ arch }}-{{ checksum "git-rev-base" }}
      - run: echo 'export PATH=${HOME}/bin:${PATH}' >> $BASH_ENV
      - attach_workspace:
          # Must be absolute path or relative path from working_directory
          at: "./"
      - run:
          name: test
          command: .ci/test.sh

  docs-and-coverage:
    <<: *defaults
    steps:
      - checkout
      - restore_cache:
          keys:
            # binary dependencies require {{ arch }} because there are different CPUs in use on the servers
            - deps-{{ arch }}-{{ checksum ".ci/install.sh" }}
      # need to init some stuff first or git will complain when sticking in base cache
      - run: git submodule init base && git submodule update base && pushd base && git submodule init && git submodule update && popd
      # we can't use command output directly for cache check so we write it to git-rev-base
      - run: pushd base && git_rev_base=$(git describe HEAD) && popd && echo $git_rev_base && echo $git_rev_base >git-rev-base
      - restore_cache:
          keys:
            - build-{{ arch }}-{{ checksum "git-rev-base" }}
      - run: echo 'export PATH=${HOME}/bin:${PATH}' >> $BASH_ENV
      - attach_workspace:
          # Must be absolute path or relative path from working_directory
          at: "./"
      - run:
          name: cleanup
          command: .ci/after_success.sh

workflows:
  version: 2
  build-and-test:
    jobs:
      - install-and-build
      - test:
          requires:
            - install-and-build
      - docs-and-coverage:
          requires:
            - install-and-build
```

* [CI] Switch to Codecov (koreader#3336)

The debug output from Coveralls was rubbish. `"Build processing error"`
Besides some basics like checking if we were actually sending
valid JSON that gives us absolutely nothing to work with.

* [feat] VirtualKeyboard cursor navigation (koreader#3290)

Still lacks Japanese due to insufficient knowledge of the language.

* README: style updates (koreader#3337)

Improved some stylistic issues.

* Updated Travis CI reference to CircleCI.
* Switched to CircleCI "shield" style badge to fit in better with the other badges.

* Small visual fixes to Menu (koreader#3338)

Rationalize horizontal construction of Menu items (TOC,
Bookmarks, Classic file views) for more even padding.
Align "x" close button diagonaly with top right border and title.

Also add forgotten scale_for_dpi to MultiConfirmBox

* Allow for disabling flashing of menu, icons and buttons  (koreader#3339)

* CircleCI: finishing touches (koreader#3340)

* junit test results; unfortunately this seems to conflict with the verbose out

* fix deps cache: two files can change independently

* verbose print obsoleted by gtest in upsream busted

* [fix] verbose_print deprecated by gtest

* [fix] gettext: die already you stupid language not found error (koreader#3341)

* Bump base (fixes landscape on Android) (koreader#3342)

Includes koreader/koreader-base#542

* Add gettext_spec stub (koreader#3343)

* dbg_spec: setVerbose test

* [chore] Rework util spec, rework util.secondsToClock: round seconds to minutes in 00:00 mode + spec

Most of the tests in util_spec were the wrong way around.
It's `assert(expected, given)`.

* [chore] Ignore empty files and tables in DocSettings (koreader#3348)

* calibre 'series' metadata fixes (koreader#3349)

Decode XML entities in series metadata and display decimal in series number
if any.

* Menu (TOC, bookmarks): add padding before right text (koreader#3350)

* FileBrowser: change page to show last file or previous subdir (koreader#3351)

When going from reader to filemanager, we are in the directory
containing the last_file. With this, we will also be on the page
showing this file.
When in filemanager and going up (".."), we will also be on the
page containing the directory we came from.

* Avoid recalculation of partial_md5_checksum at each opening (koreader#3352)

This is done by/for kosync plugin at each opening, because
the docsettings was re-opened and saved for this, but later
overwritten by the current koreader docsettings - so it was
redone each time. This correctly adds this partial_md5_checksum
to the current koreader docsettings, which will be saved on
document closing - so it will not be redone next time.
Note: this partial_md5_checksum is not (yet) used by anything.

* kodev: Add $ANDROID_ARCH to enable x86 build (koreader#3353)

You'll still have to call it with `ANDROID_ARCH=x86 ./kodev build/release/run android`.

Don't forget to `./mk-luajit.sh clean` in luajit-launcher when changing architectures.

* Bump android-luajit-launcher
  This includes the fix for Android 8. Fixes koreader#3126.
* Bump base

* [android] fix hide nav bar on activity create (koreader#3357)

* [Goodreads] lookup improvement (koreader#3354)

* add navigation button (next, prev) on the bottom of screen
* flash item when selecting
* small lookup improvements
* fix refresh "Please wait..."

* Small visual fixes to top menu (koreader#3356)

To make it more alike bottom menu:
- left and right border not displayed
- line below icons extends to screen borders
- same bottom border size
And make separator lines have same padding on both sides

* [Android] Enable DjVu (bump base) (koreader#3358)

Closes koreader#1534.

* djvu: enable color rendering (koreader#3361)

* djvu: enable color rendering

* Bump base

* README: fix "unarchieve" typo (koreader#3362)

* Bump base (koreader#3364)

*  Change JPG rendering to RGB in openJPGDocumentFromMem
*  fix build on Mac OSX and fix SDL input issues on OSX
* And some other bumps (glib, crengine, libiconv, libjpeg-turbo).

* Fix some widget title height and close button alignment (koreader#3366)

* close button alignment (koreader#3367)

* Enable Edit (rename bookmark) when tap on highlight (koreader#3369)

Also fix a few crash possibilities when unhighlighting.
Also fix bug with binary search that could not be able to remove bookmark
when there are multiple bookmarks/highlights on the same page.

* [fix] util.secondsToClock 00:60 should be 01:00 (koreader#3371)

* [Kobo] Add preliminary Kobo Aura H2O2 definition (koreader#3372)

Cf. ongoing discussion in KSM8 thread: https://www.mobileread.com/forums/showthread.php?p=3596020#post3596020

* Bump base (sdcv 0.5.2; fix *buntu 17.10 build & segfault) (koreader#3378)

Fixes koreader#3302. Fixes koreader#3071.

* kodev: add debug flags (koreader#3379)

* `--gdb=X`
* `--valgrind=X`

* Book Information: added file size (koreader#3380)

* Added util.getFriendlySize() (koreader#3381)

* Added util.getFriendlySize()

* Allow for GB

* Fix location of progress bar ticks (koreader#3382)

* Added util.getFormattedSize() (koreader#3383)

* Bump android-luajit-launcher (contains x86 nightly build fix) (koreader#3384)

* FileBrowser: optimize 'change page to show last file'

This feature, introduced some days ago, was actually
doing 2 updateItems calls: the initial one, and a second
to switch to focused_file page (cheap with classic display mode,
less cheap with CoverBrowser modes).
This change allows doing that in a single call.

* CoverBrowser: some optimizations

Speedup initialization (needs to be done only once, and avoid
saving current mode to sqlite each time) and reader-to-filebrowser
switch, by doing a single rendering (instead of 2 or 3 previously).
ListMenu: cache sidecar file parsing results (page/percent
completed) for the browsing session duration.
Fix "No choice available" when on last page and changing
to a display mode with less pages.

* Android NetworkManager (koreader#3386)

* Bump base (sdcv patch to allow enable/disable dicts) (koreader#3393)

* Fullscreen android - resize window after toggle fullscreen (koreader#3392)

* [fix, Android] adjust touch location for koreader#3392 (koreader#3394)

* NewsDownloader promptWifiOn (koreader#3388)

* [feat] SkimToWidget chapter markers & next/prev chapter/bookmark (koreader#3389)

koreader#2819 (comment)
+ refactoring code in skimtowidget
+ add chapter markers
+ add next/prev chapter buttons
+ add next/prev bookmark buttons

* [Android] Bump luajit-launcher (more lib loading debug info) (koreader#3401)

* [PocketBook] Use inkview-calls for battery-percentage/charging (koreader#3402)

* PocketBook: Basic device-detection /  PocketBook631 (Touch HD) support (koreader#3403)

Add basic device-detection via libinkview, support PocketBook631. Fixes koreader#3312

* Bump base (sdcv -u patch and framebuffer setViewport dynamically fix) (koreader#3404)

* KeyValuePage: option to align value to the right when half-overflow (koreader#3407)

Applied in FileManagerBookInfo.

* Bump base (sdcv -u patch for android) (koreader#3413)

* [UX] Fix dict title width to not go over CloseButton (koreader#3414)

* Wiki save as epub: close highlight menu when switching document (koreader#3417)

* separate strings for singular and plural (koreader#3420)

* Bump base (sdcv -u patch for android again) (koreader#3421)

* DataStorage: also create data/tessdata and minor string concat performance improvements (koreader#3430)

* [fix] Dictionary no fuzzy search flag (koreader#3433)

Upstream uses `-e` rather than the `-f` that was first introduced in our fork.

References koreader#3429 (comment)

* NewsDownloader: use feed.description as news context instead download full web page (koreader#3426)

Fixes koreader#3425.

* Bump base (updated libk2pdfopt) (koreader#3437)

* kodev: add assert_ret_zero after make android-ndk (koreader#3438)

Fixes koreader#3408.

* [build] Add -debug suffix to KODEBUG builds (koreader#3439)

* NewsDownloader: Remove NewsDownloader files from history  (koreader#3424)

* kodev: added setup_env to run android

Otherwise it didn't pick up on the -debug suffix.

* Bump android-luajit-launcher

* [Android] Wifi status in footer (koreader#3396)

Implemented through the Android API so it's reasonably cheap.

* NewsDownloader: wifi off prompt after download (koreader#3395)

* [UX] Improve ConfigDialog (koreader#3443)

* [CI] Also run coverage on plugins (koreader#3447)

* NewsDownloader: Compatibility for users with previous configuration files (koreader#3445)

* Bump base (fix wpa crash by @civalin) (koreader#3449)

For:
* [fix] filter out more bad lines in wpa scan result (koreader/lj-wpaclient#3)

Fixes koreader#3436

* [CI] apply patch for busted junit testcase time (koreader#3450)

Apply junit testcase time fix so CircleCI can actually do something useful
with the results. This can be removed once there is a busted 2.0.rc13 or final

See lunarmodules/busted@830f175

* [fix, CI] silly typo in busted junit output patch

* README: tiny grammar fix (koreader#3453)

* ConfigDialog improvements 2 (koreader#3455)

+ Fix height for ToggleWidget
+ Able ToggleWidget width more than half of width of screen

* [fix] Disable Highlight button when text is not selected (koreader#3457)

* i10n/README: small stylistic changes

* doc/Unit tests: minor extra explanation (koreader#3462)

* More contrast settings (koreader#3463)

Close: koreader#2133 
More info: koreader#2133
@poire-z
Copy link
Copy Markdown
Contributor

poire-z commented Dec 16, 2017

How would you go about adding Wikipedia lookup history?:
Use the same lookup_history file or an independant one?
If same file, different index key than ["index"] = "lookup_history"
We would need to store wikipedia language, intro or full mode, so we could launch back the exact same lookup. Would that be stored as additional data item attributes or in the word, like ["word"] = "W.fr: Magadha"?

(I am working on adding some kind of image support to TextBoxWidget to display wikipedia pages main thumbnail in lookup results - and I miss such history, i'm too often wondering what was that page i looked up earlier that caused this or that problem...)

@Frenzie
Copy link
Copy Markdown
Member Author

Frenzie commented Dec 16, 2017

I haven't given it any thought really. What you say sounds fine to me, except I think that generally speaking different index (if desired) also implies different file?

@poire-z
Copy link
Copy Markdown
Contributor

poire-z commented Dec 16, 2017

I hadn't really read luadata.lua, but looking at how it was used in readerdictionary.lua:

lookup_history = LuaData:open(DataStorage:getSettingsDir() .. "/lookup_history.lua", { name = "LookupHistory" })
lookup_history:addTableItem("lookup_history", {...})
lookup_history:has("lookup_history")
lookup_history:readSetting("lookup_history")

I thought you had planned the way to store different items in a same luadata file, and that you named it lookup_history so it's generic for all things looked up.

But it's a lot easier if I can just use a new file wikipedia_history.lua and duplicate code in readerwikipedia.lua (there are enough differences to justify that), and make the two lookup histories dealt with by independant menus (display, clean, disable).

@Frenzie
Copy link
Copy Markdown
Member Author

Frenzie commented Dec 16, 2017

That's true; reading between the lines my intentions were to potentially use something like type=wikipedia. But if you use a different index I don't think there are any advantages (in this case) to using a single file.

@fyxzlh
Copy link
Copy Markdown

fyxzlh commented Jul 6, 2018

Would you add "usage" in Dictionary Lookup History? just like vocabulary builder in kindle

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

History for searched words? [FR]Dictionary look-up history like Kindle Vocabulary Builder

4 participants