Chore: Don't cache static data tables (ui/elements, ui/data, kb layouts & co) by NiLuJe · Pull Request #12519 · koreader/koreader

NiLuJe · 2024-09-14T18:30:06Z

Using require didn't make much sense since most of them are only used in a single place anyway, and it takes care of a few weird interactions in the process (besides not polluting package.loaded with useless crap ;)).

This change is

Use dofile instead, as we don't want to cache those tables, as we or MenuSorter may insert new data into it. c.f., koreader#12372 (comment)

i.e., use dofile instead of require

sadface.

hius07 · 2024-09-14T18:59:49Z

Huh, cool.

Commodore64user · 2024-09-14T20:52:05Z

ohhh, this is what that other comment meant. I guess...

but wouldn't this also need adjusting then?

koreader/frontend/ui/menusorter.lua

Lines 27 to 29 in bf290e0

    
           if lfs.attributes(menu_order..".lua") then 
        
               return require(menu_order) or {} 
        
           end

NiLuJe · 2024-09-14T22:10:47Z

That one slipped through my regex as it's user data, but, yeah ;).

hius07 · 2024-09-15T03:36:40Z

ohhh, this is what that other comment meant. I guess...

Exactly, thanks. And wow.

Commodore64user · 2024-09-15T08:16:12Z

Exactly, thanks. And wow.

I beg your pardon?

For reference, this is the "other" comment i alluded to:

Or, better yet, just use dofile ;).

Frenzie · 2024-09-15T08:17:59Z

-        if lfs.attributes(menu_order..".lua") then
-            return require(menu_order) or {}
-        end
+        local ok, data = pcall(dofile, menu_order)


I prefer the original (not talking about the require). Okay, so the user made a typo… but that's the point! They should be immediately notified that they messed up, shouldn't they?

Concretely, this is conflating "file doesn't exist" and "file has syntax error or similar".

I think you can alternatively write it along these lines:

if not ok and not data:match(menu_order) then -- Syntax error, it'll contain the filename in a file not found error -- `cannot open bla.lua: No such file or directory` error("immediate proper feedback") end

Frenzie · 2024-09-15T08:34:56Z

Using require didn't make much sense since most of them are only used in a single place anyway

I can't seem to find the relevant page atm, but I think loadstring is only ever interpreted. Is dofile compiled into bytecode?

NiLuJe · 2024-09-15T14:55:19Z

I can't seem to find the relevant page atm, but I think loadstring is only ever interpreted. Is dofile compiled into bytecode?

dofile is usually implemented as a wrapper around loadfile.

c.f., https://github.com/LuaJIT/LuaJIT/blob/87ae18af97fd4de790bb6c476b212e047689cc93/src/luajit.c#L168-L172

And https://www.lua.org/pil/8.html for more context perhaps?

They all compile to bytecode, AFAICT, c.f., the description of load: http://luajit.org/extensions.html

Frenzie · 2024-09-15T18:03:12Z

I was referring to the Not Yet Implemented page, which seems to have vanished (and as such this copy is therefore outdated, but still mostly or entirely applicable): https://web.archive.org/web/20220708225331/http://wiki.luajit.org/NYI

Anyway, I probably partially misremembered, although of course my main point was to check. Whether the call is compiled is irrelevant. :-)

NiLuJe · 2024-09-15T18:07:50Z

Oh, yeah, no clue where that went ^^.

But on that note, fun fact: coroutines are NYI, so they're not JIT'ed ;).

NiLuJe

Reviewed 37 of 37 files at r1, 1 of 1 files at r3, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @Frenzie)

@Sonnenfee

KOReader 2024.11 "Slang" ![koreader-2024-11-slangie](https://github.com/user-attachments/assets/cb9e68bf-aa0f-47c7-a093-46fbf8bb19c3) Some of you on Android may have had dictionary issues. That should be fixed now. Connecting to Wi-Fi should be more reliable on platforms where we manage it ourselves, most notably Kobo (koreader/lj-wpaclient#11). Color is now supported when reflowing (koreader#12085). This month's logo was contributed by @Sonnenfee We'd like to thank all contributors for their efforts. Some highlights since the previous release include: * Certain network features will now work in Flatpak (<flathub/rocks.koreader.KOReader#52>) @kii-chan-reloaded * 'Use left and right keys for page turning' option added (koreader#12174) @Commodore64user * Text editor: enhancement (koreader#12176) @hius07 (1) Speed up adding/deleting chars (by avoiding a call of expensive splitToChars), noticeable on big files. (2) Indicate select mode: wider cursor. * Annotations: more fixes (koreader#12181) @hius07 (1) Allows annotations with equal datetime fields. (2) Allows page bookmarks with equal page numbers. * kopt: color support (koreader#12085) @benoit-pierre * bump base (updated build system) & fix macOS CI (koreader#12187) @benoit-pierre * AltStatusBar/Footer: add the read timer value (koreader#12002) @zwim * bump translations (koreader#12193) @pazos * CRE call cache: don't wrap setViewMode (koreader#12194) @poire-z * bump crengine: various fixes and optimizations (koreader#12200) @poire-z * Add author option for alt status bar (koreader/crengine#575) (koreader#12197) @trash-pandy * Page turns: invert vertical tap zones (koreader#12195) @hius07 * Annotations: pageno honors reference pages and hidden flows (koreader#12214) @hius07 * ToggleSwitch: fix long-press (koreader#12215) @hius07 * SDL: Return true in NetworkMgr:isConnected on !hasWifiToggle platforms (koreader#12217) @NiLuJe * Kobo: Refactor various aspects of the Kaleido/MTK support (koreader#12221) @NiLuJe * ReaderAnnotation: fix doesMatch func (koreader#12231) @hius07 * bump base (koreader#12241) @benoit-pierre * NetworkMgr: Attempt to handle wpa_supplicant rescans better (koreader#12236) @NiLuJe * miscellaneous build system changes (koreader#12242) @benoit-pierre * make: handle old build data directory (koreader#12259) @benoit-pierre * ReaderFooter: improve (koreader#12243) @hius07 * Annotations: fix exporting (koreader#12261) @hius07 * make: install missing `cr3.css`… (koreader#12262) @benoit-pierre * [fix] NetworkMgr: apply HTTP proxy setting on startup (koreader#12260) @Frenzie * Kobo: Track frontlight state more accurately on suspend/resume (koreader#12256) @NiLuJe * Make W-Fi scan more reliable (koreader#12265, koreader/lj-wpaclient#11) @NiLuJe * ReaderAnnotation: fix doesMatch func 2 (koreader#12270) @hius07 * make: fix android update sdcv (koreader#12275) @benoit-pierre * [plugin] Reader footer: check if header resp. footer are available (koreader#12264) @zwim * ReadHistory: On remove, trust the index from FileManagerHistory if possible (koreader#12253) @NiLuJe * ReaderUI: add `seamless` argument to switchDocument (koreader#12290) @ziz57 * Fix early unhighlight after Add to vocab builder (koreader#12288) @poire-z * [CoverBrowser.MosaicMenu] Increase the thickness of the mosaic’s focus underline (koreader#12189) @Commodore64user * doc: update building documentation (koreader#12295) @benoit-pierre * NetworkMgr: fix debug trace (koreader#12297) @benoit-pierre * Panel zoom: Properly handle `renderPage()` not rendering the whole page (koreader#12296) @fbriere * PowerD: Keep track of frontlight state change by interactive callers for suspend/resume purposes (koreader#12283) @NiLuJe * InputDialog and MultiInputDialog: improve enter and edit callback handling, and implement addWidget (koreader#12289) @ziz57 * ReaderSearch: "All text" improve (koreader#12287) @hius07 * Show/Hide Virtual keyboard and more keyboard shortcuts (koreader#12162) @Commodore64user * bump base; mostly thirdparty bumps (koreader#12311) @benoit-pierre * Document: Do not cache panel-zoom tiles to disk and fix their caching and rendering (koreader#12303) @NiLuJe * Disable the legacy ReaderRotation module (koreader#12309) @NiLuJe * bump luajit (koreader#12317) @benoit-pierre * [AutoWarmth] Delay front light actions after resume (koreader#12315) @zwim * Allow numbers in measure text: fixes a crash in table of contents when debugging on (koreader#12310) @zwim * ReaderFooter: Add page-turn item (koreader#12249) @Commodore64user * luajit launcher: out of tree build + don't rebuild luajit (koreader#12285) @benoit-pierre * BookInfo: rating, review (koreader#12313) @hius07 * fix missing android APK asset (koreader#12333) @benoit-pierre * ImageWidget: Don't attempt to pan if we haven't rendered the bb yet (koreader#12331) @NiLuJe * OTM: Unbreak koreader#11999 (koreader#12337) @NiLuJe * Kobo: Refactor suspend in order to be able to catch input events sent during the 2s window of ntx madness (koreader#12330) @NiLuJe * [Menu] Remove title bar buttons from layout on hasSymKey or hasScreenKB (koreader#12328) @Commodore64user * [doc] Build: update Android instructions to openjdk-17-jdk (koreader#12345) @Frenzie * Support 'Ctrl' shortcuts on hasKeyboard (koreader#12347) @Commodore64user * Annotations: fix page numbers (koreader#12324) @hius07 * Bug Fix: DPad conflict on Kindle Keyboard (koreader#12341) @Commodore64user * Nit: Make some cache keys prettier (koreader#12354) @NiLuJe * Dispatcher: minor wording fixes (koreader#12356) @hius07 * kodev rework (koreader#12338) @benoit-pierre * czmq & libzmq: update to 4.2.1 & 4.3.5 respectively (koreader#12350) @benoit-pierre * [TextViewer] Add hamburger menu to FocusManager and Menu key (koreader#12365) @Commodore64user * ci: fix test / coverage (koreader#12367) @benoit-pierre * A couple of tiny fixes (koreader#12359) @NiLuJe * android: tweak update rule (koreader#12370) @benoit-pierre * doc: update Android build instructions (koreader#12368) @benoit-pierre * Display menu preview of italic-only fonts (koreader#12351) @charrarr * ReaderStatus, Book status widget: cleanup (koreader#12343) @hius07 * ci/macos: fix homebrew packages install (koreader#12382) @benoit-pierre * Fix terminal broken cursor and backspace (koreader#12375) @zwim * Screensaver: add option to rotate to fit screen (koreader#12366) @charrarr * Minor keyboard corrections (koreader#12372) @Commodore64user * Touch menu: always go up to correct parent page (koreader#12378) @hius07 * Collections: fix context menu (koreader#12383) @hius07 * PDF: no "Contrast set to" notification on opening (koreader#12391) @hius07 * Closing Bookmark details dialog with X should also refresh list entry (koreader#12390) @fbriere * Add numerical key_codes to event_map on kindle 3 (koreader#12358) @Commodore64user * FocusManager: Fix focus_flags check in moveFocusTo, and deal with the fallout (koreader#12361) @NiLuJe * Various notification, text editor and footnote popup minor fixes (koreader#12406) @poire-z * Bug: Account for local reading order direction adjustments. (koreader#12388) @Commodore64user * android: add custom `adb logcat` formatter (koreader#12385) @benoit-pierre * kodev: improve `kodev prompt` (koreader#12408) @benoit-pierre * kodev: fix `./kodev run android…` (koreader#12407) @benoit-pierre * Readerfont: Current font on top in menu; New fonts from current session on top in font test doc (koreader#12410) @jonnyl2 * Fix a couple of potentially spurious footer refreshes (koreader#12422) @NiLuJe * FileManager: Handle the FlushSettings event (koreader#12425) @NiLuJe * PageBrowser: fix drawing of thicker thumbnail frame on tap (koreader#12432) @poire-z * FileManagerSetDefaults: Handle strings that we default to nil (koreader#12433) @NiLuJe * Input: Ignore ABS_X, ABS_Y & ABS_PRESSURE for snow_protocol (koreader#12427) @NiLuJe * Add color highlight menu (koreader#11044) @smasher816 * Exporter: choose highlight styles to be exported (koreader#12431) @hius07 * ReaderView: Don't crash on pre-koreader#11044 highlights without a color field (koreader#12438) @NiLuJe * Terminal fixes (koreader#12369) @ziz57 * Dispatcher/DeviceListener: Temporarily turn on Accelerometer/GSensor for 5 sec (koreader#12419) @jonnyl2 * fix bug preventing export to joplin; (koreader#12446) @bitterlox * bump base: blitbuffer, android build, crengine (koreader#12451) @poire-z * InputText: Unbreak onTapTextBox on an already in-focus field (koreader#12449) @NiLuJe * Update el_popup.lua (koreader#12463) @ichnilatis-gr * [Dispatcher] add [missing] conditions to dispatcher items (koreader#12453) @Commodore64user * ReaderView: Fix a couple of edge-cases in onReaderFooterVisibilityChange (koreader#12466) @NiLuJe * misc: Move cleanupSelectedText to util (koreader#12477) @NiLuJe * ReaderHighlight: Fix an old typo in the OCR help string (koreader#12479) @NiLuJe * Highlights popup dialog: add "Color" and "Details" buttons (koreader#12461) @hius07 * ButtonDialogTitle: remove (koreader#12394) @hius07 * [Dispatcher] new events for selecting links (koreader#12474) @Commodore64user * Doc: Update quickstartguide on kindle non touch (koreader#12426) @Commodore64user * [NT] Call GotoViewRel for page up/down on useDPadAsActionKeys devices (koreader#12495) @cccslater * bump base (koreader#12503) @benoit-pierre * Defaults: Deprecate DKOPTREADER_CONFIG_DOC_LANGS_TEXT (koreader#12504) @NiLuJe * ReaderAnnotation: skip old highlights with invalid xpointers (koreader#12494) @hius07 * Cloud Sync: handles potentially incompatible backup and incoming dbs (koreader#12470) @weijiuqiao * VocabBuilder.koplugin: Better handling word context (koreader#12469) @weijiuqiao * Write highlights into PDF: revisited (koreader#12509) @hius07 * SyncService: Only require online connectivity when using Dropbox (koreader#12520) @NiLuJe * Chore: Don't cache static data tables (ui/elements, ui/data, kb layouts & co) (koreader#12519) @NiLuJe * Fix page gap inconsistencies between doc open & config change (koreader#12507) @NiLuJe * Kindle: Restore hall effect sensor state on startup (koreader#12521) @NiLuJe * VocabBuilder.koplugin: save dict headword as entry (koreader#12530) @weijiuqiao * Add PocketBook Verse Pro Color (PB634K3) (koreader#12476) @ElimGarak1 * Onyx Boox Go 10.3 support (koreader#12533) @hugleo * Statistics: fix null terminated strings, add toggle stats (koreader#12547) @hius07 * input: drop use of `ffi/input` (koreader#12486) @benoit-pierre * ci/circle: fix tests parallelism (koreader#12551) @benoit-pierre * input: fix kobo init (koreader#12553) @benoit-pierre * [ReaderFooter] update status bar in real time when inverting page turning (koreader#12424) @Commodore64user * input: fix pocketbook init (koreader#12555) @benoit-pierre * use new `ffi.loadlib` helper (koreader#12545) @benoit-pierre * input: fix loading of `fbink_input` library (koreader#12560) @benoit-pierre * Fix typos in quickstart (koreader#12566) @cheywood * File search: improve (koreader#12549) @hius07 * [DictQuickLookup] shortcuts for editing searches (koreader#12539) @Commodore64user * Profiles: auto-execute on events (koreader#12564) @hius07 * Exporter: export to book folder, register to Dispatcher (koreader#12569) @hius07 * sdl: use our compiled version for the AppImage (koreader#12575) @benoit-pierre * Add kobov4 & kobov5 targets (koreader#12568) @NiLuJe * android: switch to monolibtic mode (koreader#12576) @benoit-pierre * android: fix apkanalyzer location (koreader#12563) @benoit-pierre * Disabled rotation map for Android devices where "hw media keys follow gravity" (koreader#12557) @jkl16 * Quickstart guide kindle illustrations (koreader#12577) @Commodore64user * [plugin] Exporter: fix typo (koreader#12592) @Frenzie * Dispatcher: values' range consistent with bottom menu (koreader#12580) @hius07 * metadata translations for fdroid/flathub (koreader#12581) @pazos * ProgressWidget: fix misalignment (koreader#12526) @wbstx * Add wifi support to Cervantes Touch Light devices (koreader#12148) @jsanpe * ReaderTypeset: fix margins info (koreader#12609) @hius07 * DocSettingsTweak: apply to books with status (koreader#12606) @hius07 * Menu widget: rotation handler (koreader#12573) @hius07 * Profiles: add "on rotation" auto-execute event (koreader#12612) @hius07 * ReaderHighlight: adjustable "Edit highlight" dialog position (koreader#12599) @hius07 * KOSync/Kobo: Explicitly kill Wi-Fi on suspend (koreader#12616) @NiLuJe * Kobo: Minor simplification after koreader#12616 (koreader#12629) @NiLuJe * bump luajit-launcher (koreader#12630) @benoit-pierre * Terminal: Minor cleanups (koreader#12631) @NiLuJe * Menu widget: draw focus on focused item (koreader#12583) @hius07 * Cloud storage: fix sorting non-English filenames (koreader#12644) @hius07 * Text editor: show current line number (koreader#12632) @hius07 * InputText: fix Terminal plugin input (koreader#12652) @hius07 * ReaderFont test: Error correction -- new fonts were not rendering in own font (koreader#12646) @jonnyl2 * ReaderHighlight: fix color for extended highlight in pdf (koreader#12654) @hius07 * Kindle Scribe: fix elseif for orientation code (koreader#12663) @mergen3107 * Menu: Fix fallout from koreader#12583 on hasDPad devices (koreader#12657) @NiLuJe * Misc: Some more FocusManager fallout (koreader#12667) @NiLuJe * plugin messages (koreader#12636) @pazos * And deprecation to autofrontlight (koreader#12619) @pazos * Add deprecation for unmaintained targets (koreader#12620) @pazos * Profiles: more auto-exec triggers (koreader#12651) @hius07 * PatchManagement: prettify menu (koreader#12668) @hius07 * Fix KOReader spelling in the code (koreader#12670) @mergen3107 * Update Building.md - meson install from backports (koreader#12680) @mergen3107 * OTA: Make the final ConfirmBox undismissable (koreader#12676) @NiLuJe * Text editor: rotate (koreader#12658) @hius07 * Profiles: more auto-exec triggers 2 (koreader#12691) @hius07 * Add authentication request and cookies management for news (koreader#12496) @bateast * calibre: always use custom fast parser (koreader#12714, koreader#12715) @pazos * Add Nextcloud Notes to exporter plugin (koreader#12301) @kotnik * [plugin] Wallabag: remove empty file when download fails (koreader#12723) @fenuks * [plugin] Wallabag: separate option to delete ‘on hold’ articles (koreader#12722) @fenuks * kopt: fix OCR segmentation mode (koreader#12726) @benoit-pierre * bump base (koreader#12727) @benoit-pierre Most notably dropbear: update to 2024.85 (koreader/koreader-base#1964) * bump android-luajit-launcher (koreader#12734) @pazos * ci/circle: manually cap build jobs (koreader#12751) @benoit-pierre * Adjust Minimum Font Size in Terminal Emulator (koreader#12752) @pphszx * ReaderHighlight: Highlight menu, change 'Delete' to Trash can icon (koreader#12758) @jonnyl2 * @trash-pandy made their first contribution in koreader#12197 * @fbriere made their first contribution in koreader#12296 * @charrarr made their first contribution in koreader#12351 * @smasher816 made their first contribution in koreader#11044 * @bitterlox made their first contribution in koreader#12446 * @cccslater made their first contribution in koreader#12495 * @cheywood made their first contribution in koreader#12566 * @jkl16 made their first contribution in koreader#12557 * @wbstx made their first contribution in koreader#12526 * @jsanpe made their first contribution in koreader#12148 * @fenuks made their first contribution in koreader#12723 * @pphszx made their first contribution in koreader#12752 **Full Changelog**: koreader/koreader@v2024.07...v2024.11 — [closed milestone issues](https://github.com/koreader/koreader/milestone/74?closed=1) --- **Installation instructions**: [Android](https://github.com/koreader/koreader/wiki/Installation-on-Android-devices) • [Cervantes](https://github.com/koreader/koreader/wiki/Installation-on-BQ-devices) • [ChromeOS](https://github.com/koreader/koreader/wiki/Installation-on-Chromebook-devices) • [Kindle](https://github.com/koreader/koreader/wiki/Installation-on-Kindle-devices) • [Kobo](https://github.com/koreader/koreader/wiki/Installation-on-Kobo-devices) • [PocketBook](https://github.com/koreader/koreader/wiki/Installation-on-PocketBook-devices) • [ReMarkable](https://github.com/koreader/koreader/wiki/Installation-on-ReMarkable) • [Desktop Linux](https://github.com/koreader/koreader/wiki/Installation-on-desktop-linux) • [MacOS](https://github.com/koreader/koreader/wiki/Installation-on-MacOS)

…ts & co) (koreader#12519) Using `require` didn't make much sense since most of them are only used in a single place anyway, and it takes care of a few weird interactions in the process (besides not polluting `package.loaded` with useless crap ;)).

NiLuJe added 5 commits September 14, 2024 19:50

(FM|Reader)Menu: Don't require menu elements

41adc31

Use dofile instead, as we don't want to cache those tables, as we or MenuSorter may insert new data into it. c.f., koreader#12372 (comment)

Device: Don't cache event_map & keyboard_layout

965e54d

i.e., use dofile instead of require

Don't cache data tables via require in kleyboard layouts

30c9af9

Missed a few menu elements

fa03893

Ugh, the ui/plugin/insert_menu hack requires this to be cached...

3d652a5

sadface.

NiLuJe added 3 commits September 15, 2024 00:12

Missed a spot ;p.

e43339c

Run that dofile in protected mode

df1baa9

Luacheck'ed

3ba3b4d

Frenzie reviewed Sep 15, 2024

View reviewed changes

Don't run dofile in protected mode, let it error out on syntax errors

82cb687

Frenzie approved these changes Sep 15, 2024

View reviewed changes

NiLuJe commented Sep 16, 2024

View reviewed changes

NiLuJe added this to the 2024.08 milestone Sep 16, 2024

NiLuJe merged commit da74691 into koreader:master Sep 16, 2024

Uh oh!

Uh oh!

Conversation

NiLuJe commented Sep 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hius07 commented Sep 14, 2024

Uh oh!

Commodore64user commented Sep 14, 2024

Uh oh!

NiLuJe commented Sep 14, 2024

Uh oh!

hius07 commented Sep 15, 2024

Uh oh!

Commodore64user commented Sep 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Frenzie Sep 15, 2024

Choose a reason for hiding this comment

Uh oh!

Frenzie Sep 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Frenzie commented Sep 15, 2024

Uh oh!

NiLuJe commented Sep 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Frenzie commented Sep 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NiLuJe commented Sep 15, 2024

Uh oh!

NiLuJe left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

NiLuJe commented Sep 14, 2024 •

edited

Loading

Commodore64user commented Sep 15, 2024 •

edited

Loading

Frenzie Sep 15, 2024 •

edited

Loading

NiLuJe commented Sep 15, 2024 •

edited

Loading

Frenzie commented Sep 15, 2024 •

edited

Loading