You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It seems Anki decks that have complicated css dramatically increases the data being sent which breaks things along the way (or causes crashes). The deck size is not the issue but how much cards we batch at a time. This is only the cardsInfo endpoint which is only used for modified cards (all for first time build). This makes the process slower but is negligible since future syncing will be small and in the background.
Added a new build message for the start of a build so users know that it's Anki being queried.
Reduced the batching for Yomitan as well as we can trade off a bit of speed there for less RAM.
I'm not sure if these will fixes all the issues for everyone but these changes are necessary at least for some people. We can just design around the lowest common denominator as it comes up.
Did a deeper dive on the API results and the three problematic fields are question, answer, and css. These field accounted for 45kb per card for me out of 45.5kb. So for the users having issues, this is probably a big reason, wasted RAM causing crashes.
@ShanaryS Feel free to merge whenever you're ready. For what its worth I tried the branch before you pushed the field deletion change and I didn't notice any difference in speed of initial import at all.
@killergerbah Yeah I tested it a few days ago on my fast PC at it was a difference of 23s to 27s. I just tested it on a slower PC and it went from 2m to 2m30s so negligible. For 0 modified cards, it takes about 5s to 7s for both. This is all excluding the parsing of the fields, just the getting the notes and cards.
But I think the biggest difference will be in the memory saved as I think most people were simply running out of ram. Without these fields everyone's card should be a bytes to a few kb. We could also prune the fields not configured but we can save that for later if it's needed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes #900
cssdramatically increases the data being sent which breaks things along the way (or causes crashes). The deck size is not the issue but how much cards we batch at a time. This is only thecardsInfoendpoint which is only used for modified cards (all for first time build). This makes the process slower but is negligible since future syncing will be small and in the background.I'm not sure if these will fixes all the issues for everyone but these changes are necessary at least for some people. We can just design around the lowest common denominator as it comes up.
Did a deeper dive on the API results and the three problematic fields are
question,answer, andcss. These field accounted for45kbper card for me out of45.5kb. So for the users having issues, this is probably a big reason, wasted RAM causing crashes.