"Clear cache and re-download language data files" not working

[ericjacolin]

Operating system

Linux

Joplin version

3.3.12

Desktop version info

Joplin 3.3.12 (prod, linux)

Device: linux, 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
Client ID: c9d562b412294cb3a9cfa76b0c20e0ed
Sync Version: 3
Profile Version: 47
Keychain Supported: No
Alternative instance ID: -

Revision: 4d790b6

Backup: 1.4.3
Csv Import: 1.0.1
Freehand Drawing: 3.0.1
Outline: 1.5.13
Search & Replace: 2.2.0

Current behaviour

1. I activated OCR for the first time, the scan worked (about 600 images) but only in English
2. I have a lot of Chinese content so I added Tesseract chi_sim language file (as well as French) with the procedure described in https://joplinapp.org/help/apps/ocr/
3. I tried to "Clear cache and re-download language data files", but this had no effect
4. The cached language files (English) are still there

Expected behaviour

I would expect the cache to clear and rebuild with the new set of of languages

Logs

No response

[personalizedrefrigerator]

Thank you for reporting this!

For comparison, "Clear cache and re-download language data files" seems to work for me (on Fedora 42 Linux, with Joplin 3.3.12 (dev)):

• Shows a confirmation dialog.
• After pressing OK, restarts Joplin.
• Logs the following:

  OcrDriverTesseract: Clearing cached language data...
  OcrDriverTesseract: Clearing language data with key ./eng.traineddata
  

• At this point, inspecting the keyval-store indexedDB table (using the "Application" tab in the development tools) reveals that the ./eng.traineddata cached model is no longer downloaded.

Follow-up questions:

• After clicking "Clear cache and re-download language data files", is "OcrDriverTesseract: Clearing language data with key ./eng.traineddata" added to Joplin's log file?
  • Logs can either be accessed from Joplin's development tools (Help > Toggle developer tools, includes recent logs) or from the log.txt file in the profile directory (Help > Open profile directory, includes older logs).
  • If there is an error, it should log "OCR: Failed to clear language data cache." followed by the error message.
• Are new images (not scanned previously) processed using the new OCR models?

Note

"Clear cache and re-download language data files" deletes the local eng.traineddata (and other) models, but does not re-OCR existing attachments.

[personalizedrefrigerator]

Proposed changes:

• (Maybe) Get rid of "Clear cache and re-download language data files". Instead, automatically remove the models for non-active languages (e.g. after a week or two).
  • We could also base this on available disk space.
• Add a new option, "Re-OCR all attachments", maybe with a prompt. This would be shown when the user changes the OCR URL.

[ericjacolin]

Thanks @personalizedrefrigerator this is very helpful.
Inspecting IndexedDB, I see that after clearing languages the only language that Joplin will reload from local traineddata folder is eng. Will not load chi_sim or fra

[personalizedrefrigerator]

Inspecting IndexedDB, I see that after clearing languages the only language that Joplin will reload from local traineddata folder is eng. Will not load chi_sim or fra

Thank you for the additional information!

Is the UI language set to English? If so, hanging Joplin's UI language in settings > general might help. Currently, Joplin uses the global locale setting to select the OCR language:

joplin/packages/lib/services/ocr/OcrService.ts

Line 154 in 88c95cc

const language = toIso639Alpha3(Setting.value('locale'));

As a result, if the UI language is set to English, the OCR service will try to use an eng language model.

[ericjacolin]

Thanks. I may have misunderstood the purpose of the feature.
My locale is English indeed and I was expecting to be able to scan documents in both English and simplified Chinese, using tesseract -l eng+chi_sim behind the scenes