Skip to content

Desktop: Accessibility: Simplify accessing OCR text from the Markdown editor #12237

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: dev
Choose a base branch
from

Conversation

personalizedrefrigerator
Copy link
Collaborator

@personalizedrefrigerator personalizedrefrigerator commented May 5, 2025

Summary

This pull request adds a disabled-by-default "show OCR text" button to attachments. This button's title and aria-label both include the attachment's OCR text.

Goals:

  • Make it easier for screen-reader users to quickly determine the content of an image or PDF attachment.
  • Simplify writing end-to-end tests for the OCR service.

Note: This feature was originally discussed during/after the ACB podcast.

Screenshots

New setting (disabled by default):
Screenshot: Markdown editor: Show OCR text for attachments, checked, in the "note" subsection of settings

Hovering over the "View OCR Text" button (blue outline from NVDA):
Screenshot: Mouse cursor hovers over an "eye" button in the Markdown editor. The tooltip has text that starts with "OCR Text The Project Gutenberg eBook of the Game of Logic". The "eye" button is just before the note ID

OCR text buttons in the editor:
screenshot: Two "OCR Text" buttons in the editor, each on a different attachment. Both are just after the opening "(" in the attachment markup.

Screen reader transcript

NVDA on Windows:

  • After opening a note with two attachments with text, then navigating from the bottom to the top of the note4:
Note  main landmark
Markdown editor  edit  multi line  end

After pressing up:

blank

After pressing up:

/bbf87378e6ae4ddfacc7147ea5fb85ff)

After pressing up:

![screenshot.png](  button    OCR Text Test image]  :

After pressing up:

blank

After pressing up:

/2f5c7c44f6dd4528823c5258d04eb0d6)

After pressing up:

[lewis_carroll_game_of_logic](  button    OCR Text The Project Gutenberg eBook of The Game of Logic This. The Project Gutenberg eBook of The Game of Logic This ebook is for the use of anyone anywhere in the United States and most other parts of the world at no cost and with almost no restrictio...  :

After pressing up:

blank

Above, notice that:

  • The OCR text displayed in the editor has a maximum length.
  • The object the OCR text is associated with is described as a "button". Clicking this button would show the OCR text in a text editor.
    • It's currently somewhat difficult to click the button from a keyboard. The button is in the tab order after the editor and can be accessed by pressing tab (after enabling tab indentation), but selecting the button in the editor doesn't allow clicking it.

Notes

UI:

  • Rather than the UI implemented here, it could make sense to show the "OCR Text" button in the note viewer (similar to the "Download" button for Mermaid diagrams).

@personalizedrefrigerator
Copy link
Collaborator Author

The automated test failures in d444702 may be fixed by #12216.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant