Expression Explorer Updated and On by Default

In version 3 of MathJax, the expression explorer was not active by default, and so those with assistive needs had to turn it on explicitly, and it was loaded dynamically at that time. In its place, the a11y/assistive-mml extension was used to generate a hidden MathML expression that could be voiced by those screen readers that can process MathML. This was a stop-gap measure that was both clunky and somewhat fragile, and changes in screen readers often rendered it ineffective (e.g., the current VoiceOver version doesn’t read the mathematics and skips it entirely, while Windows screen readers indicate clickable math, but then fail to read it).

Because of these problems, and because the Speech Rule Engine (SRE) that underlies MathJax’s assistive support has been substantially rewritten to improve its performance (see the Technical Details section below), we are now packaging the expression explorer and SRE as part of all the combined configuration files, while the assistive MathML extension has been removed from them (though it can still be turned on in the Options sub-menu of the MathJax contextual menu). The SRE generates speech automatically for the expressions in your page and makes these speech strings available to screen readers using aria-label and aria-braille-label attributes so that a screen reader can voice the math naturally as part of the page. Moreover, the expressions are focusable so that they can be explored interactively. These features are available by default in version 4, and should provide smoother interaction with screen readers on all platforms.

Exploring Expressions

Using the explorer remains largely the same in v4 as it was in v3: tabbing to an expression activates the explorer, and arrow keys move down into an expression, up to a higher level of an expression, or left and right among neighboring terms. New in version 4 are the following features:

  • Clicking on an expression will enter the explorer, highlight the clicked term, and cause a screen reader to speak that term; navigation via arrow keys can continue from that point.

  • Double-clicking on an expression enters the explorer at the top level of the expression (just as tabbing to it would), rather than on the clicked term.

  • In previous versions, the speech and/or braille subtitles were shown as well, but these have been turned off by default in this release, as they caused confusion for users not aware of the explorer features. They can be re-enabled using the MathJax contextual menu.

  • While exploring an expression, pressing h will open a dialog box that describes the explorer’s features. When an expression is first focused, a message informing the user of this feature is announced after the expression is read, but that can be disabled using the contextual menu.

  • In v3, if an expression was explored and tabbing is used to leave the explorer, then when that expression is refocused, the selection would remain where it was when the expression was exited. In this version, tabbing to an expression always starts at the top level of the expression, not the last selected term. The reason for this is so that if the page is read, whether as a whole or by smaller chunks, the expression will be read in full rather than just the previously selected term. There is a contextual menu item that can be used to select the older behavior, however, if a user wishes to be able to restart the explorer where they left off.

  • MathJax now provides optional auto-voicing of expressions together with step-by-step highlighting of expression while a formula is spoken. This feature is primarily aimed at users who do not normally utilize a screen reader, and in particular, as support for dyslexic users. Currently, it has to be switched on explicitly, either in the Speech sub-item of the MathJax contextual menu, or using the voicing option in the a11y sub-block of the options configuration block. Speech is generated by providing SSML annotations to the browser’s speechSynthesis API. While this makes use of the full range of prosody annotations available in SRE’s speech rules, the feature is only available in browsers that come with an implementation of the speechSynthesis API and with built-in voices.

The MathJax team tests the explorer with 13 browser/os/screen-reader combinations: Chrome, Firefox, and Safari on MacOS with VoiceOver; Chrome, Firefox, and Edge on Windows with NVDA and JAWS; and Chrome and Firefox on Linux (Ubuntu 24 and 22) with Orca. The new explorer should give a much smoother experience to screen-reader users in all these settings.

Explorer Improvements

In addition to the performance enhancements and usage changes given above, the following are improvements found in v4:

  • There is a new Korean and Afrikaans locales for speech output.

  • The explorer now handles expressions with line breaks better.

  • There are new text heuristics to distinguish genuine text elements from expressions that only use text to enforce font changes to roman or mathvariant=normal.

  • The speech output for tensor expressions has been improved.

  • A new Euro Braille output format is available that uses eight-dot Braille to output the original LaTeX for the whole expression, and during expression exploration, the LaTeX used in sub-expressions within the expression.

  • Improvements in alphabet generation and symbol translations have reduced the size of the locale files in the distribution.

Technical Details

The generation of speech strings is a resource-intensive process. In the past, that was a source of performance slowdown for pages that contain a lot of mathematics. Version 4 has moved the speech generation into a web-worker, which runs in a separate thread from the main page, so the page will remain responsive to user interaction even while speech strings are being created.

Web-workers are only available in browsers, not in node applications, so MathJax is set up to use the worker-threads library in node, along with some shim code to give it a corresponding interface to the features that MathJax uses from web-workers in the browser. This allows the same code to be used in both browsers and node applications. Currently this requires the use of the LiteDOM in node, but we may provide support in other DOM implementations in the future.

Because node applications that include speech generation now use worker-threads behind the scenes, the node application won’t exit until the MathJax worker thread is shut down, so applications may need to tell MathJax that they are finished. This is done by calling the new MathJax.done() method in applications that use MathJax components, or by calling the MathDocument’s done() method for those that use direct calls to MathJax’s modules.

Aside from moving the speech generation into a web-worker or node worker-thread, the speech-generation has been improved by adding more sophisticated semantic analysis for complex user defined structures, including improved disambiguation of function applications, ellipses, and unusual large operators. Additional locales supported are Afrikaans and Korean as well as 8-dot Euro Braille output generated from original LaTeX input formulas.