UnicodeMath -- mimic LaTeX package unicode-math for more control over font styles

[manuelbb-upb]

This is a WIP with the goal of enabling a bit more control over character shapes similar to unicode-math in LaTeX.

Potentially helps with #94 #107 #137 #124 and similar issues.

Currently, there is a submodule UnicodeMath with
a) dictionaries for mapping between different shapes/styles within a math alphabet and
b) some extra commands that are also defined in unicode-math.

Based on the dictionaries in UnicodeMath, we can define commands like \symbf, \symit etc. for the parser.
By hooking in the layout routine, it is then possible to mimic different typesetting conventions.

Currently, configuration is a bit clunky... for the sym_style function, it is done with Refs in the submodule UnicodeMath, similar to how the font family is defined globally.

Besides documentation and clean-up, I am unsure how best to proceed and could use some suggestions, @Kolaru @SimonDanisch

1. Keep UnicodeMath as a submodule in MathTeXEngine or
2. merge UnicodeMath into UnicodeFun or
3. make it an independent package and load as hard dependency or weak dependency and extension in MathTeXEngine
4. forget the idea entirely because there is something better in the pipeline.

Moreover, I am a bit unsure about licensing. unicode-math is licensed under "LaTeX Project Public License".
Can our code be MIT? If not, then option 3 seems inevitable.
Option 1 has the advantage that we could enable additional symbols to be typeset properly, like \Bbbsum in the example below, by changing package constants.

PS: I have worked with #138 because that is the font I need under Linux.

using WGLMakie
using LaTeXStrings
using MathTeXEngine
const MTE = MathTeXEngine

## define escaped underscore, used in the plot title later on
MTE.command_definitions["\\_"] = (TeXExpr(:punctuation, '_'), 0)

## choose font family, request most stuff to use :math font, and set globally
fontfam = MTE.unicode_math_fonts!(FontFamily("TeXGyrePagella"))
set_texfont_family!(fontfam)

function example_fig(cfg_symb=:tex)
	if !isnothing(cfg_symb)
		MTE.unicode_math_config!(; math_style=cfg_symb)
	end
	fig = Figure(; size=(400, 300))
	title = isnothing(cfg_symb) ? LaTeXString("\\textbf{Using default }\$\\symtt{math\\_style}\$") : LaTeXString("\\textbf{Using} \$\\symtt{math\\_style = $(Meta.quot(cfg_symb))}\$")
	xlabel = L"aA\symbf{a}\symbf{A}\symsf{a}\symsf{A}~\alpha\Alpha\symbf{\alpha}\symbf{\Alpha}\symbfsf{\alpha}\symbfsf{\Alpha}"
	ylabel = L"\sum_{j=1}^M \Bbbsum_{i=1}^{N_j} \partial_i"
	
	ax = Axis(fig[1,1]; title, xlabel, ylabel)
	
	scatter!(ax, rand(2, 20))
	text!(0.5, 0.5, text = L"\symbbit{Ddeij}\symbb{R}", align = (:center, :center))
	fig
end
example_fig(:tex)
example_fig(:iso)
example_fig(:upright)
example_fig(:literal)

[Kolaru]

This looks awesome !

Unfortunately, I won't have time to look in details and review this before next week (please ping me if I don't come back to it then).

Currently, configuration is a bit clunky... for the sym_style function, it is done with Refs in the submodule UnicodeMath, similar to how the font family is defined globally.

For a first implementation, this is probably fine.

I would like at some point to define a new LaTeXString type that could carry this information, but then there would be some more work to make sure that the info is used/usable by Makie.

Besides documentation and clean-up, I am unsure how best to proceed and could use some suggestions

I think that for now it doesn't matter much. The main interest of having it as a separate package is that it could be used by others, and you would have full control over it. From my side, depending on another packages (and/or on UnicodeFun) is not a problem.

Moreover, I am a bit unsure about licensing. unicode-math is licensed under "LaTeX Project Public License". Can our code be MIT? If not, then option 3 seems inevitable. Option 1 has the advantage that we could enable additional symbols to be typeset properly, like \Bbbsum in the example below, by changing package constants.

In principle, we can distribute different part of the software with different licenses (e.g. the fonts in the asset folder are not MIT).

How exactly is this PR derived from unicode-math ? Did you reuse any data or code from it ?

PS: I have worked with #138 because that is the font I need under Linux.

I will look into #138 as soon as possible.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 63.06306% with 82 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.23%. Comparing base (819bbec) to head (184beb8).
⚠️ Report is 33 commits behind head on master.

Files with missing lines	Patch %	Lines
src/UnicodeMath/UnicodeMath.jl	51.58%	61 Missing ⚠️
src/MathTeXEngine.jl	59.52%	17 Missing ⚠️
src/UnicodeMath/character_ranges.jl	93.87%	3 Missing ⚠️
src/engine/texelements.jl	75.00%	1 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #141      +/-   ##
==========================================
- Coverage   78.39%   71.23%   -7.17%     
==========================================
  Files          10       13       +3     
  Lines         597      869     +272     
==========================================
+ Hits          468      619     +151     
- Misses        129      250     +121     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[manuelbb-upb]

Thanks for the insights :)

Unfortunately, I won't have time to look in details and review this before next week (please ping me if I don't come back to it then).

No hurry; it is also a side quest for me, as I will need these font-style options for a thesis project.
In fact, I think you can wait for me to do a bit of refactoring, so as to not waste your time.
I'll ping you then ;)

From my side, depending on another packages (and/or on UnicodeFun) is not a problem.

I think I might try to make it a standalone package and request only minor changes to the MathTeXEngine interface.

How exactly is this PR derived from unicode-math ? Did you reuse any data or code from it ?

For the main functionality, i.e., the \sym… commands, I did manually investigate the source for the Unicode mappings and modeled styling configurations according to what I think happens with the LaTeX.
I guess we should be fine licensing this part under MIT.

The additional commands I mainly added out of curiosity by automatically scanning a source file and doing regex matching.

If we end up with a standalone package, the license does not matter much, and we could easily use LPPL or similar.
If it does not work as a standalone package, then maybe we would have to fall back to your proposal:

In principle, we can distribute different part of the software with different licenses

[manuelbb-upb]

close in favor of #143