The difficulties of displaying math symbols

Questions
1

I want to display math symbols (equations, matrices, etc.) in order to teach math so that learners can see math derivations and live results. Babylon.js doesn’t seem to have a loader to load MathML. So, I try to use SVG symbols, which are produced with tools such as MathJax. But, SVG is used in Babylon.js only as a texture, isn’t it? If so, that won’t be flexible enough to make a math learning animation.

I went to our good neighbor three.js to try its SVG loader. Unfortunately, however, it does not fully support SVG, leaving out tags such as “defs”.

Any suggestions?

Thanks.

2

you can draw svg on canvas(ctx.drawImage(image, 0, 0)), then use it as a texutre( gl.readPixels or canvas.toDataURL), and update it as you need.

or you can get the coordinate from pickInfo by BJS, then put a html <div> there, then you can render math symbols as same as on a web page.

3

Hi @DevelopDaily and welcome to the forum. Here is a rather crude use of a div to display MathML https://www.babylonjs-playground.com/#WF3VKZ#2.

4

DevelopDaily, I like your mission.

I developed MeshWriter, and used it to support SVG symbols. You can see an example of that at this playground.
https://www.babylonjs-playground.com/#NLG6KE#2

The Chinese characters were extracted from normal font files but the anchors were taken from SVG. They were both put in a font family, “Web-dings”. (Alas, turning SVG into letters is not automated.)

Also, I was unsure of what output you wanted. Your comment about textures not being flexible enough confused me a little. It might help to be able to visualize your target layout.

5

I’d like to change SVG objects in the scene as any other geometric objects. Here is a sample SVG, which is generated by the MathJax.

math.zip (2.5 KB)

6

@hjlld Do you mean I put a

as an overlay over Babylon.js scene (canvas)? But an html
cannot zoom and rotate together with the scene, can it?
7

hi, if you want to rotate/scale/translate math symbols as a 3D object in BJS scene, you can draw svg on canvas( ctx.drawImage(image, 0, 0) ), then use it as a texutre( gl.readPixels() or canvas.toDataURL() ), and update it as you need.

8

DevelopDaily, let me try to break this down. In 3D parlance there are two theoretical things you could seek to do:

  1. Make a mesh out of an equation, like MeshWriter does for fonts
  2. Put the equation on a texture

Think of a texture as being like a decal. It can be affixed to any mesh. A racing car or, in your case, a chalkboard(?).
https://doc.babylonjs.com/babylon101/materials

There is a general-purpose way to put an image into a texture. SVG “images” work great for that. I have done it many times. So in your application, the SVG could be put on a rectangular mesh and placed anywhere in the scene.
https://www.babylonjs-playground.com/#NT70VN#2

In that example, I turned your equation SVG into a dataurl and created a mesh with the same proportions as the SVG. That’s option 2.

Option 1 is harder. I have been interested in that for a long time but I suspect that is not what you want.

Meshes are first-class citizens in 3D. You can move them, size them, rotate them, light them and animate them. With the appropriate material, you will probably get what you want.

9

Indeed, both options can serve my purposes. You guys are very helpful. Thanks.

10

Hi,

I got a tool that might help you for displaying latex math string as mesh.

you will most likely find bugs here and there. But simple usage should be ok?

example usage:

import {mathmesh} from "mathmesh";
var verts = mathmesh("\\int_{a}^{b}x^2 \\,dx");
let customMesh = new Mesh("mymathmesh", scene);
let vertexData = new VertexData();

vertexData.positions = verts.positions;
vertexData.indices = verts.indices;
vertexData.applyToMesh(customMesh);

let fontmaterial = new StandardMaterial("mathmeshMat", scene);
fontmaterial.backFaceCulling = false;
fontmaterial.emissiveColor = new Color3(0, 1, 0);
customMesh.material = fontmaterial;

baby_example
baby_example1173×502 31.6 KB

11

This is coool, @PirateJC can you think of a good place in the doc?

12

Hi, thanks for the support :), but I think mathmesh is farrrrrr from being used in production. It still has a lotttttt of things to be improved/fixed. I shared it just so in case someone need some basic latex math display.