LaTeX symbols not displaying

[gdomst]

Hello everyone,
I am having some problems to display LaTeX symbols in Ipywidgets labels. Here is a screenshot.

In Jupyter it is displayed correctly.

[danielfrg]

I think we only load mathjax when there is a markdown cell, we could change that so its possible to use it everywhere.