-
-
Notifications
You must be signed in to change notification settings - Fork 7
add UNICODE_CHARACTER_TO_ASCII and CHARACTER_TO_NAME tables #169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
mmatera
merged 13 commits into
master
from
add_UNICODE_CHARACTER_TO_ASCII_and_CHARACTER_TO_NAME_tables
Apr 3, 2026
Merged
Changes from 1 commit
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
7ed9e9f
add UNICODE_CHARACTER_TO_ASCII and CHARACTER_TO_NAME tables
mmatera 582a161
using the variable
mmatera 517d49c
Merge remote-tracking branch 'origin/master' into add_UNICODE_CHARACT…
mmatera 44ba4f3
black
mmatera cd524c9
add operator-to-unicode and operator-to-ascii just if the tables are …
mmatera 5bf07a4
install mathics3 in pyodide workflow
mmatera aad225f
no isolation in mathics3-doctest workflow
mmatera 95d18ee
no build isolation for mathics3-doctst
mmatera b01e703
undo changes
mmatera aed4e80
try what happens if we comment out the new code...
mmatera eaa7a03
try what happens if we comment out the new code...
mmatera 7024b8d
resinstate changes
mmatera 5fc366d
Merge remote-tracking branch 'origin/master' into add_UNICODE_CHARACT…
mmatera File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -813,7 +813,7 @@ CapitalDelta: | |
|
|
||
| CapitalDifferentialD: | ||
| amslatex: '\mathbb{D}' | ||
| ascii: "d" | ||
| ascii: "D" | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is the symbol used in path integrals, with a capital D in the differential. |
||
| esc-alias: DD | ||
| has-unicode-inverse: true | ||
| is-letter-like: true | ||
|
|
@@ -2131,6 +2131,7 @@ DifferenceDelta: | |
| # \u2146 | ||
| DifferentialD: | ||
| amslatex: '\,d' | ||
| ascii: "d" | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This was missing. |
||
| esc-alias: dd | ||
| has-unicode-inverse: true | ||
| # This can't be letter-like because it is used in derivatives as a function | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've been thinking about this general topic, and here is my current thinking.
I am pretty sure that Mathics3's
$CharacterEncodingforUTF-8is different from WMAs. In particular, WMA has some definitions in the user-defined Unicode space that we don't follow. I think there are other symbols where we've decided that the glyph used by WMA doesn't look right.We could make those two align and change Mathics3's default
$CharacterEncodingvalue to something else.Related to this, I see that WMA defines seven encodings:
Mathematica1toMathematica7. I suppose we could do something like this, too.There seems to be a desire to have an encoding that does not map a symbol like ASCII "x" to more than one letter or named character. Strictly speaking, this is impossible to accomplish with an 8-bit code, like
ASCII, or any of theISO8859character encodings, which are defined to be 8-bit representations.For reference, UTF-8 is not a fixed 8-bit code; it is a variable-length code with a minimum of 8 bits. That's why it is possible to give all WMA named characters distinct values. (Whether we do or want to do is a different story).
Previously, I had said that WMA's
CharacterEncodingis equivalent to the older concept of a "Code Page" that was popular in computers. I now realize this is not quite right because a codepage encodes a fixed number of bytes (usually one, but sometimes two bytes).CharacterEncodingdoes this, but it also handles Unicode and possibly other variable-length encodings (if that is a thing.)So coming back for things to think and do (and when to do):
What should our default
$CharacterEncodingbe? If we call it UTF-8, what kind of UTF-8? What we use now, or what WMA uses? If it is WMA, what do we call what we do now?Does WMA's UTF-8 define different character codes for all of named characters? We have seen that operators can have several representations or that two operators can map to the same symbol. For example, the multiplication operator can be either a space or some other symbol. And both
OptionalandPatternoperators map to "?".