Skip to content

Releases: Mathics3/Mathics3-scanner

Get ready for release 10.0.0 (#175)

18 Apr 22:25

Choose a tag to compare

10.0.0

April 18, 2026

Supports Python 3.14. Python 3.10 support has been dropped. While Python 3.10 may still work, but it is not supported.

More YAML/JSON tables added. mathics-core now imports values via mathics.characters. Previously, it created JSON files on its own.

  1. PR #158 Name Pattern token introduced to handle ? and ?? operands.
  2. PR #156 Token names align more closely with CodeParser`Tokenize.
  3. PR #149 Add YAML/JSON table for box grouping characters
  4. PR #147 Disambiguate Unicode for [Rule] and [DirectedEdge].
  5. PR #145 Add Unicode to ASCII conversion table to named characters

Corrections to YAML table entries, e.g., "latex" field; additional/corrections in named characters.

Numerous spelling corrections were performed over the code.

The name Mathics3 replaces Mathics. Mathics was the monolithic Python 2-ish code. Mathics3 has rewritten several major
subcomponents, and the scanner and the character and operators tables in YAML are now in this repository.

2.0.0

28 Aug 19:24

Choose a tag to compare

Supports Python 3.13. Python 3.8 and 3.9 support has been dropped.

Note: There are incompatible changes. Use with Mathics-core 9.0.0 or greater.

Support for saving token position information was started. This is in module mathics_scanner.location. Additional location information is saved when mathics_scanner.location.TRACK_LOCATIONS is set to True.

Scanning was revised to handle more kinds of escape sequences and to be sensitive to their placement inside and outside a string. This is in module mathics_scanner.escape_sequences.

Tokenization support for Box Input operator \* was added.

Small YML changes were made to reflect a better understanding of boxing operators. In particular, \* is not a (prefix) operator.

Internals

Mathics3 scanner exceptions of class TranslateError are incompatible with previous versions, and now store error parameters, "name", "tag", and "args".

Bugs Fixed

  • #125 Hex escape sequence in string literal doesn't work.
  • #136 Is YAML file valid?

1.4.1

26 Jan 17:39

Choose a tag to compare

Re-release to include operators.yml into the tarball/wheel.

1.4.0

26 Jan 16:49

Choose a tag to compare

A new operator table, operators.yml was added to contain operator information. This information is based on information from Robert Jacobson.

See https://github.com/WLTools/LanguageSpec/blob/master/docs/Specification/Syntax/Operator%20Table.csv

From operators.yml, operators.json is created, and this holds operator information that the Mathics3 Kernel uses.

Things like operator precedence, operator arity, associativity, and AMSLaTeX equivalent notation are some of the information we store.

All of the 100 or so Unicode operators without an initial built-in meaning, .e.g., [Cup], [Cap], ... have been added.

The tokenizer and parser in the Mathics3 Kernel use more YAML table information via JSON extraction. However, more will be done in the future.

A new utility program, mathics3-tokens can be used to show tokenization of an input stream, with the -C or --CodeTokenize option, the program shows the tokens more closely in the form that the WMA CodeTokens package uses. Over time, we expect that our tokenizer will be more compliant with CodeTokens.

named-characters.yml was gone over, mostly to fill out information, such as URL links to Unicode pages.

Operator precedence values have been gone over.

Merge pull request #66 from Mathics3/release-1.3.1

09 Aug 12:01

Choose a tag to compare

Python 3.8 is now the minimum Python supported. Python 3.12 supported.
Various dependencies elsewhere force 3.8 or newer.

  • Packaging was redone to be able to support Python 3.12.
  • Files now follow current Python black formatting and isort import ordering
  • Some Python code linting

Too late for Valentines Day

25 Feb 16:02

Choose a tag to compare

  • Add escape-code sequence for 32-bit Unicode. Issue #48.
  • Correct Infix and Tilde character symbols
  • Support double backslash (\\) as a single backslash character (\).
  • Correct Unicode for ScriptN and ScriptCaptialN
  • Correct a number of is-letter-like entries.
  • Accept \u21A6 as symbol for Function.
  • Change the precedence of |->(Function symbol) to 800 so it isn't interpreted as a |
    followed by ->
  • ASCII operator tables can now be generated
  • Add DifferentialD and Integrate even though we don't have a full set of prefix operators.
  • more precedence values added to operators
  • Python 3.11 operation verified

1.2.4

10 Jul 23:28

Choose a tag to compare

  • Start adding AMSLateX names.
  • Add ApplyTo
  • Add Factorial2. PR #30
  • Adjust Tilde, and Factorial.
  • Regularize unicode equivalents.
  • Add named-characters.yml to distribution packages; Issue #32.
  • Use SPDX identifier in license; PR #31.

This time, for sure!

27 Jun 09:59

Choose a tag to compare

Redo for packaging problems.

Many thanks to Victor the packager for AUR for pointing this out.

1.2.2

26 Jun 10:50

Choose a tag to compare

Redo for packaging problems

Many thanks to Victor the packager for AUR for pointing this out quickly.

1.2.1

26 Jun 06:41

Choose a tag to compare

  • Add tables for operator precedence.
  • Start to add AMSLaTeX symbols. (A future release will finish this)
  • Revise README.rst.
  • Some small corrections: Implies
  • Make ujson optional