Reparses all macro names so that they may optionally include characters listed in allowedTokens.
This is used, for example, when parsing expl3 syntax which allows _ to be used in a macro name (even though
_ is normally stops the parsing for a macro name). Thus, a macro \foo_bar:Nn would be parsed as having
the name foo_bar:Nn rather than as foo followed by the strings _, bar, :, Nn.
Reparses all macro names so that they may optionally include characters listed in
allowedTokens
. This is used, for example, when parsing expl3 syntax which allows_
to be used in a macro name (even though_
is normally stops the parsing for a macro name). Thus, a macro\foo_bar:Nn
would be parsed as having the namefoo_bar:Nn
rather than asfoo
followed by the strings_
,bar
,:
,Nn
.