cover lexical parsing

[Anniepoo]

Thank you very much for making available your DCG tutorial.

I have a question, you write: "For complex grammars it's often worthwhile to first lexically parse tokens, and then parse that token stream for the real grammar."

However, there is no explanation what lexical parsing is, no examples, and it is not clear whether it can be done in Prolog.

It would be tremendously helpful if the tutorial could be expanded with an illustrative example or at least provide some pointers.

With best regards,
Parker

I would think that for lexical parsing it might be better to just point to ROK's book (down to the page number) than try to build it into the tutorial.