Improve handling of compound identifiers in Progress ABL

[MGilarde]

Add a new style for compound identifiers in the Progress ABL lexer. A compound identifier is of the form "identifier.identifier[.identifier ...]". A compound identifier should be styled as a single unit rather than as multiple separate identifiers.

Some examples are:

Progress.Lang.Object
System.Windows.Forms.*
_user._user-id

[nyamatongwe]

Using punctuation for scope like record.field or package::function is a common pattern and there are often different points of view and preferences amongst users. Some want to treat the compound as a unit and some want it to be treated as individual items. This intersects identifier highlighting so there may be a desire for Progress.Lang.Object to be highlighted completely as a special identifier but others may want Progress or even Progress.Lang to be recognized as special.

It will be more difficult to allow for these preferences in the future if the possibility is not addressed now.

SCE_ABL_IDENTIFIER2 is not very descriptive and could be something like SCE_ABL_IDENTIFIERCOMPOUND.

[MGilarde]

I will change SCE_ABL_IDENTIFIER2 to SCE_ABL_IDENTIFIERCOMPOUND. Thanks for the suggestion.

I understand your point about flexibility. The ABL language is quite wordy and the lexer wouldn't be able to recognize that Progress or Progress.Lang have meaning without enhancing it to understand the semantics of the language. Without semantic context there's no way to know whether Progress.Lang is part of a package name or part of a database field reference, which could be either <database.table> or <table.field>. If we can't determine what each part of a compound identifier represents, I don't see how allowing the user to highlight the parts differently would be useful in practice.

PROGRESS and OBJECT are keywords in the language so currently (before this patch) they're being styled as SCE_ABL_WORD but Lang is styled as SCE_ABL_IDENTIFIER. Our editor (I work for Progress, the main user and maintainer of this lexer) auto-cases keywords so this leads to Progress.Lang.Object being transformed to PROGRESS.Lang.OBJECT as you type, which is obviously undesirable. While the flexibility you suggest would be nice I don't think it would be achievable in practice and I believe that users would prefer to forgo the flexibility you suggest in order to gain the benefit of improved styling and auto-casing for compound identifiers which include keywords. There are about 2,000 keywords in the language (I wasn't kidding about it being wordy) so this is a common issue.

[nyamatongwe]

I misunderstood what this does initially. The description said

A compound identifier should be styled as a single unit

but that isn't what this does as illustrated by the styled examples like {7}USING{0} {7}System.{13}Windows.Forms.*.{0}.

The 'single unit' description would be better matched by {7}USING{0} {13}System.Windows.Forms.*.{0} which can be achieved by calling ChangeState when the '.' is matched.

The PR code matches a 'primary' identifier and period System. which is distinguished from qualifiers Windows.Forms.*.. This is different from past similar cases in other lexers.

If primary/qualifiers does match how you see the ABL language then it may need a different style name to be more explicit about what it means. If 'qualifiers' is used then the '.' could be part of that instead of the primary identifier.

[nyamatongwe]

Additional style values should be added to include\LexicalStyles.iface first as it is the primary source and include\SciLexer.h is generated from it.

[MGilarde]

I believe that I've resolved all of the issues you raised. All identifiers, whether single or compound are now styled as SCE_ABL_IDENTIFIER.

I apologize for the multiple commits. Thank you for the thoughtful review and suggestions for improvements.

[nyamatongwe]

The newer version makes SCE_ABL_IDENTIFIERCOMPOUND something that should never be seen in output. That may be confusing to downstream projects that will see a new state and provide theming support for that state in their UI and theme files.

If SCE_ABL_IDENTIFIERCOMPOUND is truly invisible to users then it can be defined inside LexProgress.cxx, perhaps with a non-consecutive state number (like 30) that won't interfere with a few new states.

[MGilarde]

SCE_ABL_IDENTIFIERCOMPOUND is now defined in LexProgress.cxx. I removed the definitions from the other files.