Proposal for ascii

[ivan-pi]

This module should include functions for character classification and conversion (lower, upper). I have prepared a basic implementation at https://github.com/ivan-pi/fortran-ascii.

The plan is to cover the same functionality as found in the C, C++, and D libraries:

• C : http://www.cplusplus.com/reference/cctype/
• C++: https://en.cppreference.com/w/c/string/byte
• D: https://dlang.org/phobos/std_ascii.html)

@zbeekman has already opened an issue (see ivan-pi/fortran-ascii#1) on dealing with different character kinds. The problem is that the ascii and iso_10646 character sets need not be supported by the compilers. Even if they are supported their bitwise representation might be different from the default kind.

I realized while creating these functions, that agreeing upon a style guide #3 and documentation #4 early on would be helpful to improve future pull requests. Some agreement upon unit testing will also be necessary.

cc: @jacobwilliams

[ivan-pi]

One of the simple points to discuss (connected to the style guide #3) is naming conventions. For example for the function that checks whether a character is a letter some of the possible names are:

• is_alpha
• isalpha (C/C++)
• isAlpha (D language, but in Fortran it's the same as the previous suggestion)

Personally, I like the underscored version (it matches the general verbosity of Fortran).

I have also noticed that in C++, they have a separate set of the these function for "wide" strings (e.g. iswalpha, see https://en.cppreference.com/w/c/string/wide) which if I understand correctly are Unicode characters.

[certik]

@ivan-pi let's move the naming conventions discussion into #3 (comment), where I just commented.

[zbeekman]

To address character kinds, this is my comment from the ascii proto-type repo:

If I understand correctly you are suggesting I create several copies of these function to operate on the following character kinds:

Well, not exactly, because, as you noted, they're not guaranteed to exist, and when they do exist, "DEFAULT" is often/usually the same kind as "ASCII", so you can't create overloaded functions with arguments that are "ascii" and "default". (In that case you'd have a duplicate interface.)

That's one nice thing about jin2for: It doesn't assume anything and interrogates the numeric kinds from the compiler to then generate the code. So if only one character kind is supported then your code will only have that one kind. I'll cross post this on the new issue you made.

[ivan-pi]

I see what you mean. If "default" and "ascii" weren't the same kind, then I could just create generic interfaces for all three character kinds (with conditional compilation if the compiler supports UCS using CMake).

How likely are we to meet a processor where "ascii" is not equal to "default"?
In Modern Fortran explained (2018) EBCDIC is mentioned. It seems to still be in use on some IBM mainframes: https://www.quora.com/Is-EBCDIC-replaced-by-ASCII-or-Unicode

Since the module is supposed to work for ascii characters, I think it is possible to write all the functions in such a way that they work with either "default" (non-ascii) or "ascii" straight of the box (as long as the processor supports ascii) using the achar and iachar conversion routines.

For the ascii subset of unicode characters it should be possible to create overloaded functions because they have a different kind (more bits). Since there are not so many functions in this module I could just do it manually. We can always bring in jin2for later.

[zbeekman]

For the current use case, I don't think it makes sense to use Jin2For either.

Let's start with default character kinds while we consider if there's anything clever that we can do. We can always do some introspection with CMake and then use configure_file() to achieve overloading.

[ivan-pi]

Is it possible to have the operating system somehow simulate "EBCDIC" defaults or maybe it is possible setup a Docker image? I have no means of testing non-"ascii" defaults at the moment.

[ivan-pi]

I think the way the procedures are written now (https://github.com/ivan-pi/fortran-ascii/blob/master/stdlib_ascii.f90) would work with both "default" and "ascii" characters.

[ivan-pi]

Before I submit a pull request, there are a few more issues worth discussing:

• Should these procedure be pure or elemental?
• How to handle control characters? Just as character constants? Or would it be better to simulate an enumerator using a derived type? A potential issue I see is that, the control characters constants might end up inadvertently polluting someones namespace if he does use stdlib_ascii and forgets to use the only clause. By wrapping them in a derived-type (singleton) you could only access these via something like ascii_control_char%vt (to access the vertical tab).
• Naming convention; in the other thread Style guide #3 the general agreement was to keep the longer - more explanatory names - with the verb separated by an underscore. I have prepared a table of the functions, their proposed name in the Fortran stdlib and their names in other languages:
  • I see some potential confusion between the is_white and is_blank functions, also given that they have a different name in C/C++. The is_white returns true for the characters including the space, tab, vertical tab, form feed, carriage return, and linefeed characters. The is_blank (or isblank in C) was standardized in C99 and returns true if the character is the space character ' ' or the tab character. It would be interesting to know, what was the reason to introduce this as a separate function.

• Personally, I am happy with the verbose underscored names, but I thought it is still worth discussing whether or not we would prefer the shorter seven-letter abbreviations.

Purpose	Fortran	C/C++	D	Python*
checks if a character is alphabetic	is_alpha	isalpha	isAlpha	isalpha
checks if a character is alphanumeric	is_alphanum	isalnum	isAlphaNum	isalnum
checks if a character is in the ASCII character set	is_ascii	/	isASCII	isascii
checks if a character is a control character	is_control	iscntrl	isControl
checks if a character is a digit	is_digit	isdigit	isDigit	isdigit
checks if a character is a octal character	is_octal_digit	/	isOctalDigit
checks if a character is a hexadecimal character	is_hex_digit	isxdigit	isHexDigit
checks if a character is a punctuation character	is_punctuation	ispunct	isPunctuation
checks if a character is a graphical character	is_graphical	isgraph	isGraphical
checks if a character is a printing character	is_printable	isprint	isPrintable	isprintable
checks if a character is an uppercase character	is_upper	isupper	isUpper	isupper
checks if a character is lowercase	is_lower	islower	isLower	islower
checks if a character is a whitespace character	is_white	isspace	isWhite	isspace
checks if a character is a blank character (space or tab)	is_blank	isblank	/
converts a character to lowercase	to_lower	tolower	toLower
converts a character to uppercase	to_upper	toupper	toUpper
				Note*: In Python these are bound methods of the built-in string type

• Another issue is what should be the behavior for non-ASCII characters? In D the function returns false for non-ascii characters, and the toX functions do nothing. This is probably a separate issue to discuss, once we work out the best way to also support the UCS character set.

[certik]

Thanks @ivan-pi for submitting a PR with this! Much appreciated. Let's discuss the API.

[jvdp1]

Thanks @ivan-pi for this nice implementation.
Question: should the long cases (implemented in the test) be also present in the module? or do we expect that the user would implement them for their own use?

[ivan-pi]

Perhaps I have not understood your question correctly. The long tests generally loop through all characters (therefore they are "long") and are meant only as unit tests to verify the correctness of the functions. Since the short/long adjectives might lead to some confusion I would not mind if we rename them.

The last test - test_ascii_table - reproduces the table from the C++ functions available at https://en.cppreference.com/w/cpp/string/byte

Since the functions follow the same pattern what might be worth including in the module is an abstract interface:

 abstract interface
        pure logical function validation_func_interface(c)
            character(len=1), intent(in) :: c
        end function
 end interface

although apart from my test case which uses an array of procedure pointers to loop through the character validation routines, I don't really know if their would be any use cases.

[jvdp1]

Since the short/long adjectives might lead to some confusion I would not mind if we rename them.

@ivan-pi I think they are fine.
I was not clear enough. Here is an example of what I meant:

character(len=30)::examplelower,exampleupper
example='abc'
exampleupper=to_upper(examplelower)
print*,trim(exampleupper) !it prints 'ABC'

we can discuss and see later if it could be useful or not.