Perl-style shorthands (like `\d`) not recognized, only POSIX ones (like `[[:digit:]]`)

[asarkar]

Pattern \\d+|\\b[a-zA-Z']+\\b fails to find the digits in input "testing, 1, 2 testing". The regex is correct as can be tested here https://regex101.com/r/griuTm/1.

Changing the pattern to \\b[0-9a-zA-Z']+\\b works, but it changes the intent because that makes input "123abc" would be valid. \\b[0-9]+\\b|\\b[a-zA-Z']+\\b works too.

[andreasabel]

Could you submit a small Haskell program demonstrating the problem?
Then it would be easy to compare the behavior of regex-tdfa to the other implementations, like regex-pcre, regex-posix etc.

[asarkar]

Perhaps this will help, taken from my StackOverflow question.

module WordCount (wordCount) where

import qualified Data.Char as C
import qualified Data.List as L
import Text.Regex.TDFA as R

wordCount :: String -> [(String, Int)]
wordCount xs =
  do
    let zs = R.getAllTextMatches (xs =~ "\\d+|\\b[a-zA-Z']+\\b") :: [String]
    g <- L.group $ L.sort [map C.toLower w | w <- zs]
    return (head g, length g)

[andreasabel]

What the others do:

• regex-pcre does what you want (finds the digits).
• regex-posix finds no matches

Concerning regex-tdfa, if you look up the documentation at https://hackage.haskell.org/package/regex-tdfa under section Special characters, \d is not included. Thus, you should not be surprised it is not supported.

It even says explicitly:

regex-tdfa only supports a small set of special characters and is much less featureful than some other regex engines you might be used to, such as PCRE.

So, the easiest solution for you might be to use regex-pcre.
(Not sure what your intention with filing this report was, maybe you want to PR.)

[asarkar]

I found this library looking for a regex package, and saw it mentioned in the Haskell wiki, and in a blog that’s now part of the README. I compared various libraries based on their maintainability (last commit date) and popularity (GitHub stars, issues addressed promptly), and this one came out at the top. Because of that, I’m indeed surprised that something as common as `\d‘ isn’t supported. I’m a Haskell freshman and don’t have the skills yet to start making PRs on a general-purpose library.

[andreasabel]

Predefined character classes we could support are listed here:
https://en.wikipedia.org/w/index.php?title=Regular_expression&section=13#Character_classes

One could recognize them either directly in the parser:

regex-tdfa/lib/Text/Regex/TDFA/ReadRegex.hs

Line 94 in 95d47cb

p_escaped = char '\\' >> anyChar >>= \c -> char_index >>= return . (`PEscape` c)

Maybe it is better to handle them in the translation:

regex-tdfa/lib/Text/Regex/TDFA/CorePattern.hs

Lines 537 to 538 in 95d47cb

	-- otherwise escape codes are just the escaped character
	PEscape {} -> one

[andreasabel]

There seems to be already code for POSIX character classes:

regex-tdfa/lib/Text/Regex/TDFA/TNFA.hs

Lines 798 to 805 in 95d47cb

	-- | This returns the distinct ascending list of characters
	-- represented by [: :] values in legalCharacterClasses; unrecognized
	-- class names return an empty string
	decodeCharacterClass :: PatternSetCharacterClass -> String
	decodeCharacterClass (PatternSetCharacterClass s) =
	case s of
	"alnum" -> ['0'..'9']++['a'..'z']++['A'..'Z']
	"digit" -> ['0'..'9']

These can be given to Patterns PAny and PAnyNot:

regex-tdfa/lib/Text/Regex/TDFA/Pattern.hs

Lines 45 to 46 in 95d47cb

	| PAny {getDoPa::DoPa,getPatternSet::PatternSet} -- Square bracketed things
	| PAnyNot {getDoPa::DoPa,getPatternSet::PatternSet} -- Inverted square bracketed things

@asarkar: The syntax accepted by regex-tdfa is [[:digit:]] instead of \d. See https://regex101.com/r/griuTm/1 for your whole regex.

[asarkar]

https://regex101.com/r/griuTm/1 shows \d, is that the correct link?

[andreasabel]

regex101.com/r/griuTm/1 shows \d, is that the correct link?

No, \d should be replaced by [[:digit:]]. I updated the regex, but the link didn't update.

Supporting Perl-style regexes like \d would not be hard to implement, but it would be a backward-incompatible change, because currently \d means simply d. So, I am not sure whether it is worth it. While \d is quicker to type, [[:digit:]] is easier to comprehend if you look at a regex. What is your application of regex-tdfa?