Regular expression

From 太極
Revision as of 20:39, 25 March 2020 by Brb (talk | contribs) (→‎Resources)
Jump to navigation Jump to search

Resources

Specific to R

Specific to Terminal

Using Grep & Regular Expressions to Search for Text Patterns in Linux & Extended Regular Expressions.

Grep Multiple Strings from a File in Linux

Online tools

Metacharacters

There are 12 metacharacters. It seems "]", "}" and "-" are not metacharacters. 

.   \   |   (   )   [   {   $   ^   *   +   ?

If we want to match them, we need to precede them with a double backslash. 

gsub("$", ".", "abc$def") # "abc$def"
gsub("\\$", ".", "abc$def") # "abc.def"

metachar <- scan(textConnection(".   \\   |   (   )   [   ]   {   }   $   -    ^   *   +   ?"), "")
# Read 15 items
metachar
# [1] "."  "\\" "|"  "("  ")"  "["  "]"  "{"  "}"  "$"  "-"  "^"  "*"  "+"  "?" 
nchar(metachar[2])
# [1] 1
grep("\\.", metachar, value = TRUE) # "."
grep("\\.", metachar) # 1
grep("\\\\", metachar) # 2
grep("]", metachar)  #  7
grep("}", metachar)  #  9
grep("{", metachar) 
# Error in grep("{", metachar) : 
#  invalid regular expression '{', reason 'Missing '}''
grep("\\$", metachar) # 10
grep("-", metachar)  # 11

strsplit("abc.def", "\\.")
strsplit("abc.def", ".", fixed = TRUE)
  • "." matches everything except for the empty sting "".
  • "+" the preceding item will be matched one or more times.
  • "*" the preceding item will be matched zero or more times.
  • "^" matches the empty string at the at the beginning of a line. When used in a character class means to match any character but the following ones.
  • "$" matches empty string at the end of a line.
  • "|" infix operator: OR
  • "(", ")" brackets for grouping.
  • "[", "]" character class brackets

+

gsub(pattern = "\\.\\.", replace = ".", "id..of...patient")
# [1] "id.of..patient"   NOT RIGHT, need to apply the command multiple times

gsub(pattern = "\\.+", replace = ".", "id..of...patient")
# [1] "id.of.patient"

Character classes

Replacing all values that do not contain letters or digits with NA value. The following example is from here. 

s <- c("", "  ", "3 times a day after meal", "once a day", "       ","  one per day ", "\t", "\n  ")
# Method 1
s[s==""|s=="  "|s=="       "|s=="\t"|s=="\n"]  # BAD

# Method 2
allIndices = 1:length(s)
letOrDigIndices = grep("[a-zA-Z0-9]", s)
blankInd = setdiff(allIndices, letOrDigIndices)
s[blankInd]

# Method 3
gsub("^$|^( +)$|[\t\n\r\f\v]+", NA, s)

# Method 4. Get rid of extra blank spaces
s1 = gsub("^([ \t\n\r\f\v]+)|([ \t\n\r\f\v]+)$", "", s)
gsub("^$", NA, s1)

Syntax

The following table is from endmemo.com.

Syntax Description
\\d Digit, 0,1,2 ... 9
\\D Not Digit
\\s Space
\\S Not Space
\\w Word
\\W Not Word
\\t Tab
\\n New line
^ Beginning of the string
$ End of the string
^KEY1.*KEY2$ Beginning and end of a string
\ Escape special characters, e.g. \\ is "\", \+ is "+"
| Alternation match. e.g. /(e|d)n/ matches "en" and "dn"
. OR .* Any character, except \n or line terminator
[ab] a or b
[^ab] Any character except a and b
[0-9] All Digit
[A-Z] All uppercase A to Z letters
[a-z] All lowercase a to z letters
[A-z] All Uppercase and lowercase a to z letters
i+ i at least one time (Repetition)
i* i zero or more times (Repetition)
i? i zero or 1 time (Repetition)
i{n} i occurs n times in sequence
i{n1,n2} i occurs n1 - n2 times in sequence
i{n1,n2}? non greedy match, see above example
i{n,} i occures >= n times
[:alnum:] Alphanumeric characters: [:alpha:] and [:digit:]
[:alpha:] Alphabetic characters: [:lower:] and [:upper:]
[:blank:] Blank characters: e.g. space, tab
[:cntrl:] Control characters
[:digit:] Digits: 0 1 2 3 4 5 6 7 8 9
[:graph:] Graphical characters: [:alnum:] and [:punct:]
[:lower:] Lower-case letters in the current locale
[:print:] Printable characters: [:alnum:], [:punct:] and space
[:punct:] } ~
[:space:] Space characters: tab, newline, vertical tab, form feed, carriage return, space
[:upper:] Upper-case letters in the current locale
[:xdigit:] Hexadecimal digits: 0 1 2 3 4 5 6 7 8 9 A B C D E F a b c d e f

grep()

  • Use value = TRUE will return the matching elements instead of indices
  • Use invert = TRUE will return the indices or values for elements that do not match
  • Use ignore.case = TRUE

Ref:

grepl() and fix parameter

Test if characters are in a string

sub() and gsub()

The sub function changes only the first occurrence of the regular expression, while the gsub function performs the substitution on all occurrences within the string.

To extract the filename without extension, 

sub('\\..*$', '', Filename)

regexpr() and gregexpr()

The output from these functions is a vector of starting positions of the regular expressions which were found; if no match occurred, a value of -1 is returned.

The regexpr function will only provide information about the first match in its input string(s), while the gregexpr function returns information about all matches found.

Note that in C++, the std::string::find() and Qt's QRegExp::indexIn() can do R's regexpr() does. I am not aware of any gregexpr()-equivalent function in C++.

The following example is coming from the book 'Data Manipulation with R' by Phil Spector, Chapter 7, Character Manipulation. 

tst = c('one x7 two b1', 'three c5 four b9', 'five six seven', 'a8 eight nine')
wh = regexpr('[a-z][0-9]', tst)
wh
# [1] 5 7 -1 1
# attr(,"match.length")
# [1] 2 2 -1 2

wh1 = gregexpr('[a-z][0-9]',tst) # return a list just like strsplit()
wh1

# [[1]]
# [1]  5 12
# attr(,"match.length")
# [1] 2 2
# attr(,"useBytes")
# [1] TRUE
#
# [[2]]
# [1]  7 15
# attr(,"match.length")
# [1] 2 2
# attr(,"useBytes")
# [1] TRUE
#
# [[3]]
# [1] -1
# attr(,"match.length")
# [1] -1
# attr(,"useBytes")
# [1] TRUE
#
# [[4]]
# [1] 1
# attr(,"match.length")
# [1] 2
# attr(,"useBytes")
# [1] TRUE

gregexpr("'", "|3'-5'") # find the apostrophe character
# [[1]]
# [1] 3 6
# attr(,"match.length")
# [1] 1 1
# attr(,"useBytes")
# [1] TRUE

Examples

  • sub("^.*boundary=", "", string) will substitute a substring which starts with 0 or more characters and then 'boundary=' with an empty. Here ^ means beginning, dot means any character and star means the preceding item 0 or more times.
  • grep("\\.zip$", pkgs) or grep("\\.tar.gz$", pkgs) will search for the string ending with .zip or .tar.gz
  • biocLite(suppressUpdates=c("^org\.", "^BSgenome\.")) not update any package whose name starts with "org." or "BSgenome."
  • grep("9.11", string) will search for the string containing '9', any character (to split 9 & 11) and '11'.
  • pipe metacharacter; it is translated to 'or'. flood|fire will match strings containing floor or fire.
  • [^?.]$ will match anyone ([]) not (^) ending ($) with the question mark (?) or period (.).
  • ^[Gg]ood|[Bb]ad will match strings starting with Good/good and anywhere containing Bad/bad.
  • ^([Gg]ood|[Bb]ad) will look for strings beginning with Good/good/Bad/bad.
  • ? character; it means optional. [Gg]eorge( [Ww]\.)? [Bb]ush will match strings like 'george bush', 'George W. Bush' or 'george bushes'. Note that we escape the metacharacter dot by '\.' so it becomes a literal period.
  • star and plus sign. star means any number including none and plus means at least one. For example, (.*) matches 'abc(222 )' and '()'.
  • [0-9]+ (.*) [0-9]+ will match one number and following by any number of characters (.*) and a number; e.g. 'afda1080 p' and '4 by 5 size'.
  • gsub("space:+", " ", " ab c ") will replace multiple spaces with 1 space.
  • {} refers to as interval quantifiers; specify the minimum and maximum number of match of an expression.
  • trimws() function to remove trailing/leading whitespace. The function is used in several places. 
    trimws <-
function(x, which = c("both", "left", "right"))
{
    which <- match.arg(which)
    mysub <- function(re, x) sub(re, "", x, perl = TRUE)
    if(which == "left")
        return(mysub("^[ \t\r\n]+", x))
    if(which == "right")
        return(mysub("[ \t\r\n]+$", x))
    mysub("[ \t\r\n]+$", mysub("^[ \t\r\n]+", x))
}
  • Another solution to trim leading/trailing space is 
    # returns string w/o leading whitespace
trim.leading <- function (x)  sub("^\\s+", "", x)

# returns string w/o trailing whitespace
trim.trailing <- function (x) sub("\\s+$", "", x)

# returns string w/o leading or trailing whitespace
trim <- function (x) gsub("^\\s+|\\s+$", "", x)
  • Extract/replace text between parentheses https://stackoverflow.com/a/13498914, stringr::str_replace() 
    gsub("\\(.*\\)", "", c("0.1385(+)", "0.33", "0.12(-)")

Special case: match the dot character

See Chapter 11: Strings with stringr in 'R for Data Science' by Hadley Wickham.

The printed representation of a string shows the escapes. To see the raw contents of the string, use writeLines(). 

x <- c("\"", "\\") # escape ", \
x
# [1] "\"" "\\"
writeLines(x)
# "
# \

"." matches any character. To match the dot character literally we shall use "\\.". 

# We want to match the dot character literally
writeLines("\.")
# Error: '\.' is an unrecognized escape in character string starting ""\."

# . should be represented as \. but \ itself should be escaped so
# to escape ., we should use \\.
writeLines("\\.")
# \.

Special case: match the backslash \

x <- "a\\b"
writeLines(x)
# a\b

str_view(x, "\\\\")

Approximate matching

TRE library 

> names <- c("Konrad", "Conrad", "Konard", "Connard", "con rat", "Conga rat")
> grep("(Konrad){~2}", names, value = TRUE)
[1] "Konrad" "Conrad" "Konard"
> grep("(Konrad){~1}", names, value = TRUE)
[1] "Konrad" "Conrad"

stringr package

Retrieved from "https://wiki.taichimd.us/index.php?title=Regular_expression&oldid=23957"