1Oniguruma Regular Expressions Version 5.9.1 2007/09/05 2 3syntax: ONIG_SYNTAX_RUBY (default) 4 5 61. Syntax elements 7 8 \ escape (enable or disable meta character meaning) 9 | alternation 10 (...) group 11 [...] character class 12 13 142. Characters 15 16 \t horizontal tab (0x09) 17 \v vertical tab (0x0B) 18 \n newline (0x0A) 19 \r return (0x0D) 20 \b back space (0x08) 21 \f form feed (0x0C) 22 \a bell (0x07) 23 \e escape (0x1B) 24 \nnn octal char (encoded byte value) 25 \xHH hexadecimal char (encoded byte value) 26 \x{7HHHHHHH} wide hexadecimal char (character code point value) 27 \cx control char (character code point value) 28 \C-x control char (character code point value) 29 \M-x meta (x|0x80) (character code point value) 30 \M-\C-x meta control char (character code point value) 31 32 (* \b is effective in character class [...] only) 33 34 353. Character types 36 37 . any character (except newline) 38 39 \w word character 40 41 Not Unicode: 42 alphanumeric, "_" and multibyte char. 43 44 Unicode: 45 General_Category -- (Letter|Mark|Number|Connector_Punctuation) 46 47 \W non word char 48 49 \s whitespace char 50 51 Not Unicode: 52 \t, \n, \v, \f, \r, \x20 53 54 Unicode: 55 0009, 000A, 000B, 000C, 000D, 0085(NEL), 56 General_Category -- Line_Separator 57 -- Paragraph_Separator 58 -- Space_Separator 59 60 \S non whitespace char 61 62 \d decimal digit char 63 64 Unicode: General_Category -- Decimal_Number 65 66 \D non decimal digit char 67 68 \h hexadecimal digit char [0-9a-fA-F] 69 70 \H non hexadecimal digit char 71 72 73 Character Property 74 75 * \p{property-name} 76 * \p{^property-name} (negative) 77 * \P{property-name} (negative) 78 79 property-name: 80 81 + works on all encodings 82 Alnum, Alpha, Blank, Cntrl, Digit, Graph, Lower, 83 Print, Punct, Space, Upper, XDigit, Word, ASCII, 84 85 + works on EUC_JP, Shift_JIS 86 Hiragana, Katakana 87 88 + works on UTF8, UTF16, UTF32 89 Any, Assigned, C, Cc, Cf, Cn, Co, Cs, L, Ll, Lm, Lo, Lt, Lu, 90 M, Mc, Me, Mn, N, Nd, Nl, No, P, Pc, Pd, Pe, Pf, Pi, Po, Ps, 91 S, Sc, Sk, Sm, So, Z, Zl, Zp, Zs, 92 Arabic, Armenian, Bengali, Bopomofo, Braille, Buginese, 93 Buhid, Canadian_Aboriginal, Cherokee, Common, Coptic, 94 Cypriot, Cyrillic, Deseret, Devanagari, Ethiopic, Georgian, 95 Glagolitic, Gothic, Greek, Gujarati, Gurmukhi, Han, Hangul, 96 Hanunoo, Hebrew, Hiragana, Inherited, Kannada, Katakana, 97 Kharoshthi, Khmer, Lao, Latin, Limbu, Linear_B, Malayalam, 98 Mongolian, Myanmar, New_Tai_Lue, Ogham, Old_Italic, Old_Persian, 99 Oriya, Osmanya, Runic, Shavian, Sinhala, Syloti_Nagri, Syriac, 100 Tagalog, Tagbanwa, Tai_Le, Tamil, Telugu, Thaana, Thai, Tibetan, 101 Tifinagh, Ugaritic, Yi 102 103 104 1054. Quantifier 106 107 greedy 108 109 ? 1 or 0 times 110 * 0 or more times 111 + 1 or more times 112 {n,m} at least n but not more than m times 113 {n,} at least n times 114 {,n} at least 0 but not more than n times ({0,n}) 115 {n} n times 116 117 reluctant 118 119 ?? 1 or 0 times 120 *? 0 or more times 121 +? 1 or more times 122 {n,m}? at least n but not more than m times 123 {n,}? at least n times 124 {,n}? at least 0 but not more than n times (== {0,n}?) 125 126 possessive (greedy and does not backtrack after repeated) 127 128 ?+ 1 or 0 times 129 *+ 0 or more times 130 ++ 1 or more times 131 132 ({n,m}+, {n,}+, {n}+ are possessive op. in ONIG_SYNTAX_JAVA only) 133 134 ex. /a*+/ === /(?>a*)/ 135 136 1375. Anchors 138 139 ^ beginning of the line 140 $ end of the line 141 \b word boundary 142 \B not word boundary 143 \A beginning of string 144 \Z end of string, or before newline at the end 145 \z end of string 146 \G matching start position 147 148 1496. Character class 150 151 ^... negative class (lowest precedence operator) 152 x-y range from x to y 153 [...] set (character class in character class) 154 ..&&.. intersection (low precedence at the next of ^) 155 156 ex. [a-w&&[^c-g]z] ==> ([a-w] AND ([^c-g] OR z)) ==> [abh-w] 157 158 * If you want to use '[', '-', ']' as a normal character 159 in a character class, you should escape these characters by '\'. 160 161 162 POSIX bracket ([:xxxxx:], negate [:^xxxxx:]) 163 164 Not Unicode Case: 165 166 alnum alphabet or digit char 167 alpha alphabet 168 ascii code value: [0 - 127] 169 blank \t, \x20 170 cntrl 171 digit 0-9 172 graph include all of multibyte encoded characters 173 lower 174 print include all of multibyte encoded characters 175 punct 176 space \t, \n, \v, \f, \r, \x20 177 upper 178 xdigit 0-9, a-f, A-F 179 word alphanumeric, "_" and multibyte characters 180 181 182 Unicode Case: 183 184 alnum Letter | Mark | Decimal_Number 185 alpha Letter | Mark 186 ascii 0000 - 007F 187 blank Space_Separator | 0009 188 cntrl Control | Format | Unassigned | Private_Use | Surrogate 189 digit Decimal_Number 190 graph [[:^space:]] && ^Control && ^Unassigned && ^Surrogate 191 lower Lowercase_Letter 192 print [[:graph:]] | [[:space:]] 193 punct Connector_Punctuation | Dash_Punctuation | Close_Punctuation | 194 Final_Punctuation | Initial_Punctuation | Other_Punctuation | 195 Open_Punctuation 196 space Space_Separator | Line_Separator | Paragraph_Separator | 197 0009 | 000A | 000B | 000C | 000D | 0085 198 upper Uppercase_Letter 199 xdigit 0030 - 0039 | 0041 - 0046 | 0061 - 0066 200 (0-9, a-f, A-F) 201 word Letter | Mark | Decimal_Number | Connector_Punctuation 202 203 204 2057. Extended groups 206 207 (?#...) comment 208 209 (?imx-imx) option on/off 210 i: ignore case 211 m: multi-line (dot(.) match newline) 212 x: extended form 213 (?imx-imx:subexp) option on/off for subexp 214 215 (?:subexp) not captured group 216 (subexp) captured group 217 218 (?=subexp) look-ahead 219 (?!subexp) negative look-ahead 220 (?<=subexp) look-behind 221 (?<!subexp) negative look-behind 222 223 Subexp of look-behind must be fixed character length. 224 But different character length is allowed in top level 225 alternatives only. 226 ex. (?<=a|bc) is OK. (?<=aaa(?:b|cd)) is not allowed. 227 228 In negative-look-behind, captured group isn't allowed, 229 but shy group(?:) is allowed. 230 231 (?>subexp) atomic group 232 don't backtrack in subexp. 233 234 (?<name>subexp), (?'name'subexp) 235 define named group 236 (All characters of the name must be a word character.) 237 238 Not only a name but a number is assigned like a captured 239 group. 240 241 Assigning the same name as two or more subexps is allowed. 242 In this case, a subexp call can not be performed although 243 the back reference is possible. 244 245 2468. Back reference 247 248 \n back reference by group number (n >= 1) 249 \k<n> back reference by group number (n >= 1) 250 \k'n' back reference by group number (n >= 1) 251 \k<-n> back reference by relative group number (n >= 1) 252 \k'-n' back reference by relative group number (n >= 1) 253 \k<name> back reference by group name 254 \k'name' back reference by group name 255 256 In the back reference by the multiplex definition name, 257 a subexp with a large number is referred to preferentially. 258 (When not matched, a group of the small number is referred to.) 259 260 * Back reference by group number is forbidden if named group is defined 261 in the pattern and ONIG_OPTION_CAPTURE_GROUP is not setted. 262 263 264 back reference with nest level 265 266 level: 0, 1, 2, ... 267 268 \k<n+level> (n >= 1) 269 \k<n-level> (n >= 1) 270 \k'n+level' (n >= 1) 271 \k'n-level' (n >= 1) 272 273 \k<name+level> 274 \k<name-level> 275 \k'name+level' 276 \k'name-level' 277 278 Destinate relative nest level from back reference position. 279 280 ex 1. 281 282 /\A(?<a>|.|(?:(?<b>.)\g<a>\k<b+0>))\z/.match("reer") 283 284 ex 2. 285 286 r = Regexp.compile(<<'__REGEXP__'.strip, Regexp::EXTENDED) 287 (?<element> \g<stag> \g<content>* \g<etag> ){0} 288 (?<stag> < \g<name> \s* > ){0} 289 (?<name> [a-zA-Z_:]+ ){0} 290 (?<content> [^<&]+ (\g<element> | [^<&]+)* ){0} 291 (?<etag> </ \k<name+1> >){0} 292 \g<element> 293 __REGEXP__ 294 295 p r.match('<foo>f<bar>bbb</bar>f</foo>').captures 296 297 298 2999. Subexp call ("Tanaka Akira special") 300 301 \g<name> call by group name 302 \g'name' call by group name 303 \g<n> call by group number (n >= 1) 304 \g'n' call by group number (n >= 1) 305 \g<-n> call by relative group number (n >= 1) 306 \g'-n' call by relative group number (n >= 1) 307 308 * left-most recursive call is not allowed. 309 ex. (?<name>a|\g<name>b) => error 310 (?<name>a|b\g<name>c) => OK 311 312 * Call by group number is forbidden if named group is defined in the pattern 313 and ONIG_OPTION_CAPTURE_GROUP is not setted. 314 315 * If the option status of called group is different from calling position 316 then the group's option is effective. 317 318 ex. (?-i:\g<name>)(?i:(?<name>a)){0} match to "A" 319 320 32110. Captured group 322 323 Behavior of the no-named group (...) changes with the following conditions. 324 (But named group is not changed.) 325 326 case 1. /.../ (named group is not used, no option) 327 328 (...) is treated as a captured group. 329 330 case 2. /.../g (named group is not used, 'g' option) 331 332 (...) is treated as a no-captured group (?:...). 333 334 case 3. /..(?<name>..)../ (named group is used, no option) 335 336 (...) is treated as a no-captured group (?:...). 337 numbered-backref/call is not allowed. 338 339 case 4. /..(?<name>..)../G (named group is used, 'G' option) 340 341 (...) is treated as a captured group. 342 numbered-backref/call is allowed. 343 344 where 345 g: ONIG_OPTION_DONT_CAPTURE_GROUP 346 G: ONIG_OPTION_CAPTURE_GROUP 347 348 ('g' and 'G' options are argued in ruby-dev ML) 349 350 351 352----------------------------- 353A-1. Syntax depend options 354 355 + ONIG_SYNTAX_RUBY 356 (?m): dot(.) match newline 357 358 + ONIG_SYNTAX_PERL and ONIG_SYNTAX_JAVA 359 (?s): dot(.) match newline 360 (?m): ^ match after newline, $ match before newline 361 362 363A-2. Original extensions 364 365 + hexadecimal digit char type \h, \H 366 + named group (?<name>...), (?'name'...) 367 + named backref \k<name> 368 + subexp call \g<name>, \g<group-num> 369 370 371A-3. Lacked features compare with perl 5.8.0 372 373 + \N{name} 374 + \l,\u,\L,\U, \X, \C 375 + (?{code}) 376 + (??{code}) 377 + (?(condition)yes-pat|no-pat) 378 379 * \Q...\E 380 This is effective on ONIG_SYNTAX_PERL and ONIG_SYNTAX_JAVA. 381 382 383A-4. Differences with Japanized GNU regex(version 0.12) of Ruby 1.8 384 385 + add character property (\p{property}, \P{property}) 386 + add hexadecimal digit char type (\h, \H) 387 + add look-behind 388 (?<=fixed-char-length-pattern), (?<!fixed-char-length-pattern) 389 + add possessive quantifier. ?+, *+, ++ 390 + add operations in character class. [], && 391 ('[' must be escaped as an usual char in character class.) 392 + add named group and subexp call. 393 + octal or hexadecimal number sequence can be treated as 394 a multibyte code char in character class if multibyte encoding 395 is specified. 396 (ex. [\xa1\xa2], [\xa1\xa7-\xa4\xa1]) 397 + allow the range of single byte char and multibyte char in character 398 class. 399 ex. /[a-<<any EUC-JP character>>]/ in EUC-JP encoding. 400 + effect range of isolated option is to next ')'. 401 ex. (?:(?i)a|b) is interpreted as (?:(?i:a|b)), not (?:(?i:a)|b). 402 + isolated option is not transparent to previous pattern. 403 ex. a(?i)* is a syntax error pattern. 404 + allowed incompleted left brace as an usual string. 405 ex. /{/, /({)/, /a{2,3/ etc... 406 + negative POSIX bracket [:^xxxx:] is supported. 407 + POSIX bracket [:ascii:] is added. 408 + repeat of look-ahead is not allowed. 409 ex. /(?=a)*/, /(?!b){5}/ 410 + Ignore case option is effective to numbered character. 411 ex. /\x61/i =~ "A" 412 + In the range quantifier, the number of the minimum is omissible. 413 /a{,n}/ == /a{0,n}/ 414 The simultanious abbreviation of the number of times of the minimum 415 and the maximum is not allowed. (/a{,}/) 416 + /a{n}?/ is not a non-greedy operator. 417 /a{n}?/ == /(?:a{n})?/ 418 + invalid back reference is checked and cause error. 419 /\1/, /(a)\2/ 420 + Zero-length match in infinite repeat stops the repeat, 421 then changes of the capture group status are checked as stop condition. 422 /(?:()|())*\1\2/ =~ "" 423 /(?:\1a|())*/ =~ "a" 424 425 426A-5. Disabled functions by default syntax 427 428 + capture history 429 430 (?@...) and (?@<name>...) 431 432 ex. /(?@a)*/.match("aaa") ==> [<0-1>, <1-2>, <2-3>] 433 434 see sample/listcap.c file. 435 436 437A-6. Problems 438 439 + Invalid encoding byte sequence is not checked. 440 441 ex. UTF-8 442 443 * Invalid first byte is treated as a character. 444 /./u =~ "\xa3" 445 446 * Incomplete byte sequence is not checked. 447 /\w+/ =~ "a\xf3\x8ec" 448 449// END 450
