Python regex split on U+2028 LINE SEPARATOR and U+2029 PARAGRAPH SEPARATOR splits ordinary prose
A Python text-splitting utility used a regex alternation intended to treat U+2028 LINE SEPARATOR and U+2029 PARAGRAPH SEPARATOR as line breaks: re.compile(r'\r\n|[\n\v\f\r\x2028\x2029\x85]'). Instead of splitting on the Unicode separators, it silently split ordinary prose on substrings like ' 28': splitting 'February 28, 2026' returned ['February', ', 2026'], corrupting text that contained no line breaks at all. No error or warning was raised anywhere; the pattern compiled fine and looked correct in review, so the corruption shipped and lurked for years. Testing with actual U+2028 characters was not done because the escape read as obviously covering that codepoint.
Python's re (and string literals generally) define \x as consuming exactly two hex digits. \x2028 is parsed as \x20 (a space) followed by the literal characters 2 and 8 — so inside a character class [\x2028\x2029] you actually get the set {space, '2', '8', '0', '9'}, and in an alternation \x2028|\x2029 you match the three-character strings ' 28' and ' 29'. That is why 'February 28' splits: the pattern matches the space-plus-28.
Fix: use the four-digit Unicode escape, which is what these codepoints require:
# wrong: matches ' 28' / ' 29'
re.compile('\\x2028|\\x2029')
# right: matches U+2028 LINE SEPARATOR / U+2029 PARAGRAPH SEPARATOR
re.compile('\\u2028|\\u2029')This trap is especially common when porting regexes or line-splitting logic from other ecosystems, or when an author half-remembers the JavaScript idiom (\u2028 is famous in JS for breaking JSON-in-<script>). A quick audit heuristic: any \x escape followed by more than two hex digits in a Python pattern is almost certainly a bug — grep for \\x[0-9a-fA-F]{3,}. Note str.splitlines() already treats U+2028/U+2029 as line boundaries natively, which is a safer baseline when applicable.