Sentence Break
Sentence Break is a text segmentation rule that determines where sentences start and end in a string of text, based on punctuation, spacing, and certain character classes like uppercase letters, numbers, or emoji. It operates by assigning each character a specific Break value, such as ATerm, Lower, Upper, or Sep, and then applying algorithmic rules to these values to avoid false breaks after abbreviations like “Mr.” or initials, while still splitting after terminal punctuation like periods, question marks, or exclamation points followed by appropriate whitespace. This enables software to correctly iterate through sentences for tasks like line wrapping, text selection, or natural language processing, ensuring that complex typographic conventions across languages, including those without spaces, are handled consistently.
- Abbr
- AT
- Count
- 4
ATerm is a Sentence Break value marking a period that ends a sentence only when followed by certain conditions, like a space and uppercase letter.
- Abbr
- CL
- Count
- 197
Close is a sentence break value for punctuation that typically pairs with an opening mark, like a closing bracket or quote.
- Abbr
- CR
- Count
- 1
CR is a Sentence Break value that marks a carriage return, forcing a sentence boundary immediately after it.
- Abbr
- EX
- Count
- 2680
Extend is a Sentence Break value that marks characters which continue a sentence without starting a new one, such as combining marks.
- Abbr
- FO
- Count
- 60
Format is used for characters that control formatting, like zero-width joiners, which are ignored when determining sentence boundaries.
- Abbr
- LF
- Count
- 1
LF is a control character that forces a hard line break, terminating a sentence without requiring a following space.
- Abbr
- LO
- Count
- 2693
Lower is a sentence break value indicating a lowercase letter that typically begins a new sentence, preventing breaks before it.
- Abbr
- NU
- Count
- 785
Numeric is absent for all Sentence Break values, as they are categorical labels like ATerm, Upper, and Sep, not numeric indicators.
- Abbr
- LE
- Count
- 154159
OLetter is a sentence break class indicating characters that form words, including letters and digits, used to define sentence boundaries for text segmentation.
- Abbr
- XX
- Count
- 147528
Other is the default Sentence Break value for characters that don’t fit any specific rule, like symbols, punctuation, and unassigned code points.
- Abbr
- SC
- Count
- 31
SContinue is a Sentence Break value indicating that a character continues a sentence without starting a new one, like commas or quotation marks.
- Abbr
- SE
- Count
- 3
Sep is the Sentence Break value assigned to characters that act as separators, like line or paragraph terminators, to mark sentence boundaries.
- Abbr
- SP
- Count
- 20
Sp is a numeric value indicating the likelihood that a sentence ends after a given character, based on statistical analysis.
- Abbr
- ST
- Count
- 168
STerm is a binary value marking characters that can end a sentence, aiding in text segmentation for line breaks and processing.
- Abbr
- UP
- Count
- 2011
Upper is a reserved sentence break value used for uppercase letters, indicating they start a new sentence when no explicit boundary exists.