Split a text into sentences

Question

How can I split a text into an array of sentences?

Example text:

Fry me a Beaver. Fry me a Beaver! Fry me a Beaver? Fry me Beaver no. 4?! Fry me many Beavers... End

Should output:

0 => Fry me a Beaver.
1 => Fry me a Beaver!
2 => Fry me a Beaver?
3 => Fry me Beaver no. 4?!
4 => Fry me many Beavers...
5 => End

I tried some solutions that I've found on SO through search, but they all fail, especially at the 4th sentence.

/(?<=[!?.])./

/\.|\?|!/

/((?<=[a-z0-9)][.?!])|(?<=[a-z0-9][.?!]\"))(\s|\r\n)(?=\"?[A-Z])/

/(?<=[.!?]|[.!?][\'"])\s+/    // <- closest one

The sentence #4 doesn't follow standard syntax. You need a class of `Terminators` - tokens that mark the end of a sentence. If you use one of the terminators as a regular symbol, then it's either not a terminator or you're misforming the sentences. You can't have your cake and eat it too, to put it simply. — Shark, May 04 '13 at 18:14
I make cakes and eat them all the time :P Can a regex look ahead like 2 characters and if 2nd character is not uppercase A-Z it means that the punctuation before is not valid — thelolcat, May 04 '13 at 18:16
@thelolcat you are better off with your own parser..a single regex won't do! You have to consider sentences which contains `Mr.thelolcat`, `no.1` — Anirudha, May 04 '13 at 18:32
what computer in this world should know that this: `no. 4?!` is the end of a sentence? What if it's `no. 4 (the number after 3)?!` You currently entering spheres which are reserved for Chuck Norris — hek2mgl, May 04 '13 at 18:40
@lolcat what your asking can be done with regexes, what you need is a zero width assertion, also the last regex you gave seems to work, what do you think is wrong with it — aaronman, May 04 '13 at 18:41

HamZa · Accepted Answer · 2013-05-04T19:10:54.003

27

Since you want to "split" sentences why are you trying to match them ?

For this case let's use preg_split().

Code:

$str = 'Fry me a Beaver. Fry me a Beaver! Fry me a Beaver? Fry me Beaver no. 4?! Fry me many Beavers... End';
$sentences = preg_split('/(?<=[.?!])\s+(?=[a-z])/i', $str);
print_r($sentences);

Output:

Array
(
    [0] => Fry me a Beaver.
    [1] => Fry me a Beaver!
    [2] => Fry me a Beaver?
    [3] => Fry me Beaver no. 4?!
    [4] => Fry me many Beavers...
    [5] => End
)

Explanation:

Well to put it simply we are spliting by grouped space(s) \s+ and doing two things:

(?<=[.?!]) Positive look behind assertion, basically we search if there is a point or question mark or exclamation mark behind the space.
(?=[a-z]) Positive look ahead assertion, searching if there is a letter after the space, this is kind of a workaround for the no. 4 problem.

edited May 04 '13 at 19:10

answered May 04 '13 at 18:47

HamZa

13,530
11
51
70

1

just a question: shouldn't `\s` be `\s+` ? I mean to ignore multiple spaces grouped toghether – thelolcat May 04 '13 at 19:09
@thelolcat Well you're right in case there is multiple spaces ! – HamZa May 04 '13 at 19:09
@HamZa : what would it translate to be, in java? I tried the same thing in java but it doesn't work. Can you guide me ? – voidMainReturn Jul 15 '13 at 12:47
@tejas I would guess you need to use double backslash instead of one `(?<=[.?!])\\s+(?=[a-z])` – HamZa Jul 15 '13 at 12:51
yes this is what I am using : str.split("(?<=[.?!])\\s+(?=[a-z])"); But of no use. – voidMainReturn Jul 15 '13 at 12:54
@tejas What do you mean "not correctly" ? Do you mind to join me in the [regex chatroom](http://chat.stackoverflow.com/rooms/25767) ? – HamZa Jul 15 '13 at 12:58
actually, the following one worked : str.split("(?<=[.?!])\\s+(?=[a-zA-Z])") – voidMainReturn Jul 15 '13 at 13:00
@tejas You see that little `i` after `/` ? That means match case insensitive. I think you could use my expression and add `(?i)` to the beginning of it :) – HamZa Jul 15 '13 at 13:02
yeah ok. I didn't know it's used as ?i in java. I tried using it as /i and it didn't work – voidMainReturn Jul 15 '13 at 13:06
@tejas not only in Java, it's possible in much more languages. – HamZa Jul 15 '13 at 13:08
1

Thank you! Add it to my helper library - https://github.com/Cosmologist/Gears/blob/master/src/Gears/StringType/Text.php – Cosmologist Oct 06 '16 at 13:50
I love this and would love it even more if I could disqualify `...` from counting as the end of a sentence and *include* `.)` as the end of a sentence. Ideas? Thanks. – Ryan Feb 11 '17 at 00:35
1

@Ryan quick [`(? – HamZa Feb 11 '17 at 00:39
1

@HamZa Holy smokes, you are incredible! Thank you so much!! – Ryan Feb 11 '17 at 00:55
3

Based on what I learned from yours, I was able to edit it to handle even more corner cases that I'm running into: https://regex101.com/r/e4NYyd/4 Cool stuff. – Ryan Feb 11 '17 at 01:39
This doesn't work. Try adding "i.e. " to the sentence, this regex fails at this – Richard May 17 '18 at 19:42
Also it doesn't work for sentences with Mr. and !" at the end of sentence. – holden321 Dec 21 '18 at 16:20
"2020 is the year the system failed." Sentences may start with a digit... which makes avoiding "See (A. 1) for reference." more complex. – Markus AO Sep 16 '20 at 21:07

Split a text into sentences

1 Answers1

Linked