Count word occurrence in a list of strings

Question

How to count how many times a word appears in a list of strings?

For example:

['This is a sentence', 'This is another sentence']

and the result for the word "sentence" is 2

Could you paste your first attempt that didn't work so we can help you track down the problem? — JoshG79, Aug 14 '13 at 12:20
Related : [item frequency count in python](http://stackoverflow.com/questions/893417/item-frequency-count-in-python) — Ashwini Chaudhary, Aug 14 '13 at 12:23
nice community you got here..perhaps instead of being mean you could try answering the question or freakin' skip it — user2578185, Aug 14 '13 at 12:39
You clearly didn't read instructions about making a proper question nor show any effort of trying to solve the problem yourself. — jester112358, Aug 14 '13 at 13:13
sum(i.count('sentence') for i in ['This is a sentence', 'This is another sentence']) — Ray, Apr 16 '14 at 12:39

Martijn Pieters · Accepted Answer · 2013-08-14T12:29:32.267

12

Use a collections.Counter() object and split your words on whitespace. You probably want to lowercase your words as well, and remove punctuation:

from collections import Counter

counts = Counter()

for sentence in sequence_of_sentences:
    counts.update(word.strip('.,?!"\'').lower() for word in sentence.split())

or perhaps use a regular expression that only matches word characters:

from collections import Counter
import re

counts = Counter()
words = re.compile(r'\w+')

for sentence in sequence_of_sentences:
    counts.update(words.findall(sentence.lower()))

Now you have a counts dictionary with per-word counts.

Demo:

>>> sequence_of_sentences = ['This is a sentence', 'This is another sentence']
>>> from collections import Counter
>>> counts = Counter()
>>> for sentence in sequence_of_sentences:
...     counts.update(word.strip('.,?!"\'').lower() for word in sentence.split())
... 
>>> counts
Counter({'this': 2, 'is': 2, 'sentence': 2, 'a': 1, 'another': 1})
>>> counts['sentence']
2

edited Aug 14 '13 at 12:29

answered Aug 14 '13 at 12:20

Martijn Pieters

889,049
245
3,507
2,997

Instead of trying to write out all the punctuation symbols, I'd use regex in the first place. `re.findall('\w+', sentence)` – Oleh Prypin Aug 14 '13 at 12:23
@OlehPrypin: Oh, yeah, nice idea too. – Martijn Pieters Aug 14 '13 at 12:29
@OlehPrypin The problem with that regex is that it doesn't account for words with `-`, it will count as 2 words – Rafael Almeida May 20 '16 at 12:55
@RafaelAlmeida: then update the regex to include those: `r'[\w-]+'` would count dashes as word characters. – Martijn Pieters May 20 '16 at 14:32

score 3 · Answer 2 · answered Aug 14 '13 at 12:56

You could do what you want pretty easily with a little regex and a dictionary.

import re

dict = {}
sentence_list = ['This is a sentence', 'This is a sentence']
for sentence in sentence_list:
    for word in re.split('\s', sentence): # split with whitespace
        try:
            dict[word] += 1
        except KeyError:
            dict[word] = 1
print dict

Count word occurrence in a list of strings

2 Answers2

Linked

Related