Txt File-Dictionary-Frequency

Question

How do I count the number of times each five letter word appears in a text file and then print the five most frequent and least frequent five letter words?

So far this is what I have written going off of some of the answers shown to me so far. I can't get it to give me the words with five letters and print the most frequent and least frequent words.

counter = {}

in_file = open('tale_of_two_cities_ascii.txt', 'r')
content = in_file.read()


for line in in_file:
    for word in line.split():
        if len(word) != 5: continue

        if word not in counter:
            counter[word] = 0
            counter[word] += 1

words = sorted(counter, key=counter.get)
print("The five most frequent words:", ','.join(words[-5:]))
print("The five least frequent words:", ','.join(words[:5]))

Possible duplicate of [item frequency count in python](http://stackoverflow.com/questions/893417/item-frequency-count-in-python) — TessellatingHeckler, Oct 31 '15 at 06:31
the `counter[word]+=1` should not be indented under the if statement. — ytpillai, Oct 31 '15 at 17:05

score 1 · Answer 1 · answered Oct 31 '15 at 06:25

Have a try about collections.Counter:

>>> Counter('abracadabra').most_common(3)  # most common three items
[('a', 5), ('r', 2), ('b', 2)]
>>> Counter('abracadabra').most_common()[:-4:-1] # least common three items
[('d', 1), ('c', 1), ('b', 2)]

so, the solution maybe like this:

import re
from collections import Counter

with open('your_text_file') as f:
    content = f.read()
    words = re.findall(r'\w+', content)
    counter = Counter(words)
    most_common = [item[0] for item in counter.most_common() if len(item[0]) == 5][:5]
    least_common = [item[0] for item in counter.most_common() if len(item[0]) == 5][:-6:-1]

score 0 · Answer 2 · answered Oct 31 '15 at 06:13

counter = {}

with open('tale_of_two_cities_ascii.txt') as infile:
    for line in infile:
        for word in line.strip():
            if len(word) != 5: continue
            if word not in counter: counter[word] = 0
            counter[word] += 1

words = sorted(counter, key=counter.__get__)
print("The five most common words are:", ','.join(words[-5:]))
print("The five least common words are:", ','.join(words[:5]))

score 0 · Answer 3 · answered Oct 31 '15 at 07:19

Check this out

>>> import re
>>> from collections import Counter
>>> # 1st the text tokenizer
>>> TOKENS = lambda x: re.findall('[a-zA-Z]+', x)
>>> # 2nd counts the tokens with exactly 5 letters
>>> COUNTS = lambda txt: Counter([t for t in TOKENS(txt) if len(t) == 5])

Demo 1 read text from file

>>> # read some text file
>>> text = open('README.txt').read()
>>> # prints the most common 5 words in the counter
>>> print(COUNTS(text).most_common(5))
[('words', 3), ('Words', 3), ('model', 3), ('small', 2), ('Given', 1)]

Demo 2 with short text

>>> demo = '''fives!! towes towes.. another fives cools, words NLP python fives'''
>>> print(COUNTS(demo).most_common(5))
[('fives', 3), ('towes', 2), ('words', 1), ('cools', 1)]

you may also change the TOKENS to the pattern you like, e.g to lower case '[a-z]+', x.lower().

Txt File-Dictionary-Frequency

3 Answers3