Pandas str.contains - Search for multiple values in a string and print the values in a new column

Question

I just started coding in Python and want to build a solution where you would search a string to see if it contains a given set of values.

I've find a similar solution in R which uses the stringr library: Search for a value in a string and if the value exists, print it all by itself in a new column

The following code seems to work but i also want to output the three values that i'm looking for and this solution will only output one value:

#Inserting new column
df.insert(5, "New_Column", np.nan)

#Searching old column
df['New_Column'] = np.where(df['Column_with_text'].str.contains('value1|value2|value3', case=False, na=False), 'value', 'NaN')

------ Edit ------

So i realised i didn't give that good of an explanation, sorry about that.

Below is an example where i match fruit names in a string and depending on if it finds any matches in the string it will print out either true or false in a new column. Here's my question: Instead of printing out true or false i want to print out the name it found in the string eg. apples, oranges etc.

import pandas as pd
import numpy as np

text = [('I want to buy some apples.', 0),
         ('Oranges are good for the health.', 0),
         ('John is eating some grapes.', 0),
         ('This line does not contain any fruit names.', 0),
         ('I bought 2 blueberries yesterday.', 0)]
labels = ['Text','Random Column']

df = pd.DataFrame.from_records(text, columns=labels)

df.insert(2, "MatchedValues", np.nan)

foods =['apples', 'oranges', 'grapes', 'blueberries']

pattern = '|'.join(foods)

df['MatchedValues'] = df['Text'].str.contains(pattern, case=False)

print(df)

Result

                                          Text  Random Column  MatchedValues
0                   I want to buy some apples.              0           True
1             Oranges are good for the health.              0           True
2                  John is eating some grapes.              0           True
3  This line does not contain any fruit names.              0          False
4            I bought 2 blueberries yesterday.              0           True

Wanted result

                                          Text  Random Column  MatchedValues
0                   I want to buy some apples.              0           apples
1             Oranges are good for the health.              0           oranges
2                  John is eating some grapes.              0           grapes
3  This line does not contain any fruit names.              0          NaN
4            I bought 2 blueberries yesterday.              0           blueberries

adr · Answer 1 · 2018-02-26T20:43:12.157

11

You need to set the regex flag (to interpret your search as a regular expression):

whatIwant = df['Column_with_text'].str.contains('value1|value2|value3',
                                                 case=False, regex=True)

df['New_Column'] = np.where(whatIwant, df['Column_with_text'])

------ Edit ------

Based on the updated problem statement, here is an updated answer:

You need to define a capture group in the regular expression using parentheses and use the extract() function to return the values found within the capture group. The lower() function deals with any upper case letters

df['MatchedValues'] = df['Text'].str.lower().str.extract( '('+pattern+')', expand=False)

edited Feb 26 '18 at 20:43

answered Feb 05 '18 at 22:55

adr

947
5
14

1

Did this solve your problem? – adr Feb 09 '18 at 20:42
No, but i have edited my post now to make it more clear on what i'm aiming to do. Your help is much appreciated! – Feb 12 '18 at 20:49
Just wondering, did you try my updated (one line) solution? – adr Feb 21 '18 at 20:47

score 8 · Accepted Answer · answered Feb 13 '18 at 18:19

Here is one way:

foods =['apples', 'oranges', 'grapes', 'blueberries']

def matcher(x):
    for i in foods:
        if i.lower() in x.lower():
            return i
    else:
        return np.nan

df['Match'] = df['Text'].apply(matcher)

#                                           Text        Match
# 0                   I want to buy some apples.       apples
# 1             Oranges are good for the health.      oranges
# 2                  John is eating some grapes.       grapes
# 3  This line does not contain any fruit names.          NaN
# 4            I bought 2 blueberries yesterday.  blueberries

Pandas str.contains - Search for multiple values in a string and print the values in a new column

2 Answers2

Linked