Python regex works in regex tester but not in practical

Question

I need to match the text inside <p> element excluding the first <strong> element from the text.

When tested on an online regex tester like https://regexr.com/ it works fine, but when I run on my python script, it doesn't matches anything.

Regex:

<br>(.*)<\/p>$

Text:

<p><strong>Ackee and Saltfish Fritters</strong><br>\n2 cup salted Cod fish, soaked overnight<br>\n2 cloves garlic<br>\n½ medium onion<br>\n1 tsp thyme<br>\n1 tbsp cilantro<br>\n1 scallion, finely chopped<br>\n¼ scotch bonnet pepper, seeds removed<br>\n1 cup all purpose flour<br>\n1 tsp baking powder<br>\n½ cup ackee<br>\n¾ cup water<br>\nCanola oil</p>

Python code:

re.search(r"<br>(.*)<\/p>$", target_string)

Desired result in matches group 1:

\n2 cup salted Cod fish, soaked overnight<br>\n2 cloves garlic<br>\n½ medium onion<br>\n1 tsp thyme<br>\n1 tbsp cilantro<br>\n1 scallion, finely chopped<br>\n¼ scotch bonnet pepper, seeds removed<br>\n1 cup all purpose flour<br>\n1 tsp baking powder<br>\n½ cup ackee<br>\n¾ cup water<br>\nCanola oil

Too many problems with your regex, and besides you should be using the Beautiful Soup library here, not regex. — Tim Biegeleisen, Oct 01 '20 at 05:16
@TimBiegeleisen I used scrapy to get the `
` text, but need to extract the lines from it now without the `` element. — Biswajit Chopdar, Oct 01 '20 at 05:18
You shouldn't use regex to parse HTML: https://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags/1732454#1732454 — Christian Baumann, Oct 01 '20 at 05:23

score 1 · Answer 1 · answered Oct 01 '20 at 05:26

Since you have already isolated the one <p> tag you want, you could just use re.sub here to string off the <strong> tags, e.g.

# -*- coding: utf-8 -*-
import re

txt = "<p><strong>Ackee and Saltfish Fritters</strong><br>\n2 cup salted Cod fish, soaked overnight<br>\n2 cloves garlic<br>\n½ medium onion<br>\n1 tsp thyme<br>\n1 tbsp cilantro<br>\n1 scallion, finely chopped<br>\n¼ scotch bonnet pepper, seeds removed<br>\n1 cup all purpose flour<br>\n1 tsp baking powder<br>\n½ cup ackee<br>\n¾ cup water<br>\nCanola oil</p>"
txt = re.sub(r'<p>(.*)</p>', '\\1', re.sub(r'<strong>.*?</strong>', '', txt), flags=re.DOTALL)
print(txt)

This prints:

<br>
2 cup salted Cod fish, soaked overnight<br>
2 cloves garlic<br>
½ medium onion<br>
1 tsp thyme<br>
1 tbsp cilantro<br>
-1 scallion, finely chopped<br>
¼ scotch bonnet pepper, seeds removed<br>
1 cup all purpose flour<br>
1 tsp baking powder<br>
½ cup ackee<br>
¾ cup water<br>
Canola oil

score 1 · Accepted Answer · answered Oct 01 '20 at 06:16

Although Tim's answer solves the problem, I found out what was causing the regex to not work in the my script.

It was the \n character in the text that was causing the issue.

After removing the \n characters, regex worked fine. I think it's because python sees it as new line.

Posting it if anyone faces the same problem:

text = text.replace("\n", "")
lines = re.search("<br>(.*)<\/p>", text).groups(0)[0]
print(lines)

This prints:

2 cup salted Cod fish, soaked overnight<br>2 cloves garlic<br>½ medium onion<br>1 tsp thyme<br>1 tbsp cilantro<br>1 scallion, finely chopped<br>¼ scotch bonnet pepper, seeds removed<br>1 cup all purpose flour<br>1 tsp baking powder<br>½ cup ackee<br>¾ cup water<br>Canola oil

Python regex works in regex tester but not in practical

2 Answers2