Questions tagged [ply]

PLY is an implementation of lex and yacc parsing tools for Python.

About PLY

PLY is a parser generator tool that uses reflection to read token definitions and production rules written in pure Python. You can, for example, define tokens with a simple string attribution or with methods containing a regular expression in its docstring.

To install PLY on your machine for python2/3, follow the steps outlined below:

  1. Download the source code from here.
  2. Unzip the downloaded zip file
  3. Navigate into the unzipped ply-3.10 folder
  4. Run the following command in your terminal: python setup.py install

If you completed all the above, you should now be able to use the PLY module. You can test it out by opening a python interpreter and typing import ply.lex.

Note: Do not use pip to install PLY, it will install a broken distribution on your machine.


Examples of token definitions with code to interpret value:

def t_BOOLEAN(token):
    r'(?:true|false)'
    token.value = token.value == "true"
    return token
def t_NUMBER(token):
    r'[0-9]+'
    token.value = int(token.value)
    return token


Related tags:

317 questions
6
votes
2 answers

How to ignore comments inside string literals

I'm doing a lexer as a part of a university course. One of the brain teasers (extra assignments that don't contribute to the scoring) our professor gave us is how could we implement comments inside string literals. Our string literals start and end…
Konsta
  • 79
  • 1
  • 7
6
votes
1 answer

What is the role of the Empty production for PEGs?

The empty production rule nonterminal -> epsilon is useful in lex-yacc LR bottom up parser generators (e.g. PLY). In what context should one use Empty productions in PEG parsers e.g. pyparsing ?
Frankie Ribery
  • 10,943
  • 11
  • 46
  • 62
6
votes
2 answers

How to get PLY to ignore case of a regular expression?

I'm working on a simple translator from SQL INSERT statements to a dataset XML file to be used with DbUnit. My current definition looks like this: def t_INSERT(token): r'INSERT\s+INTO' return token Now, I want to support case insensitive…
Elias Dorneles
  • 19,202
  • 9
  • 63
  • 95
5
votes
1 answer

Why does PLY treat regular expressions differently from Python/re?

Some background: I am writing a parser to retrieve information from sites with a markup language. Standard libraries as wikitools, ... do not work for me as I need to be more specific and adapting them to my needs puts a layer of complexity between…
programkai
  • 53
  • 3
5
votes
6 answers

md5 module error

I'm using an older version of PLY that uses the md5 module (among others): import re, types, sys, cStringIO, md5, os.path ... although the script runs but not without this error: DeprecationWarning: the md5 module is deprecated; use hashlib…
eozzy
  • 58,300
  • 96
  • 249
  • 396
5
votes
2 answers

Does Pyparsing Support Context-Sensitive Grammars?

Forgive me if I have the incorrect terminology; perhaps just getting the "right" words to describe what I want is enough for me to find the answer on my own. I am working on a parser for ODL (Object Description Language), an arcane language that as…
HardlyKnowEm
  • 3,040
  • 17
  • 30
4
votes
2 answers

Python PLY multiple rules

In ply how can we distinguish between which rule was used - example :- ''' p : a b | c | d ''' so suppose we need to write different code for different rules. Then is there any elegant way…
4
votes
2 answers

How to discard non-terminal in grammar file with PLY (Python Lex-Yacc)

I have faced a problem when using PLY. I want to create a call graph generator by PLY. In some situation, I need to discard some tokens in the grammar file. That is because I need to do something when the parser recognize that token before I discard…
wani
  • 167
  • 2
  • 6
4
votes
1 answer

How to understand and fix conflicts in PLY

I am working on a SystemVerilog parser and I am running into many ply conflicts (both shift/reduce and reduce/reduce). I currently have like 170+ conflicts and the problem I have is that I don't really understand the parser.out file generated by…
viterbi
  • 349
  • 3
  • 10
4
votes
1 answer

How do I change the parsing order of the rules?

I'm developing a programming language, and I'm having trouble dealing with condition statements. Here's the code in my language: x = 4 -> ? 2 > 5 ?> -> [o] <- x -> Here's the specific part of the grammar that defines condition…
Ericson Willians
  • 6,608
  • 10
  • 47
  • 97
4
votes
5 answers

Match unicode in ply's regexes

I'm matching identifiers, but now I have a problem: my identifiers are allowed to contain unicode characters. Therefore the old way to do things is not enough: t_IDENTIFIER = r"[A-Za-z](\\.|[A-Za-z_0-9])*" In my markup language parser I match…
Cheery
  • 21,751
  • 13
  • 55
  • 80
4
votes
1 answer

Parsing Java Source code using plyj in Python

I am trying to parse Java source to get the method names, their invocations, variable names, etc. I was looking for a pre-built or extensible module in Python and stumbled upon plyj (https://github.com/musiKk/plyj). I want to find out a method, then…
krish7919
  • 804
  • 2
  • 13
  • 26
4
votes
5 answers

Should I use Lex or a home-brewed solution to parse a formula?

I'm in the process of writing a small, rule-based 'math' engine. I realize this is unclear, so I'll provide a small example. Let's say you have some variable a, that holds an integer. You also have some functions you can apply to the number,…
FreeMemory
  • 8,142
  • 7
  • 32
  • 48
4
votes
2 answers

Parsing python with PLY, how to code the indent and dedent part

I was trying to parse the function definition for the python language with PLY. I am encountering issues related to the indentation. For instance for a for statement, I would like to be able to know when the block ends. I read the python grammar…
joetde
  • 1,336
  • 1
  • 14
  • 25
4
votes
1 answer

PLY yacc parsing IF-ELSE IF-ELSE nested statements

can somebody help me with writing correct grammar rules for nested if statements? In my language, I am able to write constructions like this: (IF CONDITION) some statements (IF CONDITION) some statements (ELSE IF CONDITION) …
JoshuaBoshi
  • 1,196
  • 1
  • 13
  • 24
1
2
3
21 22