Is there a way we can define the following code (a classic example for recursion) via generators in Python? I am using Python 3.
def fac(n):
if n==0:
return 1
else:
return n * fac(n-1)
I tried this, no success:
In [1]: def fib(n):
...: if n == 0:
...: yield 1
...: else:
...: n * yield (n-1)
File "<ipython-input-1-bb0068f2d061>", line 5
n * yield (n-1)
^
SyntaxError: invalid syntax
Classic recursion in Python leads to Stack Overflow
This classic example leads to a stack overflow on my machine for an input of n=3000
. In the Lisp dialect "Scheme" I'd use tail recursion and avoid stack overflow. Not possible in Python. That's why generators come in handy in Python. But I wonder:
Why no stack overflow with generators?
Why is there no stack overflow with generators in Python? How do they work internally? Doing some research leads me always to examples showing how generators are used in Python, but not much about the inner workings.
Update 1: yield from my_function(...)
As I tried to explain in the comments secion, maybe my example above was a poor choice for making a point. My actual question was targeted at the inner workings of generators used recursively in yield from
statements in Python 3.
Below is an (incomplete) example code that I use to proces JSON files generatred by Firebox bookmark backups. At several points I use yield from process_json(...)
to recursively call the function again via generators.
Exactly in this example, how is stack overflow avoided? Or is it?
# (snip)
FOLDERS_AND_BOOKMARKS = {}
FOLDERS_DATES = {}
def process_json(json_input, folder_path=""):
global FOLDERS_AND_BOOKMARKS
# Process the json with a generator
# (to avoid recursion use generators)
# https://stackoverflow.com/a/39016088/5115219
# Is node a dict?
if isinstance(json_input, dict):
# we have a dict
guid = json_input['guid']
title = json_input['title']
idx = json_input['index']
date_added = to_datetime_applescript(json_input['dateAdded'])
last_modified = to_datetime_applescript(json_input['lastModified'])
# do we have a container or a bookmark?
#
# is there a "uri" in the dict?
# if not, we have a container
if "uri" in json_input.keys():
uri = json_input['uri']
# return URL with folder or container (= prev_title)
# bookmark = [guid, title, idx, uri, date_added, last_modified]
bookmark = {'title': title,
'uri': uri,
'date_added': date_added,
'last_modified': last_modified}
FOLDERS_AND_BOOKMARKS[folder_path].append(bookmark)
yield bookmark
elif "children" in json_input.keys():
# So we have a container (aka folder).
#
# Create a new folder
if title != "": # we are not at the root
folder_path = f"{folder_path}/{title}"
if folder_path in FOLDERS_AND_BOOKMARKS:
pass
else:
FOLDERS_AND_BOOKMARKS[folder_path] = []
FOLDERS_DATES[folder_path] = {'date_added': date_added, 'last_modified': last_modified}
# run process_json on list of children
# json_input['children'] : list of dicts
yield from process_json(json_input['children'], folder_path)
# Or is node a list of dicts?
elif isinstance(json_input, list):
# Process children of container.
dict_list = json_input
for d in dict_list:
yield from process_json(d, folder_path)
Update 2: yield
vs yield from
Ok, I get it. Thanks to all the comments.
- So generators via
yield
create iterators. That has nothing to do with recursion, so no stack overflow here. - But generators via
yield from my_function(...)
are indeed recursive calls of my function, albeit delayed, and only evaluated if demanded.
This second example can indeed cause a stack overflow.