46

I would like to get a list of names of built-in modules in python such that I can test the popularity of function's naming conventions (underline, CamelCase or mixedCase).

I know there is a Global Module Index but I am wondering if there is a list of strings, which is easier to use :)

Update:

len(dir(__builtins__)) = 145  
len(stdlib_list("2.7")) = 430  
help('modules') = 508 # counting manually the output
Drake Guan
  • 12,774
  • 11
  • 52
  • 92
  • 7
    You might also want to check [PEP8](http://www.python.org/dev/peps/pep-0008/) and Naming Conventions – Vincent Savard Dec 03 '11 at 19:29
  • 1
    "underline" and "underscore" are the same thing. There are conventions for the use of `_name` and `__name__`. The number of function/method/names in the built-in library that have uppercase characters is vanishingly small. What's your objective? – John Machin Dec 03 '11 at 20:26
  • Thx, Vincent. I do like to get my hand on traverse the whole built-in modules' functions to see the current naming situation :) – Drake Guan Dec 04 '11 at 06:30

7 Answers7

49

The compiled-in module names are in sys.builtin_module_names. For all importable modules, see pkgutil.iter_modules.

Run these in a clean virtualenv to get (almost) only the modules that come with Python itself.

Note that a “popularity poll” will necessarily include modules that use old, discouraged naming conventions because they were written before today's guidelines were put in place, and can't change because need to be backwards compatible. It might be useful for something, but not for answering best-practice questions such as “How should I name my functions?”. For that, see the PEP8, the Python style guide, especially the “Naming Conventions” section.

Petr Viktorin
  • 58,535
  • 6
  • 72
  • 77
  • Added a note inspired by @VincentSavard's comment on the question. Thanks, Vincent. – Petr Viktorin Dec 03 '11 at 20:27
  • These don't seem to be complete. Neither sys.builtin_module_names nor pkgutil.iter_modules() contains "collections" or "os", for example (at least in Python 2.7). Can anyone say why that is? Is there a solution that will return everything listed in the Global Module Index from the original question? Edit: to answer my first question, os and collections are part of the standard library, not builtins. – Boon Jul 28 '20 at 07:11
  • 1
    That's interesting. On my Python 2.7 (and 3.8 as well), `pkgutil.iter_modules()` does include entries for `collections` and `os`. What kind of modules do you get? – Petr Viktorin Jul 29 '20 at 08:18
  • That's embarrassing. User error. I was simply testing `'collections' in list(pkgutil.iter_modules())` and it was returning `False`. – Boon Jul 31 '20 at 00:16
23

How about this? Though, this gets a list of built-in functions and variables rather than modules...

dir(__builtins__)

help('modules') will give you a list of all modules, according to How can I get a list of locally installed Python modules?. Not a list of strings, though.

ChillarAnand
  • 22,858
  • 8
  • 98
  • 114
U-DON
  • 1,996
  • 11
  • 13
12

Now there is a 3rd party package for this. It scrapes the TOC of the Standard Library page in the official Python docs and builds a list.

You can install it using pip

pip install stdlib_list

and got get a list of libraries

In [12]: from stdlib_list import stdlib_list

In [13]: libraries = stdlib_list("3.5")

In [14]: libraries[4:12]
Out[14]: ['abc', 'aifc', 'argparse', 'array', 'ast', 'asynchat', 'asyncio', 'asyncore']

You can find source code here.

ChillarAnand
  • 22,858
  • 8
  • 98
  • 114
  • 2
    This is quite amazing. Thanks for this information. I also put the result (the number of packages by different ways) into my original post. – Drake Guan May 10 '16 at 09:04
7

>>>dir (__builtins__)

or

>>>help (__builtins__)

shengnan.wang
  • 79
  • 1
  • 1
2

From the CPython`s docs:

All known built-in modules are listed in sys.builtin_module_names

Names of modules in sys.builtin_module_names is actual only for used a Python interpreter:

A tuple of strings giving the names of all modules that are compiled into this Python interpreter

Each built-in module use the special loader while importing: BuiltinImporter

In [65]: import itertools, sys, gc

In [66]: itertools.__loader__, sys.__loader__, gc.__loader__
Out[66]: 
(_frozen_importlib.BuiltinImporter,
 _frozen_importlib.BuiltinImporter,
 _frozen_importlib.BuiltinImporter)

In the Python 3 the number of built-in modules has slightly increased 

$ python2.7 -c "import sys; print('Count built-in modules: %d' %len(sys.builtin_module_names)); print(sys.builtin_module_names)"
Count built-in modules: 51
('__builtin__', '__main__', '_ast', '_bisect', '_codecs', '_collections', '_functools', '_heapq', '_io', '_locale', '_md5', '_random', '_sha', '_sha256', '_sha512', '_socket', '_sre', '_struct', '_symtable', '_warnings', '_weakref', 'array', 'binascii', 'cPickle', 'cStringIO', 'cmath', 'datetime', 'errno', 'exceptions', 'fcntl', 'gc', 'grp', 'imp', 'itertools', 'marshal', 'math', 'operator', 'posix', 'pwd', 'select', 'signal', 'spwd', 'strop', 'sys', 'syslog', 'thread', 'time', 'unicodedata', 'xxsubtype', 'zipimport', 'zlib')
$ python3.4 -c "import sys; print('Count built-in modules: %d' %len(sys.builtin_module_names)); print(sys.builtin_module_names)"
Count built-in modules: 54
('_ast', '_bisect', '_codecs', '_collections', '_datetime', '_elementtree', '_functools', '_heapq', '_imp', '_io', '_locale', '_md5', '_operator', '_pickle', '_posixsubprocess', '_random', '_sha1', '_sha256', '_sha512', '_socket', '_sre', '_stat', '_string', '_struct', '_symtable', '_thread', '_tracemalloc', '_warnings', '_weakref', 'array', 'atexit', 'binascii', 'builtins', 'errno', 'faulthandler', 'fcntl', 'gc', 'grp', 'itertools', 'marshal', 'math', 'posix', 'pwd', 'pyexpat', 'select', 'signal', 'spwd', 'sys', 'syslog', 'time', 'unicodedata', 'xxsubtype', 'zipimport', 'zlib')

As the CPython is implemented (primary) on the C programming language, so it is not easy to find it, as example location the Python`s module sys (based on this answer):

$ locate sysmodule | grep python
/usr/include/python2.7/sysmodule.h
/usr/include/python3.4m/sysmodule.h
/usr/local/include/python3.5m/sysmodule.h

More information about getting an information about all available modules is the CPython, look in my answer here.

Community
  • 1
  • 1
PADYMKO
  • 3,291
  • 2
  • 31
  • 36
1

It can be done using the given block of code below and it is the most effective way as per me.

import sys
a = sys.builtin_module_names
print(a)

The last line to be included if you want to print them. Here, a is a tuple and so it can access all the functionalities of a tuple.

You can have a look at sys.builtin_module_names for further help https://docs.python.org/3/library/sys.html

Akash Kumar
  • 79
  • 6
0

I was working on a similar problem when I learned that 'builtin' means something like "there is no source file associated with this object".

Here's a solution based on checking the /lib and /Dlls folders manually. The use of "unique" may be redundant; it's there because I'm not sure if DLLs is strictly for packages which come with python/it was useful for a different problem I was trying to solve (finding out the requirements for a source package).

from typing import Generator, Iterable
import os, re, sys

def unique(iterable:Iterable, yielded:list=None) -> Generator:
    """
    Iterate over unique elements of an iterable
    examples:
        >>> [*unique('abbcccdddd')]
        ['a', 'b', 'c', 'd']
        >>> [*unique('abcd')]
        ['a', 'b', 'c', 'd']
    """
    yielded = yielded if not isinstance(yielded, type(None)) else []
    for i in iterable:
        if not i in yielded:
            yield i
            yielded.append(i)

def native_modules() -> Generator:
    """
    Every module found:
        under your installation's /lib and /DLLs directories; excuding any under /lib/site-packages/*
        in sys.builtin_module_names
    """
    omitables = 'site-packages __pycache__ .pyo .ico .dll .pdb'.split()
    
    path = lambda folder: os.path.join(sys.exec_prefix, folder)
    tidy = lambda pth: os.path.split(os.path.splitext(pth)[0])[-1] # separate the name from the path and extension, if present
    discriminant = lambda pth: not any(re.match(i, pth) for i in map(re.escape, omitables))
    
    lib = map(tidy, filter(discriminant, os.listdir(path('lib'))))
    dlls = map(tidy, filter(discriminant, os.listdir(path('DLLs'))))

    yielded = []
    yield from yielded
    yield from unique(sys.builtin_module_names, yielded)
    yield from unique(lib, yielded)
    yield from unique(dlls, yielded)
kendfss
  • 173
  • 1
  • 7