How to draw complex networkx DiGraph in a readable way?

Question

I'd like to use networkx to study the architecture of a fairly large project but the test i've done so far are not so good, here's a minimal example of all my research:

import matplotlib.pyplot as plt
import networkx as nx
from networkx.readwrite import node_link_graph

G = node_link_graph({'directed': True, 'multigraph': False, 'graph': {}, 'nodes': [{'id': 'build'}, {'id': 'root'}, {'id': 'utils'}, {'id': 'codegen'}, {'id': 'codegen.templates'}, {'id': 'nodes.shapes'}, {'id': 'codegen.c_types'}, {'id': 'nodes'}, {'id': 'containers'}, {'id': 'distutils'}, {'id': 'wheel'}, {'id': 'tools.testing'}, {'id': 'finalizations'}, {'id': 'importing'}, {'id': 'plugins'}, {'id': 'freezer'}, {'id': 'tree'}, {'id': 'specs'}, {'id': 'optimizations'}, {'id': 'plugins.standard'}, {'id': 'tools.general.dll_report'}, {'id': 'tools.specialize'}, {'id': 'tools.testing.compare_with_cpython'}, {'id': 'tools.testing.find_sxs_modules'}, {'id': 'tools.testing.measure_construct_performance'}, {'id': 'tools.testing.run_root_tests'}, {'id': 'tools'}], 'links': [{'source': 'build', 'target': 'root'}, {'source': 'build', 'target': 'utils'}, {'source': 'root', 'target': 'root'}, {'source': 'root', 'target': 'containers'}, {'source': 'root', 'target': 'utils'}, {'source': 'root', 'target': 'finalizations'}, {'source': 'root', 'target': 'freezer'}, {'source': 'root', 'target': 'plugins'}, {'source': 'root', 'target': 'nodes.shapes'}, {'source': 'utils', 'target': 'root'}, {'source': 'utils', 'target': 'utils'}, {'source': 'codegen', 'target': 'codegen'}, {'source': 'codegen', 'target': 'codegen.templates'}, {'source': 'codegen', 'target': 'root'}, {'source': 'codegen', 'target': 'codegen.c_types'}, {'source': 'codegen', 'target': 'utils'}, {'source': 'codegen', 'target': 'nodes.shapes'}, {'source': 'codegen', 'target': 'nodes'}, {'source': 'codegen', 'target': 'containers'}, {'source': 'codegen.templates', 'target': 'root'}, {'source': 'nodes.shapes', 'target': 'codegen.c_types'}, {'source': 'nodes.shapes', 'target': 'codegen'}, {'source': 'nodes.shapes', 'target': 'root'}, {'source': 'nodes.shapes', 'target': 'nodes.shapes'}, {'source': 'codegen.c_types', 'target': 'codegen.templates'}, {'source': 'codegen.c_types', 'target': 'codegen.c_types'}, {'source': 'codegen.c_types', 'target': 'codegen'}, {'source': 'nodes', 'target': 'containers'}, {'source': 'nodes', 'target': 'utils'}, {'source': 'nodes', 'target': 'nodes.shapes'}, {'source': 'nodes', 'target': 'importing'}, {'source': 'nodes', 'target': 'root'}, {'source': 'nodes', 'target': 'optimizations'}, {'source': 'nodes', 'target': 'tree'}, {'source': 'nodes', 'target': 'nodes'}, {'source': 'nodes', 'target': 'specs'}, {'source': 'containers', 'target': 'root'}, {'source': 'distutils', 'target': 'wheel'}, {'source': 'distutils', 'target': 'tools.testing'}, {'source': 'tools.testing', 'target': 'root'}, {'source': 'tools.testing', 'target': 'utils'}, {'source': 'tools.testing', 'target': 'tools.testing'}, {'source': 'finalizations', 'target': 'finalizations'}, {'source': 'finalizations', 'target': 'root'}, {'source': 'finalizations', 'target': 'importing'}, {'source': 'finalizations', 'target': 'plugins'}, {
                    'source': 'importing', 'target': 'containers'}, {'source': 'importing', 'target': 'plugins'}, {'source': 'importing', 'target': 'root'}, {'source': 'importing', 'target': 'utils'}, {'source': 'importing', 'target': 'importing'}, {'source': 'importing', 'target': 'tree'}, {'source': 'plugins', 'target': 'root'}, {'source': 'plugins', 'target': 'containers'}, {'source': 'plugins', 'target': 'utils'}, {'source': 'plugins', 'target': 'plugins'}, {'source': 'freezer', 'target': 'codegen'}, {'source': 'freezer', 'target': 'codegen.templates'}, {'source': 'freezer', 'target': 'root'}, {'source': 'freezer', 'target': 'utils'}, {'source': 'freezer', 'target': 'containers'}, {'source': 'freezer', 'target': 'importing'}, {'source': 'freezer', 'target': 'nodes'}, {'source': 'freezer', 'target': 'plugins'}, {'source': 'freezer', 'target': 'tree'}, {'source': 'freezer', 'target': 'freezer'}, {'source': 'tree', 'target': 'root'}, {'source': 'tree', 'target': 'plugins'}, {'source': 'tree', 'target': 'utils'}, {'source': 'tree', 'target': 'tree'}, {'source': 'tree', 'target': 'nodes'}, {'source': 'tree', 'target': 'optimizations'}, {'source': 'tree', 'target': 'freezer'}, {'source': 'tree', 'target': 'importing'}, {'source': 'tree', 'target': 'specs'}, {'source': 'specs', 'target': 'root'}, {'source': 'specs', 'target': 'specs'}, {'source': 'specs', 'target': 'utils'}, {'source': 'optimizations', 'target': 'root'}, {'source': 'optimizations', 'target': 'importing'}, {'source': 'optimizations', 'target': 'nodes'}, {'source': 'optimizations', 'target': 'nodes.shapes'}, {'source': 'optimizations', 'target': 'tree'}, {'source': 'optimizations', 'target': 'utils'}, {'source': 'optimizations', 'target': 'optimizations'}, {'source': 'optimizations', 'target': 'plugins'}, {'source': 'plugins.standard', 'target': 'root'}, {'source': 'plugins.standard', 'target': 'plugins'}, {'source': 'plugins.standard', 'target': 'containers'}, {'source': 'plugins.standard', 'target': 'utils'}, {'source': 'tools.general.dll_report', 'target': 'freezer'}, {'source': 'tools.general.dll_report', 'target': 'utils'}, {'source': 'tools.specialize', 'target': 'codegen'}, {'source': 'tools.specialize', 'target': 'root'}, {'source': 'tools.testing.compare_with_cpython', 'target': 'root'}, {'source': 'tools.testing.compare_with_cpython', 'target': 'tools.testing'}, {'source': 'tools.testing.compare_with_cpython', 'target': 'utils'}, {'source': 'tools.testing.find_sxs_modules', 'target': 'tools.testing'}, {'source': 'tools.testing.find_sxs_modules', 'target': 'root'}, {'source': 'tools.testing.find_sxs_modules', 'target': 'utils'}, {'source': 'tools.testing.measure_construct_performance', 'target': 'tools.testing'}, {'source': 'tools.testing.run_root_tests', 'target': 'tools'}, {'source': 'tools.testing.run_root_tests', 'target': 'tools.testing'}, {'source': 'tools.testing.run_root_tests', 'target': 'utils'}]})
nx.draw(G, with_labels=True)
plt.show()

As you can see, drawing the graph in this naive way will produce a totally useless&unreadable output such as:

Thing is, after all my research reading the networkx tutorial/docs, checking some google references I've been unable to figure out a proper way to acomplish the task. I've got graphviz installed but I've failed miserably trying to build&run pygraphivz/pydot on windows, anyway...

Question: How can I draw a complex graph using networkx in some sort of hierachical & clean way where the nodes are dispersed uniformly between them? Below you can see the type of output I'd like to achieve here:

As you can see, nodes are dispersed, cycles are shown properly and the different levels of the hierarchy are completely clear top/down... It'd be great if something like this (or similar) could be achieved with networkx.

In fact, what's described in this paper is exactly the type of output I'd like to achieve here

Ns. Image example borrowed from this site

Have you tried drawing with a layout? *edit: layouts will makes your graphs cleaner but won't give you the flow chart organization seen in the paper you specified. To do that, you need to hard-code the node positions using the "pos" argument. Layouts in case they're helpful: https://networkx.github.io/documentation/stable/reference/drawing.html#module-networkx.drawing.layout — M-Wi, Apr 12 '20 at 01:27
@M-Wi Yeah, i've tried few things and none of them did work, that's why I've asked this in case someone with real experience using networkx would know how to do it :) — BPL, Apr 12 '20 at 06:03

warped · Accepted Answer · 2020-04-13T13:47:58.153

The various drawfunctions of networkx take a pos argument, which is a dictionary that has the node name as key, and x,y coords as values.

You can generate this yourself. If you know the hierarchy that you want to impose, you can translate hierarchy into y positions, and then just add fill in x positions as you go:

# exctracting nodes from dictionary into list:
nodes = [{'id': 'build'}, {'id': 'root'}, {'id': 'utils'}, {'id': 'codegen'}, {'id': 'codegen.templates'}, {'id': 'nodes.shapes'}, {'id': 'codegen.c_types'}, {'id': 'nodes'}, {'id': 'containers'}, {'id': 'distutils'}, {'id': 'wheel'}, {'id': 'tools.testing'}, {'id': 'finalizations'}, {'id': 'importing'}, {'id': 'plugins'}, {'id': 'freezer'}, {'id': 'tree'}, {'id': 'specs'}, {'id': 'optimizations'}, {'id': 'plugins.standard'}, {'id': 'tools.general.dll_report'}, {'id': 'tools.specialize'}, {'id': 'tools.testing.compare_with_cpython'}, {'id': 'tools.testing.find_sxs_modules'}, {'id': 'tools.testing.measure_construct_performance'}, {'id': 'tools.testing.run_root_tests'}, {'id': 'tools'}]

nodelist = []
for n in nodes:
    for k, v in n.items():
        nodelist.append(v)

# hierarchy here is arbitrarily defined based on the index of hte node in nodelist. 
# {hierarchy_level : number_of_nodes_at_that_level}
hierarchy = {
    0:4,
    1:10,
    2:5,
    3:5,
    4:3
}

coords = []
for y, v in hierarchy.items():
    coords += [[x, y] for x in list(range(v))]

# map node names to positions 
# this is based on index of node in nodelist.
# can and should be tailored to your actual hierarchy    
positions = {}
for n, c in zip(nodelist, coords):
    positions[n] = c

fig = plt.figure(figsize=(15,5))
nx.draw_networkx_nodes(G, pos=positions, node_size=50)
nx.draw_networkx_edges(G, pos=positions, alpha=0.2)

# generate y-offset for the labels, s.t. they don't lie on the nodes
label_positions = {k:[v0, v1-.25] for k, (v0,v1) in positions.items()}
nx.draw_networkx_labels(G, pos=label_positions, font_size=8)
plt.show()

Node labels overlap somewhat, but this can be adjusted with font size, additional offsetting of via the figure dimensions

EDIT:

Rotate node labels to avoid text overlap:

text = nx.draw_networkx_labels(G, pos=label_positions, font_size=8)
for _, t in text.items():
    t.set_rotation(20)

This is a nice answer and almost almost exactly what I want to achieve here. As you've already pointed out it'd be great to figure out an automatic method so you can avoid node overlapping. The graph I've provided above is an extremely small subset of what I want to analize (think about any kinda large python project).... Also, what would be the options if the graph was really large? I'm using matplotlib but maybe that's no the best choice... So, would it be possible to render it to pdf (or svg) maybe? Anyway, the answer is very good and it deserves an upvote already. Thanks sir. — BPL, Apr 13 '20 at 08:00
matplotlib has functionality to save figures as pdf or as svg. As for automatically generating the hierarchy: this is a bit difficult to do as I do not know what you are after. you could eg classify a node based on the number of incoming/outgoing nodes, or based on the distance from terminal nodes, so this is mainly a choice you make based on what you want to demonstrate. As for overlapping text, you can rotate the node labels by something like 20 degrees to avoid overlap. — warped, Apr 13 '20 at 13:44
Well, the main goal is creating a little tool that allows to analize analize large architectures (cycles, coupling, levelization, etc...) so basically I'm still tring to figure out what networkx is offering me to achieve this task in the nicer possible way. I think from your current answer I can start testing a bit more how the final output will be when the graph is much larger. I liked the idea of rotation now at least you can nicely what's going on there... as for hierarchical structuring like in the posted paper well... that'd be awesome but I think at this point your answer is valid — BPL, Apr 13 '20 at 14:02
Just for the record, I think I can work from here to continue exploring by myself... but if you ever figure out how to create nice hierarchical structures please don't hesitate to edit your answer as I wouldn't mind to bounty in the future... this subject is quite interesting to do so... Anyway, tyvm :) — BPL, Apr 13 '20 at 14:03

How to draw complex networkx DiGraph in a readable way?

1 Answers1

EDIT: