Arangodb AQL recursive graph traversal

Question

I have a graph with three collections which items can be connected by edges. ItemA is a parent of itemB which in turn is a parent of itemC. Elements only can be connected by edges in direction

"_from : child, _to : parent"

Currently I can get only "linear" result with this AQL query:

LET contains = (FOR v IN 1..? INBOUND 'collectionA/itemA' GRAPH 'myGraph' RETURN v)

     RETURN {
        "root": {
            "id": "ItemA",
            "contains": contains
       }
   }

And result looks like this:

"root": {
    "id": "itemA",
    "contains": [
        {
            "id": "itemB"
        },
        {
            "id": "itemC"
        }
    ]
}

But I need to get a "hierarchical" result of graph traversal like that:

"root": {
    "id": "itemA",
    "contains": [
        {
            "id": "itemB",
            "contains": [
                {
                    "id": "itemC"
                }
            }
        ]
    }

So, can I get this "hierarchical" result running an aql query?

One more thing: traversal should run until leaf nodes will be encountered. So depth of the traversal is unknown in advance.

Answer 1

I have found solution. We decided to use UDF (user defined functions).

Here is a few steps to construct the proper hierarchical structure:

1. Register the function in arango db.
2. Run your aql query, that constructs a flat structure (vertex and corresponding path for this vertex). And pass result as input parameter of your UDF function. Here my function just append each element to its parent

In my case: 1) Register the function in arango db.

db.createFunction(
        'GO::LOCATED_IN::APPENT_CHILD_STRUCTURE',
            String(function (root, flatStructure) {
                if (root && root.id) {
                    var elsById = {};
                    elsById[root.id] = root;

                    flatStructure.forEach(function (element) {
                        elsById[element.id] = element;
                        var parentElId = element.path[element.path.length - 2];
                        var parentEl = elsById[parentElId];

                        if (!parentEl.contains)
                            parentEl.contains = new Array();

                        parentEl.contains.push(element);
                        delete element.path;
                    });
                }
                return root;
            })
        );

2) Run AQL with udf:

    LET flatStructure = (FOR v,e,p IN 1..? INBOUND 'collectionA/itemA' GRAPH 'myGraph' 
       LET childPath = (FOR pv IN p.vertices RETURN pv.id_source)
    RETURN MERGE(v, childPath))

    LET root = {"id": "ItemA"} 

    RETURN GO::LOCATED_IN::APPENT_CHILD_STRUCTURE(root, flatStructure)

Note: Please don't forget the naming convention when implement your functions.

Answer 2

I also needed to know the answer to this question so here is a solution that works.

I'm sure the code will need to be customised for you and could do with some improvements, please comment accordingly if appropriate for this sample answer.

The solution is to use a Foxx Microservice that supports recursion and builds the tree. The issue I have is around looping paths, but I implemented a maximum depth limit that stops this, hard coded to 10 in the example below.

To create a Foxx Microservice:

1. Create a new folder (e.g. recursive-tree)
2. Create the directory scripts
3. Place the files manifest.json and index.js into the root directory
4. Place the file setup.js in the scripts directory
5. Then create a new zip file with these three files in it (e.g. Foxx.zip)
6. Navigate to the ArangoDB Admin console
7. Click on Services | Add Service
8. Enter an appropriate Mount Point, e.g. /my/tree
9. Click on Zip tab
10. Drag in the Foxx.zip file you created, it should create without issues
11. If you get an error, ensure the collections myItems and myConnections don't exist, and the graph called myGraph does not exist, as it will try to create them with sample data.
12. Then navigate to the ArangoDB admin console, Services | /my/tree
13. Click on API
14. Expand /tree/{rootId}
15. Provide the rootId parameter of ItemA and click 'Try It Out'
16. You should see the result, from the provided root id.

If the rootId doesn't exist, it returns nothing If the rootId has no children, it returns an empty array for 'contains' If the rootId has looping 'contains' values, it returns nesting up to depth limit, I wish there was a cleaner way to stop this.

Here are the three files: setup.js (to be located in the scripts folder):

'use strict';
const db = require('@arangodb').db;
const graph_module =  require("org/arangodb/general-graph");

const itemCollectionName = 'myItems';
const edgeCollectionName = 'myConnections';
const graphName = 'myGraph';

if (!db._collection(itemCollectionName)) {
  const itemCollection = db._createDocumentCollection(itemCollectionName);
  itemCollection.save({_key: "ItemA" });
  itemCollection.save({_key: "ItemB" });
  itemCollection.save({_key: "ItemC" });
  itemCollection.save({_key: "ItemD" });
  itemCollection.save({_key: "ItemE" });

  if (!db._collection(edgeCollectionName)) {
    const edgeCollection = db._createEdgeCollection(edgeCollectionName);
    edgeCollection.save({_from: itemCollectionName + '/ItemA', _to: itemCollectionName + '/ItemB'});
    edgeCollection.save({_from: itemCollectionName + '/ItemB', _to: itemCollectionName + '/ItemC'});
    edgeCollection.save({_from: itemCollectionName + '/ItemB', _to: itemCollectionName + '/ItemD'});
    edgeCollection.save({_from: itemCollectionName + '/ItemD', _to: itemCollectionName + '/ItemE'});
  }

  const graphDefinition = [ 
    { 
      "collection": edgeCollectionName, 
      "from":[itemCollectionName], 
      "to":[itemCollectionName]
    }
  ];

  const graph = graph_module._create(graphName, graphDefinition);
}

mainfest.json (to be located in the root folder):

{
  "engines": {
    "arangodb": "^3.0.0"
  },
  "main": "index.js",
  "scripts": {
    "setup": "scripts/setup.js"
  }
}

index.js (to be located in the root folder):

'use strict';
const createRouter = require('@arangodb/foxx/router');
const router = createRouter();
const joi = require('joi');

const db = require('@arangodb').db;
const aql = require('@arangodb').aql;

const recursionQuery = function(itemId, tree, depth) {
  const result = db._query(aql`
    FOR d IN myItems
    FILTER d._id == ${itemId}
    LET contains = (
      FOR c IN 1..1 OUTBOUND ${itemId} GRAPH 'myGraph' RETURN { "_id": c._id }
    )
    RETURN MERGE({"_id": d._id}, {"contains": contains})
  `);

  tree = result._documents[0];

  if (depth < 10) {
    if ((result._documents[0]) && (result._documents[0].contains) && (result._documents[0].contains.length > 0)) {
        for (var i = 0; i < result._documents[0].contains.length; i++) {
        tree.contains[i] = recursionQuery(result._documents[0].contains[i]._id, tree.contains[i], depth + 1);
        }
    }
  }
  return tree;
}

router.get('/tree/:rootId', function(req, res) {
  let myResult = recursionQuery('myItems/' + req.pathParams.rootId, {}, 0);
  res.send(myResult);
})
  .response(joi.object().required(), 'Tree of child nodes.')
  .summary('Tree of child nodes')
  .description('Tree of child nodes underneath the provided node.');

module.context.use(router);

Now you can invoke the Foxx Microservice API end point, providing the rootId it will return the full tree. It's very quick.

The example output of this for ItemA is:

{
  "_id": "myItems/ItemA",
  "contains": [
    {
      "_id": "myItems/ItemB",
      "contains": [
        {
          "_id": "myItems/ItemC",
          "contains": []
        },
        {
          "_id": "myItems/ItemD",
          "contains": [
            {
              "_id": "myItems/ItemE",
              "contains": []
            }
          ]
        }
      ]
    }
  ]
}

You can see that Item B contains two children, ItemC and ItemD, and then ItemD also contains ItemE.

I can't wait until ArangoDB AQL improves the handling of variable depth paths in the FOR v, e, p IN 1..100 OUTBOUND 'abc/def' GRAPH 'someGraph' style queries. Custom visitors were not recommended for use in 3.x but haven't really be replaced with something as powerful for handling wild card queries on the depth of a vertex in a path, or handling prune or exclude style commands on path traversal.

Would love to have comments/feedback if this can be simplified.