Is there a streaming API for JSON?

Question

Is DOM the only way to parse JSON?

Although those giving answers may have recommended parsers, the OP *did **not*** ask anyone to "recommend or find a tool, library, or favorite off-site resource". — iconoclast, Aug 20 '14 at 15:16
...and to finally answer the OQ which got so prematurely and unjustifiably closed yet received so many fine and helpful answers, no, "there is" no "streaming API for JSON" in the sense that "there is" (a standardized and universally adopted, single) API for, say, representing web pages in a browser (i.e. the DOM etc). There are a lot of different solutions you can choose from (a fact which justifies the answers that tried to be helpful but didn't answer the OQ in a literal way). — flow, Jan 04 '15 at 18:06

score 59 · Answer 1 · edited May 23 '17 at 12:03

59

Some JSON parsers do offer incremental ("streaming") parser; for Java, at least following parsers from json.org page offer such an interface:

Jackson (pull interface)
Json-simple (SAX-style push interface)

(in addition to Software Monkey's parser referred to by another answer)

Actually, it is kind of odd that so many JSON parsers do NOT offer this simple low-level interface -- after all, they already need to implement low-level parsing, so why not expose it.

EDIT (June 2011): Gson too has its own streaming API (with gson 1.6)

edited May 23 '17 at 12:03

Community

1
1

answered May 05 '09 at 06:41

StaxMan

102,903
28
190
229

Another: http://www.salsify.com/blog/json-streaming-parser-for-php – Rob Gonzalez Apr 12 '13 at 17:34
With Python you can get a [SAX-like push parser that I wrote here](https://github.com/kashifrazzaqui/json-streamer) – kashif Dec 19 '14 at 11:41

Lawrence Dol · Answer 2 · 2017-04-11T21:55:18.817

By DOM, I assume you mean that the parser reads an entire document at once before you can work with it. Note that saying DOM tends to imply XML, these days, but IMO that is not really an accurate inference.

So, in answer to your questions - "Yes", there are streaming API's and "No", DOM is not the only way. That said, processing a JSON document as a stream is often problematic in that many objects are not simple field/value pairs, but contain other objects as values, which you need to parse to process, and this tends to end up a recursive thing. But for simple messages you can do useful things with a stream/event based parser.

I have written a pull-event parser for JSON (it was one class, about 700 lines). But most of the others I have seen are document oriented. One of the layers I have built on top of my parser is a document reader, which took about 30 LOC. I have only ever used my parser in practice as a document loader (for the above reason).

I am sure if you search the net you will find pull and push based parsers for JSON.

EDIT: I have posted the parser to my site for download. A working compilable class and a complete example are included.

EDIT2: You'll also want to look at the JSON website.

score 14 · Answer 3 · answered Oct 07 '10 at 19:18

As stefanB mentioned, http://lloyd.github.com/yajl/ is a C library for stream parsing JSON. There are also many wrappers mentioned on that page for other languages:

yajl-ruby - ruby bindings for YAJL

yajl-objc - Objective-C bindings for YAJL

YAJL IO bindings (for the IO language)

Python bindings come in two flavors, py-yajl OR yajl-py

yajl-js - node.js bindings (mirrored to github).

lua-yajl - lua bindings

ooc-yajl - ooc bindings

yajl-tcl - tcl bindings

some of them may not allow streaming, but many of them certainly do.

score 8 · Answer 4 · answered Feb 05 '14 at 13:37

Disclaimer: I'm suggesting my own project.

I maintain a streaming JSON parser in Javascript which combines some of the features of SAX and DOM:

Oboe.js website

The idea is to allow streaming parsing, but not require the programmer to listen to lots of different events like with raw SAX. I like SAX but it tends to be quite low level for what most people need. You can listen for any interesting node from the JSON stream by registering JSONPath patterns.

The code is on Github here:

Oboe.js Github page

Oboe.js website link appears to require authentication. – jacobq Apr 08 '16 at 00:00 — jacobq, Apr 08 '16 at 00:00

score 7 · Answer 5 · answered Jan 14 '12 at 03:46

If you want to use pure javascript and a library that runs both in node.js and in the browser you can try clarinet:

https://github.com/dscape/clarinet

The parser is event-based, and since it’s streaming it makes dealing with huge files possible. The API is very close to sax and the code is forked from sax-js.

score 3 · Answer 6 · answered Aug 30 '09 at 18:10

LitJSON supports a streaming-style API. Quoting from the manual:

"An alternative interface to handling JSON data that might be familiar to some developers is through classes that make it possible to read and write data in a stream-like fashion. These classes are JsonReader and JsonWriter.

"These two types are in fact the foundation of this library, and the JsonMapper type is built on top of them, so in a way, the developer can think of the reader and writer classes as the low-level programming interface for LitJSON."

score 3 · Answer 7 · answered Oct 16 '13 at 18:50

3

Here's a NodeJS NPM library for parsing and handling streams of JSON: https://npmjs.org/package/JSONStream

answered Oct 16 '13 at 18:50

Tom Chapin

2,500
20
16

score 3 · Answer 8 · answered Nov 08 '13 at 12:29

If you are looking specifically for Python, then ijson claims to support it. However, it is only a parser, so I didn't come across anything for Python that can generate json as a stream.

For C++ there is rapidjson that claims to support both parsing and generation in a streaming manner.

score 2 · Answer 9 · answered Dec 14 '13 at 15:46

For Python, an alternative (apparently lighter and more efficient) to ijson is jsaone (see that link for rough benchmarks, showing that jsaone is approximately 3x faster).

DISCLAIMER: I'm the author of jsaone, and the tests I made are very basic... I'll be happy to be proven wrong!

score 1 · Answer 10 · answered Feb 16 '10 at 22:57

Answering the question title: YAJL a JSON parser library in C:

YAJL remembers all state required to support restarting parsing. This allows parsing to occur incrementally as data is read off a disk or network.

So I guess using yajl to parse JSON can be considered as processing stream of data.

score -7 · Answer 11 · answered Jan 14 '09 at 19:47

-7

In reply to your 2nd question, no, many languages have JSON parsers. PHP, Java, C, Ruby and many others. Just Google for the language of your choice plus "JSON parser".

answered Jan 14 '09 at 19:47

Ryan Doherty

37,071
3
51
62

2

This does not answer to question: question author asked not about general JSON parser at all, but about streaming parser. – Alexey Tigarev Mar 31 '12 at 09:25
1

What is a streaming parser? – Koray Tugay Feb 05 '14 at 13:45
a non-streaming parser loads everything into memory at once. Bad. A streaming parser pulls its input one byte at a time into a parser, and it calls a callback each time that input achieves some goal. That way the file can be arbitrarily long, and read in On time, without loading it all into memory. That causes thrashing. – Phlip May 19 '14 at 16:37

Is there a streaming API for JSON?

11 Answers11

Linked

Related