Streaming data with Node.js

Question

I want to know if it is possible to stream data from the server to the client with Node.js. I want to post a single AJAX request to Node.js, then leave the connection open and continuously stream data to the client. The client will receive this stream and update the page continuously.

Update:

As an update to this answer - I cannot get this to work. The response.write is not sent before you call close. I have set up an example program that I use to achieve this:

Node.js:

var sys = require('sys'), 
http = require('http');
http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/html'});
    var currentTime = new Date();
    setInterval(function(){
        res.write(
            currentTime.getHours()
            + ':' + 
            currentTime.getMinutes()
            + ':' +
            currentTime.getSeconds()
        );
    },1000);
}).listen(8000);

HTML:

<html>
    <head>
        <title>Testnode</title>
    </head>

    <body>
        <!-- This fields needs to be updated -->
        Server time: <span id="time">&nbsp;</span>

        <!-- import jQuery from google -->
        <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js"></script>

        <!-- import jQuery -->
        <script type="text/javascript">
            $(document).ready(function(){
            // I call here node.localhost nginx ports this to port 8000
                $('#time').load('http://node.localhost');
            });
        </script>
    </body>
</html>

Using this method I don't get anything back until I call close(). Is this possible or should I go with a long poll approach instead where I call the load function again as one comes in?

Also I don't think this is streaming, this is just chunking stuff in bits. Yes, node will send parts of an answer, but every html server does this. The streaming goodness you get from node refers to something else. — Zlatko, Jun 24 '14 at 01:44
Also, in your example you don't have the pause, resume, flush methods. The (read) stream is something that is giving you data, as fast as it can (or as fast as you can). And you tell it to pause, then it pauses. And you tell it to continue etc. — Zlatko, Jun 24 '14 at 01:45

score 27 · Accepted Answer · edited Oct 08 '13 at 22:11

27

It is possible. Just use response.write() multiple times.

var body = ["hello world", "early morning", "richard stallman", "chunky bacon"];
// send headers
response.writeHead(200, {
  "Content-Type": "text/plain"
});

// send data in chunks
for (piece in body) {
    response.write(body[piece], "ascii");
}

// close connection
response.end();

You may have to close and reopen connection every 30 seconds or so.

EDIT: this is the code I actually tested:

var sys = require('sys'),
http = require('http');
http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/html'});
    var currentTime = new Date();
    sys.puts('Starting sending time');
    setInterval(function(){
        res.write(
            currentTime.getHours()
            + ':' +
            currentTime.getMinutes()
            + ':' +
            currentTime.getSeconds() + "\n"
        );

        setTimeout(function() {
            res.end();
        }, 10000);

    },1000);
}).listen(8090, '192.168.175.128');

I connected to it by Telnet and its indeed gives out chunked response. But to use it in AJAX browser has to support XHR.readyState = 3 (partial response). Not all browsers support this, as far as I know. So you better use long polling (or Websockets for Chrome/Firefox).

EDIT2: Also, if you use nginx as reverse proxy to Node, it sometimes wants to gather all chunks and send it to user at once. You need to tweak it.

edited Oct 08 '13 at 22:11

Vinay

5,510
5
34
52

answered Apr 01 '10 at 20:30

Kuroki Kaze

7,142
3
34
48

I am testing this approach, but I am having trouble accessing the node.js script trough Ajax on my page. I think this has something to do with some cross domain thing. I am going to work on this later on and see if this works as expected. – Saif Bechan Apr 02 '10 at 03:16
3

`jquery.load` waits for all body to arrive before firing callback. You need to use something different. See http://api.jquery.com/load/ – Kuroki Kaze Apr 02 '10 at 10:43
Do you have any idea on what kind of difference? – Saif Bechan Apr 02 '10 at 10:51
Some low-level stuff, I think. `XMLHTTPRequest` maybe. Anything that can deal with chunked responses. – Kuroki Kaze Apr 02 '10 at 11:46
Are you sure this should work. Or are you just thinking theoretically it should work. Because I am trying to get it chunked but its just not displaying. The Transfer-Encoding is chunked but Node.js only sends it when you call close. Else i should just use a long poll solution. – Saif Bechan Apr 02 '10 at 13:47
1

I'll check it on my node machine (0.1.31) – Kuroki Kaze Apr 02 '10 at 13:55
1

Yes, maybe longpoll will be better. – Kuroki Kaze Apr 02 '10 at 13:55
is it possible to stream binary data? – KJW Jun 08 '11 at 08:08
3

To make this sample work: move "var currentTime = new Date();" into setinterval function. – jmav Jul 12 '12 at 17:01
Is `response.close` a function? I believe you mean `response.end`: http://nodejs.org/api/http.html#http_response_end_data_encoding – Vinay Oct 08 '13 at 22:07
@Vinay It was in 2012 :) – Kuroki Kaze Oct 09 '13 at 08:53

score 18 · Answer 2 · answered May 25 '11 at 20:34

18

Look at Sockets.io. It provides HTTP/HTTPS streaming and uses various transports to do so:

WebSocket
WebSocket over Flash (+ XML security policy support)
XHR Polling
XHR Multipart Streaming
Forever Iframe
JSONP Polling (for cross domain)

And! It works seamlessly with Node.JS. It's also an NPM package.

https://github.com/LearnBoost/Socket.IO

https://github.com/LearnBoost/Socket.IO-node

answered May 25 '11 at 20:34

BMiner

14,934
11
47
52

Socket.IO is inefficient for this because he is looking for a one-way stream from the server to the client. Something SSE's are perfect for. – Druska May 06 '13 at 15:06
This solution helps me a lot. I'd like to stream a webcam video + microphone audio from client to server. How might I do that? http://stackoverflow.com/questions/27712381/how-to-send-video-from-getusermedia-to-node-js-server – Costa Jan 02 '15 at 05:24

score 5 · Answer 3 · answered Feb 18 '12 at 21:56

You can also abort the infinite loop:

app.get('/sse/events', function(req, res) {
    res.header('Content-Type', 'text/event-stream');

    var interval_id = setInterval(function() {
        res.write("some data");
    }, 50);

    req.socket.on('close', function() {
        clearInterval(interval_id);
    }); 
});

This is an example of expressjs. I believe that without expressjs will be something like.

Streaming data with Node.js

Update:

3 Answers3

Linked