How can I stream a GET request line-by-line?

Question

I would like to send a GET request to a server that is streamed almost in 'realtime' using Chunk Transfer Encoding that I can completley modify line-by-line. For example:

SendChunks = SomeHTTPLibrary.SendData;
SendChunks(Example.org, "5\r\n")
SendChunks(Example.org, "Hello\r\n")
SendChunks(Example.org, "7\r\n")
SendChunks(Example.org, "Goodbye\r\n")
SendChunks(Example.org, "0\r\n")

Where I am right now, I don't even care about listening for a response. It doesn't need to be in C++, I'm comfortable with Python, Javascript, PHP or anything similar.

"completley modify line-by-line" - so... you could use whatever language's TCP library and write whatever you like - Chunk Transfer Encoding or otherwise - you don't have an actual coding problem, and this site's not for arguments about language and library selection.... — Tony Delroy, Oct 31 '14 at 07:16
I mean line-by-line for the actual stream like almost a console sending commands but instead it's the chunks of the GET request. — Aaron Kleiman, Oct 31 '14 at 07:28
so? if you get a TCP `socket()` descriptor, `connect()` to the web server, you can `send()` line-by-line if you like (though the TCP library's [Nagle or similar algorithm](http://en.wikipedia.org/wiki/Nagle%27s_algorithm) and/or buffering may effectively combine or split the content you `send()`, so you're not guaranteed 1:1 `recv`s will happen on the server side. But, so what? TCP is a byte stream protocol - it doesn't honour any sender's notion of message boundaries... they must be encoded in the data sent (e.g. the `\n`s). — Tony Delroy, Oct 31 '14 at 07:31
GET requests don't have a body. What is it that you want to chunk-encode? — avakar, Oct 31 '14 at 08:19
"streamed almost in 'realtime' " why not use a socket connection ? — astroanu, Oct 31 '14 at 09:51

score 1 · Answer 1 · edited May 23 '17 at 11:57

Firstly, you shouldn't be sending a request body along with a GET request. I think technically you can, but if the server does anything with it then it's non-compliant. See https://stackoverflow.com/a/983458/241294.

From you question it looks as though you already know that you need chunked transfer encoding. Here is a crude example of how you can achieve this in python, but with a POST request instead of a GET request (code hacked from here):

import httplib

conn = httplib.HTTPConnection('Example.org')
conn.connect()
conn.putrequest('POST', '/post')
conn.putheader('Transfer-Encoding', 'chunked')
conn.endheaders()

conn.send("5\r\n")
conn.send("hello\r\n")
conn.send("7\r\n")
conn.send("Goodbye\r\n")
conn.send("0\r\n")

resp = conn.getresponse()
print(resp.status, resp.reason, resp.read())
conn.close()

For a nicer example with a python chunking function see How to force http.client to send chunked-encoding HTTP body in python?.

How can I stream a GET request line-by-line?

1 Answers1