2

How could I stream files(images) using Apache Thrift? I have searched a lot about Thrift and did not find any well written documentation regarding to it. Why did Facebook open sourced this project without docs?

JensG
  • 12,102
  • 4
  • 40
  • 51
Mahdi
  • 223
  • 2
  • 10
  • "*did not find any well written documentation regarding to it*" - http://thrift.apache,org/docs and [this answer](http://stackoverflow.com/questions/20653240/what-is-rpc-framework-and-apache-thrift/20664706#20664706) don't fit your needs? "*Why did Facebook open sourced this project without docs*" - [They clearly didn't](https://thrift.apache.org/static/files/thrift-20070401.pdf). Maybe the Google index is outdated so you were not able to find those links? – JensG Nov 05 '14 at 09:00
  • I mean an API Doc listed classes and Methods – Mahdi Nov 05 '14 at 11:44
  • The Tutorials and the Test server/client pair do a much better job explaining Thrift than any API class list would do ever. I also highly recommend [Randy's forthcoming Thrift book](http://www.manning.com/abernethy/), already available via MEAP. More info here http://stackoverflow.com/questions/20653240/what-is-rpc-framework-and-apache-thrift/20664706#20664706 - BTW, one of the links above is incorrect, it is http://thrift.apache.org/docs – JensG Nov 05 '14 at 15:56
  • Thanks @JensG, since I'm a newbie in network applications, I'm searching for more detail tutorial using Thrift. – Mahdi Nov 05 '14 at 18:01

1 Answers1

4

The way I would recommend is to set up your service to deliver data in chunks, like so:

struct DataChunk {
  1 : binary data
  2 : bool haveMoreData
}

service {
  DataChunk  GetChunk( 1 : string resource, 2: i32 offset, 3: i32 size)
}

It seems a good idea to either limit the size to some sane value (needs to be checked on the server side), or remove the size argument at all and always deliver chunks of a fixed, predefined size to circumvent clients asking for insanely large data blocks.

Note that the whole process needs to follow the pull model, there is no built-in push feature. However, you still can do push, you just need to run a Thrift server on the client side and pass the necessary connection info. Although this will not work in all scenarios (especially transports), it is a fully working solution where it is possible.

JensG
  • 12,102
  • 4
  • 40
  • 51