20

I have a Node app which accesses a static, large (>100M), complex, in-memory data structure, accepts queries, and then serves out little slices of that data to the client over HTTP.

Most queries can be answered in tenths of a second. Hurray for Node!

But, for certain queries, searching this data structure takes a few seconds. This sucks because everyone else has to wait.

To serve more clients efficiently, I would like to use some sort of parallelism.

But, because this data structure is so large, I would like to share it among the workers or threads or what have you, so I don't burn hundreds of megabytes. This would be perfectly safe, because the data structure is not going to be written to. A typical 'fork()' in any other language would do it.

However, as far as I can tell, all the standard ways of doing parallelism in Node explicitly make this impossible. For safety, they don't want you to share anything.

But is there a way?

Background:

It is impractical to put this data structure in a database, or use memcached, or anything like that.

WebWorker API libraries and similar only allow short serialized messages to be passed in and out of the workers.

Node's Cluster uses a call named 'fork', but it is not really a fork of the existing process, it is spawning a new one. So once again, no shared memory.

Probably the really correct answer would be to use filesystem-like access to shared memory, aka tmpfs, or mmap. There are some node libraries that make mount() and mmap() available for exactly something like this. Unfortunately then one has to implement complex data structure access on top of synchronous seeks and reads. My application uses arrays of arrays of dicts and so on. It would be nice to not have to reimplement all that.

NeilK
  • 726
  • 1
  • 6
  • 15
  • Can't you pre-empt your search (using `process.nextTick` perhaps) so it won't block the rest? – robertklep Feb 14 '13 at 08:21
  • 3
    `It is impractical to put this data structure in a database, or use memcached, or anything like that.` The what?? Since when? – freakish Feb 14 '13 at 08:25
  • freakish: We are checking if each item is a subset of a query. Imagine we have a string "fooquux", and we want to check if "ox" is in that string. There is no way that I know of to do that efficiently with ordinary database operations. But it's super easy if you have access to it as an ordinary data structure. Then a large number of these are sorted and ranked and these are "pointers" to more data, again impractical unless we have them in a data structure. – NeilK Feb 14 '13 at 18:21
  • @NeilK First of all: you should've told us about 512MB limitation and that the data is actually 208MB, not 100MB. It is a big difference. Secondly: there are hundreds of tools which do what you are trying to do ( and I'm talking about your string example, because you didn't give us enough details here as well ). Maybe you are trying to use wrong tools? You might have a look for example at Apache Solr. And finally: ordinary database operations? I don't what that is. But I'm pretty sure that you can do that efficiently with any (proper) database as long as you configure it to be in-memory. – freakish Feb 15 '13 at 08:55
  • Are you certian that your data access is properly asynchronous? How many clients are we talking about here? If yes, then the next level of performance would be node cluster. – Eric Feb 03 '15 at 20:22
  • Here: https://stackoverflow.com/q/10965201/632951 – Pacerier Jul 21 '17 at 14:54

4 Answers4

5

I tried write a C/C++ binding of shared memory access from nodejs. https://github.com/supipd/node-shm

Still work in progress (but working for me), maybe usefull, if bug or suggestion, inform me.

supipd
  • 279
  • 3
  • 9
  • This is a great idea @supipd, but is it possible for you to have this working on the current version? I believe `node::ObjectWrap` is not linked to `` in the newer versions. (users will receive an error on compiling) Anyways, this should be the accepted answer. :) – NiCk Newman Sep 07 '15 at 20:32
0

building with waf is old style (node 0.6 and below), new build is with gyp.

You should look at node cluster (http://nodejs.org/api/cluster.html). Not clear this is going to help you without having more details, but this runs multiple node processes on the same machine using fork.

Pascal Belloncle
  • 10,538
  • 3
  • 53
  • 56
  • 1
    Cluster doesn't really fork - it is just spawning a new process. The reason why I'd like to fork is to share memory that I know is immutable and won't be changed by the children. – NeilK Feb 14 '13 at 08:07
  • 2
    Right from the entry you linked to: "These child Nodes are still whole new instances of V8. Assume at least 30ms startup and 10mb memory for each new Node. That is, you cannot create many thousands of them." And in my case I have to load >100MB into each. – NeilK Feb 14 '13 at 08:18
  • @NeilK >100MB is not a big deal ( unless by >100MB you understand 10GB :D ). What is more important is to keep that data synchronized between instances. For that I suggest using dedicated server for keeping that data. See my answer. – freakish Feb 14 '13 at 08:23
  • correct, it uses child_process, and spawns a new node process. You may want to consider using process.nextTick to break down your search and give other queries a chance to run. – Pascal Belloncle Feb 14 '13 at 08:26
  • as @freakish pointed out, 100Mb is really no big deal, and you only get slower start time. Once your system is up and running, you should be ok. – Pascal Belloncle Feb 14 '13 at 08:28
  • an other avenue to consider, although a long road, you could consider removing the spawn here https://github.com/joyent/node/blob/master/lib/child_process.js#L475 and use fork instead. Not clear if this would work out of the box. I sor of doubt it. – Pascal Belloncle Feb 14 '13 at 08:30
  • @Neilk I don't think forking [shares](https://groups.google.com/forum/#!topic/nodejs/Q0fE5ul68F0) memory (or it might have changed?) – NiCk Newman Sep 07 '15 at 19:05
0

Actually Node does support spawning processes. I'm not sure how close Node's fork is to real fork, but you can try it:

http://nodejs.org/api/child_process.html#child_process_child_process_fork_modulepath_args_options

By the way: it is not true that Node is unsuited for that. It is as suited as any other language/web server. You can always fire multiple instances of your server on different ports and put a proxy in front.

If you need more memory - add more memory. :) It is as simple as that. Also you should think about putting all of that data on a dedicated in-memory database like Redis or Memcached ( or even Couchbase if you need complex queries ). You won't have to worry about duplicating that data any more.

freakish
  • 48,318
  • 8
  • 114
  • 154
  • 2
    Read the documentation you just linked me to. All the way to the end. :) Also, while it might be doable to have several processes with copies of a 100M data structure at a modest budget, I'm doing this as a side project with a free Heroku dyno, which has a limit of 512 MB. Having two workers blows my memory budget, and Heroku shuts it down. But more importantly, wasting memory like this offends me. – NeilK Feb 14 '13 at 18:22
  • Um and lest you think I can't do arithmetic, in reality the data structure is 208 MB - 100 MB was an example – NeilK Feb 14 '13 at 18:28
  • @NeilK But 512MB limit does not offend you? :D Besides: even with fork child process does copy the data unless it is read only. – freakish Feb 15 '13 at 08:59
0

Most web applications spend the majority of their life waiting for network buffers and database reads. Node.js is designed to excel at this io bound work. If your work is truly bound by the CPU, you might be served better by another platform.

With that out of the way...

  1. Use process.nextTick (perhaps even nested blocks) to make sure that expensive CPU work is properly asynchronous and not allowed to block your thread. This will make sure one client making expensive requests doesn't negatively impact all the others.

  2. Use node.js cluster to add a worker process for each CPU in the system. Worker processes can all bind to a single HTTP port and use Memcached or Redis to share memory state. Workers also have a messaging API that can be used to keep an in-process memory cache synchronized, however it has some consistency limitations.

Eric
  • 1,680
  • 13
  • 19