How can I parse records on demand with Node.js?

Question

Is there a Node module that can parse a specific number of records from a CSV file? The use case is to parse a large log file and deliver records to a paging client as requested.

node-csv can't yet do this, and the closest I've found is to read lines one by one, which requires reinventing the CSV parsing wheel, and will break on multi-line records.

But let's lower the bar: how can I parse single-line CSV records one by one with Node.js? Pretty trivial task in most other languages.

What do you mean by "single-line CSV records"? Because obviously it's not just 1 line, as my reply got downvoted. — Slavo, Oct 28 '13 at 09:32
@Slavo: no idea who downvoted your answer (wasn't me). By "single-line CSV records", I mean comma-separated tuples without newlines in any of the fields. As in, the regular type of CSV data. — Dan Dascalescu, Oct 28 '13 at 11:56

robertklep · Answer 1 · 2013-10-29T08:35:31.787

1

Parsing a single 'line' (which can also have embedded newlines):

var csv = require('csv'); // node-csv

csv()
  .from.string(SINGLE_LINE_OF_CSV)
  .to.array(function(record) {
    console.log('R', record);
  });

I'm not sure what you mean by 'a specific number of records from a CSV file', or what the issue is exactly. Once you've read the amount you need, just send the response back to the client and you're done.

EDIT: if you want to implement paging, you can use node-csv too:

var csv   = require('csv');
var skip  = 100;
var limit = 10;

csv()
  .from.path('file.csv')
  .on('record', function(row, index) {
    if (index >= skip && index < (skip + limit))
      console.log('R', index);
  });

edited Oct 29 '13 at 08:35

answered Oct 25 '13 at 13:06

robertklep

174,329
29
336
330

I seem to have trouble parsing the sample code. How is "SINGLE_LINE_OF_CSV" read, such that it has embedded newlines? And how is the `csv` function defined? By "specific number of records", I mean reading synchronously N records from a CSV file, instead of parsing the entire file in memory and then picking the desired records. – Dan Dascalescu Oct 28 '13 at 11:58
@DanDascalescu sorry, you lowered the bar :) The example code parses single-line CSV records (and you can call it multiple times to achieve the 'one-by-one' aspect). You didn't mention where they should come from. And `csv` is `node-csv`. Perhaps you should just make your question more clear as everyone seems to be failing to implement your intention... – robertklep Oct 28 '13 at 16:46

score 1 · Answer 2 · answered Oct 25 '13 at 13:06

1

As far as I understand, you just want to parse the comma-separated line of values into an array? If so, try this one:

https://npmjs.org/package/csvrow

answered Oct 25 '13 at 13:06

Slavo

14,445
10
42
59

How can I parse records on demand with Node.js?

2 Answers2

Linked