CopyManager cp;
//loop until 1 million rows have been persisted
StringBuilder sb = new StringBuilder(); //build 10k entries, comma separated
cp.copyIn("COPY tableName FROM STDIN WITH CSV", new StringReader(sb.toString()));
The copyIn
itself takes about 30 seconds. Would you consider this reasonable performance? Is there anything I could do to optmize the performance?
Reason: I have to batch insert a few hundred million rows a few times a day. Thus, when importing like 500 million rows, that would last 4h!
How could I increase the speed? Switch to a different DB? Create multi threads an import in parallel?