How to append bytes, multi-bytes and buffer to ArrayBuffer in javascript?

Question

Javascript ArrayBuffer or TypedArrays dont have any kind of appendByte(), appendBytes(), or appendBuffer() methods. So if I want to fill an ArrayBuffer one value at a time, how do I do it?

var firstVal = 0xAB;              // 1 byte
var secondVal = 0x3D7F            // 2 bytes
var anotherUint8Array = someArr;

var buffer = new ArrayBuffer();   // I don't know the length yet
var bufferArr = new UInt8Array(buffer);

// following methods do not exist. What are the alternatives for each??
bufferArr.appendByte(firstVal);
bufferArr.appendBytes(secondVal);
bufferArr.appendBuffer(anotherUint8Array);

it's an array, use array syntax `r[i]=x` ex: https://github.com/rndme/download/blob/master/download.js#L69 also review the syntax for the Uint8Array constructor: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array#Syntax - you need a size on that array buffer to be able to use it like that.... — dandavis, Nov 13 '15 at 22:44

Paul S. · Answer 1 · 2015-11-13T23:11:10.213

21

You can create a new TypedArray with a new ArrayBuffer, but you can't change the size of an existing buffer

function concatTypedArrays(a, b) { // a, b TypedArray of same type
    var c = new (a.constructor)(a.length + b.length);
    c.set(a, 0);
    c.set(b, a.length);
    return c;
}

Now can do

var a = new Uint8Array(2),
    b = new Uint8Array(3);
a[0] = 1; a[1] = 2;
b[0] = 3; b[1] = 4;
concatTypedArrays(a, b); // [1, 2, 3, 4, 0] Uint8Array length 5

If you want to use different types, go via Uint8Array as the smallest unit is a byte, i.e.

function concatBuffers(a, b) {
    return concatTypedArrays(
        new Uint8Array(a.buffer || a), 
        new Uint8Array(b.buffer || b)
    ).buffer;
}

This means .length will work as expected, you could now convert this to your typed array of choice (make sure it's a type that would accept the .byteLength of the buffer though)

From here, you could now implement any method you like for concatenating your data, e.g.

function concatBytes(ui8a, byte) {
    var b = new Uint8Array(1);
    b[0] = byte;
    return concatTypedArrays(ui8a, b);
}

var u8 = new Uint8Array(0);
u8 = concatBytes(u8, 0x80); // [128]

edited Nov 13 '15 at 23:11

answered Nov 13 '15 at 22:59

Paul S.

58,277
8
106
120

Thanks @Paul S. It would have been nice for such basic functions to be part of the spec and implemented natively, but your solution is pretty good. One question I still have how would you append a multi-byte, lets say a single 4-byte value to a typedarray. We can loop and call your suggested `concatBytes` function multiple times for each single byte, but can it be accomplished in better way? – codneto Nov 14 '15 at 01:30
@codneto how are your multi-bytes stored? e.g. if you're using ints, how do you know the difference between `0x3D7F` and `0x00003D7F`? And yes, I agree there should be some native `concat`, but I don't think there should be a native `push` or _length_ changing - that's not how typed arrays work – Paul S. Nov 14 '15 at 01:41
S, I have some functions which return 4-byte value which I need to add to buffer. I also am reading some bytes from a stream and appending to buffer, and sometimes based on some flags I have to read 1, 2 or 4 byte value, all of which need to be appended to buffer. How should I accomplish appending these multi-byte values to buffer? – codneto Nov 15 '15 at 10:15
@codneto write it as a loop. Best place to loop would be creating a `n` long _Uint8Array_ for the new data and using the loop to set the vales on it before concatenating with your previous data. – Paul S. Nov 15 '15 at 15:20
Based on your suggestion, I changed `concatBytes` as following. I think it will work. **function concatBytes(ui8a, bytes) { var b = new Uint8Array(bytes.length); bytes.forEach(function (byte, index) { b[index] = byte; }); return concatTypedArrays(ui8a, b); } var u8 = new Uint8Array(0); // to append 0x80C83B u8 = concatBytes(u8, [0x80, 0xC8, 0x3B]); // 128, 200, 59]** – codneto Nov 16 '15 at 21:57
Can't post formatted code in comments, therefore code in above comment is not readable so created a jsfiddle here: http://jsfiddle.net/pLb3haqr/ – codneto Nov 16 '15 at 22:06

score 5 · Answer 2 · answered May 12 '17 at 09:06

Paul's answer allows you to concatenate one TypedArray to an existing TypedArray. In ES6, you can use the following function to concatenate multiple TypedArrays:

function concatenate(resultConstructor, ...arrays) {
    let totalLength = 0;
    for (const arr of arrays) {
        totalLength += arr.length;
    }
    const result = new resultConstructor(totalLength);
    let offset = 0;
    for (const arr of arrays) {
        result.set(arr, offset);
        offset += arr.length;
    }
    return result;
}

const ta = concatenate(Uint8Array,
    Uint8Array.of(1, 2), Uint8Array.of(3, 4));
console.log(ta); // Uint8Array [1, 2, 3, 4]
console.log(ta.buffer.byteLength); // 4

To append a new byte is:

const byte = 3;
concatenate(Uint8Array, Uint8Array.of(1, 2), Uint8Array.of(byte));

This method is found in ExploringJS.

How to append bytes, multi-bytes and buffer to ArrayBuffer in javascript?

2 Answers2