A buffer is a specific location in raw memory. It serves as a temporary storage space for excess binary data that the processing unit cannot accept at that particular moment.
Node.js includes a Buffer class. It can deal with binary data when managing TCP (Transfer Control Protocol) streams and read-write operations on a file system.
Learn how to create, read, and change the contents of a buffer.
Creating a Buffer
To create a Buffer in Node.js, you will use the alloc() or from() methods. The alloc() method creates a new buffer, specifying its size during creation as the first and only required parameter. It’s useful when you don’t have any data to store at the time of buffer creation.
Specify the buffer size parameter in bytes when you create a buffer with the alloc() method. For example:
const buf = Buffer.alloc(8);
console.log(buf);
The Buffer class automatically adds zeroes as placeholder values for new data when you create it with the alloc() method.
The Buffer class expresses each 0 value as 00, using hexadecimal format. In this example, it contains a total of eight values.
To initialize the Buffer with different placeholder values, pass a second fill parameter:
const buf_filled = Buffer.alloc(8, 5);
console.log(buf_filled);
This object cites a portion in memory that stores 8 bytes of the value 05. Note that although you passed a number as the fill parameter, buffers only store data in binary.
After allocating memory to the Buffer, write data by calling the write() method:
const buf = Buffer.alloc(8);buf.write("v", "utf-8");
console.log(buf)
buf.write("va","utf-8");
console.log(buf)
The write() method uses character encoding to convert the first parameter, using utf-8 and then writes the string to the Buffer. Adding a second character to the string will fill up the second byte.
To extract data from existing data types like strings or arrays, use the from() method. This method creates buffers from strings and arrays.
For example:
const stringBuf = Buffer.from('string')
console.log(stringBuf)
const arrayBuf = Buffer.from([97, 114, 114, 97, 121], 'hex')
console.log(arrayBuf);
The from() method takes the input as its first parameter, calculates the number of bytes it needs to encode the data, and then sends the result to the Buffer. By supplying another encoding format as the second parameter, you can override the default encoding (UTF-8).
Passing numbers to the from() method will result in an error.
Reading a Buffer
Although Buffers are similar to arrays, they are not resizable and can deal with binary computer data thanks to built-in methods.
The Buffer class allows us to read individual bytes of its data using JavaScript’s square bracket syntax.
For example:
const myBuf = Buffer.from('Mine');
console.log(MyBuf[1]);console.log(MyBuf[3]);
console.log(MyBuf[5]);
The code block above employs the square bracket syntax to obtain the values of the first and third bytes in their decimal representation. Attempting to get an invalid byte will result in an undefined error.
To access all its data, the Buffer class comes with methods toJSON() and toString(), which obtain the contents in two different formats.
The toString() method outputs a string as the buffer contents:
const myBuf = Buffer.from('Mine');
console.log(myBuf.toString());const numberBuf = Buffer.from([123]);
console.log(numberBuf.toString())
const emptyBuf = Buffer.alloc(5);
console.log(emptyBuf.toString());
The first call initializes the Buffer with the value “Mine”, which the call to toString replicates. The second example uses a single-int array for initialization, which has a string representation as the “{” character. In the final case, a Buffer with five null values returns the string “\x00\x00\x00\x00\x00”. The string \x00 is the hexadecimal representation of null.
The toString() method always outputs the result in string format, no matter what type of data you initialize the Buffer with.
The .toJSON() method returns the decimal representation of the Buffer data, regardless of the data you used to initialize the Buffer.
For example:
const myBuf = Buffer.from('Mine');
console.log(myBuf.toJSON());
// output: { type: 'Buffer', data: [ 77, 105, 110, 101 ] }
The JSON output has a type property with a value of Buffer to indicate its origin. Its data property stores an array of decimals that represent the original byte array.
Modifying a Buffer
Similar to accessing the individual bytes of a Buffer, you could also modify individual bytes of a Buffer’s content using the square bracket syntax.
When using the square bracket syntax to change an individual content, you can only assign the decimal representation of the value.
For example:
myBuf[0] = 70
console.log(myBuf.toString())
Because buffers are binary data, you cannot give a specific portion of a buffer a string. If you try to set an individual byte to a string, Buffer will translate it into a null character.
For example:
myBuf[0] = 'F';
console.log(myBuf.toString());
Alternatively, you can change the entire contents of a buffer using the write() method.
Consider inserting an index outside the Buffer’s length. Rather than returning an error, Buffer ignores the invalid index and keeps the original Buffer content intact.
For example, try to set the fifth element of myBuf to r via its decimal representation of 114:
myBuf[4] = 114;
console.log(myBuf.toString());
Notice that the toString() method returns the same value ‘Mine’.
Since you cannot resize a Buffer, trying to write more data than one can hold will result in it discarding the extra data. For example:
const buf1 = Buffer.alloc(5)
buf1.write('number');
console.log(buf1.toString())
Using the toString() method to confirm the buffer data, it returns ‘numbe’ rather than ‘number’. Which is the inserted argument inside the write() method.
Buffers write in a serial fashion beginning at index zero. The write() method serially adds bytes to a Buffer, overwriting any previous data.
For example:
const buf2 = Buffer.alloc(6);buf2.write('member');
console.log(buf2.toString())
buf2.write('hi');
console.log(buf2.toString());
The code above creates a six-byte buffer and adds the string “member” to it using the write() method.
It then updates the buffer with new content that takes up less memory space than the earlier content.
This results in the creation of a new string with the first two bytes overwritten and the remaining bytes left unaltered.
Many APIs and Data Structures Use Buffers
Now you know how to create a buffer, write to one, read its contents, and modify it with the appropriate methods.
There are several other methods available to work with the Node.js Buffer class.
You should know these methods and understand Buffers to grasp how different concepts like streams and file systems work.