← archive

Building a crude Node.js from scratch

October 10, 2017
javascript

Node is powered by the JavaScript engine used in Google Chrome, called V8[1]. In this post I'm going to guide you through two steps:

  • making a "Hello World" example in V8
  • making a crude Node runtime with support for 3 statements: console.log, console.error and quit for quitting the process

In our new crude runtime we'll execute the following script:

console.log("🎉");
b=4+4;
console.error(b);
quit();

console.log("never reach this");

Setting up the hello world example

First things first. Let's execute a JavaScript string concatenation in V8! We'll write an example that takes a JavaScript statement as a string argument, executes it as JavaScript code, and prints the result to standard out. The string will be js 'Hello' + ', World!'

The setup will loosely follow this gist in combination with the V8 getting started with embedding wiki.

We're going to use version 5.8 of V8. I'm going to assume you're using MacOS with git, Python 2.7 and Xcode installed and that you're using Bash as your shell of choice.

Clone the v8 source

git clone https://github.com/v8/v8.git

Install depot tools and add it to our path

git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
cd depot_tools
echo "export PATH=`pwd`:\"$PATH\"" >> ~/.bashrc # replace with ~/.zshrc if using ZSH
source ~/.bashrc # or ~/.zshrc
cd -

Make sure you have clang and clang++ installed

which clang && which clang++ # should output paths to clangs

If you don't have clang and clang++ make sure you have installed Xcode.


Add environment libraries for clang and clang++

cat <<EOT >> ~/.bashrc # replace with ~/.zshrc if using ZSH
export CXX="`which clang++`"
export CC="`which clang`"
export CPP="`which clang` -E"
export LINK="`which clang++`"
export CXX_host="`which clang++`"
export CC_host="`which clang`"
export CPP_host="`which clang` -E"
export LINK_host="`which clang++`"
export GYP_DEFINES="clang=1"
EOT

source ~/.bashrc # or ~/.zshrc

Update .git/config of V8 to fetch remotes and tags

cd v8
vim .git/config

Update config for remote origin to this

[remote "origin"]
  url = https://chromium.googlesource.com/v8/v8.git
  fetch = +refs/branch-heads/*:refs/remotes/branch-heads/*
  fetch = +refs/tags/*:refs/tags/*

Exit vim[2] and fetch origin.

git fetch

Checkout to a new branch for version 5.8 of V8

git checkout -b 5.8 -t branch-heads/5.8

Sync this git repo

gclient sync

Create build configuration

tools/dev/v8gen.py x64.release

Edit the default build configuration

gn args out.gn/x64.release

And add these two lines to that configuration:

is_component_build = false
v8_static_library = true

Build v8

You'll have to detect a number of cores your CPU has. Go to Activity Monitor and click on CPU LOAD section. Window with graphs should pop up - the number of cores is the number of panels on that window.

For my 2015 i7 CPU it's 4, so I'm running the following command: bash make native -j 4 If you're not sure just run it without -j flag

make native

Alternatively you can just use Ninja like this:

ninja -C out.gn/x64.release

Add hello world

vim hello_world.cpp

Save this in hello_world.cpp[3]:

// Copyright 2015 the V8 project authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "include/libplatform/libplatform.h"
#include "include/v8.h"
using namespace v8;
int main(int argc, char* argv[]) {
  // Initialize V8.
  V8::InitializeICUDefaultLocation(argv[0]);
  V8::InitializeExternalStartupData(argv[0]);
  Platform* platform = platform::CreateDefaultPlatform();
  V8::InitializePlatform(platform);
  V8::Initialize();
  // Create a new Isolate and make it the current one.
  Isolate::CreateParams create_params;
  create_params.array_buffer_allocator =
      v8::ArrayBuffer::Allocator::NewDefaultAllocator();
  Isolate* isolate = Isolate::New(create_params);
  {
    Isolate::Scope isolate_scope(isolate);
    // Create a stack-allocated handle scope.
    HandleScope handle_scope(isolate);
    // Create a new context.
    Local<Context> context = Context::New(isolate);
    // Enter the context for compiling and running the hello world script.
    Context::Scope context_scope(context);
    // Create a string containing the JavaScript source code.
    Local<String> source =
        String::NewFromUtf8(isolate, "'Hello' + ', World!'",
                            NewStringType::kNormal).ToLocalChecked();
    // Compile the source code.
    Local<Script> script = Script::Compile(context, source).ToLocalChecked();
    // Run the script to get the result.
    Local<Value> result = script->Run(context).ToLocalChecked();
    // Convert the result to an UTF8 string and print it.
    String::Utf8Value utf8(result);
    printf("%s\n", *utf8);
  }
  // Dispose the isolate and tear down V8.
  isolate->Dispose();
  V8::Dispose();
  V8::ShutdownPlatform();
  delete platform;
  delete create_params.array_buffer_allocator;
  return 0;
}

Copy snapshots

cp out.gn/x64.release/*.bin .

Compile the hello world example

clang++ -Iinclude out/native/*.a hello_world.cpp -o hello_world

Run the hello world example

./hello_world

You should see

Hello, World!

in your shell. Now that we have a hello world example up and running, we can start adding support for console.log, console.error and quit.

Add support for console and quit statements

Add the following to run_script.cpp[4]:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fstream>
#include <iostream>
#include <sstream>
#include "include/libplatform/libplatform.h"
#include "include/v8.h"

using namespace v8;

// Define a quit function that exits.
void quit(const v8::FunctionCallbackInfo<v8::Value>& args) {
  std::exit(0);
}

// Store isolate in a global variable.
Isolate* isolate_;
Isolate* GetIsolate() { return isolate_; }

// Define a Console class.
class Console { };

// Extracts a C string from a V8 Utf8Value.
const char* ToCString(const v8::String::Utf8Value& value) {
  return *value ? *value : "<string conversion failed>";
}

// Define a log function that prints to stdout.
void log(const FunctionCallbackInfo<Value>& args){
  v8::String::Utf8Value str(args[0]);
  const char* cstr = ToCString(str);
  printf("%s\n", cstr);
}

// Define an error function that prints to stderr.
void error(const FunctionCallbackInfo<Value>& args){
  v8::String::Utf8Value str(args[0]);
  const char* cstr = ToCString(str);
  fprintf(stderr,"%s\n", cstr);
}

Local<Object> WrapConsoleObject(Console *c) {
  EscapableHandleScope handle_scope(GetIsolate());

  Local<ObjectTemplate> class_t;
  Local<ObjectTemplate> raw_t = ObjectTemplate::New(GetIsolate());
  raw_t->SetInternalFieldCount(1);

  // Set log method.
  raw_t->Set(
      v8::String::NewFromUtf8(GetIsolate(), "log", v8::NewStringType::kNormal).ToLocalChecked(),
      v8::FunctionTemplate::New(GetIsolate(), log));

  // Set error method.
  raw_t->Set(
      v8::String::NewFromUtf8(GetIsolate(), "error", v8::NewStringType::kNormal).ToLocalChecked(),
      v8::FunctionTemplate::New(GetIsolate(), error));
  class_t = Local<ObjectTemplate>::New(GetIsolate(),raw_t);

  // Create instance.
  Local<Object> result = class_t->NewInstance(GetIsolate()->GetCurrentContext()).ToLocalChecked();

  // Create wrapper.
  Local<External> ptr = External::New(GetIsolate(),c);
  result->SetInternalField(0,ptr);
  return handle_scope.Escape(result);
}


int main(int argc, char* argv[]) {
  // Initialize V8.
  V8::InitializeICUDefaultLocation(argv[0]);
  V8::InitializeExternalStartupData(argv[0]);
  Platform* platform = platform::CreateDefaultPlatform();
  V8::InitializePlatform(platform);
  V8::Initialize();

  // Get JavaScript script file from the first argument.
  FILE* file = fopen(argv[1],"r");
  fseek(file, 0, SEEK_END);
  size_t size = ftell(file);
  rewind(file);
  char* fileScript = new char[size + 1];
  fileScript[size] = '\0';
  for (size_t i = 0; i < size;) {
    i += fread(&fileScript[i], 1, size - i, file);
  }
  fclose(file);

  // Create a new Isolate and make it the current one.
  Isolate::CreateParams create_params;
  create_params.array_buffer_allocator =
    v8::ArrayBuffer::Allocator::NewDefaultAllocator();
  Isolate* isolate = Isolate::New(create_params);
  {
    Isolate::Scope isolate_scope(isolate);
    isolate_ = isolate;

    // Create a stack-allocated handle scope.
    HandleScope handle_scope(isolate);

    // Create a template.
    v8::Local<v8::ObjectTemplate> global = v8::ObjectTemplate::New(isolate);

    // Set a quit statement to global context.
    global->Set(
        v8::String::NewFromUtf8(isolate, "quit", v8::NewStringType::kNormal)
        .ToLocalChecked(),
        v8::FunctionTemplate::New(isolate, quit));

    // Create a new context.
    Local<Context> context = Context::New(isolate, NULL, global);

    // Enter the context for compiling and running the hello world script.
    Context::Scope context_scope(context);

    // Create a JavaScript console object.
    Console* c = new Console();
    Local<Object> con = WrapConsoleObject(c);
    // Set a console statement to global context.
    context->Global()->Set(String::NewFromUtf8(isolate, "console", NewStringType::kNormal).ToLocalChecked(),
        con);

    // Create a string containing the JavaScript source code.
    Local<String> source =
      String::NewFromUtf8(isolate, fileScript,
          NewStringType::kNormal).ToLocalChecked();

    // Compile the source code.
    Local<Script> script = Script::Compile(context, source).ToLocalChecked();

    // Run the script to get the result.
    Local<Value> result = script->Run(context).ToLocalChecked();
  }
  // Dispose the isolate and tear down V8.
  isolate->Dispose();
  V8::Dispose();
  V8::ShutdownPlatform();
  delete platform;
  delete create_params.array_buffer_allocator;
  return 0;
}

After that, set up a JavaScript script at test.js:

console.log("🎉");
b=4+4;
console.error(b);
quit();

console.log("never reach this");

Compile our new program with

clang++ -Iinclude out/native/*.a run_script.cpp -o run_script

And run it with

./run_script test.js

You should see the following output

🎉
8

Phew! That's it - congratulate yourself for building a crude version of Node that only supports console.log, console.error and quit statements.

Having a global function quit like this is a little weird, so implementing a more normal process.exit is left as an exercise for the reader.


If this article sparked your interest about Node internals, I strongly suggest you continue reading article named How does NodeJS work, which explains the concepts used in this article in greater detail.


  1. More info about V8 can be found at their wiki.
  2. :wq or try searching Stack Overflow.
  3. Taken from V8 repo here.
  4. This example is based on this gist.
Want to talk more about this or any other topic? Email me. I welcome every email.