Bundling Node.js / Express apps with Webpack

Installation all of Node.js dependencies and build sources on production host could be painful. You might want to make a bundle on development host and deploy single bundle on production.

The idea came after experiencing 502 error on main page of CorpGlory.com for 15 minutes because of fogrotten npm install in our CI process.

Your production server should have as less software / files as possible. The post describes tricks and technics you can use to build simillar node.js app. Maybe it's a better idea to use a http framework ready for bundling with Webpack, but we will use Express.js here.

TL;DR: we pack in dev mode by hacking externals and subsituting require() for node_modules. In prod mode we hack dynimic import views with ContextReplacementPlugin

Test project

You can find sources on GitHub. We use webpack v4.39.1 and express v4.17.1 here.

Lets list files we interested in:

express-webpack-example
|- /build                   // webpack configs folder
  |- webpack.base.conf.js
  |- webpack.dev.conf.js
  |- webpack.prod.conf.js
|- /dist                    // destination folder with bundle 
  |                         // (available after running `npm run dev`)
  |- app.js                 // target bundle
|- /node_modules            // default node.js / npm modules folder
|- app.js                   // our express.js application
|- test_module.js           // a test module for `require()`
|- template.ejs             // a test .ejs template to render 
|- package.json             // default node.js / npm dependecy config

Development mode

Idea is that we don't want to rebuild whole bundle each time because it's going to be slow...Why? Because Webpack bundles all modules required by the app. So most of files from node_modules will be imported into bundle. That's why building process will be slow.

Let's import only local modeles but other import by require():

const webpack = require('webpack');

var base = require('./webpack.base.conf');

base.externals.push(
  function(context, request, callback) {
    if(request[0] == '.') {
      callback();
    } else {
      callback(null, "require('" + request + "')");
    }
  }
);

module.exports = base;

The hack is to use externals feature. It takes ... and we local but don't touch packages from node_modules and allow node.js resolve it requires in runtime.

If you open file dist/app.js (the bundle produced by webpack) you will find lines like:

/***/ "express":
/*!*************************************!*\
  !*** external "require('express')" ***!
  \*************************************/
/*! no static exports found */
/***/ (function(module, exports) {

eval("module.exports = require('express');\n\n//# sourceURL=webpack:///external_%22require('express')%22?");

/***/ })

/******/ });

So, we don't have problem with dynamic require because webpack doesn't go into express module so doesn't find any dynamic require. but file test_module.js will be imported as usual:

/***/ "./test_module.js":
/*!************************!*\
  !*** ./test_module.js ***!
  \************************/
/*! exports provided: MODULE_CONST */
/***/ (function(module, __webpack_exports__, __webpack_require__) {

"use strict";
eval("__webpack_require__.r(__webpack_exports__);\n/* harmony export (binding) */ __webpack_require__.d(__webpack_exports__, \"MODULE_CONST\", function() { return MODULE_CONST; });\nconst MODULE_CONST = \"Hello from the test module\";\r\n\n\n//# sourceURL=webpack:///./test_module.js?");

/***/ }),

Production mode

Dynamic views problem

When you start to build this, you will get error:

WARNING in ../node_modules/express/lib/view.js
81:13-25 Critical dependency: the request of a dependency is an expression
 @ ../node_modules/express/lib/view.js
 @ ../node_modules/express/lib/application.js
 @ ../node_modules/express/lib/express.js
 @ ../node_modules/express/index.js
 @ ./app.ts

so there is a problem in ../node_modules/express/lib/view.js, because there is:

// ...
  if (!opts.engines[this.ext]) {
    // load engine
    var mod = this.ext.substr(1)
    debug('require "%s"', mod)

    // default engine export
    var fn = require(mod).__express

    if (typeof fn !== 'function') {
      throw new Error('Module "' + mod + '" does not provide a view engine.')
    }

    opts.engines[this.ext] = fn
  }
// ...

so webpack can't resolve require(mod).__express because it's dynamic. It's made this way because express.js can't know which library you will use for rendering.

You can find something interesting in your build log:


 Asset      Size  Chunks             Chunk Names
app.js  1.05 MiB    main  [emitted]  main
Entrypoint main = app.js
[0] multi ./app.js 28 bytes {main} [built]
[./app.js] 281 bytes {main} [built]
[./node_modules/express/lib sync recursive] ./node_modules/express/lib sync 160 bytes {main} [built]
[./test_module.js] 59 bytes {main} [built]
[buffer] external "buffer" 42 bytes {main} [built]
[crypto] external "crypto" 42 bytes {main} [built]
[events] external "events" 42 bytes {main} [built]
[fs] external "fs" 42 bytes {main} [built]
[http] external "http" 42 bytes {main} [built]
[net] external "net" 42 bytes {main} [built]
[path] external "path" 42 bytes {main} [built]
[querystring] external "querystring" 42 bytes {main} [built]
[stream] external "stream" 42 bytes {main} [built]
[url] external "url" 42 bytes {main} [built]
[util] external "util" 42 bytes {main} [built]
    + 108 hidden modules

Notice [./node_modules/express/lib sync recursive] ./node_modules/express/lib sync 160 bytes {main} [built]. Looks unusual, right? Webpack tryes to say that it's going to import all files in ./node_modules/express/lib into bundle.

If you don't really use views feature then you will be able to make a bundle with the warning above and run dist/app.js.

Ok. So we compiled dist somehow. What Webpack does is substitutes

var fn = require(mod).__express

to

var fn = __webpack_require__("./node_modules/express/lib sync recursive")(mod).__express

We got a critical error, let's help Webpack to handle __webpack_require__("./node_modules/express/lib sync recursive") here.

This expression is a bit different, we expect context here.

Now let's add following to production build (webpack.prod.conf.js):

base.plugins = [
  new webpack.ContextReplacementPlugin(// we want to replace context
    /express\/lib/,                    // and replace all searches in
                                       // express/lib/*
    resolve('node_modules'),           // to look in folder 'node_modules'
    {                                  // and return a map
      'ejs': 'ejs'                     // which resolves request for 'ejs'
    }                                  // to module 'ejs'
  )                                    // __webpack_require__(...)(mod)
]                                      // we set `mod = 'ejs'`

Check out a nice post to learn more about ContextReplacementPlugin

EJS: Require.extension

After we included ejs modeuls in bundle, we get warning:

WARNING in ./node_modules/ejs/lib/ejs.js 917:4-22
require.extensions is not supported by webpack. Use a loader instead.
 @ ./node_modules sync ejs
 @ ./node_modules/express/lib/view.js
 @ ./node_modules/express/lib/application.js
 @ ./node_modules/express/lib/express.js
 @ ./node_modules/express/index.js
 @ ./app.js
 @ multi ./app.js

It's a deprecated feature of node.js, which ejs tries to use. So it we will not try to do something require('template.ejs'), that we are fine. As Webpack, suggests, it's better to use loaders. So, it is our responsobility to deal with code like require('template.ejs').

We can say that we aware what we are dealing with and add following warning supression into webpack.prod.conf.js:

base.stats = {
  warningsFilter: /require\.extensions/
}

Result

In result, we have a node.js project which we can bundle into signle javascript file and send to to production to run. No need to care about node_modules and package.json. Only right version on node.js is necessary. Yes, we also should care about assets and *.ejs files, but let's make it in another post.

Other

Do you like o write in node.js? Would you practice your skills by contributing to open-source? Checkout our node.js app. We used similar approach there, but based on Koa.js. We wanted to avoid hacks with packing Express.js as described here.