Services Blog Visit Gitea LinkedIn Contact Us
Services Blog Contact Us

Bundling Node.js / Express apps with Webpack

Installation of all Node.js dependencies and building sources on the production host is painful. You might want to make a bundle on development host and deploy a single bundle on production.

The idea came after experiencing a 502 error on the main page of CorpGlory.com for 15 minutes, because of forgotten npm install in our CI process.

Your production server should have as little software/files as possible. The post describes tricks and technics you can use to build similar node.js app. Maybe it's a better idea to use an HTTP framework ready for bundling with Webpack, but we will use Express.js here.

TL;DR: we pack in dev mode by hacking externals and subsituting require() for node_modules. In production mode, we hack dynamic import views with ContextReplacementPlugin

Test project

You can find sources on GitHub. We use webpack v4.39.1 and express v4.17.1 here.

Lets list files we interested in:

express-webpack-example
|- /build                   // webpack configs folder
  |- webpack.base.conf.js
  |- webpack.dev.conf.js
  |- webpack.prod.conf.js
|- /dist                    // destination folder with bundle 
  |                         // (available after running `npm run dev`)
  |- app.js                 // target bundle
|- /node_modules            // default node.js / npm modules folder
|- app.js                   // our express.js application
|- test_module.js           // a test module for `require()`
|- template.ejs             // a test .ejs template to render 
|- package.json             // default node.js / npm dependecy config

Development mode

The idea is we don't want to rebuild the whole bundle each time - it's going to be slow... Why? Because Webpack bundles all modules required by the app. So, most of the files from node_modules will be imported into bundle. That's why the building process will be slow.

Let's import only local models but other import by require():

const webpack = require('webpack');

var base = require('./webpack.base.conf');

base.externals.push(
  function(context, request, callback) {
    if(request[0] == '.') {
      callback();
    } else {
      callback(null, "require('" + request + "')");
    }
  }
);

module.exports = base;

The hack is to use externals features. It takes ... and we are local, but doesn’t touch packages from node_modules and allows node.js to resolve it requires in runtime..

If you open file dist/app.js (the bundle produced by webpack) you will find lines like:

/***/ "express":
/*!*************************************!*\
  !*** external "require('express')" ***!
  \*************************************/
/*! no static exports found */
/***/ (function(module, exports) {

eval("module.exports = require('express');\n\n//# sourceURL=webpack:///external_%22require('express')%22?");

/***/ })

/******/ });

So, we don't have a problem with dynamic require because webpack doesn't go into express module, so it doesn't find any dynamic require. but the file test_module.js will be imported as usual:

/***/ "./test_module.js":
/*!************************!*\
  !*** ./test_module.js ***!
  \************************/
/*! exports provided: MODULE_CONST */
/***/ (function(module, __webpack_exports__, __webpack_require__) {

"use strict";
eval("__webpack_require__.r(__webpack_exports__);\n/* harmony export (binding) */ __webpack_require__.d(__webpack_exports__, \"MODULE_CONST\", function() { return MODULE_CONST; });\nconst MODULE_CONST = \"Hello from the test module\";\r\n\n\n//# sourceURL=webpack:///./test_module.js?");

/***/ }),

Production mode

Dynamic views problem

When you start to build this, you'll get the error:

WARNING in ../node_modules/express/lib/view.js
81:13-25 Critical dependency: the request of a dependency is an expression
 @ ../node_modules/express/lib/view.js
 @ ../node_modules/express/lib/application.js
 @ ../node_modules/express/lib/express.js
 @ ../node_modules/express/index.js
 @ ./app.ts

It means you have a problem in ../node_modules/express/lib/view.js, because there is:

// ...
  if (!opts.engines[this.ext]) {
    // load engine
    var mod = this.ext.substr(1)
    debug('require "%s"', mod)

    // default engine export
    var fn = require(mod).__express

    if (typeof fn !== 'function') {
      throw new Error('Module "' + mod + '" does not provide a view engine.')
    }

    opts.engines[this.ext] = fn
  }
// ...

so webpack can't resolve require(mod).__express because it's dynamic. It's made this way because express.js can't know which library you will use for rendering.

You can find something interesting in your build log:


 Asset      Size  Chunks             Chunk Names
app.js  1.05 MiB    main  [emitted]  main
Entrypoint main = app.js
[0] multi ./app.js 28 bytes {main} [built]
[./app.js] 281 bytes {main} [built]
[./node_modules/express/lib sync recursive] ./node_modules/express/lib sync 160 bytes {main} [built]
[./test_module.js] 59 bytes {main} [built]
[buffer] external "buffer" 42 bytes {main} [built]
[crypto] external "crypto" 42 bytes {main} [built]
[events] external "events" 42 bytes {main} [built]
[fs] external "fs" 42 bytes {main} [built]
[http] external "http" 42 bytes {main} [built]
[net] external "net" 42 bytes {main} [built]
[path] external "path" 42 bytes {main} [built]
[querystring] external "querystring" 42 bytes {main} [built]
[stream] external "stream" 42 bytes {main} [built]
[url] external "url" 42 bytes {main} [built]
[util] external "util" 42 bytes {main} [built]
    + 108 hidden modules

Notice [./node_modules/express/lib sync recursive] ./node_modules/express/lib sync 160 bytes {main} [built]. Looks unusual, right? Webpack tries to say that it's going to import all files in ./node_modules/express/lib into bundle.

If you don't really use the views feature then you will be able to make a bundle with the warning above and run dist/app.js.

Ok. So we compiled dist somehow. What Webpack does is substitutes

var fn = require(mod).__express

to

var fn = __webpack_require__("./node_modules/express/lib sync recursive")(mod).__express

We got a critical error, let's help Webpack to handle __webpack_require__("./node_modules/express/lib sync recursive") here.

This expression is a bit different, we expect context here.

Now let's add following to production build (webpack.prod.conf.js):

base.plugins = [
  new webpack.ContextReplacementPlugin(// we want to replace context
    /express\/lib/,                    // and replace all searches in
                                       // express/lib/*
    resolve('node_modules'),           // to look in folder 'node_modules'
    {                                  // and return a map
      'ejs': 'ejs'                     // which resolves request for 'ejs'
    }                                  // to module 'ejs'
  )                                    // __webpack_require__(...)(mod)
]                                      // we set `mod = 'ejs'`

Check out a nice post to learn more about ContextReplacementPlugin

EJS: Require.extension

After we included ejs modeuls in bundle, we get a warning:

WARNING in ./node_modules/ejs/lib/ejs.js 917:4-22
require.extensions is not supported by webpack. Use a loader instead.
 @ ./node_modules sync ejs
 @ ./node_modules/express/lib/view.js
 @ ./node_modules/express/lib/application.js
 @ ./node_modules/express/lib/express.js
 @ ./node_modules/express/index.js
 @ ./app.js
 @ multi ./app.js

It's a deprecated feature of node.js, which ejs tries to use. So, we will not try to do something require('template.ejs'), that we are fine. As Webpack, suggests, it's better to use loaders. So, it's our responsobility to deal with code like require('template.ejs').

We can say that we are aware of what we are dealing with and add the following warning supression into webpack.prod.conf.js:

base.stats = {
  warningsFilter: /require\.extensions/
}

Result

IAs the result, we have a node.js project which we can bundle into a single JavaScript file and send it to production to run. No need to care about node_modules and package.json. Only the right version on node.js is necessary. Yes, we also should care about assets and *.ejs files, but let's make it in another post.

Other

Do you like to write in node.js? Would you practice your skills by contributing to open-source? Checkout our node.js app. We used similar approach there but based on Koa.js. We wanted to avoid hacks by packing Express.js as described here.