Bundling Node.js / Express apps with Webpack
Installation of all Node.js dependencies and building sources on the production host is painful. You might want to make a bundle on development host and deploy a single bundle on production.
The idea came after experiencing a 502 error on the main page of CorpGlory.com for 15 minutes, because of forgotten npm install
in our
CI process.
Your production server should have as little software/files as possible. The post describes tricks and technics you can use to build similar node.js app. Maybe it's a better idea to use an HTTP framework ready for bundling with Webpack, but we will use Express.js here.
TL;DR: we pack in dev mode by hacking externals
and subsituting require()
for node_modules
.
In production mode, we hack dynamic import views with ContextReplacementPlugin
Test project ¶
You can find sources on GitHub.
We use webpack v4.39.1
and express v4.17.1
here.
Lets list files we interested in:
express-webpack-example
|- /build // webpack configs folder
|- webpack.base.conf.js
|- webpack.dev.conf.js
|- webpack.prod.conf.js
|- /dist // destination folder with bundle
| // (available after running `npm run dev`)
|- app.js // target bundle
|- /node_modules // default node.js / npm modules folder
|- app.js // our express.js application
|- test_module.js // a test module for `require()`
|- template.ejs // a test .ejs template to render
|- package.json // default node.js / npm dependecy config
Development mode ¶
The idea is we don't want to rebuild the whole bundle each time - it's going to be slow... Why? Because Webpack bundles all modules required by the app. So, most of the files from
node_modules
will be imported into bundle. That's why the building process will be slow.
Let's import only local models but other import by require()
:
const webpack = require('webpack');
var base = require('./webpack.base.conf');
base.externals.push(
function(context, request, callback) {
if(request[0] == '.') {
callback();
} else {
callback(null, "require('" + request + "')");
}
}
);
module.exports = base;
The hack is to use externals features. It takes ... and we are local, but doesn’t touch packages from node_modules and allows node.js to resolve it requires in runtime..
If you open file dist/app.js
(the bundle produced by webpack)
you will find lines like:
/***/ "express":
/*!*************************************!*\
!*** external "require('express')" ***!
\*************************************/
/*! no static exports found */
/***/ (function(module, exports) {
eval("module.exports = require('express');\n\n//# sourceURL=webpack:///external_%22require('express')%22?");
/***/ })
/******/ });
So, we don't have a problem with dynamic require
because webpack
doesn't go into express
module, so it doesn't find any dynamic require
.
but the file test_module.js
will be imported as usual:
/***/ "./test_module.js":
/*!************************!*\
!*** ./test_module.js ***!
\************************/
/*! exports provided: MODULE_CONST */
/***/ (function(module, __webpack_exports__, __webpack_require__) {
"use strict";
eval("__webpack_require__.r(__webpack_exports__);\n/* harmony export (binding) */ __webpack_require__.d(__webpack_exports__, \"MODULE_CONST\", function() { return MODULE_CONST; });\nconst MODULE_CONST = \"Hello from the test module\";\r\n\n\n//# sourceURL=webpack:///./test_module.js?");
/***/ }),
Production mode ¶
Dynamic views problem ¶
When you start to build this, you'll get the error:
WARNING in ../node_modules/express/lib/view.js
81:13-25 Critical dependency: the request of a dependency is an expression
@ ../node_modules/express/lib/view.js
@ ../node_modules/express/lib/application.js
@ ../node_modules/express/lib/express.js
@ ../node_modules/express/index.js
@ ./app.ts
It means you have a problem in ../node_modules/express/lib/view.js
, because
there is:
// ...
if (!opts.engines[this.ext]) {
// load engine
var mod = this.ext.substr(1)
debug('require "%s"', mod)
// default engine export
var fn = require(mod).__express
if (typeof fn !== 'function') {
throw new Error('Module "' + mod + '" does not provide a view engine.')
}
opts.engines[this.ext] = fn
}
// ...
so webpack can't resolve require(mod).__express
because it's dynamic.
It's made this way because express.js can't know which library you will use for rendering.
You can find something interesting in your build log:
Asset Size Chunks Chunk Names
app.js 1.05 MiB main [emitted] main
Entrypoint main = app.js
[0] multi ./app.js 28 bytes {main} [built]
[./app.js] 281 bytes {main} [built]
[./node_modules/express/lib sync recursive] ./node_modules/express/lib sync 160 bytes {main} [built]
[./test_module.js] 59 bytes {main} [built]
[buffer] external "buffer" 42 bytes {main} [built]
[crypto] external "crypto" 42 bytes {main} [built]
[events] external "events" 42 bytes {main} [built]
[fs] external "fs" 42 bytes {main} [built]
[http] external "http" 42 bytes {main} [built]
[net] external "net" 42 bytes {main} [built]
[path] external "path" 42 bytes {main} [built]
[querystring] external "querystring" 42 bytes {main} [built]
[stream] external "stream" 42 bytes {main} [built]
[url] external "url" 42 bytes {main} [built]
[util] external "util" 42 bytes {main} [built]
+ 108 hidden modules
Notice [./node_modules/express/lib sync recursive] ./node_modules/express/lib sync 160 bytes {main} [built]
. Looks unusual, right? Webpack tries to say that it's going to import all files in ./node_modules/express/lib
into bundle.
If you don't really use the views feature then you will be able to make a bundle with the warning above and run dist/app.js
.
Ok. So we compiled dist somehow. What Webpack does is substitutes
var fn = require(mod).__express
to
var fn = __webpack_require__("./node_modules/express/lib sync recursive")(mod).__express
We got a critical error, let's help Webpack to handle
__webpack_require__("./node_modules/express/lib sync recursive")
here.
This expression is a bit different, we expect context here.
Now let's add following to production build (webpack.prod.conf.js
):
base.plugins = [
new webpack.ContextReplacementPlugin(// we want to replace context
/express\/lib/, // and replace all searches in
// express/lib/*
resolve('node_modules'), // to look in folder 'node_modules'
{ // and return a map
'ejs': 'ejs' // which resolves request for 'ejs'
} // to module 'ejs'
) // __webpack_require__(...)(mod)
] // we set `mod = 'ejs'`
Check out a nice post to learn more about ContextReplacementPlugin
EJS: Require.extension ¶
After we included ejs modeuls in bundle, we get a warning:
WARNING in ./node_modules/ejs/lib/ejs.js 917:4-22
require.extensions is not supported by webpack. Use a loader instead.
@ ./node_modules sync ejs
@ ./node_modules/express/lib/view.js
@ ./node_modules/express/lib/application.js
@ ./node_modules/express/lib/express.js
@ ./node_modules/express/index.js
@ ./app.js
@ multi ./app.js
It's a deprecated feature of node.js,
which ejs tries to use. So, we will not try to do something
require('template.ejs')
, that we are fine. As Webpack, suggests, it's
better to use loaders. So, it's our responsobility to deal with code like require('template.ejs')
.
We can say that we are aware of what we are dealing with and add the following
warning supression into webpack.prod.conf.js
:
base.stats = {
warningsFilter: /require\.extensions/
}
Result ¶
IAs the result, we have a node.js project which we can bundle into a single JavaScript file and send it to production to run. No need to care about node_modules
and package.json
.
Only the right version on node.js is necessary. Yes, we also should care about assets and *.ejs
files, but let's make it in another post.
Other ¶
Do you like to write in node.js? Would you practice your skills by contributing to open-source? Checkout our node.js app. We used similar approach there but based on Koa.js. We wanted to avoid hacks by packing Express.js as described here.