Bundling Node.js / Express apps with Webpack
Installation all of Node.js dependencies and build sources on production host could be painful. You might want to make a bundle on development host and deploy single bundle on production.
The idea came after experiencing 502 error on main page of CorpGlory.com for 15 minutes
because of fogrotten npm install in our
CI process.
Your production server should have as less software / files as possible. The post describes tricks and technics you can use to build simillar node.js app. Maybe it's a better idea to use a http framework ready for bundling with Webpack, but we will use Express.js here.
TL;DR: we pack in dev mode by hacking externals
and subsituting require() for node_modules.
In prod mode we hack dynimic import views with ContextReplacementPlugin
Test project
You can find sources on GitHub.
We use webpack v4.39.1 and express v4.17.1 here.
Lets list files we interested in:
express-webpack-example
|- /build // webpack configs folder
|- webpack.base.conf.js
|- webpack.dev.conf.js
|- webpack.prod.conf.js
|- /dist // destination folder with bundle
| // (available after running `npm run dev`)
|- app.js // target bundle
|- /node_modules // default node.js / npm modules folder
|- app.js // our express.js application
|- test_module.js // a test module for `require()`
|- template.ejs // a test .ejs template to render
|- package.json // default node.js / npm dependecy config
Development mode
Idea is that we don't want to rebuild whole bundle each time
because it's going to be slow...Why? Because Webpack bundles
all modules required by the app. So most of files from
node_modules will be imported into bundle. That's why building
process will be slow.
Let's import only local modeles but other import by require():
const webpack = require('webpack');
var base = require('./webpack.base.conf');
base.externals.push(
function(context, request, callback) {
if(request[0] == '.') {
callback();
} else {
callback(null, "require('" + request + "')");
}
}
);
module.exports = base;
The hack is to use externals feature. It takes ... and we local but don't touch packages from node_modules and allow node.js resolve it requires in runtime.
If you open file dist/app.js (the bundle produced by webpack)
you will find lines like:
/***/ "express":
/*!*************************************!*\
!*** external "require('express')" ***!
\*************************************/
/*! no static exports found */
/***/ (function(module, exports) {
eval("module.exports = require('express');\n\n//# sourceURL=webpack:///external_%22require('express')%22?");
/***/ })
/******/ });
So, we don't have problem with dynamic require because webpack
doesn't go into express module so doesn't find any dynamic require.
but file test_module.js will be imported as usual:
/***/ "./test_module.js":
/*!************************!*\
!*** ./test_module.js ***!
\************************/
/*! exports provided: MODULE_CONST */
/***/ (function(module, __webpack_exports__, __webpack_require__) {
"use strict";
eval("__webpack_require__.r(__webpack_exports__);\n/* harmony export (binding) */ __webpack_require__.d(__webpack_exports__, \"MODULE_CONST\", function() { return MODULE_CONST; });\nconst MODULE_CONST = \"Hello from the test module\";\r\n\n\n//# sourceURL=webpack:///./test_module.js?");
/***/ }),
Production mode
Dynamic views problem
When you start to build this, you will get error:
WARNING in ../node_modules/express/lib/view.js
81:13-25 Critical dependency: the request of a dependency is an expression
@ ../node_modules/express/lib/view.js
@ ../node_modules/express/lib/application.js
@ ../node_modules/express/lib/express.js
@ ../node_modules/express/index.js
@ ./app.ts
so there is a problem in ../node_modules/express/lib/view.js, because
there is:
// ...
if (!opts.engines[this.ext]) {
// load engine
var mod = this.ext.substr(1)
debug('require "%s"', mod)
// default engine export
var fn = require(mod).__express
if (typeof fn !== 'function') {
throw new Error('Module "' + mod + '" does not provide a view engine.')
}
opts.engines[this.ext] = fn
}
// ...
so webpack can't resolve require(mod).__express because it's dynamic.
It's made this way because express.js can't know which library you will use for rendering.
You can find something interesting in your build log:
Asset Size Chunks Chunk Names
app.js 1.05 MiB main [emitted] main
Entrypoint main = app.js
[0] multi ./app.js 28 bytes {main} [built]
[./app.js] 281 bytes {main} [built]
[./node_modules/express/lib sync recursive] ./node_modules/express/lib sync 160 bytes {main} [built]
[./test_module.js] 59 bytes {main} [built]
[buffer] external "buffer" 42 bytes {main} [built]
[crypto] external "crypto" 42 bytes {main} [built]
[events] external "events" 42 bytes {main} [built]
[fs] external "fs" 42 bytes {main} [built]
[http] external "http" 42 bytes {main} [built]
[net] external "net" 42 bytes {main} [built]
[path] external "path" 42 bytes {main} [built]
[querystring] external "querystring" 42 bytes {main} [built]
[stream] external "stream" 42 bytes {main} [built]
[url] external "url" 42 bytes {main} [built]
[util] external "util" 42 bytes {main} [built]
+ 108 hidden modules
Notice [./node_modules/express/lib sync recursive] ./node_modules/express/lib sync 160 bytes {main} [built]. Looks unusual, right? Webpack tryes to say that it's going to import all files in ./node_modules/express/lib into bundle.
If you don't really use views feature then you will be able to
make a bundle with the warning above and run dist/app.js.
Ok. So we compiled dist somehow. What Webpack does is substitutes
var fn = require(mod).__express
to
var fn = __webpack_require__("./node_modules/express/lib sync recursive")(mod).__express
We got a critical error, let's help Webpack to handle
__webpack_require__("./node_modules/express/lib sync recursive") here.
This expression is a bit different, we expect context here.
Now let's add following to production build (webpack.prod.conf.js):
base.plugins = [
new webpack.ContextReplacementPlugin(// we want to replace context
/express\/lib/, // and replace all searches in
// express/lib/*
resolve('node_modules'), // to look in folder 'node_modules'
{ // and return a map
'ejs': 'ejs' // which resolves request for 'ejs'
} // to module 'ejs'
) // __webpack_require__(...)(mod)
] // we set `mod = 'ejs'`
Check out a nice post to learn more about ContextReplacementPlugin
EJS: Require.extension
After we included ejs modeuls in bundle, we get warning:
WARNING in ./node_modules/ejs/lib/ejs.js 917:4-22
require.extensions is not supported by webpack. Use a loader instead.
@ ./node_modules sync ejs
@ ./node_modules/express/lib/view.js
@ ./node_modules/express/lib/application.js
@ ./node_modules/express/lib/express.js
@ ./node_modules/express/index.js
@ ./app.js
@ multi ./app.js
It's a deprecated feature of node.js,
which ejs tries to use. So it we will not try to do something
require('template.ejs'), that we are fine. As Webpack, suggests, it's
better to use loaders. So, it is
our responsobility to deal with code like require('template.ejs').
We can say that we aware what we are dealing with and add following
warning supression into webpack.prod.conf.js:
base.stats = {
warningsFilter: /require\.extensions/
}
Result
In result, we have a node.js project which we can bundle into signle javascript file
and send to to production to run. No need to care about node_modules and package.json.
Only right version on node.js is necessary. Yes, we also should care about assets and *.ejs
files, but let's make it in another post.
Other
Do you like o write in node.js? Would you practice your skills by contributing to open-source? Checkout our node.js app. We used similar approach there, but based on Koa.js. We wanted to avoid hacks with packing Express.js as described here.