Installation of all Node.js dependencies and building sources on the production host is painful. You might want to make a bundle on development host and deploy a single bundle on production.
The idea came after experiencing a 502 error on the main page of CorpGlory.com for 15 minutes, because of forgotten npm install in our
CI process.
Your production server should have as little software/files as possible. The post describes tricks and technics you can use to build similar node.js app. Maybe it's a better idea to use an HTTP framework ready for bundling with Webpack, but we will use Express.js here.
TL;DR: we pack in dev mode by hacking externals
and subsituting require() for node_modules.
In production mode, we hack dynamic import views with ContextReplacementPlugin
You can find sources on GitHub.
We use webpack v4.39.1 and express v4.17.1 here.
Lets list files we interested in:
express-webpack-example
|- /build // webpack configs folder
|- webpack.base.conf.js
|- webpack.dev.conf.js
|- webpack.prod.conf.js
|- /dist // destination folder with bundle
| // (available after running `npm run dev`)
|- app.js // target bundle
|- /node_modules // default node.js / npm modules folder
|- app.js // our express.js application
|- test_module.js // a test module for `require()`
|- template.ejs // a test .ejs template to render
|- package.json // default node.js / npm dependecy config
The idea is we don't want to rebuild the whole bundle each time - it's going to be slow... Why? Because Webpack bundles all modules required by the app. So, most of the files from
node_modules will be imported into bundle. That's why the building process will be slow.
Let's import only local models but other import by require():
const webpack = require('webpack');
var base = require('./webpack.base.conf');
base.externals.push(
function(context, request, callback) {
if(request[0] == '.') {
callback();
} else {
callback(null, "require('" + request + "')");
}
}
);
module.exports = base;
The hack is to use externals features. It takes ... and we are local, but doesn’t touch packages from node_modules and allows node.js to resolve it requires in runtime..
If you open file dist/app.js (the bundle produced by webpack)
you will find lines like:
/***/ "express":
/*!*************************************!*\
!*** external "require('express')" ***!
\*************************************/
/*! no static exports found */
/***/ (function(module, exports) {
eval("module.exports = require('express');\n\n//# sourceURL=webpack:///external_%22require('express')%22?");
/***/ })
/******/ });
So, we don't have a problem with dynamic require because webpack
doesn't go into express module, so it doesn't find any dynamic require.
but the file test_module.js will be imported as usual:
/***/ "./test_module.js":
/*!************************!*\
!*** ./test_module.js ***!
\************************/
/*! exports provided: MODULE_CONST */
/***/ (function(module, __webpack_exports__, __webpack_require__) {
"use strict";
eval("__webpack_require__.r(__webpack_exports__);\n/* harmony export (binding) */ __webpack_require__.d(__webpack_exports__, \"MODULE_CONST\", function() { return MODULE_CONST; });\nconst MODULE_CONST = \"Hello from the test module\";\r\n\n\n//# sourceURL=webpack:///./test_module.js?");
/***/ }),
When you start to build this, you'll get the error:
WARNING in ../node_modules/express/lib/view.js
81:13-25 Critical dependency: the request of a dependency is an expression
@ ../node_modules/express/lib/view.js
@ ../node_modules/express/lib/application.js
@ ../node_modules/express/lib/express.js
@ ../node_modules/express/index.js
@ ./app.ts
It means you have a problem in ../node_modules/express/lib/view.js, because
there is:
// ...
if (!opts.engines[this.ext]) {
// load engine
var mod = this.ext.substr(1)
debug('require "%s"', mod)
// default engine export
var fn = require(mod).__express
if (typeof fn !== 'function') {
throw new Error('Module "' + mod + '" does not provide a view engine.')
}
opts.engines[this.ext] = fn
}
// ...
so webpack can't resolve require(mod).__express because it's dynamic.
It's made this way because express.js can't know which library you will use for rendering.
You can find something interesting in your build log:
Asset Size Chunks Chunk Names
app.js 1.05 MiB main [emitted] main
Entrypoint main = app.js
[0] multi ./app.js 28 bytes {main} [built]
[./app.js] 281 bytes {main} [built]
[./node_modules/express/lib sync recursive] ./node_modules/express/lib sync 160 bytes {main} [built]
[./test_module.js] 59 bytes {main} [built]
[buffer] external "buffer" 42 bytes {main} [built]
[crypto] external "crypto" 42 bytes {main} [built]
[events] external "events" 42 bytes {main} [built]
[fs] external "fs" 42 bytes {main} [built]
[http] external "http" 42 bytes {main} [built]
[net] external "net" 42 bytes {main} [built]
[path] external "path" 42 bytes {main} [built]
[querystring] external "querystring" 42 bytes {main} [built]
[stream] external "stream" 42 bytes {main} [built]
[url] external "url" 42 bytes {main} [built]
[util] external "util" 42 bytes {main} [built]
+ 108 hidden modules
Notice [./node_modules/express/lib sync recursive] ./node_modules/express/lib sync 160 bytes {main} [built]. Looks unusual, right? Webpack tries to say that it's going to import all files in ./node_modules/express/lib into bundle.
If you don't really use the views feature then you will be able to make a bundle with the warning above and run dist/app.js.
Ok. So we compiled dist somehow. What Webpack does is substitutes
var fn = require(mod).__express
to
var fn = __webpack_require__("./node_modules/express/lib sync recursive")(mod).__express
We got a critical error, let's help Webpack to handle
__webpack_require__("./node_modules/express/lib sync recursive") here.
This expression is a bit different, we expect context here.
Now let's add following to production build (webpack.prod.conf.js):
base.plugins = [
new webpack.ContextReplacementPlugin(// we want to replace context
/express\/lib/, // and replace all searches in
// express/lib/*
resolve('node_modules'), // to look in folder 'node_modules'
{ // and return a map
'ejs': 'ejs' // which resolves request for 'ejs'
} // to module 'ejs'
) // __webpack_require__(...)(mod)
] // we set `mod = 'ejs'`
Check out a nice post to learn more about ContextReplacementPlugin
After we included ejs modeuls in bundle, we get a warning:
WARNING in ./node_modules/ejs/lib/ejs.js 917:4-22
require.extensions is not supported by webpack. Use a loader instead.
@ ./node_modules sync ejs
@ ./node_modules/express/lib/view.js
@ ./node_modules/express/lib/application.js
@ ./node_modules/express/lib/express.js
@ ./node_modules/express/index.js
@ ./app.js
@ multi ./app.js
It's a deprecated feature of node.js,
which ejs tries to use. So, we will not try to do something
require('template.ejs'), that we are fine. As Webpack, suggests, it's
better to use loaders. So, it's our responsobility to deal with code like require('template.ejs').
We can say that we are aware of what we are dealing with and add the following
warning supression into webpack.prod.conf.js:
base.stats = {
warningsFilter: /require\.extensions/
}
IAs the result, we have a node.js project which we can bundle into a single JavaScript file and send it to production to run. No need to care about node_modules and package.json.
Only the right version on node.js is necessary. Yes, we also should care about assets and *.ejs
files, but let's make it in another post.
Do you like to write in node.js? Would you practice your skills by contributing to open-source? Checkout our node.js app. We used similar approach there but based on Koa.js. We wanted to avoid hacks by packing Express.js as described here.
Like what we do? Check out services.