Let Gulp Take the Tedium out of Tasks

JavaScript automation

If you’ve spent any time writing JavaScript in the last couple of years, you’ve likely at least heard of Gulp.js. Smashing Magazine has an excellent introduction to Gulp that I strongly recommend reading if you are new to this build tool.

Here, I will describe the latest Gulp file I built for a MEAN stack app, explaining and justifying each section of the code. Originally our team simply wanted to automate the process of linting, minifying, and concatenating our client-side code and assets. Because we wanted to store the built files in a separate public folder, we also wanted to clean out this folder before each complete build.

In the Beginning

We first require the following into our gulpfile.js:

var gulp = require('gulp'),
  
  //to lint our code
  jshint = require('gulp-jshint'), 

  //to clean the public folder
  del = require('del'),

  //to minify our javascript
  uglify = require('gulp-uglify'),

  //to concatenate files together
  concat = require('gulp-concat'),

  //to minify css
  cssmin = require('gulp-cssmin'),

  //to minify images
  imagemin = require('gulp-imagemin'),
  pngquant = require('imagemin-pngquant');

Next, we install everything via npm and save to the dev-dependencies of our package.json file:

npm install --save-dev gulp gulp-jshint gulp-uglify gulp-concat gulp-cssmin gulp-imagemin gulp-imagemin-pngquant

We also created an object by which to refer to common file paths:

var paths = {
  scripts: ['./client/app/**/*.js','./server/**/*.js', './index.js','./spec/**/*.js','./gulpfile.js'],
  clientScripts: ['./client/app/**/*.js', '!./client/app/spec/**/*.js'],
  styles: ['./client/assets/**/*.css'],
  partials: ['client/app/**/*.html'],
  images: ['client/assets/**/*.png', 'client/assets/**/*.jpg', 'client/assets/**/*.jpeg', 'client/assets/**/*.gif', 'client/assets/**/*.svg', 'client/**/*.ico'],
  backendTests: ['specs/server/**/*.js']
};

And now the main code:

// JSHint task
gulp.task('lint', function() {
  gulp.src(paths.scripts)
  .pipe(jshint())
  .pipe(jshint.reporter('default'));
});

// Clean task — cleans the contents of the public folder
gulp.task('clean', function (callback) {
  del([
    'public/**/*'
  ], callback);
  
});

//Concatentate and minify client-side js files
gulp.task('js', function () {
  gulp.src(paths.clientScripts)
    .pipe(concat('app.min.js'))
    .pipe(uglify())
    .pipe(gulp.dest('public/'));
});

//minify html files
gulp.task('html', function() {
  gulp.src(paths.partials)
  .pipe(minifyHTML())
  .pipe(gulp.dest('public/'));
});

// Images task — copy all images to public folder and minify
gulp.task('images', function() {
  gulp.src('client/assets/favicon.ico')
  .pipe(gulp.dest('public/assets/'));
  
  gulp.src(paths.images)
  .pipe(imagemin({
      progressive: true,
      svgoPlugins: [{removeViewBox: false}],
      use: [pngquant()]
  }))
  .pipe(gulp.dest('public/assets/'));
});

// Copy and minify css files from the client folder to public/assets folder
gulp.task('styles', function() {
  gulp.src(paths.styles)
  .pipe(autoprefixer("last 2 versions", "> 1%", "ie 8"))
  .pipe(cssmin())
  .pipe(concat('styles.min.css'))
  .pipe(gulp.dest('public/assets/'));
});

//Put it all together
gulp.task('production', ['clean', 'lint', 'js', 'html', 'styles', 'images']);

Unfortunately, this simple build task did not work quite as we had hoped. Firstly, some of our Angular dependencies were not injected correctly after concatenating and minifying the javascript files. To fix this, we used a plugin called gulp-ng-annotate. We also realized that all of the production tasks were running synchronously, resulting in some generated files being cleaned out. We used run-sequence to deal with this issue; notice we had to return something from each gulp task for this to work.

npm install --save-dev gulp-ng-annotate run-sequence

//add additional requirements
  ngAnnotate = require('gulp-ng-annotate'),
  runSequence = require('run-sequence');

//add return statements to all build functions
gulp.task('clean', function (callback) {
  return del([
    'public/**/*'
  ], callback);
});

gulp.task('js', function () {
  return gulp.src(paths.clientScripts)
    .pipe(concat('app.min.js'))
    //to ensure Angular dependencies are injected correctly
    .pipe(ngAnnotate())
    .pipe(uglify())
    .pipe(gulp.dest('public/'));
});

gulp.task('html', function() {
  return gulp.src(paths.partials)
  .pipe(minifyHTML())
  .pipe(gulp.dest('public/'));
});

gulp.task('images', function() {
  gulp.src('client/assets/favicon.ico')
  .pipe(gulp.dest('public/assets/'));
  
  return gulp.src(paths.images)
  .pipe(imagemin({
      progressive: true,
      svgoPlugins: [{removeViewBox: false}],
      use: [pngquant()]
  }))
  .pipe(gulp.dest('public/assets/'));
});

gulp.task('styles', function() {
  return gulp.src(paths.styles)
  .pipe(autoprefixer("last 2 versions", "> 1%", "ie 8"))
  .pipe(cssmin())
  .pipe(concat('styles.min.css'))
  .pipe(gulp.dest('public/assets/'));
});

//Refactored production task that uses runSequence
gulp.task('production', function(callback){
  //Any tasks in brackets are run synchronously with each other, but wait for all tasks prior
  runSequence('clean',
    'lint', 
    ['js', 'html', 'styles', 'images'],
    callback); 
});

Build Tasks 2.0

After making those minor fixes, we realized we needed to change the source paths for any script files or other dependencies, as well as the base href (for using Angular’s html5 mode) in our index.html file. We began by replacing any local library dependencies with those hosted on a cdn. We then used gulp-html-replace to update the index.html with the correct source paths in the 'html' task. We also had to add a few key comments to the index.html file.

npm install --save-dev gulp-html-replace

//require statements
  htmlreplace = require('gulp-html-replace');

  // Update index.html to use built js and css files and use correct base href; 
  //minify html files
  gulp.task('html', function() {
    return gulp.src(paths.partials)
    .pipe(htmlreplace({
        'css': 'assets/styles.min.css',
        'js': 'app.min.js', 
        'base': '<base href="/">'
    }))
    .pipe(minifyHTML())
    .pipe(gulp.dest('public/'));
  });

The gulp-html-replace plugin searches html files for comments in the form of:

<!-- build:[name] -->
<unbuilt html>Stuff</endtag>
<!-- endbuilt -->

It then replaces anything from within the comments with the text specified in the guilp task. Here is what ours looked like:

<!-- build:base-->
<base href="/app/">
<!-- endbuild -->

<!-- build:css -->
<link rel="stylesheet" href="../assets/styles.css">
<!-- endbuild -->

<!-- build:js -->
<script src="app.js"></script>
<script src="user/loginController.js"></script>
<script src="ui/menuController.js"></script>
<script src="lists/listController.js"></script>
<script src="lists/pantryController.js"></script>
<script src="general/landingController.js"></script>
<script src="general/seasonalFactory.js"></script>
<script src="household/householdController.js"></script>
<script src="recipes/recipeController.js"></script>
<!-- endbuild -->

Notice the build names in the html match the keys in the object passed to html replace. The values in that object are what replace the source paths for the css and script tags. Note that the entire base tag is replaced.

The Real Fun

So far, we have a convenient little build task that concatenates, minifies, and replaces code for us. But we wanted to do more. For example, to view our app in the browser, we first had to start the server with nodemon, start mongod to access our database, and open the browser with the correct address. All of this can be automated with Gulp.

npm install --save-dev child_process gulp-nodemon gulp-open

//add to our requirements
  exec = require('child_process').exec,
  nodemon = require('gulp-nodemon'),
  open = require('gulp-open');

//Set up function to allow running scripts
function runCommand(command) {
  return function (cb) {
    exec(command, function (err, stdout, stderr) {
      console.log(stdout);
      console.log(stderr);
      cb(err);
    });
  };
}

//Start mongodb
gulp.task('mongo', runCommand('mongod'));

//Start nodemon
gulp.task('nodemon', function () {
  nodemon({ script: 'index.js' })
    .on('restart', function () {
      console.log('restarted!');
    });
});

//Open the correct url for the app
gulp.task('open-dev', function(){
  //use setTimeout to decrease chance of opening app
  //before mongod and server are up and running
  setTimeout(function(){
    gulp.src(__filename)
    .pipe(open({uri: 'http://localhost:1337/app/'}));
  }, 1500);
});

//put it all together--toss in the "watch" task for good measure
gulp.task('run-app', ['mongo','nodemon','watch', 'open-dev'],function(){
});

A quick note about mongod

If you currently have to run sudo mongod, this won’t work for you. You need to get that situation figured out, because you really shouldn’t need to use sudo for this task anyway.

Additionally, when you are done and want to stop the nodemon and mongod processes, you should be able to simply use ^C. However sometimes this will not stop mongod if you use this gulp task. You’ll know, because the next time you try to start mongod it will give you angry messages. To solve this, you’ll first need to find the mongod process id and then kill it:

#retrieve the id
ps ax | grep mongo

#kill the process
kill [id]

The Cherry On Top

We can now run gulp production to compile everything and gulp run-app to view our app in the browser with the unbuilt files…but sometimes it might be necessary to view the app from the built files–especially in the beginning when you want to make sure the build worked. However, this requires serving these files instead of the unbuilt versions as if we were running in the production environment. Easy enough—we just create a task that changes the value of process.env.NODE_ENV to fit our needs. The nice thing about this is that we don’t even need to install any additional plugins!

//Set environment variables for production or development
gulp.task('set-dev-node-env', function() {
    process.env.NODE_ENV = 'development';
});

gulp.task('set-prod-node-env', function() {
    process.env.NODE_ENV = 'production';
});

//Open the correct url for production environment
gulp.task('open-prod', function(){
  setTimeout(function(){
    gulp.src(__filename)
    .pipe(open({uri: 'http://localhost:1337/'}));
  }, 1500);
});

//Run development environment
gulp.task('run-dev', ['set-dev-node-env','mongo','nodemon','watch', 'open-dev'],function(){
});

//Run production environment
gulp.task('run-prod', ['set-prod-node-env', 'production', 'mongo','nodemon', 'watch', 'open-prod']);

And that’s it! We now have a set of gulp tasks that let us quickly and easily switch between our production and development environments while linting our code! See the complete file here.

Written on September 20, 2015