Rafael G. Blanes
Español
← Back
Software Development7 min read

A Deep Dive into NodeJS 0.12

A practical experience with NodeJS 0.12: backend modules, testing, cloud services, task automation, and principles for organizing a professional application.

NodeJS

I was drawn to NodeJS from the start, when I first heard about server-side code written in JavaScript. After learning the basics a couple of years ago, integrating it with many modules, and nearly finishing a complete project with this framework—a project I’ve been working on over the past year—I can say that despite seeing both its pros and cons, it remains a wonderful platform for building scalable, maintainable applications.

Framework, platform, technology… the concepts get a little mixed up. NodeJS gives you the essential core or framework on which to build applications, along with an event-driven programming model and a set of basic but very important modules.

What truly brings a NodeJS project to life is the huge ecosystem of modules around it. They began appearing as soon as NodeJS arrived on the scene, and they’re still growing and evolving today.

This is typical and part of software’s natural evolution. Drupal, WordPress, and the freshly released ASP.NET 5 web application framework would not be what they are today without their constellations of modules that extend or improve their functionality.

But it’s not just a matter of choosing and integrating a set of modules. You need tools to manage your project’s configuration properly and put your defined workflows into practice, as well as continually applying sound development principles, clean code, and ongoing refactoring.

On this web project, which has a fairly substantial backend and which I’ve been working on for a year, I’ve had the chance to use NodeJS in depth: reading many books and posts and articles about best practices, and searching for and evaluating the most popular or mature modules to add all the necessary functionality.

Here’s a little about my experience, which has basically revolved around the recurring questions how do I do this? and how can I do this other thing even better?—and so on, until the project begins to reach a certain maturity, all the work starts to converge into something more tangible, and you begin to see a project that offers some real value. Or at least that’s what you hope.

I’ve taken a lean approach to the project: I’ll finish a clearly defined set of features, and then the project itself will be validated by users (the first ones are always your friends and family) to see whether it’s useful… whether to keep going (;-), pivot (;-|, or start something else entirely (;-(.

I find the lean approach fascinating because, in reality, you build experiments that the market then validates. If you don’t know what the lean project approach is, start by reading Eric Ries’s classic book.

As for the modules I’m using, I’ll group them by function.

At the application’s core, the main modules break down as follows:

  • Express as the framework for the essentials of a web application. It’s practically the standard in the NodeJS world and has a huge ecosystem of middleware for session management (express-session), handling request or post parameters (body-parser), cookies (cookie-parser), routing (handled directly by Express), security (CSRF protection with csurf, and so on. Although there are more flexible options, I used EJS as the templating tool. I also used compression to compress content sent with each request.
  • File compression with archiver.
  • Image handling with fabric.js (yes, it has a module that lets you use Fabric on the server side) as well as canvas.
  • Generating unique values and hashes with node-uuid, hashids (for elegantly generating short links), and md5 (for creating hash values from text strings).
  • A user-login framework with Passport; its extensions let you easily integrate login with Facebook, Google+, or Twitter.
  • Q as a library for using promises and avoiding nested asynchronous functions (callback hell), which make readable code difficult to write.
  • A client for using Redis as a cache object server and storing user sessions in it.
  • Search-index as a simple, useful way to index site content and enable searches.
  • The SendGrid client module for sending email through that mailer.
  • cron for running recurring site management and maintenance operations.
  • winston as an advanced logging library.
  • To write more readable, maintainable code, I’ve used Lodash extensively (any JavaScript developer should consider it essential, although there are similar alternatives).
  • html-minifier to minify content as it’s sent with each request.

I didn’t choose all these modules at random. I tested, evaluated, and experimented with them, read many blogs, and looked at what the community uses most often.

And what about testing? No software project can be considered mature and professional without tests to back it up.

For that, naturally, I used Chai as the assertion library and Mocha to run the tests. The results and my experience with both libraries have been—and continue to be—excellent.

The site manages many different types of files (no, it’s not just another page of links and downloads…), so after evaluating various cloud options such as CDNs, I chose Amazon AWS and its aws-sdk package to integrate its REST API with NodeJS.

The site also relies on a MySQL database, so I used the mysql client to integrate it with the application.

Application development isn’t just about making the application work, with more or less test coverage. You also need to define workflows for generating the project assets that will eventually be deployed to production.

For that, of course, I used Grunt and many basic modules to build the deployable application. Their names give away their purpose, but here are the most important ones:

Whew! More than one person coming to NodeJS for the first time will think that’s a lot of external modules, libraries, and frameworks to use to build a complete, professional application. It is—and that’s exactly the good part: because everything is so modular, you can choose only what suits you best or what you like and know. In the end, you have a production application with no overhead from things it doesn’t use, and it’s lightweight because it has neither more nor less than it needs.

However, more important than all of the above—including using and choosing the right modules—is working hard to keep the distribution and organization of all the modules and assets in your project as simple and readable as possible. Very good projects with very poor organization end up becoming a headache because their assets aren’t organized properly.

Everyone eventually develops their own guidelines, based on how others do things. I’ve followed these rules:

  • Most of the modules I’ve implemented in the solution are very simple and short, and each provides a focused, coherent feature. Many emerged through continuous refactoring. The simpler and more focused they are, the easier they are to test.
  • The frontend assets (based on AngularJS) are completely separate from the backend. Yes, it’s obvious, but I have to say it…
  • I’ve defined a consistent way of naming test files in relation to the modules they test, so they’re easy to find.
  • I’ve used my own dependency-injection mechanism.
  • All modules related to system maintenance are clearly separated.

You don’t have to do things this way just because it’s a NodeJS project; this is simply an application of principles that should be present in any software project, as I mentioned some time ago in this post and discuss at length in The Programmer’s Black Book.

What’s more, especially at the beginning, you realize that the newer the technology or environment is to you, the more you need to refactor continuously. As you learn more and explore it in depth, you discover ways to improve everything you’ve done so far.

I’m very happy with this new project, which I’ve been advancing through microtasks on nights, weekends, and vacation days. As I often say, you can only learn something by doing it, not by constantly theorizing and reading about it (learning by doing).