We want you to be MEAN ! (Part 2)

In this second installment on the MEAN stack,  we are going to complete an integration between a service layer module written in NodeJS, and MongoDB as our document persistence.  We will be using the NodeJS Mongoose module to integrate NodeJS with MongoDB, and we will also be writing some integration tests using Mocha.  (You do write tests for all your code?  Don’t you?)

Scenario

The application model is simple.   Users can perform basic CRUD operations on entries in a notes database.  Each note contains a name of the author,  the date of the entry, and the actual contents of the note.  The interface to the service layer will be note-like operations,  and the service layer implementation will do the actual integration with the data store. (MongoDB.)  In this post, we will NOT be looking further up the stack than the service layer interface.   That will be saved for the next post!

Our service layer functionality will support the following:

  1. save – The caller specifies his/her name and the note text.  The date will be supplied by the service layer, before saving to the data store.
  2. findAll – find all notes in the data store.
  3. findByDate –  find all notes in the data store within a date range.
  4. findByName – find all notes authored by a specified name.
  5. findByNameAndDate – find all notes authored by a specified name within a date range.
  6. removeAll –  remove all notes from the data store.
  7. removeByName – remove all notes authored by a specified name.

Setup

Before we start, complete the following setup steps.  See the previous post in this series for help if necessary.

  1. Verify that your instance of MongoDB is running.
  2. Create an ExpressJs project called ‘notes’ using express-generator.
  3. Use npm to install Mongoose locally in the above project.  (npm install mongoose –save).  Mongoose is middleware that our service code will use to interact with the Mongo database.  The –save option will automatically update your package.json to include a dependency for the mogoose version you are installing.  This saves you the trouble of having to independently edit package.json after you install the dependency.
  4. Use npm to install Mocha locally in the above project.  (npm install mocha –save-dev).  Mocha is the test framework we will use to test our service code.  The –save-dev option will automatically update your package.json to include a development dependency for the mocha  version you are installing.  This saves you the trouble of having to independently edit package.json after you install the development dependency.  A development dependency (like mocha) differs from a runtime dependency in that it is NOT required for others to download and run your application.  (Hence it is a development only dependency.)  You can supress the installation of development dependencies with the command npm install –production.
  5. Create a directory ‘services’ in your project.  Your service layer code will be placed here.
  6. Create a directory ‘test’ in your project.  Your integration tests will be placed here.  When you run Mocha tests,  a directory by this name is the default location for the test cases.
  7. Create a directory ‘conf’ in your project.  Configuration files will live here.  In this project, the URI for the MongoDB is the only configuration setting.

Your project structure should look something like the following after the setup steps are completed.

Screen Shot 2015-12-21 at 10.27.53 AM

Application Code

The app code we will add at this point is contained in 3 modules (files).  Three may be a bit of overkill for the application we are writing, but it is a good way of demonstrating some of the features of NodeJS.

  • config.js – the configuration module
  • db.js – the MongoDB management module
  • noteservice.js – the actual service implementation

config.js

Create a file in the conf directory, called config.js, with the configuration data.

'use strict';
var config = {
    db : {
        url: 'mongodb://localhost/notes'
    }
}
module.exports = config;

 

‘use strict’ is a declaration indicating that the code should be executed in strict mode. Strict mode prohibits many constructs that you could “get away with” in earlier versions of javascript, such as using undeclared variables.  Good practice in NodeJS is to begin all modules with ‘use strict’.

The url for the mongodb uses the default port of 27017, and does not specify a username/password for the database notes.  More details on the MongoDB connection uri can be found here.

NodeJS uses a module structure in which modules ‘export’ objects that are visible to other modules.   The other modules then ‘require’ the external module.  In the module above, we are exporting the config object to external modules.  We will see how the requiring side handles things in the next section.

 db.js

Create a file in the services directory, called db.js.  This module manages the connection to our MongoDB instance.

'use strict';
var mongoose = require('mongoose');
var dbconfig = require('../conf/config').db;
var db = {
    connect : function() {
        mongoose.connect (dbconfig.url);
    },
    disconnect :function() {
        mongoose.disconnect();
    }
}
module.exports=db

We are ‘requiring’ 2 external modules here.  The first module, mongoose, is installed in our node_modules directory.  Therefore, the module name may be specified without a path.  The second module, config, is a module in our project structure.   Therefore, the module name is specified relative to this (db.js) file. Note that the .js suffix is not specified in the require string, as it is optional.

The return value from the require function is what was ‘exported’ from the module being required.   In the second case, you recall that we exported the ‘config’ object, but note that we are setting the variable dbconfig to the db member of the config object.

The local db object, specifies a connect and disconnect function which uses mongoose to manage the connection.  Note that dbconfig.url (from the config.js module) specifies the MongoDB URI.

Finally, this modules export the db object containing the connect and disconnect functions.

noteservice.js

Create a file in the services directory called noteservice.js. This is the module that is the service layer implementation.

 

'use strict';
var mongoose = require('mongoose');
var Note = null;
var noteservice = {
    private: {
        init: function () {
            var schema = {
                name: {type: String, required: true},
                date: {type: Date, required: true},
                message: {type: String, required: true}
            }
            // not permissible to recompile an existing schema
            try {
                Note = mongoose.model('Note');
            } catch (ex) {
                Note = mongoose.model('Note', new mongoose.Schema(schema));
            }
        }
    },
    public: {
        save: function (name, message, date, callback) {
            var useDate = date || new Date();
            var json = {'name': name, date: useDate, 'message': message};
            var doc = new Note(json);
            doc.save(callback);
        },
        findByName: function (name, callback) {
            Note.find({'name': name}, callback);
        },
        findByNameAndDate: function (name, date, callback) {
            Note.find({'name': name, date:{$gte: startDate, $lte: endDate }}, callback);
        },
        findByDate: function (startDate, endDate, callback) {
            Note.find({date: {$gte: startDate, $lte: endDate }}, callback);
        },
        findAll: function (callback) {
            Note.find(callback);
        },
        removeAll: function (callback) {
            Note.remove({},callback);
        },
        removeByName: function(name,callback) {
            Note.remove({'name':name },callback);
        }
    }
}
noteservice.private.init();
module.exports = noteservice.public;

 

Looking at the last couple of lines of this module, you see that it calls the init function, which is part of the private object within the noteservice object.  It then exports the public object, which contains the methods available to clients of the noteservice.  (The point being that the init function is not available externally to the noteservice module.)

The init function prepares a mongoose schema definition for note entries in the database.  If you take a look at the schema variable, you will see it define a schema that has three members:

  • name, of type String
  • date, of type Date
  • message, of type String.

All 3 members are required (can not be undefined or null).

The schema is then compiled and saved by the model function, and given the identifier name ‘Note’.  Once compile, the Note object can be used for a variety of CRUD operations as defined by functions within the public object.

The public object functions are:

  • save
  • findByName
  • findByNameAndDate
  • findByDate
  • findAll
  • removeAll
  • removeByName

 

This is a good time to be aware of an EXTREMELY important architectural concept of NodeJS (if you aren’t already).  NodeJS is implemented as a single threaded, asynchronous I/O, event loop.  Compare this to a traditional  implementation using multi-threaded, blocking I/O.  That’s a mouthful!  Entire books have been dedicated to considering the pros and cons of this architectural approach, and we are not going to go there now!  But this is the right time to point out this important behavior of NodeJS:   Each method above completes immediately, and  the thread of control immediately returns to the caller.   The callback parameter  passed to the service function, is a function that will be invoked by NodeJS, when the operation requested by the function completes (at some point in the future.)  If you are familiar with the concept of promises and/or futures in Java, this is quite similar.  So, these service methods, do NOT block while the I/O (such as I/O to MongoDB!) is completing.

Let’s dissect this a bit more for the findAll method:

 

findAll: function (callback) {
        Note.find(callback);

        }

When a client calls findAll,  it passes a callback function to the findAll function. The findAll function calls the find function for the Note object and passes that same callback function to find. The thread of control then merrily proceeds to return from findAll to the client, and the client will do whatever it does next. The find function will asynchronously communicate with MongoDB and get the results of the query – at which time the callback function will be called with the results.  In this post, our client code will be an integration test written in Mocha.  In a future post, our client code will be a RESTful services layer written in NodeJS!

 

Test Code

Let’s write our test code now using a NodeJS test framework called Mocha.  We already installed Mocha in the setup above, so let’s have at it.

Create a file in the test directory called noteservicetest.js.

'use strict';
var noteservice = require('../services/noteservice');
var assert = require('assert');
var db = require('../services/db');
describe('noteservice tests', function () {
    before(function(done) {
        db.connect();
        noteservice.removeAll(function(){
            done();
        })
    })
    after(function() {
        db.disconnect();
    })
    it('create 2 notes from Cindy on the 9th and 11th', function (done) {
        var morningDate = new Date(2015,11,9);
        var eveningDate = new Date(2015,11,11);
        noteservice.save('Cindy', 'good morning', morningDate, function (err, result) {
            if (err) {
                throw err;
            }
            assert.equal(result.name, 'Cindy');
            assert.equal(result.message, 'good morning');
            assert.equal(result.date.getTime(), morningDate.getTime());
            noteservice.save('Cindy', 'good evening', eveningDate, function(err,result) {
                if (err) {
                    throw err;
                }
                assert.equal(result.name, 'Cindy');
                assert.equal(result.message, 'good evening');
                assert.equal(result.date.getTime(), eveningDate.getTime());
                done();
            })
        });
    })
    it('create a note from Paul on the 10th', function (done) {
        var date = new Date(2015,11,10);
        noteservice.save('Paul', 'have a nice day', date, function (err, result) {
            if (err) {
                throw err;
            }
            assert.equal(result.name, 'Paul');
            assert.equal(result.message, 'have a nice day');
            assert.equal(result.date.getTime(), date.getTime());
            done();
        });
    });
    it('expect an exception when message is null', function (done) {
        noteservice.save("Ryan", null,null, function (err, result) {
            if (err) {
                done();
            }
            else {
                throw new Error('An excpetion was expected');
            }
        });
    });
    it('expect to find 2 notes from Cindy', function (done) {
        noteservice.findByName('Cindy', function (err, result) {
            if (err) {
                throw err;
            }
            assert.equal(result.length, 2);
            done();
        });
    });
    it('expect to find 0 notes from Mark', function (done) {
        noteservice.findByName('Mark', function (err, result) {
            if (err) {
                throw err;
            }
            assert.equal(result.length, 0);
            done();
        })
    })
    it('expect to find 3 total notes', function (done) {
        noteservice.findAll(function (err, result) {
            if (err) {
                throw err;
            }
            assert.equal(result.length, 3);
            done();
        });
    });
    it('expect to find 0 total notes between the 1st and 3rd', function (done) {
        noteservice.findByDate(new Date(2015,11,1), new Date(2015,11,3), function (err, result) {
            if (err) {
                throw err;
            }
            assert.equal(result.length, 0);
            done();
        });
    });
    it('expect to find 2 total notes between the 10th and 13th', function (done) {
        noteservice.findByDate(new Date(2015,11,10), new Date(2015,11,13), function (err, result) {
            if (err) {
                throw err;
            }
            assert.equal(result.length, 2);
            done();
        });
    });
    it('expect to find 1  note for Cindy on the 9th', function (done) {
        noteservice.findByNameAndDate('Cindy', new Date(2015,11,9), new Date(2015,11,9), function (err, result) {
            if (err) {
                throw err;
            }
            assert.equal(result.length, 1);
            assert.equal(result[0].name, 'Cindy');
            assert.equal(result[0].message, 'good morning');
            assert.equal(result[0].date.getTime(), new Date(2015,11,9).getTime());
            done();
        });
    });
    it('expect only notes for Cindy, after Pauls notes are removed', function (done) {
        noteservice.removeByName('Paul', function (err, result) {
            if (err) {
                throw err;
            }
            noteservice.findAll(function (err, result) {
                if (err) {
                    throw err;
                }
                assert.equal(result.length, 2);
                assert.equal(result[0].name, 'Cindy');
                assert.equal(result[1].name, 'Cindy');
                done();
            })
        });
    });
})

 

You can execute the test module with the following line from the project home directory:

mocha

If you want your test to run each time you make a change in the source code, without having to re-type the above command, give this a try:

mocha -w

At the beginning of the test module,  we require three libraries.  Two of them are code we described in the previous section.

  • noteservice – the module under test.
  • db – a utility module to manage MongoDB connections
  • assert – an assert library to use in our test module.  There are several good assertion libraries that work with Mocha as well as other javascript testing frameworks.  For example, some (like Chai) support BDD/TDD assertions.  In the above example, we will use the basic assert library that is part of the NodeJS distribution.  In the next post, we will write test cases using the Chai library for assertions.

Before we consider an individual test, let’s look at the structure of the test module.   describe is used to group a collection of test cases together.  It’s first argument is a descriptive string of what the collection is testing.  it is used to define a single test case.  it as well has a descriptive string about the test case as its first argument.

In addition, Mocha  has hooks that you can call to do set-up and tear-down type operations.  before and after are used to do ‘global’ set-up and tear-down. before executes before any tests in the describe block, and after executes after all tests in the describe block complete.  In the above tests module, we use before to connect to MongoDB and remove any content from the ‘Note’ collection under test.  after is used to disconnect from MongoDB at the conclusion of all the tests.   Mocha also supports a beforeEach and afterEach function to include set-up and tear-down that applies before and after each individual test case.

Take a look at the 2nd argument to any of the test cases.  It is a function with a single parameter done,  which is a callback function.  The test author should callback to done, when the test case is completed.  This callback tells Mocha that the asynchronous portion of the test case has fully completed, and it can move on to the next test case.   Of course, done would not be required if the code under test is synchronous, or you specifically intend to run additional test cases before the asynchronous completion of a pending test case.

Coming Attractions

In this post, we took a deeper dive into NodeJS and MongoDB.  In the next post, we’ll move up the MEAN stack and implement our controller layer in ExpressJS.  We’ll then test the completed server-side implementation using a REST client (Postman), and an integration test suite.

About the Author

Object Partners profile.

One thought on “We want you to be MEAN ! (Part 2)

  1. Kevin says:

    Hi Larry,

    Thanks for posting this, it really helps get a hands-on introduction to the MEAN stack. I know the code is meant for an example, but in case you were interested, the findByNameAndDate function has a parameter mismatch, and the test will fail (a single date vs. start and end dates).

    This was a good example to work through to help familiarize myself, thanks again.

    -Kevin

    1. Larry says:

      Thanks for the feedback Kevin. It’s good fun to pick up on MEAN technology – glad this helped. I’ll take a look at the error you reported. Appreciate the response.

Leave a Reply to Larry Cancel reply

Your email address will not be published.

Related Blog Posts
Natively Compiled Java on Google App Engine
Google App Engine is a platform-as-a-service product that is marketed as a way to get your applications into the cloud without necessarily knowing all of the infrastructure bits and pieces to do so. Google App […]
Building Better Data Visualization Experiences: Part 2 of 2
If you don't have a Ph.D. in data science, the raw data might be difficult to comprehend. This is where data visualization comes in.
Unleashing Feature Flags onto Kafka Consumers
Feature flags are a tool to strategically enable or disable functionality at runtime. They are often used to drive different user experiences but can also be useful in real-time data systems. In this post, we’ll […]
A security model for developers
Software security is more important than ever, but developing secure applications is more confusing than ever. TLS, mTLS, RBAC, SAML, OAUTH, OWASP, GDPR, SASL, RSA, JWT, cookie, attack vector, DDoS, firewall, VPN, security groups, exploit, […]