mb23565

aMEBA Application

Blog Post created by mb23565 on May 23, 2017

Hello!

Today I would like to share will you all my take on creating what I would like to call a aMEBA (uh-mee-buh) Application: A Mongo DB, ExpressJS(NodeJS), BBRest API Application.

 

Scientific Definition:

A simple application that is categorized by consuming requests and returning data.

 

So how do we create such an application?

Let me first by stating that although that is focused on creating a simple NodeJS application, the concepts can be applied to any language and framework at the discretion of the developer. Also note ,everything in this article is more of a "getting started" approach to using NodeJS, ExpressJS, MongoDB, and making Bb Rest API calls using the Request Module (a HTTP Client for NodeJS). Ok, now that is out of the way, let's begin. I will be mixing *nix terminal commands to start off the project creation then will switch to and IDE later on. I use Atom, but you can use whichever IDE you prefer.

 

Prerequisites:

 

Optional:

  • Typescript: TypeScript - JavaScript that scales.
    • Super set of JS and has many features that are not in regular JS or ES6
  • Babel: https://babeljs.io/
    • ES6 to JS transpiler that allows you to take advantages in newer JS features that are not easily doable in regular JS.
  • Yarn: Yarn
    • A NPM overlay package manager that keeps the install process quick and efficient as it optimizes your node modules by caching your dependencies.
  • nodemon: nodemon

 

 

Getting Started!

After you have installed NodeJS for your OS, begin to create a project directory for our little aMEBA. For my examples I will be using $HOME/CodeProjects as my project home directory that will hold all my code projects.

 

 

$ mkdir $HOME/CodeProjects/aMEBA-app
$ cd $HOME/CodeProjects/aMEBA-app
$ pwd
[home]/CodeProjects/aMEBA-app
$

 

Let's generate our NPM (or Yarn) init package.json

NPM: For now, just press enter to apply the defaults. We can always edit the package.json file later.

 

$ npm init
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sensible defaults.


See `npm help json` for definitive documentation on these fields
and exactly what they do.


Use `npm install <pkg> --save` afterwards to install a package and
save it as a dependency in the package.json file.


Press ^C at any time to quit.
name: (aMEBA-app) ameba-app
version: (1.0.0) 
description: 
test command: 
git repository: 
keywords: 
author: 
license: (MIT) 
About to write to /home/elmiguel/CodeProjects/aMEBA-app/package.json:


{
  "name": "ameba-app",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "description": ""
}


Is this ok? (yes) 






 

 

or Yarn alt:

 

$ yarn init
yarn init v0.24.4
question name (aMEBA-app): 
question version (1.0.0): 
question description: 
question entry point (index.js): 
question repository url: 
question author: 
question license (MIT): 
success Saved package.json
Done in 11.56s.

 

 

Let's take a peek at the package.json file (Yarn init version):

 

{
  "name": "aMEBA-app",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT"
}



 

For further detail information about understanding what package.json is and what it can do: https://docs.npmjs.com/files/package.json

 

If you know already, or not, you can add a scripts key to add an object of key:value pairs to execute tasks via the terminal to help you in your application building and development process. Let's create a scripts key now if not already with the following:

 

{
  "name": "aMEBA-app",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "scripts": {
    "start": "node app.js",
    "start:dev": "nodemon app.js",
    "start:ts-dev": "nodemon --exec ./node_modules/.bin/ts-node -- ./src/app.ts"
  }
}



 

Notice that I have created some custom scripts: start:dev and start:ts-dev, these are here if you have decided to use nodemon (highly recommended!) and also if you are going with the ts-node option. Please note, that if you are going to go with Typescript, then I highly recommend that you place your .ts scripts in a src folder within your projects and setup a tsconfig.json file to compile and output to a distribution folder ("dist" by convention). More information here: tsconfig.json · TypeScript. If you go this route, then make sure you update your scripts to point to the appropriate paths. You can also build your .ts via the npm scripts as such:

 

"scripts": {
  "build:ts": "tsc"
}

 

Then run it as:

 

$ npm run build:ts

 

Naturally you can just do tsc in your terminal but one good thing about placing it into the scripts object is that you can use this to keep your commands cleaner by stacking the npm scripts:

 

{
  "name": "aMEBA-app",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "scripts": {
    "start": "node app.js",
    "start:dev": "npm run build:ts && nodemon app.js",
    "start:ts-dev": "nodemon --exec ./node_modules/.bin/ts-node -- ./src/app.ts",
    "build:ts": "tsc"
  }
}



 

So now, if you want to add options to the tsc typescript build cli, you can do so and keep the start:dev script clean. Read more on tsc options here: Compiler Options · TypeScript. Optional package npm-run-all: npm-run-all this is a nice package to use if you wish to run a series of custom scripts. You can even run them in parallel! Great for doing SASS builds along side your server side compiling.

 

Installing project dependencies.

Ok, now that we got our feet wet in setting up our application project folder, let's start coding! First let's install some dependencies:

I will be using Yarn:

 

$ yarn add ts-node typescript request nodemon

 

This should only take a couple of seconds. Once that is complete, you can check your package.json file to see dependencies and version releases:

 

$ vim package.json

 

Result:

{
  "name": "aMEBA-app",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "scripts": {
    "start": "node app.js",
    "start:dev": "nodemon app.js",
    "start:ts-dev": "nodemon --exec ./node_modules/.bin/ts-node -- ./src/app.ts"
  },
  "dependencies": {
    "nodemon": "^1.11.0",
    "request": "^2.81.0",
    "ts-node": "^3.0.4",
    "typescript": "^2.3.3"
  }
}

 

If you are using Typescript or Babel, then you will have to make sure you install the required dependencies in order to have your code compile correctly for NodeJS to understand.

Note that I am using Typescript, so I have installed ts-node and typescript as dependencies. I will add the type definitions to take advantage of using typescript for creating my node application.

 

Using yarn to save it to my devDependencies:

 

$ yarn add --dev @types/node @types/request

 

NPM:

 

$ npm install --save-dev @types/node @types/request

 

 

We'll be using Express JS to help with our web application along side with the types:

 

$ yarn add express body-parser method-override
$ yarn add --dev @types/express @types/body-parser @types/method-override 

 

 

 

Results:

 

  "devDependencies": {
    "@types/body-parser": "^1.16.3",
    "@types/node": "^7.0.22",
    "@types/method-override": "^0.0.29",
    "@types/request": "^0.0.43"
  }

 

 

Since I am using Typescript, here is my tsconfig.json settings:

 

{
  "compileOnSave": false,
  "compilerOptions": {
    "target": "es5",
    "noLib": false,
    "lib": ["es2016", "dom"],
    "module": "commonjs",
    "moduleResolution": "node",
    "noImplicitAny": true,
    "removeComments": true,
    "preserveConstEnums": true,
    "sourceMap": false,
    "typeRoots": ["node_modules/@types"],
    "rootDir": "src/server",
    "outDir": "dist"
  },
  "include": [
    "src/**/*"
  ],
  "exclude": [
    "node_modules",
    "**/*.spec.ts"
  ]
}






 

Pay attention to the typeRoots key, this will point to the @types directory that holds the type definitions you installed. This will be auto-referenced during the compiling process.

 

Now once we create our app.ts file in our src directory and run tsc, we will have a file called app.js in our dist directory. Make sure to read up on the tsconfig options! tsconfig.json · TypeScript  Note: setting noLib and lib is redundant: noLib will include all available libraries in your application while lib will only include the libraries in which you want to target: es2016 (es6) features, and dom (js features for regular dom accessors) for example.

 

Git init project creation.

Next let's create a git init folder and add node_modules and our dist directory to it.

 

$ git init
$ touch .gitignore
$ echo "node_modules" >> .gitignore && echo "dist" >> .gitignore

 

This will ignore these two directories to help save on project space in GitHub or GitLab for your online repository. If you use yarn, then you can also add the yarn.lock file to keep the cache clean for later rebuilds.

 

Creating the actual Node JS Application.

Create a src directory to place all our scripts in, if you are not going with Typescript or Babel, then you can setup the project however you wish but in my honest option, try Typescript or Babel

Note: If you are not going to be coding in Typescript or Babel, then you can replace the import statements to just const <var_name> = require('<module_name>'). NodeJS uses the v8 engine which is a limited version of ES6. You can enable more ES6 features by running node with the --harmony flag. More on that here: ECMAScript 2015 (ES6) and beyond |

Node.js

 

 

 mkdir src

 

Now in the src directory, create a file call app.ts (app.js for non Typescripters).

 

$ touch src/app.ts

 

You should now see your file listed in your src directory.

 

Open of the newly created app file and let's enter (or copy paste ):

 

"use strict"
import * as express from 'express'
import * as path from 'path'
import * as bodyParser from 'body-parser'
import * as methodOverride from 'method-override'
import * as request from 'request'

// some variable setup
const env = process.env.NODE_ENV || 'development'
const port = process.env.PORT || 3000


// Express app creation and configuration
const app = express()
// Do not display: Express JS in the x-powered-by header
app.set('x-powered-by', false)


// Allow trust from proxies and SSL support from your load balancer
// Especially if you are running on NGINX and upstreams
app.enable('trust proxy')


// setup body-parser and method-override lets express have access to body posts
// and PUT AND PATCH methods.
app.use(bodyParser.json())
app.use(bodyParser.urlencoded({ extended: true }))
app.use(methodOverride((req: any) => {
  if (req.body && typeof req.body === 'object' && '_method' in req.body) {
    // look in urlencoded POST bodies and delete it
    var method = req.body._method
    delete req.body._method
    return method
  }
}))


// mount our api router
app.use('/api', require('./routes').router)
app.get('/', (req: any, res: any) => res.redirect('/api'))


// error handling
// development will print stacktrace esle an empty object
app.use((err: any, req: any, res: any, next: any) => {
  res.status(err.status || 500).json({
    message: err.message,
    error: env === 'development' ? err : {}
  })
})


//now that the app is setup, start the server
app.listen(port, () => {
  console.log(`Server is listening on port: ${port}`)
})







 

You may notice that this is slightly different from you would normally see when coding in regular JS but in ES6 or Typescript we have the import feature that allows us to import modules like other OOP languages. (Yes I know, JS/ES6/Typescript is more prototypal in structure but still OOP non-the-less). Also, no semicolons! You can use them if you wish, but with ES6 and above these are optional as the compiler will insert them automatically if not present. I have not seen any compiling issues concerning time when using semicolons or not.

 

Creating our router file.

Express lets us create mount points to urls which will easily allow us to delegate certain url paths to specific router functions. Create a file called router.ts (router.js). Please note that I am explicitly assigning the req (request) and res (response) arguments to any. I have also set noImplicitAny in my tsconfig.json to allow the tsc compiler warn of any errors pertaining to variable that is implicit to any instead of a expected type. As this can be annoying at times, but is very helpful later when refactoring later for a more production quality app. For instance, req: express.Request, res: express.Response. Explicit type checking is helpful to future developers so that they will know what type the arguments are expected to be. For this small example, I will just use :any as this will just compile and use whatever type that is passed. This will also help if you decide to use node modules that may not have type definitions. You can always change it later during your refactoring. Also, note that I am not coding entirely in Typescript convention: Classes, interfaces, implements, extends etc. This small example, in my opinion, wouldn't really benefit from it anyways. If you were expand on this by making multiple modules that would utilizing those features, then I would say that would be a more appropriate use case for them.

 

"use strict"
import * as express from 'express'


const router = express.Router()


router.get('/', (req: any, res: any) => {
  res.json({ message: "Hello from aMEBA!" })
})


module.exports = router




 

Running our application.

Now that we have our aMEBA in it basic form, let's see if it works!

 

$ npm run start:ts-dev


> aMEBA-app@1.0.0 start:ts-dev /home/elmiguel/CodeProjects/aMEBA-app
> nodemon --exec ./node_modules/.bin/ts-node -- ./src/app.ts


[nodemon] 1.11.0
[nodemon] to restart at any time, enter `rs`
[nodemon] watching: *.*
[nodemon] starting `./node_modules/.bin/ts-node ./src/app.ts`
Server is listening on port: 3000






 

So far so go, lets go to our browser and see what we get:

 

Awesome! it works!

 

So at this point we should have our project setup something similar to this:

 

If you managed to get this far, congratulations! You just made your first NodeJS + ExpressJS Web API.

 

Setting up MongoDB and Creating Mongoose Models and Middleware.

This section we'll cover going over the ins and outs of creating a mongdb connection with NodeJS and using Models to interact with our document storage. We will implement a small middleware function on our router to intercept our calls to our local API and make sure that we have the correct authentication to talk to the BbRest API. This is what I like to call a passthrough API setup. Basically will not be trying to reinvent the BbRest API but merely pre-authenticating our request locally and letting app keep the token session alive. Each time we make call we either use the stored token or request a new one. Locally we never authenticate to our app, the app is doing the transfer and only relaying the information. The purpose of our router is to only allow what requests we deem necessary for the end result of the app. For example, if we never plan on sending any data to Bb, then we only have to relay GET requests. In the example, we'll set the router to except all http methods: router.all()

 

Prerequisites:

 

Once your MongoDB install and running press Ctrl+C to stop the app if you have it still running and let's install some more dependencies:

 

$ yarn add express-session connect-mongo mongoose bluebird
$ yarn add --dev @types/express-session @types/connect-mongo @types/mongoose

 

Config.ts

Create another script file called config.ts (config.js) and paste the following:

 

"use strict"
export let mongooseConfig: any = {
  database: 'mongodb://localhost:27017/aMEBA'
}

 

Back in our app.ts file let modify it to connect to MongoDB and create our new database: aMEBA (you can make this any name you wish but I like to keep it that same name as the app.)

 

Edit app.ts

 

"use strict"
import * as express from 'express'
import * as session from 'express-session'
import * as path from 'path'
import * as bodyParser from 'body-parser'
import * as methodOverride from 'method-override'
import * as request from 'request'
import * as mongoStore from 'connect-mongo'
import { mongooseConfig } from './config'
mongoStore(session)


const mongoose = require('mongoose')
mongoose.Promise = require('bluebird')


mongoose.connection.once('open', () => {
  mongoose.connection.on('error', (err: any) => {
    console.log(err)
  })
})
mongoose.connect(mongooseConfig.database)


//...rest of code below 



 

Now if you run the app again, it should connect and the app will load. Now if you were to check the mongodb instance running, you will see that there is not aMEBA database:

 

$ mongo
MongoDB shell version: 3.2.12
connecting to: test
> show databases;
local                0.000GB


 

This is because connect-mongo will not create the database until there is an actual interaction.

 

Let's fix that shall we!

 

Models

Create a folder in your src directory called models, this will be the place where we will be making all our models in to keep the project clean.

 

token.ts

 

import * as mongoose from 'mongoose'
const Schema = mongoose.Schema


const TokenSchema = new Schema({
  access_token: String,
  token_type: String,
  expires_in: String
})
TokenSchema.set('timestamps', true)
TokenSchema.set('capped', { size: 1024, max: 1 })


TokenSchema.statics.getToken = function(cb: any) {
  return this.findOne({}, (err: any, token: any) => {
    if (err) throw (err)
    cb(token)
  })
}


TokenSchema.methods.isValid = function(): boolean {
  // console.log(this)
  let createdAt = new Date(this.createdAt)
  // console.log('createAt:', createdAt)
  let expires_in = new Date(createdAt + this.expires_in + '1000').toISOString()
  // console.log('calculated expires_in:', expires_in)
  let now = new Date().toISOString()
  // console.log('now:', now)


  return now <= expires_in
}


module.exports = mongoose.model('Token', TokenSchema, 'tokens')





 

Now that we have our token model, we can now be able to store our request auth tokens. Note that I have this collection capped to 1 document (line 11). This is so that we are only storing one token and that we can allow the application to look at this token only to check its validity.

 

 

config.ts

"use strict"
export const mongooseConfig: any = {
  database: 'mongodb://localhost:27017/aMEBA'
}

export const BbConfig = {
  key: '<YOUR_APP_KEY>',
  secret: '<YOUR_APP_SECRET>',
  credentials: 'client_credentials',
  cert_path: './trusted/keytool_crt.pem',
  url: 'https://<YOUR_BB_INSTANCE>/learn/api/public/v1',
  auth: ''
}


BbConfig.auth = new Buffer(BbConfig.key + ":" + BbConfig.secret).toString("base64")




BbRest API Middleware

Let's create a simple middleware file. Create a folder call middleware and also a file called bbrest-api.ts (bbrest-api.js):

 

 

import { BbConfig } from '../config'
const Token = require('../models/token')
const request = require('request')


export const api = (req: any, res: any, next: any) => {
  if (process.env.NODE_DEBUG) {
    console.log('[ bbrest-api:bbApiUrl ]\t', req.app.locals.bbApiUrl)
    console.log('[ bbrest-api:bbconfig ]\t', req.app.locals.bbpayload)
  }
  next()
}


export const setToken = (cb: any) => {
  request(
    {
      method: 'post',
      url: `${BbConfig.url}/oauth2/token`,
      headers: {
        "Authorization": `Basic ${BbConfig.auth}`
      },
      form: {
        grant_type: 'client_credentials'
      },
      json: true
    },
    (err: any, res: any, body: any) => {
      if (err) {
        console.log('Oops!')
        throw (err.message)
      }
      Token.create(body, (err: any, token: any) => {
        if (err) throw (err)
        cb(token)
      })
    }
  )
}






 

 

Let's update our app.ts.

 

app.ts

 

"use strict"
import * as express from 'express'
import * as session from 'express-session'
import * as path from 'path'
import * as bodyParser from 'body-parser'
import * as methodOverride from 'method-override'
import * as request from 'request'
import * as mongoStore from 'connect-mongo'
import { mongooseConfig, BbConfig } from './config'
import { setToken } from './middleware/bbrest-api'


mongoStore(session)


const mongoose = require('mongoose')
mongoose.Promise = require('bluebird')


mongoose.connection.once('open', () => {
  mongoose.connection.on('error', (err: any) => {
    console.log(err)
  })
})
mongoose.connect(mongooseConfig.database)
// require our models here
const Token = require('./models/token')




// some variable setup
const env = process.env.NODE_ENV || 'development'
const port = process.env.PORT || 3000




process.env.NODE_DEBUG == '*' ? true : false


// Express app creation and configuration
const app = express()
// Do not display: Express JS in the x-powered-by header
app.set('x-powered-by', false)


// Allow trust from proxies and SSL support from your load balancer
// Especially if you are running on NGINX and upstreams
app.enable('trust proxy')


// Add our BbConfig to our app
app.locals.bbApiUrl = BbConfig.url
Token.getToken((token: any) => {
  if (token && token.isValid()) {
    app.locals.bbpayload = token
  } else {
    setToken((token: any) => {
      app.locals.bbpayload = token
    })
  }
})
// setup body-parser and method-override lets express have access to body posts
// and PUT AND PATCH methods.
app.use(bodyParser.json())
app.use(bodyParser.urlencoded({ extended: true }))
app.use(methodOverride((req: any) => {
  if (req.body && typeof req.body === 'object' && '_method' in req.body) {
    // look in urlencoded POST bodies and delete it
    var method = req.body._method
    delete req.body._method
    return method
  }
}))


// mount our api router
app.use('/api', require('./routes'))
app.get('/', (req: any, res: any) => res.redirect('/api'))


// error handling
// development will print stacktrace esle an empty object
app.use((err: any, req: any, res: any, next: any) => {
  res.status(err.status || 500).json({
    message: err.message,
    error: env === 'development' ? err : {}
  })
})


//now that the app is setup, start the server
app.listen(port, () => {
  console.log(`Server is listening on port: ${port}`)
})


















 

Now if you re-run your app:

$ npm run start:ts-dev

 

The application should run without any errors. If you open another terminal do the following, you should see our aMEBA database along with our tokens collection that hold our token:

 

$ mongo
> show databases;
aMEBA                0.000GB
local                0.000GB
> use aMEBA;
switched to db aMEBA
> show collections
tokens
> db.tokens.find();
{ "_id" : ObjectId("592587cfc137e1196f2ad1f1"), "updatedAt" : ISODate("2017-05-24T13:17:03.221Z"), "createdAt" : ISODate("2017-05-24T13:17:03.221Z"), "access_token" : "<TOKEN_HERE>", "token_type" : "bearer", "expires_in" : "1444", "__v" : 0 }
> 





 

YAY! Success!!

 

Now for our router refactor and passthrough API. We will create the passthrough API by capturing the first part the path and dynamically load the correct model to set the correct BbRest API route.

 

Refactoring the router and adding a BbUser Model

router.ts

Update the router to the following:

 

"use strict"
import * as express from 'express'
import * as request from 'request'


const router = express.Router()
const BbUser = require('./models/bbuser')
const MODELS: any = {
  users: BbUser,
  // more models would go here
}


router.get('/', (req: any, res: any) => {
  res.json({ message: "Hello from aMEBA!" })
})


router.all('/users/:userId?', (req: any, res: any) => {
  if (process.env.NODE_DEBUG) console.log(req.params)
  if (process.env.NODE_DEBUG) {
    console.log(req.params.userId)
    console.log('isMany?:', req.params.userId == undefined ? true : false)
  }
  doApi(req, res, req.params.userId == undefined ? true : false)
})




function doApi(req: any, res: any, isMany: boolean) {
  if (process.env.NODE_DEBUG) {
    console.log('[ bbapi:req.path   ]\t', req.path)
    console.log('[ bbapi:req.params ]\t', req.params)
    console.log('[ bbapi:req.query  ]\t', req.query)
    console.log('[ bbapi:req.body   ]\t', req.body)
  }


  let params: any = Object.assign({}, req.params)
  let p: string[] = req.path.split('/').slice(1)


  let model: any
  model = MODELS[p[0]]


  // let isMany:boolean = (typeof params != undefined && params[0] != undefined && params != {}) ? false : true
  if (!isMany) {
    for (let key in req.params) {
      let _key = key
      if (key == 'userId') {
        // check for externalId else userName
        if (/^\d+/.test(req.params[key])) {
          _key = 'externalId'
        } else {
          _key = 'userName'
        }
        delete params.userId
      }
      params[_key] = req.params[key].replace(/^(\w+:)/gi, '')
    }
  }


  if (process.env.NODE_DEBUG) console.log(model.name, p, params)


  let docs = model.find(params).exec()
  docs.then((docs: any) => {
    if (process.env.NODE_DEBUG) console.log(docs)
    console.log('should have documents here.....')
    // if no docs then get docs from bb
    console.log(docs.length)
    // if (!docs && docs == []) {
    if (docs.length == 0) {
      // save retrieved to mongo
      console.log('I should be getting the data from Bb here!')
      getDataAndSave(req, res, model, params)
    } else {
      // return docs to client
      console.log('We have some docs...')
      console.log(docs)
      sendJson(res, null, docs)
    }
  })
}


function getDataAndSave(req: any, res: any, model: any, params: any) {
  if (process.env.NODE_DEBUG) console.log('[bbpi.doApi] [document.then()] retrieving document from Bb')
  let url = req.app.locals.bbApiUrl + req.path
  console.log(req.app.locals.bbpayload.access_token)
  if (process.env.NODE_DEBUG) console.log(url)
  request({
    method: req.method,
    url: url,
    headers: {
      "Authorization": `Bearer ${req.app.locals.bbpayload.access_token}`
    },
    qs: req.query,
    form: req.body,
    json: true
  }, (err: any, resp: any, body: any) => {
    // console.log(resp)
    console.log(body)
    // if (body.status != 404) {
    if (body) {
      console.log('docs loaded')
      model.create(body, (err: any, docs: any) => {
        // model.insertMany((err: any, docs: any) => {
        if (err) throw (err)
        sendJson(res, null, docs)
      })
    } else {
      // doc(s) do(es) not exist in Bb
      // pass the result(s) back to the client
      sendJson(res, err, body)
    }
  })
}


function sendJson(res: any, err: any, data: any) {
  if (err || !data) {
    res.status(404).json({ message: 'No records could be found' });
  } else {
    // console.log(data)
    res.status(200).json(data);
  }
}




module.exports = router














































 

BbUser Model

 

Create a new model in the models directory:

bbuser.ts

import crypto = require('crypto')


const mongoose = require('mongoose')
const fieldsAliasPlugin = require('mongoose-aliasfield')
const Schema = mongoose.Schema


/**
 * User Schema
 */


let BbUserSchema = new Schema({
  id: { type: String, alias: 'userId' },
  uuid: String,
  externalId: String,
  dataSourceId: String,
  userName: { type: String },
  educationLevel: String,
  gender: String,
  created: String,
  lastLogin: String,
  systemRoleIds: [],
  availability: {
    available: String
  },
  name: {
    given: String,
    family: String,
    title: String
  },
  contact: {
    email: String
  }
})


let handleE11000 = function(error: any, doc: any, next: any) {
  if (error.name === 'MongoError' && error.code === 11000) {
    next(new Error('There was a duplicate key error'))
  } else {
    next(error);
  }
}


BbUserSchema.post('save', handleE11000);
BbUserSchema.post('update', handleE11000);
BbUserSchema.post('findOneAndUpdate', handleE11000);
BbUserSchema.post('insertMany', handleE11000);




BbUserSchema.plugin(fieldsAliasPlugin)
BbUserSchema.set('timestamps', true)


const default_select = 'id uuid externalId userName name email'


/**
 * Virtuals
 */


BbUserSchema
  .virtual('password')
  .set((password: string) => {
    this._password = password
    this.salt = this.makeSalt()
    this.hashed_password = this.encryptPassword(password)
  })
  .get(() => {
    return this._password
  })


BbUserSchema.set('toJSON', {
  transform: function(doc: any, ret: any, options: any) {
    delete ret.password; return ret;
  }
})




BbUserSchema.pre('save', (next: any) => {
  //if (!this.isNew) return next()


  //if (!validatePresenceOf(this.password)) {
  //    next(new Error('Invalid password'))
  //} else {
  next()
  //}
})


/**
 * Methods
 */


BbUserSchema.methods = {


  /**
   * Authenticate - check if the passwords are the same
   *
   * @param {String} plainText
   * @return {Boolean}
   * @api public
   */


  authenticate: (plainText: string) => {
    // coming from LDAP, if we get here, we are already authenticated
    // TODO: remove when confirmed
    return true
  },


  /**
   * Make salt
   *
   * @return {String}
   * @api public
   */


  makeSalt: function() {
    return Math.round((new Date().valueOf() * Math.random())) + ''
  },


  /**
   * Encrypt password
   *
   * @param {String} password
   * @return {String}
   * @api public
   */


  encryptPassword: (password: string) => {
    if (!password) return ''
    try {
      //noinspection JSUnresolvedVariable
      return crypto
        .createHmac('sha1', this.salt)
        .update(password)
        .digest('hex')
    } catch (err) {
      return ''
    }
  }
}


/**
 * Statics
 */


BbUserSchema.statics = {


  /**
   * Load
   *
   * @param {Object} options
   * @param {Function} cb
   * @api private
   */


  load: function(options: any, cb: any) {
    let criteria = options.criteria || {}
    let select = options.select || default_select
    let isPrimaryId: boolean
    try {
      console.log('[BbUser.load() [options.criteria]', criteria)
      isPrimaryId = /^_\d+_1/.test(criteria.userId)
    } catch (ex) {
      console.log(ex)
      isPrimaryId = false
    } finally {
      console.log(isPrimaryId)
    }
    let _criteria: any = {}
    if (!isPrimaryId) {
      for (let key in criteria) {
        if (key == 'userId') {
          // check for externalId else userName
          if (/^\d+/.test(criteria.userId)) {
            _criteria.externalId = criteria.userId
          } else {
            _criteria.userName = criteria.userId
          }
        } else {
          _criteria[key] = criteria[key]
        }
      }
    } else {
      _criteria = Object.assign({}, criteria)
      _criteria.id = criteria.userId
      delete _criteria.userId
    }
    console.log('[BbUser.load()] [_criteria]:', _criteria)
    return this.findOne(_criteria)
      .select(select)
      .exec(cb)
  },
  list: function(options: any, cb: any) {
    var criteria = options.criteria || {}
    var select = options.select || default_select
    var sort = options.sort || 1
    var page = options.page || 0
    var limit = options.limit || 30
    return this.find(criteria)
      .select(select)
      .populate('users', select)
      .sort({ _id: sort })
      .limit(limit)
      .skip(limit * page)
      .exec(cb)
  },
  findOrCreate: function(options: any, cb: any) {
    var criteria = options.criteria || {}
    var select = options.select || default_select
    let isPrimaryId: boolean
    try {
      console.log('[BbUser.load() [options.criteria]', options.criteria)
      isPrimaryId = /^_\d+_1/.test(options.criteria.userId)
    } catch (ex) {
      console.log(ex)
      isPrimaryId = false
    } finally {
      console.log(isPrimaryId)
    }
    let _criteria: any = {}
    if (!isPrimaryId) {
      for (let key in options) {
        if (key == 'userId') {
          _criteria.userName = options.criteria.userId
        } else {
          _criteria[key] = options.criteria[key]
        }
      }
    } else {
      _criteria = Object.assign({}, criteria)
    }
    console.log('[BbUser.findOrCreate()] [_criteria]:', _criteria)
    this.findOne(_criteria, (err: any, user: any) => {
      if (err) throw (err)
      if (user) {
        cb(err, user)
      } else {
        let _user = new this(criteria)
        cb(null, _user.save())
      }
    })
  }
}


module.exports = mongoose.model('BbUser', BbUserSchema, 'bbusers')

 

 

Note that most of the code is not needed: encryptPassword and validation, but I have it there in case you wish to implement a local login system. This would allow you to add users by first calling the BbRest API, saving them locally to our mongodb instance, then having the local login system check to see if a user exists and is valid.

 

The take away from here would be the main flow: MongoDB First, if not already exists then get from BbRest API, If found save.

 

Now if you run the app, and go to your app url: http://localhost:3000/api/users/userName:<ENTER_A_USERNAME>

 

What happens here is that we are just passing the end route path to the BbRest API and then printing the results that it gave. All method and actions are actually BbRest API Specs: Explore APIs

 

MAGIC!

 

MongoDB Magic!

Checking our mongodb after the API Call:

 

> show collections;
bbusers
tokens
> db.bbusers.find();
{ "_id" : ObjectId("5925cae83f2ee1264e928e23"), "updatedAt" : ISODate("2017-05-24T18:03:20.544Z"), "createdAt" : ISODate("2017-05-24T18:03:20.544Z"), "id" : "_<USER_PK>_1", "uuid" : "<UUID>", "externalId" : "<USER_BATCH_UID>", "dataSourceId" : "_<DATA_SOURCE_PK>_1", "userName" : "mbechtel", "educationLevel" : "Unknown", "gender" : "Unknown", "created" : "2013-07-02T15:02:36.000Z", "lastLogin" : "2017-05-24T11:42:05.000Z", "contact" : { "email" : "mbechtel@irsc.edu" }, "name" : { "given" : "Michael", "family" : "Bechtel", "title" : "System Admin" }, "availability" : { "available" : "Yes" }, "systemRoleIds" : [ "SystemAdmin" ], "__v" : 0 }
> 



 

 

YAY! It Works!

 

Let's step this up one more notch: More User routes and Courses!

Ok, so we have a simple router that only listens to /user/:userId? This means we can only interact with the /users routes. This can be easily expanded on now with just creating more models and routes for those models. Let's create a course.ts model in src/models and then add some route functionality to interact with the BbRest API.

 

routes.ts

Let's just add a quick route for the users:

router.all('/users/:userId/courses', (req: any, res: any) => {
  doApi(req, res, true)
})

 

This route is really listed under the course membership routes but I personally think that it belongs to the user routes. Ok back to courses.

 

courses.ts

import crypto = require('crypto')


const mongoose = require('mongoose')
const Schema = mongoose.Schema


/**
 * User Schema
 */


let CourseSchema = new Schema(
  {
    id: String,
    uuid: String,
    externalId: String,
    dataSourceId: String,
    courseId: String,
    name: String,
    created: Date,
    organization: Boolean,
    ultraStatus: String,
    allowGuests: Boolean,
    readOnly: Boolean,
    availability: {
      available: String,
      duration: {
        type: { type: String }
      }
    },
    enrollment: {
      type: { type: String }
    },
    locale: {
      force: Boolean
    }
  })
CourseSchema.set('timestamps', true)


const default_select = 'id uuid externalId courseId courseName organization availability'




/**
 * Statics
 */


CourseSchema.statics = {


  /**
   * Load
   *
   * @param {Object} options
   * @param {Function} cb
   * @api private
   */


  load: function(options: any, cb: any) {
    var select = options.select || default_select
    console.log('[Course.load()] [criteria]:', options.criteria)
    return this.findOne(options.criteria)
      .select(select)
      .exec(cb)
  },
  list: function(options: any, cb: any) {
    var criteria = options.criteria || {}
    var select = options.select || default_select
    var sort = options.sort || 1
    var page = options.page || 0
    var limit = options.limit || 30
    return this.find(criteria)
      .select(select)
      .populate('users', select)
      .sort({ _id: sort })
      .limit(limit)
      .skip(limit * page)
      .exec(cb)
  },
  findOrCreate: function(options: any, cb: any) {
    var criteria = options.criteria || {}
    var select = options.select || default_select
    this.findOne(options.criteria, (err: any, user: any) => {
      if (err) throw (err)
      if (user) {
        cb(err, user)
      } else {
        let _user = new this(criteria)
        cb(_user.save())
      }
    })
  }
}


module.exports = mongoose.model('Course', CourseSchema, 'courses')



































 

Pretty standard stuff. A little tip, currently I am exporting this as a module package and exporting it as a mongoose model. What this means is that the mongoose model attaches itself to the default mongoose connection directly. If you are going to have multiple mongo connections that will share these models, then you need to export just the schemas and then register them to the connection variable.

 

Example: Taken from the mongoose documentation from the GitHub repository: GitHub - Automattic/mongoose: MongoDB object modeling designed to work in an asynchronous environment.

This ties the model directly to the connection instance that was assigned to a variable.

let conn = mongoose.createConnection('your connection string'),
  MyModel = conn.model('ModelName', schema),
  m = new MyModel;
m.save(); // works


 

So now you can just do MyModel.db.host, MyModel.db.port, MyModel.db.name to get the database information.

 

Now we register this to our router MODELS variable:

 

router.ts

 

Change:

 

const BbUser = require('./models/bbuser')
const MODELS: any = {
  users: BbUser,
  // more models would go here
}

 

To:

const BbUser = require('./models/bbuser')
const Course = require('./models/course')
const MODELS: any = {
  users: BbUser,
  courses: Course
}


 

We have added the new Course model and registered this to our MODELS variable. Since the router is just a passthrough, what we are doing is capturing the first route path and loading the correct model on the fly! Now we just have to let our express app know what to do with /api/courses.

 

Add these routes under your users routers:

 


router.all('/courses/:courseId?', (req: any, res: any) => {
  doApi(req, res, req.params.courseId == undefined ? true : false)
})


router.all('/courses/:courseId/users/:userId?', (req: any, res: any) => {
  doApi(req, res, req.params.userId == undefined ? true : false)
})









 

Now let's test it out, run the app and look up a course:

 

back in mongo:

 

> show collections;
bbusers
courses
tokens
> db.courses.find();
{ "_id" : ObjectId("5926d358192c007e9c63f2e7"), "updatedAt" : ISODate("2017-05-25T12:51:36.346Z"), "createdAt" : ISODate("2017-05-25T12:51:36.346Z"), "id" : "_2841_1", "uuid" : "ad842f780aa845879438a8cb0dd530dd", "externalId" : "LOR-CC-mbechtel", "dataSourceId" : "_1186_1", "courseId" : "LOR-CC-mbechtel", "name" : "LOR-CC-mbechtel", "created" : ISODate("2014-03-04T14:24:14Z"), "organization" : false, "ultraStatus" : "Classic", "allowGuests" : true, "readOnly" : false, "locale" : { "force" : false }, "enrollment" : { "type" : "InstructorLed" }, "availability" : { "available" : "Yes", "duration" : { "type" : "Continuous" } }, "__v" : 0 }
> 



 

YAY!

 

Conclusion

Well I hope you enjoyed my take on creating a small Node JS Express web application and making BbRest API calls. From here, you should be able to create awesome web apps with NodeJS For Bb!

 

Now that I had you read through this entire article....here is the source code: GitHub - elmiguel/aMEBA: A MongoDB, Express JS, BbRest API Application

Outcomes