This is Part 25 of our Real World App series. For other parts, see links at the bottom of this post or go to https://blog.realworldfullstack.io/. The app is currently under development and can be viewed at https://bitwiser.io and the github repo can be found here.

Why I chose NativeScript?

After looking at several alternates (see this and several other comparisons on the web), in order to leverage maximum code reuse and not rewrite our app completely for mobile platforms, it came down to 2 choices -

• Hybrid PWA app with Ionic
• Native app with NativeScript

The easiest choice would have been to go with the hybrid approach and use Ionic as it would have required less effort and leveraged the existing tech stack to accomplish this. For me however, performance was a big consideration, especially for a growing game app like ours, so I decided to go with NativeScript.

The vast majority of code changes would involve rewriting the components, along with it’s templates and css, since NativeScript uses a markup language in XML format that is different from HTML to compose the UI for the native platforms.

In addition, since the firebase plugin for NativeScript has a different API surface, we’ll create an interface for interacting with Firebase with different implementations for web and mobile. We can then inject the appropriate implementation based on the platform.

Installing NativeScript on Mac OS

In order to simulate, debug and deploy to iOS platform, I switched to Mac OS for development.

Installing NativeScript on Mac -

• https://docs.nativescript.org/angular/start/quick-setup
• https://docs.nativescript.org/angular/start/ns-setup-os-x

1. Install Xcode from App Store
2. Use nvm to install Nodejs. nvm allows you to install and switch between multiple versions of node. nvm also takes away the need to use sudo for installing npm packages.

curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.11/install.sh | bash

Close terminal window and install latest version of node 8

nvm --version
nvm install 8.14.0

3. Install Homebrew (homebrew helps in installation of other packages)

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

4. Install other dependencies for iOS

a. Install Xcode command line tools, if not already installed.

Verify Xcode command line tools -

xcodebuild -version

If not installed, start Xcode and you will be prompted to install it.

b. Install the xcodeproj ruby gem with the following command

sudo gem install xcodeproj

c. Install & setup CocoaPods

sudo gem install cocoapods
pod setup

d. Install python six package (if not already installed)

(got this: Requirement already satisfied: six in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (1.4.1))

sudo easy_install pip
pip install six

5. Install Android dependencies

a. Install JDK 8.

brew tap caskroom/versions
brew cask install java8

b. Set JAVA_HOME

export JAVA_HOME=$(/usr/libexec/java_home -v 1.8)

c. Install Android Studio from https://developer.android.com/studio/install

d. Add this to ~/.bash_profile

export ANDROID_HOME="$HOME/Library/Android/sdk"

6. Install NativeScript cli

npm i -g nativescript

Restart terminal

7. Verify all installation was ok.

tns doctor

At this point, NativeScript and all iOS and Android dependencies are installed and we’re ready to get going with our project

Add NativeScript to our application

Let’s start by adding NativeScript support to our app -

ng add @nativescript/schematics

This will the necessary npm modules, tsconfig and the module files. We would add the remaining dependencies to our package.json manually.

Building the apps

To build our mobile apps, we will use the nativescript-dev-webpack plugin. The build process handles replacing the web files with the ones with “.tns” extension when available.

For example, “core.module.tns.ts” will replace “core.module.ts” when building for mobile; while the tns files will be ignored for web builds.

Here is the complete webpack.config.js for nativescript build.

The tsconfig.app.json is also modified to ignore mobile specific files.

To build the mobile apps we use the following commands -

tns run ios --bundle
tns run android --bundle

Mobile UI

The UI for our mobile app will be slightly different from our web application ui with some different interactions to leverage mobile platform.

NativeScript Components

Similar to an Angular component, a typical NativeScript-Angular component consists of a component class, a template and a css(scss) file. Ideally, the component logic remains the same and the same component class can be used for both web & mobile platforms.

More often than not, there are subtle differences between the implementation. In those cases, we can create a base class for the common stuff and have 2 separate component classes — one for web and the other for mobile to implement the differences.

Let’s look at a couple of the examples -

The question.component has similar logic for both the web and mobile versions. So we have use the same class in question.component.ts file shared between them while creating a tns file for the mobile version -

<!-- question.component.tns.html -->

<CardView class="cardStyle" margin="10" android:elevation="20" ios:elevation="40" radius="5" class="gameplay" backgroundColor="white">
    <StackLayout class="cardStyle Roboto-Medium" backgroundColor="#ffffff" height="100%" width="100%" margin="15">
        <Label text="Question of the day!" color="#000" fontSize="20" class="text-center" textWrap="true"></Label>

        <Label [text]="question?.questionText" color="#263238" fontSize="18" fontWeight="500" margin="10" textWrap="true"></Label>

        <StackLayout orientation="horizontal" margin="10" *ngIf="question">
            <Label text="Tags:" marginRight="5" color="#1B1C1C" fontSize="15" fontWeight="500" textWrap="true"></Label>
            <Label text="{{question.tags?.toString()}}" class="Roboto-Regular" color="#1B1C1C" fontSize="15" textWrap="true"></Label>
        </StackLayout>
        <StackLayout *ngIf="question">
            <StackLayout class="question-answered Hind-Medium" margin="10" *ngFor="let answer of question.answers"
            [class.players-answer]="answeredText === answer.answerText" [class.wrong]="answeredText === answer.answerText && correctAnswerText !== answer.answerText"
            [class.right]="answeredText !== '' && correctAnswerText === answer.answerText" (tap)="answerButtonClicked(answer)" [isUserInteractionEnabled] = "doPlay">
            <Label text="{{answer?.answerText}}" horizontalAlignment="center" fontSize="17" fontWeight="500" marginTop="15"
                marginBottom="15" textWrap="true"></Label>
        </StackLayout>
        </StackLayout>
        <StackLayout orientation="vertical" *ngIf="question" margin="10">
            <author [userDict]="userDict" [userId]="question.created_uid"></author>
            <button class="Roboto-Regular" backgroundColor="#8ac541" color="white" margin="20" (tap)="getNextQuestion()" *ngIf="answeredText!==''"
                width="100%" text="Try Another" fontSize="17" padding="15"></button>
        </StackLayout>
    </StackLayout>

</CardView>

On the other hand, the dashboard component has slightly different interactions in the mobile and web versions. So we not only have a different tns.html file, but also a different component class that is derived from a common base class.

You can find the base class, the web version and the mobile version here.

NativeScript - Firebase

The nativescript-plugin-firebase allows integration with firebase. However, the plugin does not provide the same interface as the web version of firebase package. Also, the plugin returns Promises instead of the Observables that the firebase library for web does.

We define a common interface that can be used for all platforms -

// db.service.ts

import { Injectable, Inject, NgZone } from '@angular/core';
import { Observable, of } from 'rxjs';

@Injectable()
export abstract class DbService {

    constructor() {

    }

    public createDoc(collectionName: string, document: any): any {
    }

    public setDoc(collectionName: string, docId: any, document: any): any {

    }

    public updateDoc(collectionName: string, docId: any, document: any) {

    }
    public valueChanges(collectionName: string, path?: any, queryParams?: any): Observable<any> {
        return of();
    }

    public createId(): any {

    }
    public getFireStorageReference(filePath: string): any {

    }

    public getFireStore(): any {

    }

    public getDoc(collectionName: string, docId: any): any {

    }

    public upload(filePath: string, imageBlob: any): any {

    }
}

And then the implementation for web and mobile.

ngZone

The nativescript-firebase plugin runs outside Angular. In order for Angular to run it’s change detection, we need to call ngZone.run when listening to a firebase event. We use this in implementation of the mobile version for DbService.

// db.service.ts

@Injectable()
export class TNSDbService extends DbService {

    constructor(private zone: NgZone,
        protected _store: Store<any>,
        private userActions: UserActions) {
        super();
    }
...

    public valueChanges(collectionName: string, path?: any, queryParams?: any): Observable<any> {

        let query = firebaseApp.firestore().collection(collectionName);

        if (queryParams) {
            for (const param of queryParams.condition) {
                query = query.where(param.name, param.comparator, param.value);
            }
            if (queryParams.orderBy) {
                for (const param of queryParams.orderBy) {
                    query = query.orderBy(param.name, param.value);
                }
            }
            if (queryParams.limit) {
                query = query.limit(queryParams.limit);
            }
        }
        if (path) {
            query = query.doc(path);
        }
        return Observable.create(observer => {
            const unsubscribe = query.onSnapshot((snapshot: any) => {
                let results = [];
                if (snapshot && snapshot.forEach) {
                    snapshot.forEach(doc => results.push({
                        id: doc.id,
                        ...doc.data()
                    }));
                } else {
                    results = snapshot.data();
                }
                this.zone.run(() => {
                    if (results !== undefined) {
                        observer.next(results);
                    } else {
                        observer.next(undefined);
                    }
                });
            });
            return () => unsubscribe();
        });
    }
...
}

And we do the same for FirebaseAuthService -

// firebase-auth.service.ts

import { Observable } from 'rxjs';

export abstract class FirebaseAuthService {
    abstract createUserWithEmailAndPassword(email, password);
    abstract authState(): Observable<any>;
    abstract signOut();
    abstract showLogin();
    abstract getIdToken(user, forceRefresh: boolean);
    abstract refreshToken(forceRefresh: boolean): Promise<string>;
    abstract signInWithEmailAndPassword(email: string, password: string);
    abstract sendEmailVerification(user): Promise<any>;
    abstract sendPasswordResetEmail(email: string): Promise<any>;
    abstract firebaseAuth(): any;
    abstract googleLogin(): Promise<any>;
    abstract facebookLogin(): Promise<any>;
    abstract twitterLogin(): Promise<any>;
    abstract githubLogin(): Promise<any>;
}

And it’s implementation here - web & mobile.

Modules and Routing

We then go on to create a mobile version of the modules and routes to include only the necessary modules for the mobile app. The complete code for this part is here.

Testing the app on the simulator

To test the app in the simulator -

tns run android--bundle

Similarly, for iOS

tns run ios --bundle

Next up

In this part, we started to build out the native version of the app and tested it using the iOS and Android simulator. In a follow up part to this 25[b], we’ll see how we can deploy this to the actual devices.

If you enjoyed this article please recommend and share and feel free to provide your feedback on the comments section.

Real World App Series (quick links) :

Series: https://blog.realworldfullstack.io

• 0: From zero to cli-ng, 1: Not another todo list!
• 2: It’s a Material World!, 3: Form Formation
• 4: State of my SPA, 5: Light my fire
• 6: 3Rs — Roles, Rules & Routes, 6.1: Upgrading to 4.0.0-rc.2
• 7: Split my lazy loaded code, 8: Just Ahead of in Time
• 9, 9.1 and 9.2 : Unit Testing; X: Upgrading to Angular 4
• 11: Gameplay with Angular; 12: Cloud Functions for Firebase
• 13: Elasticsearch on google cloud; 14: Faceted Search
• 15: Material UI Design; 16: Firebase to firestore
• 17: Firebase Cloud Storage; 18: Revisiting ngrx
• 19: Two Player; 20: Upgrading to Angular 6
• 21: PWA; 22: e2e Testing; 23: SSR with Angular Universal
• 24: Angular Workspace

References

• https://www.konstantinfo.com/blog/react-native-vs-flutter-vs-ionic-vs-nativescript-vs-pwa/
• https://brew.sh/
• https://github.com/creationix/nvm
• https://nativescript.org
• https://docs.nativescript.org/angular/start/introduction
• https://blog.angular.io/apps-that-work-natively-on-the-web-and-mobile-9b26852495e7
• https://github.com/NathanWalker/angular-seed-advanced/wiki/How-to-integrate-Firebase-across-all-platforms-(web---nativescript---desktop)

Real world App: Part 25[a]- NativeScript, Angular and Firebase was originally published in Real World Full Stack on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is Part 24 of our Real World App series. For other parts, see links at the bottom of this post or go to https://blog.realworldfullstack.io/. The app is currently under development and can be viewed at https://bitwiser.io and the github repo can be found here.

Angular workspace

A workspace is a way to split up your application into smaller units called projects. There are 2 types of projects - library and application.

A workspace library is a logical grouping of modules that can be re-used across workspace projects. A library can also be published & distributed to be re-used in other applications.

A workspace application is a single-page application (SPA) that may or may not consume one or more libraries.

Angular introduced the concept of workspaces in version 6. Prior to version 6, the concept of workspaces was introduced by the nrwl.io team - https://nrwl.io/nx.

Why workspaces? Why now?

The admin module of our application was never meant to be part of the main app. Even though the module was lazy loaded and behind a route guard, the admin users are different from the users of the rest of the site. Also, the functionality of this module is so distinct from the main site, that it’ll almost never be used along the main site. The state that supports this functionality is different than the state that’s on the main site.

As our main app is reaching minimum viable functionality, we also need to start development of our mobile app. This will only add to the complexity of the existing application. The mobile app will not have any of the admin functionality.

This part of the series will focus on simply refactoring our existing application into 3 Angular projects - we will create 2 application projects (trivia and trivia-admin) and one library project, for now, that’s shared between the two. Let’s get right into it.

Generating the new application

First we’ll rename our project in angular.json temporarily, so we can create another project with the same name. Let’s rename our project to triviatmp in angular.json

Next, let’s generate a new one -

ng generate application --name=trivia

This would generate a new project under /projects/trivia and create a corresponding section in angular.json. Let’s the /projects/trivia/src with our /src folder, and then replace the /projects/trivia-e2e/src with contents of /e2e.

Next, we modify angular.json, project trivia to match triviatmp (including styles, assets, serviceworker, dev build configuration), but keeping the folder paths the same. Let’s try it out -

ng serve

Note: I did have some incorrect import paths that I needed to fix.

Remove the tsconfigs and typings from inside the projects/trivia/src folder as they are now at the project/trivia level.

For unit testing

I also had to modify the jest setting in package.json (path to setup and tsconfig.spec), had to move the setup-jest out of src and modify the path to mocks.

For e2e testing

Simply remove protractor.conf.js from root folder and then use command “ng e2e trivia-e2e”

Trivia project folder structure -

Admin & Shared library projects

Let’s create the other projects. This will also modify our angular.json -

ng generate application --name=trivia-admin --prefix=app
ng generate library --name=shared-library --prefix=stl

We would then carefully replace the contents of each of the src folders of these projects with what we need from our main project.

The core module along with the services and the shared module with our model will be moved to the shared-library project.

Admin and bulk (for admin) modules will be moved to the admin project. Our application folder structure for the 3 projects will looks something like this -

To run these applications -

ng serve trivia

ng server trivia-admin

The complete code for this part is here. Even though the PR shows a daunting 852 files changed, most of this is just moving the files around and changing import paths.

Next up

In this part, we broke up our app into more maintainable workspace projects. In the next part, we will introduce NativeScript and start developing our mobile app.

If you enjoyed this article please recommend and share and feel free to provide your feedback on the comments section.

Real World App Series (quick links) :

Series summary and index: Part X

• 0: From zero to cli-ng, 1: Not another todo list!
• 2: It’s a Material World!, 3: Form Formation
• 4: State of my SPA, 5: Light my fire
• 6: 3Rs — Roles, Rules & Routes, 6.1: Upgrading to 4.0.0-rc.2
• 7: Split my lazy loaded code, 8: Just Ahead of in Time
• 9, 9.1 and 9.2 : Unit Testing; X: Upgrading to Angular 4
• 11: Gameplay with Angular; 12: Cloud Functions for Firebase
• 13: Elasticsearch on google cloud; 14: Faceted Search
• 15: Material UI Design; 16: Firebase to firestore
• 17: Firebase Cloud Storage; 18: Revisiting ngrx
• 19: Two Player; 20: Upgrading to Angular 6
• 21: PWA; 22: e2e Testing; 23: SSR with Angular Universal

References

• https://blog.angularindepth.com/creating-a-library-in-angular-6-87799552e7e5
• https://angular.io/guide/file-structure
• https://nitayneeman.com/posts/understanding-the-angular-cli-workspace-file/
• https://medium.com/@angularlicious/angular-6-workspace-test-drive-cfe24bbceeb3
• https://nrwl.io/nx

Real World App: Part 24 - Angular Workspace was originally published in Real World Full Stack on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is Part 23 of our Real World App series. For other parts, see links at the bottom of this post or go to https://blog.realworldfullstack.io/. The app is currently under development and can be viewed at https://bitwiser.io and the github repo can be found here.

Server side rendering with Angular

(see https://angular.io/guide/universal)

A normal Angular application executes in the browser, rendering pages in the DOM in response to user actions and events. Server side rendering (SSR) is a process of generating static application pages on the server.

Angular Universal can generate and serve those SSR pages in response to requests from browsers. It can also pre-generate pages as HTML files that you serve later.

Why use SSR?

The three main reasons to create a SSR version of your app.

• Facilitate web crawlers (SEO)
• Improve performance on mobile and low-powered devices
• Show the first page quickly

Search Engine Optimization (SEO)

Search engines like Google & Bing & social media sites like Facebook & Twitter rely on web crawlers to index your application content and make that content searchable on the web. These web crawlers may be unable to navigate and index your highly interactive Angular application as a human user could do, even though google’s engine does a pretty decent job of executing javascript.

An SSR application can generate a static version of your app that is easily searchable, linkable, and navigable without JavaScript. It also makes a site preview available since each URL returns a fully rendered page.

As it’s important for our app to be discoverable by search engines, this was the primary reason, I wanted to have SSR early in our development cycle. The primary page that needs to be server-side rendered is the dashboard.

Angular universal allows us to render on the server (without a need for web browser) with minimal change to our existing code.

Universal with Express/NodeJS

Angular universal provides a template engine for Express on NodeJS which works for us as our firebase functions run on node engine. Let’s get right into it.

We’ll start by adding dependencies to package.json (and also update existing ones) -

"dependencies": {
...
 "@angular/platform-server": "^6.0.7",
 "@nguniversal/express-engine": "^6.0.0",
 "@nguniversal/module-map-ngfactory-loader": "^6.0.0",
 "domino": "^2.0.2",
 "ts-loader": "^4.2.0",
 "webpack-cli": "^3.0.8",
 "xmlhttprequest": "^1.8.0"
...
}

Let’s see what each of them does -

• platform-server: server implementation of DOM
• express-engine: Express Engine for running Angular Apps on server for SSR
• module-map-ngfactory-loader: lazy-loading on server
• domino (DOM in node): additional support for server side DOM manipulation
• xmlhttprequest: wrapper for the built-in http client to emulate the browser XMLHttpRequest object
• webpack-cli: for configuration and compilation of the server package
• ts-loader: typescript loader for webpack

Let’s add a server.ts that’ll run in our express engine to render the application on the server. Note the use of xmlhttprequest that’ll be used for http requests on the server to emulate a browser’s XmlHttpRequest. Also we use domino for any references to window for SSR.

// server.ts
import 'zone.js/dist/zone-node';
import 'reflect-metadata';

import { enableProdMode } from '@angular/core';

import * as express from 'express';
import { ngExpressEngine } from '@nguniversal/express-engine';
import { provideModuleMap } from '@nguniversal/module-map-ngfactory-loader';
import { resolve } from 'path';

const domino = require('domino');
const win = domino.createWindow('');

global['window'] = win;
global['document'] = win.document;
global['XMLHttpRequest'] = require('xmlhttprequest').XMLHttpRequest;

console.log(process.cwd());
const DIST_FOLDER = resolve(process.cwd(), './dist');
console.log(DIST_FOLDER);

const {
  AppServerModuleNgFactory,
  LAZY_MODULE_MAP
} = require(`./functions/dist/server/main`);

enableProdMode();

const app = express();

// Set the engine
app.engine(
  'html',
  ngExpressEngine({
    bootstrap: AppServerModuleNgFactory,
    providers: [provideModuleMap(LAZY_MODULE_MAP)]
  })
);

app.set('view engine', 'html');

app.set('views', DIST_FOLDER);

// Point all routes to Universal
app.get('*', (req, res) => {
  res.render('index', { req });
});


exports.app = app;

Next we add a root module for the server -

// app.server.module.ts

import { NgModule } from '@angular/core';
import { ServerModule, ServerTransferStateModule } from '@angular/platform-server';
import { ModuleMapLoaderModule } from '@nguniversal/module-map-ngfactory-loader';
import { FlexLayoutServerModule } from '@angular/flex-layout/server';

import { AppModule } from './app.module';
import { AppComponent } from './components/app/app.component';

@NgModule({
  imports: [
    AppModule,
    ServerModule,
    ModuleMapLoaderModule,
    ServerTransferStateModule,
    FlexLayoutServerModule,
  ],
  providers: [
    // Add universal-only providers here
  ],
  bootstrap: [AppComponent],
})
export class AppServerModule { }

Add main.server.ts -

// main.server.ts

export { AppServerModule } from './app/app.server.module';

and modify our app.module to use server transition -

// app.module.ts
import { BrowserModule, BrowserTransferStateModule } from '@angular/platform-browser';
...

imports [
...
BrowserModule.withServerTransition({ appId: 'trivia' }),
...

We would then add a tsconfig.server.json that extends our client-side tsconfig -

// tsconfig.server.json

{
  "extends": "../tsconfig.json",
  "compilerOptions": {
    "outDir": "../out-tsc/app",
    "baseUrl": "./",
    "module": "commonjs",
    "types": []
  },
  "exclude": [
    "test.ts",
    "**/*.spec.ts"
  ],
  "angularCompilerOptions": {
    "entryModule": "app/app.server.module#AppServerModule"
  }
}

Also add a new build target to angular.json

// angular.json
...
"server": {
 "builder": "@angular-devkit/build-angular:server",
 "options":
 {
    "outputPath": "functions/dist/server",
    "main": "src/main.server.ts",
    "tsConfig": "src/tsconfig.server.json"
 }
...

And finally we add our webpack configuration -

// webpack.server.config.js

const path = require('path');
const webpack = require('webpack');



module.exports = {
  entry: { server: './server.ts' },
  resolve: { extensions: ['.js', '.ts'] },
  mode: 'none',
  target: 'node',
  externals: [/(node_modules|main(\\|\/)..*(\\|\/).js)/],
  output: {
    path: path.join(__dirname, `functions/server/functions`),
    filename: '[name].js'
  },
  module: {
    rules: [
      { test: /\.ts$/, loader: 'ts-loader', exclude: /^(?!.*\.spec\.ts$).*\.ts$/  },
      {
        test: /(\\|\/)@angular(\\|\/)core(\\|\/).+\.js$/,
        parser: { system: true }
      }
    ]
  },
  plugins: [
    new webpack.ContextReplacementPlugin(
      /(.+)?angular(\\|\/)core(.+)?/,
      path.join(__dirname, 'src'), // location of your src
      {} // a map of your routes
    ),
    new webpack.ContextReplacementPlugin(
      /(.+)?express(\\|\/)(.+)?/,
      path.join(__dirname, 'src'),
      {}
    )
  ]
};

To build our server bundle we can use this -

webpack --config webpack.server.config.js

SSR for Firebase functions

However, in order to deploy our bundle to firebase functions, we need to take care of a few more things.

First, we will upgrade our functions to run on node version 8 by modifying our package.json. Node v8 will allow us to use ES6 style imports, that some of our dependent libraries use -

// package.json

...
"engines": {    "node": "8"  },
...

We’ll then create an ssr.ts, separate from our app.ts to host our server-side angular bundle (note that we’ve upgraded the memory required for our ssr) -

// ssr.ts

const ssrFunction = require('firebase-functions');
const ngApp = require('./server').app;

const runtimeOpts = {
  memory: '512MB'
}

exports.ssr = ssrFunction.runWith(runtimeOpts).https.onRequest(ngApp);

Next, we’ll update our firebase.json so we can redirect our requests to the ssr version of our application -

// firebase.json

{
  "hosting": {
    "public": "dist",
    "rewrites": [
      {
        "source": "/app/**",
        "function": "app"
      },   
      {
        "source": "**",
        "function": "ssr"
      }
    ]
  }
}

Deployment to firebase

Although the firebase-cli supports deployment of multiple functions, I found out that we can deploy only one function at a time. Since, we have our original app function and now the new ssr function, we’ll use a bit of a hack to do this.

We’ll rename our original functions/index.js to functions/app-functions.js and add a new functions/ssr-functions.js -

// ssr-functions.js

exports.ssr = require('./server/functions/ssr').ssr;

At the time of deployment, we’ll copy the appropriate functions.js file as index.js and then deploy to firebase.

Let’s create a bunch of compilation and deployment handlers for our application in package.json -

// package.json
...

"compile-functions": "rmdir /s/q functions\\server & tsc --project functions",

"webpack:server": "webpack --config webpack.server.config.js",

"compile-ssr-functions": "npm run compile-functions && npm run webpack:server && cpx dist/index.html functions/dist && rimraf dist/index.html",

"pre-compile-app-functions": "rm -rf functions/index.js && cp functions/app-functions.js functions/index.js",

"pre-compile-ssr-functions": "rm -rf functions/index.js && cp functions/ssr-functions.js functions/index.js",

"prod:build:ssr": "ng build --prod --configuration=production && ng run trivia:server",

"prod:deploy-functions-build": "npm run prod:build:ssr && npm run compile-ssr-functions",

"prod:deploy-functions-app": "npm run pre-compile-app-functions && firebase deploy -P production --only functions",

"prod:deploy-functions-ssr": "npm run pre-compile-ssr-functions && firebase deploy -P production --only functions:ssr",

"prod:set-index": "firebase -P production functions:config:set elasticsearch.index.production=true",

"prod:deploy-apps-to-firebase": "firebase deploy -P production --only hosting",

"prod:deploy-functions": "npm run prod:deploy-functions-build && npm run prod:set-index && npm run prod:deploy-functions-app && npm run prod:deploy-functions-ssr && prod:deploy-apps-to-firebase",

...

The prod:deploy-functions command will compile the ssr application & to the task of deploying everything including the ssr & app functions and the front-end hosting app.

Note the removal of index.html from the client hosting app as firebase will not redirect to the server side index.html if it finds this file.

And that’s all we need for deploying our SSR app to firebase functions. Let’s give it a try.

Detecting server vs browser

Not all functionality need be made available to our server-side application. For example, game-play, profile settings and pretty much any functionality that requires a user to login will remain a client-side function. In order to detect whether the app is running in a browser or on the server at runtime, , Angular provides PLATFORM_ID that can be checked in our code -

// authentication.provider.ts 
...

constructor( ... 
  @Inject(PLATFORM_ID) private platformId: Object,
...)

...

ensureLogin = function (url?: string) {
   if (!this.isAuthenticated) {
    if (isPlatformBrowser(this.platformId)) {
      this.showLogin(url);
      if (!this.isAuthenticated) {
        this.showLogin(url);
      }
    }
   }
};
...

Our web application is now a fully functional SSR app. If you do a view source on the browser, you’ll now see the entire html generated by angular universal rendered on the server.

SEO and social meta tags

We take this opportunity to add some SEO specific headers and social tags to our index.html to help facebook & twitter to better crawl our site.

// index.html
...
<head>
...
  <!-- Schema.org markup for Google+ -->
  <meta itemprop="name" content="bitwiser.io: get wiser - bit by bit">
  <meta itemprop="description" content="Open source tech trivia game. by, for and of the developers">
  <meta itemprop="image" content="https://cdn-images-1.medium.com/max/1600/1*5RH1reNYpyBJOKauMMX2tw.png">

  <!-- Facebook Open graph -->
  <meta property="og:url" content="https://bitwiser.io" />
  <meta property="og:type" content="article" />
  <meta property="og:title" content="bitwiser.io: get wiser - bit by bit" />
  <meta property="og:description" content="Open source tech trivia game. by, for and of the developers" />
  <meta property="og:image" content="https://cdn-images-1.medium.com/max/1600/1*5RH1reNYpyBJOKauMMX2tw.png" />
  <meta property="fb:app_id" content="1436607329691182" />
  <meta property="fb:admins" content="10156413230908290" />
  <!--meta property="og:site_name" content="Site Name, i.e. Moz" /-->

  <!-- Twitter Card data -->
  <meta name="twitter:card" content="summary">
  <!--meta name="twitter:site" content="@publisher_handle"-->
  <meta name="twitter:title" content="bitwiser.io: get wiser - bit by bit">
  <meta name="twitter:description" content="Open source tech trivia game. by, for and of the developers">
  <!--meta name="twitter:creator" content="@author_handle"-->
  <!-- Twitter summary card with large image must be at least 280x150px -->
  <meta name="twitter:image:src" content="https://cdn-images-1.medium.com/max/1600/1*5RH1reNYpyBJOKauMMX2tw.png">
---
</head>

Google analytics

At this time, I also added google analytics to our site so we can start capturing more data about the user behavior. For now, we’ll just add the basic tag manager script so we can analyze the hits to the site. At a later point, we can add more specific data points for analyzing user behavior on our site.

For this, we must first go to google analytics and obtain a GA tracking code for our site. We can then plug this into our index.html -

// index.html
...

<script async src="https://www.googletagmanager.com/gtag/js?id=UA-122807814-1"></script>
  <script>
    (function (i, s, o, g, r, a, m) {
      i['GoogleAnalyticsObject'] = r;
      i[r] = i[r] || function () {
        (i[r].q = i[r].q || []).push(arguments)
      }, i[r].l = 1 * new Date();
      a = s.createElement(o),
        m = s.getElementsByTagName(o)[0];
      a.async = 1;
      a.src = g;
      m.parentNode.insertBefore(a, m)
    })(window, document, 'script', 'https://www.google-analytics.com/analytics.js', 'ga');
    ga('create', '<GA Tracking ID>', 'auto');// add your tracking ID here.
    ga('send', 'pageview');
    window.dataLayer = window.dataLayer || [];
    function gtag() { dataLayer.push(arguments); }
    gtag('js', new Date());
    gtag('config', '<GA Tracking ID>');
  </script>

Complete code for this part can be hound here.

Alternates to Angular universal

An alternate to SSR with angular universal is using a pre-renderer such as Rendertron (https://github.com/GoogleChrome/rendertron) or phantomJS (http://phantomjs.org/)

We can then check if the request header to see if the user-agent calling our page is a webcrawler bot and return the pre-rendered page instead of our client-side SPA. You can check an example of this here - https://github.com/anihalaney/rwa-trivia/blob/rendertron/functions/ssr.ts.

Next up

In this part we saw how we can use Angular universal with Express/NodeJS to enable server-side rendering for our app. SSR is a key element of our application as it enables search engines and social media sites to discover and crawl our web application.

In the next part, we would see how we can use workspaces (introduced in Angular 6) to separate out the admin piece of our application into it’s own SPA. Workspace will also help us when we decide to create a separate mobile app as it’s own application.

If you enjoyed this article please recommend and share and feel free to provide your feedback on the comments section.

Real World App Series (quick links) :

Series summary and index: Part X

• 0: From zero to cli-ng, 1: Not another todo list!
• 2: It’s a Material World!, 3: Form Formation
• 4: State of my SPA, 5: Light my fire
• 6: 3Rs — Roles, Rules & Routes, 6.1: Upgrading to 4.0.0-rc.2
• 7: Split my lazy loaded code, 8: Just Ahead of in Time
• 9, 9.1 and 9.2 : Unit Testing; X: Upgrading to Angular 4
• 11: Gameplay with Angular; 12: Cloud Functions for Firebase
• 13: Elasticsearch on google cloud; 14: Faceted Search
• 15: Material UI Design; 16: Firebase to firestore
• 17: Firebase Cloud Storage; 18: Revisiting ngrx
• 19: Two Player; 20: Upgrading to Angular 6
• 21: PWA; 22: e2e Testing

References

• https://angular.io/guide/universal
• https://hackernoon.com/deploying-angular-universal-v6-with-firebase-c86381ddd445
• https://medium.freecodecamp.org/what-exactly-is-client-side-rendering-and-hows-it-different-from-server-side-rendering-bd5c786b340d
• https://angular.io/guide/universal
• https://dev.to/stereobooster/server-side-rendering-or-ssr-what-is-it-for-and-when-to-use-it-2cpg

Real World App - Part 23: SSR with Angular Universal was originally published in Real World Full Stack on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is Part 22 of our Real World App series. For other parts, see links at the bottom of this post or go to https://blog.realworldfullstack.io/. The app is currently under development and can be viewed at https://bitwiser.io and the github repo can be found here.

Recap

In Part 9 of the series, I had introduced unit tests to our app. Unfortunately due to time constraints I was not able to keep them up to date as we moved along with development. In this part, I would start fresh first by purging all unit tests and then adding a few unit and end to end tests. As we move along the development, I would incrementally add more tests in the future.

Unit Tests with Jest

Since the unit tests were outdated, I decided to first delete all the existing tests to avoid confusion. As we are starting fresh, we decided to go with Jest - an open source testing platform from Facebook.

Karma/Jasmine vs Jest

One of the advantages of using Jest over Jasmine/Karma was speed. Tests with Jest are significantly faster (2.5x times faster).

Some other advantages of Jest include out of the box code coverage, snapshot testing, and ability to run tests in parallel. Unlike karma, Jest does not run in a browser, but instead uses jsdom. See references for links.

Jest with Angular

Setting up jest with Angular for testing is a breeze. Simply install the dependencies -

npm install --save-dev @types/jest jest jest-preset-angular

To package.json, add a jest section -

"jest": {
  "preset": "jest-preset-angular",
  "setupTestFrameworkScriptFile": "<rootDir>/src/setup-jest.ts"
}

and the jest run commands -

"scripts" : {
...
"test": "jest",
"test:watch": "jest --watch",
...
}

Let’s also add the setup-jest.ts

import 'jest-preset-angular';
import './jestGlobalMocks';

And we’ll add our global mocks here to jestGlobalMocks -

const mock = () => {
  let storage = {};
  return {
    getItem: key => key in storage ? storage[key] : null,
    setItem: (key, value) => storage[key] = value || '',
    removeItem: key => delete storage[key],
    clear: () => storage = {},
  };
};

Object.defineProperty(window, 'localStorage', { value: mock() });
Object.defineProperty(window, 'sessionStorage', { value: mock() });
Object.defineProperty(window, 'getComputedStyle', {
  value: () => ['-webkit-appearance']
});

Jest unit test

You’ll see that tests written in Jest are very similar to the ones we wrote in Jasmine in Part 9. Here is a sample test file -

// newsletter.component.spec.ts

import { TestBed, ComponentFixture, async, fakeAsync } from '@angular/core/testing';
import { ReactiveFormsModule, FormsModule } from "@angular/forms";
import { NewsletterComponent } from './newsletter.component';
import { StoreModule, Store } from '@ngrx/store';
import { User, Subscription } from '../../../model';
import { TEST_DATA } from '../../../testing/test.data';
import { subscribeOn } from 'rxjs/internal/operators/subscribeOn';

describe('Component: NewsletterComponent', async () => {

    let component: NewsletterComponent;
    let fixture: ComponentFixture<NewsletterComponent>;
    let _store: any;
    let spy: any;
    let user: User;

    beforeEach(() => {

        // refine the test module by declaring the NewsletterComponent component
        TestBed.configureTestingModule({
            imports: [ReactiveFormsModule, FormsModule, StoreModule.forRoot({})],
            providers: [Store],
            declarations: [NewsletterComponent]
        });

        // create component and NewsletterComponent fixture
        fixture = TestBed.createComponent(NewsletterComponent);

        // get NewsletterComponent component from the fixture
        component = fixture.componentInstance;
        component.user = user;

        // get the injected instances
        _store = fixture.debugElement.injector.get(Store);

        // get object of action
        spy = spyOn(_store, 'dispatch');

        // dispatch service to get TotalSubscriber count
        spy.and.callFake((action: any) => {
            expect(action.GetTotalSubscriber);
        });

        component.ngOnInit();

        expect(_store.dispatch).toHaveBeenCalled();

    });

    it('form invalid when empty', () => {
        expect(component.subscriptionForm.valid).toBeFalsy();
    });

    it('email field validity', () => {
        let errors = {};
        const email = component.subscriptionForm.controls['email'];

        // Email field is required
        errors = email.errors || {};
        expect(errors['required']).toBeTruthy();

        // Set incorrect value to email
        email.setValue('test');
        errors = email.errors || {};
        expect(errors['required']).toBeFalsy();
        expect(errors['pattern']).toBeTruthy();

        // Set correct value to email
        email.setValue('demo@example.com');
        errors = email.errors || {};
        expect(errors['required']).toBeFalsy();
        expect(errors['pattern']).toBeFalsy();
    });

    it('subscription for normal user', () => {
        expect(component.subscriptionForm.valid).toBeFalsy();
        component.subscriptionForm.controls['email'].setValue('test@test.com');
        expect(component.subscriptionForm.valid).toBeTruthy();

        // dispatch service to save subscribe email

        const subscription = new Subscription();
        subscription.email = component.subscriptionForm.controls['email'].value;
        if (user) {
            subscription.userId = user.userId;
        }
        spy.and.callFake((action: any) => {
            expect(action.AddSubscriber);
            expect(action.payload.subscription).toEqual(subscription);
        });

        // Trigger the subscribe function
        component.onSubscribe();

        expect(_store.dispatch).toHaveBeenCalled();

        // Now we can check to make sure the emitted value is correct
        expect(component.subscriptionForm.get('email').value).toBe('test@test.com');
    });

    it('subscription for logged in user', () => {
        expect(component.subscriptionForm.valid).toBeFalsy();
        user = { ...TEST_DATA.userList[0] };
        component.user = user;
        component.subscriptionForm.controls['email'].setValue(user.email);
        expect(component.subscriptionForm.valid).toBeTruthy();

        // dispatch service to save subscribe email

        const subscription = new Subscription();
        subscription.email = user.email;
        if (user) {
            subscription.userId = user.userId;
        }

        spy.and.callFake((action: any) => {
            expect(action.AddSubscriber);
            expect(action.payload.subscription).toEqual(subscription);
        });

        // Trigger the subscribe function
        component.onSubscribe();

        expect(_store.dispatch).toHaveBeenCalled();

        // Now we can check to make sure the emitted value is correct
        console.log("expected" + JSON.stringify(user));
        expect(component.subscriptionForm.get('email').value).toBe(user.email);
        expect(component.user.userId).toBe(user.userId);
    });
});

Complete code for this part.

To run the test, we can simply execute “npm run test”.

End to End Testing with Protractor

The cli has already configured our project for end to end testing with Protractor, including the tsconfig and protractor.conf.js.

In order to write e2e tests, we need 2 files.

A Page Object file that contains a class which describes a high level view of our component or page. This class contains logic to find all the elements on the component (and how to navigate to the page, if applicable) -

// newsletter.po.ts

import { browser, element, by, Key } from 'protractor';
export class NewsLetterPage {
    getTitle() {
        const titleElement = element(by.css('.subscrib h2'));
        if (titleElement.isPresent) {
            return titleElement.getText();
        }
    }

    getSubTitle() {
        const subtitleElement = element(by.css('.subscrib p'));
        if (subtitleElement.isPresent) {
            return subtitleElement.getText();
        }
    }

    getEmailElement() {
        return element(by.css('input[formControlName=email]'));
    }

    getSubscribeButton() {
        return element(by.cssContainingText('button', 'Subscribe'));
    }

    getRequiredMessage() {
        const errorElement = element(by.css('.errorSpan'));
        if (errorElement.isPresent) {
            return errorElement.getText();
        }
    }

    getResponseMessage() {
        const responseElement = element(by.css('.subscrib-form p'));
        if (responseElement.isPresent) {
            return responseElement.getText();
        }
    }
}

An e2e test spec file using Jasmine to write our test cases. We first initialize our page object and use that to test the functionality of our page.

// newsletter.e2e-spec.ts

import { NewsLetterPage } from './newsletter.po';
import { browser, element, by, Key } from 'protractor';

describe('NewsLetterPage', () => {

    let page: NewsLetterPage;
    page = new NewsLetterPage();
    // page.navigateTo();

    it('Should display title', () => {
        browser.waitForAngularEnabled(false);
        expect(page.getTitle()).toMatch('Sign up for our newsletter to stay connected');
    });

    it('Should display subtitle', () => {
        browser.waitForAngularEnabled(false);
        expect(page.getSubTitle()).toMatch('Subscribe to our bi-weekly email newsletter for useful tips and valuable resources.');
    });

    it('Should check email validation by clicking Subscribe Button', () => {
        browser.waitForAngularEnabled(false);
        browser.driver.sleep(500);
        page.getSubscribeButton().click();
        expect(page.getRequiredMessage()).toMatch('Email is required!');
        browser.driver.sleep(2000);

        page.getEmailElement().sendKeys('test')
        page.getSubscribeButton().click();
        expect(page.getRequiredMessage()).toMatch('Invalid Email!');
        browser.driver.sleep(2000);

        page.getEmailElement().clear();
        browser.driver.sleep(500);
        page.getEmailElement().sendKeys('test@test.com');
        page.getSubscribeButton().click();
        expect(page.getResponseMessage()).toBeDefined();
        browser.driver.sleep(5000);
    });
});

We can now test this using “npm run e2e”. Note that the site must be running on the port specified in our protractor.conf.js, prior to running the e2e tests.

Complete code for this part.

We have added some basic unit tests and e2e tests in this part. I would continue to add more tests as we touch various components as part of the continued

Next Up

Server side rendering

If you enjoyed this article please recommend and share and feel free to provide your feedback on the comments section.

Real World App Series (quick links) :

Series summary and index: Part X

• 0: From zero to cli-ng, 1: Not another todo list!
• 2: It’s a Material World!, 3: Form Formation
• 4: State of my SPA, 5: Light my fire
• 6: 3Rs — Roles, Rules & Routes, 6.1: Upgrading to 4.0.0-rc.2
• 7: Split my lazy loaded code, 8: Just Ahead of in Time
• Parts 9, 9.1 and 9.2 : Unit Testing
• 11: Gameplay with Angular; 12: Cloud Functions for Firebase
• 13: Elasticsearch on google cloud; 14: Faceted Search
• 15: Material UI Design; 16: Firebase to firestore
• 17: Firebase Cloud Storage; 18: Revisiting ngrx
• 19: Two Player; 20: Upgrading to Angular 6
• 21: PWA;

References:

• https://jestjs.io/
• https://semaphoreci.com/community/tutorials/testing-angular-2-and-continuous-integration-with-jest
• https://www.xfive.co/blog/testing-angular-faster-jest/
• https://izifortune.com/unit-testing-angular-applications-with-jest/
• https://github.com/jest-community/vscode-jest
• https://coryrylan.com/blog/introduction-to-e2e-testing-with-the-angular-cli-and-protractor
• https://angular.io/guide/testing

Real World App - Part 22: Angular Testing with Protractor, Jasmine and Jest was originally published in Real World Full Stack on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is Part 21 of our Real World App series. For other parts, see links at the bottom of this post or go to https://blog.realworldfullstack.io/. The app is currently under development and can be viewed at https://bitwiser.io and the github repo can be found here.

Recap

In Part 20 of the series, we upgraded our app to Angular 6. In this part we’ll implement service workers to make our app a Progressive Web App. Final code for this part here.

What are Progressive Web Apps?

Progressive Web Apps (PWA) are apps built using a series of web technologies, patterns and APIs that allow a web application to appear like a native mobile app. For more details see - https://developer.mozilla.org/en-US/Apps/Progressive/Introduction

Service Workers

Service workers are an essential element of a PWA.

A service worker is type of a web worker (executes script in a background thread separate from the main execution thread of a web application) that intercepts network requests, and can cache or retrieve resources from the cache. Service workers also allow delivering push notifications to the browser.

Manifest file

The web app manifest provides information about an application (such as its name, author, icon, and description) in a JSON text file. The manifest informs details for websites installed on the homescreen of a device, providing users with quicker access and a richer experience. In addition, Chrome on Android will proactively suggest that the user install the web app, via a web app install banner.

For benefits of PWA and details of service worker architecture check the links in the references section.

Service Workers in Angular

Angular ships with service worker implementation since version 5. The cli makes adding service worker to your project really easy -

ng add  @angular/pwa --project trivia

The above command does the following:

1. Adds the @angular/service-worker package to your project.
2. Enables service worker build support in the CLI.
3. Imports and registers the service worker in the app module.
4. Updates the index.html file:

• Includes a link to add the manifest.json file.
• Adds meta tags for theme-color.

5. Installs icon files to support the installed Progressive Web App (PWA).

6. Creates the service worker configuration file called ngsw-config.json, which specifies the caching behaviors and other settings.

You can see the changes in github.

manifest.json file

A manifest.json file is created with name, icons and other settings. We can change this file as create new icons for our app as needed.

Service worker configuration

The ngsw-config.json provides the configuration for service worker. It specifies the caching behavior of various assets used in the application with wild cards. When the app is built by the cli with prod flag, it will use this config and use this to generate ngsw.json file with names of each file it finds matching these wild cards. This json file is used as the run time configuration by the ngsw-worker.js

Debugging with Chrome dev tools

Once deployed with prod flag enabled, you can test the service worker in using Application tab in the Chrome dev tools.

You can see how the static assets are cached in Cache Storage. You can also try to take your app Offline and reload to see how the app behaves when the assets are cached.

You can also look at the manifest in Chrome dev tools -

Testing with Chrome on Android

When you test the app on Android using a Chrome browser, you‘ll see that Chrome prompts you to “Add to Home Screen”

Summary

Angular provides an easy way to add service worker and PWA to our application. In addition to the option of adding to home screen on mobile, service workers provide several benefits such as caching for performance and offline support, push notifications, etc. We’ll also look at lighthouse plugin as a tool for testing our PWA.

In later parts we will revisit service workers and see how we can add push notifications and measure performance improvements and smoothen out the offline support. Complete code for this part, which also includes bug fixes and other minor enhancements here.

Next Up

In Part 9, we introduced unit testing, but I haven’t been able to keep the tests up to date. In the next part, I’ll start adding these back and also start with some end-to-end test cases.

If you enjoyed this article please recommend and share and feel free to provide your feedback on the comments section.

Real World App Series (quick links) :

Series summary and index: Part X

• 0: From zero to cli-ng, 1: Not another todo list!
• 2: It’s a Material World!, 3: Form Formation
• 4: State of my SPA, 5: Light my fire
• 6: 3Rs — Roles, Rules & Routes, 6.1: Upgrading to 4.0.0-rc.2
• 7: Split my lazy loaded code, 8: Just Ahead of in Time
• Parts 9, 9.1 and 9.2 : Unit Testing
• 11: Gameplay with Angular; 12: Cloud Functions for Firebase
• 13: Elasticsearch on google cloud; 14: Faceted Search
• 15: Material UI Design; 16: Firebase to firestore
• 17: Firebase Cloud Storage; 18: Revisiting ngrx
• 19: Two Player; 20: Upgrading to Angular 6

References:

• https://developer.mozilla.org/en-US/Apps/Progressive/Introduction
• https://developers.google.com/web/fundamentals/primers/service-workers/
• https://developers.google.com/web/ilt/pwa/introduction-to-service-worker
• https://www.youtube.com/watch?v=jVfXiv03y5c
• https://angular.io/guide/service-worker-intro
• https://blog.angular-university.io/angular-service-worker/

Real World App - Part 21: Service Workers (PWA) with Angular was originally published in Real World Full Stack on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is Part 20 of our Real World App series. For other parts, see links at the bottom of this post or go to https://blog.realworldfullstack.io/. The app is currently under development and can be viewed at https://bitwiser.io and the github repo can be found here.

Recap

In Part 19 of the series, we built out all the major functional pieces of our app. In this part, as we do from time to time, we’ll upgrade to the latest versions of our dependencies. Final code for this part here.

What’s new?

Angular & cli 6

The Angular 6 update is less of an update in terms of the features, but more of an update of the underlying tool set around Angular. As part of the update, the angular team brought out updates to angular cli and material+cdk, while the ngrx team updated their project also to version 6. In addition we have a new version of RxJS (also 6, even though it’s version is not directly related to that of Angular).

New features of Angular includes -

• Schematics - Allows ease of development and updates to your dependencies
• Elements - Allows you to ship Angular components as custom elements and use them in the browser without the framework dependency
• Workspaces - Support for multiple workspaces in a single repo. The angular-cli.json is replaced with angular.json and supports build configuration of multiple workspaces

A summary of changes can be seen here - https://blog.angular.io/version-6-of-angular-now-available-cc56b0efa7a4. See references for more links on details.

RxJS 6

The two breaking changes introduced in RxJS are -

• Due to internal structure changes the import statements need to change
• Use of pipe as a method to chain operators

Fortunately, for the 6.x release RxJS provides rxjs-compat library that enables backward compatibility with the prior version with minimal code change. However, this will be deprecated with RxJS 7, so we’ll ensure everything is working without the compat library by the end of this part.

Let’s start our upgrade …

Upgrading to 6

We start by updating the global version of angular-cli by uninstalling the current version and then installing the latest 6.x.x. We will also update all our local packages to the latest versions (except for typescript - we’ll use the latest version currently supported by Angular 6 - 2.7.2).

We’ll then execute the ng update commands

ng update @angular/cli
ng update @angular/core
ng update

RxJS (compat)

In order to not modify all the operators, I used rxjs-compat so we can get everything to compile first

npm install --save rxjs-compat

When I tried to compile this, I got a bunch of errors on using ngrx select, when selecting from a slice of the store. So, everywhere we used the “select“ from the stores’ slice had to be changed as follows -

this.store.select(adminState).select(s => s.questionsSearchResults);

to a pipeable select. Note, we had to import select from ngrx/store

import { select } from '@ngrx/store';
...
this.questionsSearchResultsObs = this.store.select(adminState).pipe(select(s => s.questionsSearchResults));

csv parser

After resolving the above I was still getting a Typescript error on the csv parser - Module not found: Error: Can’t resolve ‘stream’ in csv-parse. The csv parser library we were using (csv) was essentially a NodeJs library. We decided to remove our dependency on this and instead use the papaparse library

npm uninstall csv
npm install papaparse

And changed the code to use this library

Angularfire2

The getDownloadUrl() method on AngularFireUploadTask was removed, instead we have to subscribe to the snapshotChanges observable on finalize. (https://github.com/angular/angularfire2/blob/master/docs/storage/storage.md#monitoring-upload-percentage) This needed to be changed -

const socialShareImageObj = this.storage.upload(`${this.basePath}/${userId}/${this.folderPath}/${new Date().getTime()}`, imageBlob);
return socialShareImageObj.downloadURL().map(url => url);

was changed to -

const filePath = `${this.basePath}/${userId}/${this.folderPath}/${new Date().getTime()}`;
const fileRef = this.storage.ref(filePath);

const socialShareImageObj = this.storage.upload(filePath, imageBlob);
return socialShareImageObj.snapshotChanges().pipe(
  finalize(() => fileRef.getDownloadURL())
);

Finally, I was able to compile after resolving these.

Runtime errors (Angular Material)

After resolving all the compile errors, I thought that was it, but when trying to execute, I started getting some runtime errors -

Uncaught Error: Template parse errors:
‘mat-input-container’ is not a known element:

From cli change log: The following deprecated CSS classes have been removed:

• mat-input-container instead use mat-form-field

So, I changed all occurrences of “mat-input-container”s to “mat-form-field”. (I later realized, I could’ve used “ng update @angular/material” and that should’ve taken care of this issue. Oh well!)

And that was it! Everything worked again!

RxJs (removing rxjs-compat)

Finally, I removed the dependency on rxjs-compat and fixed all the imports and then operators to use pipes. Excellent concise article here — https://www.academind.com/learn/javascript/rxjs-6-what-changed/

Final code on the repo.

We’re getting pretty close to a working system and fixing bugs as we go along.

Next Up

PWA / Service Workers.

If you enjoyed this article please recommend and share and feel free to provide your feedback on the comments section.

Real World App Series (quick links) :

Series summary and index: Part X

• 0: From zero to cli-ng, 1: Not another todo list!
• 2: It’s a Material World!, 3: Form Formation
• 4: State of my SPA, 5: Light my fire
• 6: 3Rs — Roles, Rules & Routes, 6.1: Upgrading to 4.0.0-rc.2
• 7: Split my lazy loaded code, 8: Just Ahead of in Time
• Parts 9, 9.1 and 9.2 : Unit Testing
• 11: Gameplay with Angular; 12: Cloud Functions for Firebase
• 13: Elasticsearch on google cloud; 14: Faceted Search
• 15: Material UI Design; 16: Firebase to firestore
• 17: Firebase Cloud Storage; 18: Revisiting ngrx
• 19: Two Player

References

Angular 6

• https://blog.angular.io/version-6-of-angular-now-available-cc56b0efa7a4
• https://blog.angular.io/schematics-an-introduction-dc1dfbc2a2b2
• https://angular.io/guide/elements
• https://github.com/angular/angular-cli/wiki/angular-workspace

RxJS 6

• https://github.com/ReactiveX/rxjs/blob/master/docs_app/content/guide/v6/migration.md
• https://auth0.com/blog/whats-new-in-rxjs-6/
• https://www.academind.com/learn/javascript/rxjs-6-what-changed/

AngularFire2

• https://github.com/angular/angularfire2/blob/master/docs/storage/storage.md#monitoring-upload-percentage

Angular Material 6

• https://github.com/angular/material2/blob/master/CHANGELOG.md

Real World App - Part 20: Angular, ngrx & cli version 6 was originally published in Real World Full Stack on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is Part 19 of our Real World App series. For other parts, see links at the bottom of this post or go to https://blog.realworldfullstack.io/. The app is currently under development and can be viewed at https://bitwiser.io and the github repo can be found here.

Recap

In Part 18 of the series, we upgraded to ngrx 5. In this part we will focus on adding several features to our application including two player game play, revising our single player mode, revamping our dashboard, fixing bulk upload, stats and leaderboard and much more. In essence it will be a feature complete beta version.

We will also do a major refactor of our firebase functions codebase. Since this is essentially a feature rich part of development, I’ll discuss the features here rather than the code itself. Check out the code for this part here.

Firebase functions

As the functionality of our firebase functions was growing it became increasingly unmanageable to have a single app.ts file for all http functions. So we decided to break down our app into a traditional express app with routes, controllers, middleware and services. This gave us a set of manageable smaller modules. The new functions folder can be seen here - https://github.com/anihalaney/rwa-trivia/tree/part-19/functions.

Game Play mechanics

Two player

A two player game could be played either with a random player or a friend. Once the game is started with a random player, the system will try to match you with another player who has also started the game at around the same time. With each correct question the player earns a badge. When a player answers incorrectly, the turn goes to the other player. The first to earn 5 badges wins the game. The game can last up to 16 rounds. If no one wins 5 badges in 16 rounds, then the game ends in a tie. Each player has 32 hours to play their turn, else the player defaults on the game.

Right now, the mechanism to earn a badge and random player selection are pretty basic. As we grow in content (questions and answers) and broaden the user base, we will incorporate a more intelligent match between random players.

Single player

The single player game play was also modified to be in line with the 2 player for consistency, albeit a much simpler version of it. 5 correct answers will win 5 badges and the game. Answer 3 questions incorrectly and you lose the game.

The game play mechanism involves significant code changes both to the front-end and firebase functions. A schedule job also checks for the 32 hour turn limit and marks the game over flag if applicable.

The dashboard is changed to reflect the badges & rounds.

Friends

Invitation to friends can be send by entering their email address. We currently use gmail smtp to send invitations.

Stats & Leaderboard

Stats are computed at a game level, user level and at system level. We use firebase triggers to compute these stats. See code here. The top 10 category leaders are shown on the dashboard.

Blog links

The links to blogs are acquired using a firebase function that fetches blogs from an rss feed. In our case the feed link is https://blog.realworldfullstack.io/feed. The function extracts the top 3 latest blog and the relevant fields from the feed and stores it in the database. The firebase function is called on a scheduler. Code here.

Social Share

Social share buttons are now active on the site. However, since our code is all client-side, the social sites are unable to render these when the page is shared. We will fix that with Angular universal and server-side rendering in our future posts.

This part has by far the biggest feature set we have done in our series, which is why the code changes are significant, even though we haven’t introduced much new technical concepts. It is also the reason why I haven’t posted an update in a long time. I’ll try to post smaller incremental updates from now on.

Next Up

Upgrade to Angular 6.

If you enjoyed this article please recommend and share and feel free to provide your feedback on the comments section.

Real World App Series (quick links) :

Series summary and index: Part X

• 0: From zero to cli-ng, 1: Not another todo list!
• 2: It’s a Material World!, 3: Form Formation
• 4: State of my SPA, 5: Light my fire
• 6: 3Rs — Roles, Rules & Routes, 6.1: Upgrading to 4.0.0-rc.2
• 7: Split my lazy loaded code, 8: Just Ahead of in Time
• Parts 9, 9.1 and 9.2 : Unit Testing
• 11: Gameplay with Angular; 12: Cloud Functions for Firebase
• 13: Elasticsearch on google cloud; 14: Faceted Search
• 15: Material UI Design; 16: Firebase to firestore
• 17: Firebase Cloud Storage; 18: Revisiting ngrx

References

• https://www.terlici.com/2014/08/25/best-practices-express-structure.html

Real World App - Part 19: Ready Player Two was originally published in Real World Full Stack on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is Part 18 of our Real World App series. For other parts, see links at the bottom of this post or go to https://blog.realworldfullstack.io/. The app is currently under development and can be viewed at https://bitwiser.io and the github repo can be found here.

Recap

In Part 4 of the series, I explained how we managed state of our application using ngrx. ngrx has undergone quite a few changes since and we’ll upgrade our application to reflect on those changes.

In this part we will upgrade our app to improve our state management using ngrx 5 and the new features it provides

In upgrading our application store to ngx 5 we will look at the following features -

• Using selectors
• Split up store into feature modules
• Replace action functions with action classes
• Using pipeable functions in effects
• router-store
• Router actions in effects
• Use of entities (as needed)

Using selectors

Selectors are methods used for obtaining slices of store state.

A couple of links that explain it in depth -

• https://github.com/ngrx/platform/blob/master/docs/store/selectors.m
• https://toddmotto.com/ngrx-store-understanding-state-selectors

I would start by refactoring categoryDictionary, making it a selector, instead of a reducer -

// categories.reducer.ts

...
export const getCategoryDictionary = (state: Category[]) => {
  let categoryDict: {[key: number]: Category} = {};
    state.forEach(category => {
    categoryDict[category.id] = category;
  });
  return categoryDict;
}
...

and in app-store.ts

// app-store.ts

...
export const getState = (state: AppStore) => state;

//Categories selector
export const getCategories = createSelector(getState, (state: AppStore) => state.categories);
export const categoryDictionary = createSelector(getCategories, getCategoryDictionary);
...

We can then refer to the selector by simply using store.select(categoryDictionary)

Code for this commit.

Split up store into feature modules

ngrx 2 allowed us to have the state of our application in a central store, but didn’t have any easy mechanism to split that into feature modules.

This limitation meant we had to put our entire store in our root module, instead of placing it in relevant modules, even for lazy loaded features where at times we may never need those slices of the store to be loaded in our app. We start by moving the game-play part of the store in the game-play module.

I’ll not go into details of the code as most of it is simply moving the relevant store parts from core feature to game-play feature. A couple of things to note -

The game-play.module.ts will create a StoreModule for feature

// game-play.module.ts
...

  imports: [ ...

//ngrx feature store
    StoreModule.forFeature('gameplay', reducer),

//ngrx effects
    EffectsModule.forFeature(effects),

],
...

And we create a feature selector in app-store

//app-store.ts
...
export const gameplayState = createFeatureSelector<GamePlayState>('gameplay');
...

We can then refer to our gameplayState using ->

this.gameObs = store.select(gameplayState).select(s => s.currentGame)

Complete code for this commit.

Replace action functions with action classes

The payload property was removed from the Action interface as it was a source of type-safety issues. We had temporarily gotten around this by providing a ActionWithPayload interface in Part 14.

Here we completely replace these functions with classes and use the constructor to provide the payload.

//game-play.actions.ts
...
export enum GamePlayActionTypes {
  RESET_NEW = '[GamePlay] ResetNew',
  CREATE_NEW = '[GamePlay] CreateNew',
...
}

export class ResetNewGame implements Action {
  readonly type = GamePlayActionTypes.RESET_NEW;
  payload = null;
}

export class CreateNewGame implements Action {
  readonly type = GamePlayActionTypes.CREATE_NEW;
  constructor(public payload: {gameOptions: GameOptions, user: User}) {}
}
...
export type GamePlayActions
  = ResetNewGame
  | CreateNewGame
...

and we use it by creating a new instance -

this.store.dispatch(new gameplayactions.CreateNewGame({gameOptions: gameOptions, user: user}))

Complete code for the commit.

Using pipeable operators in effects

While pipeable operators are a new RxJs feature, rather than an ngrx feature, we use the opportunity to refactor our effects to use these operators.

Pipeable operators offer a mechanism of composing observable chains by using pure functions that can be used as standalone operators instead of methods on an observable. They’re lightweight and can decrease your overall build size by only including the operators being used. See reference section for further reading on this.

Sample effect -

//game-play.effects.ts
...
@Effect() 
  startNewGame$ = this.actions$
      .ofType(GamePlayActionTypes.CREATE_NEW)
      .pipe(
        switchMap((action: gameplayactions.CreateNewGame) => 
          this.svc.createNewGame(action.payload.gameOptions, action.payload.user).pipe(
            map((gameId: string) => new gameplayactions.CreateNewGameSuccess(gameId))
            //catchError(error => new)
          )
        )
      );
...

router-store

Our application currently stores only the domain entities in the store. The router-store allows us to store the url and the router state in the store based on a route change action on the angular router. We can then have effects listen to these actions and invoke our services to load data as needed. The components would no longer need to listen to route changes, but simply observe on the store for changes.

A great explanation of the states of our application is covered by Victor Savkin in his post - https://blog.nrwl.io/using-ngrx-4-to-manage-state-in-angular-applications-64e7a1f84b7b.

The video series by Todd Motto listed in the reference section is also a great resource to understanding how this all comes together.

Let’s start by adding the router state and a custom serializer to our reducer -

//reducers/index.ts
import ...

export interface RouterStateUrl {
  url: string;
  queryParams: Params;
  params: Params;
}
export interface State {
  routerReducer: fromRouter.RouterReducerState<RouterStateUrl>
}

export const reducers: ActionReducerMap<State> = {
  routerReducer: fromRouter.routerReducer
}

export const routerState = 
  createFeatureSelector<fromRouter.RouterReducerState<RouterStateUrl>>('routerReducer');

export class CustomSerializer implements fromRouter.RouterStateSerializer<RouterStateUrl> {
  
  serialize(routerState: RouterStateSnapshot): RouterStateUrl {
    const {url} = routerState;
    const {queryParams} = routerState.root;

  let state: ActivatedRouteSnapshot = routerState.root;
    while (state.firstChild) {
      state = state.firstChild;
 }

    const {params} = state;
    return { url, queryParams, params };
  }
}

We can then list our CustomSerializer as a provider for RouterStateSerializer in the app.module.

Using router actions in effects

We can now observe on these router actions in our effects to load data.

//game.play.effects.ts
...
  //load from router
  @Effect()
  // handle location update
  loadGame$ = this.actions$
    .ofType('ROUTER_NAVIGATION')
    .map((action: any): RouterStateUrl => action.payload.routerState)
    .filter((routerState: RouterStateUrl) => 
      routerState.url.toLowerCase().startsWith('/game-play/') && 
      routerState.params.gameid
    )
    .pipe(
      switchMap((routerState: RouterStateUrl) =>
        this.svc.getGame(routerState.params.gameid).pipe(
          map((game: Game) => new gameplayactions.LoadGameSuccess(game))
        )
      )
    );
...

As we can see above, we can look for the game-play route to load our game. We can remove the dispatch action for the route from our component.

We’ll go ahead and implement these changes across the application. We’ll add stores for core and bulk feature modules. The complete code for this part is here.

In this part, we refactored our store into smaller feature module stores and used the router-store to manage our route states. We also fixed some bugs and made UI changes to cleanup our bulk upload.

Next Up

Two player game play.

If you enjoyed this article please recommend and share and feel free to provide your feedback on the comments section.

Real World App Series (quick links) :

Series summary and index: Part X

• 0: From zero to cli-ng, 1: Not another todo list!
• 2: It’s a Material World!, 3: Form Formation
• 4: State of my SPA, 5: Light my fire
• 6: 3Rs — Roles, Rules & Routes, 6.1: Upgrading to 4.0.0-rc.2
• 7: Split my lazy loaded code, 8: Just Ahead of in Time
• Parts 9, 9.1 and 9.2 : Unit Testing
• 11: Gameplay with Angular; 12: Cloud Functions for Firebase
• 13: Elasticsearch on google cloud; 14: Faceted Search
• 15: Material UI Design; 16: Firebase to firestore
• 17: Firebase Cloud Storage

References

ngrx

• https://github.com/ngrx/platform
• https://github.com/ngrx/platform/blob/master/MIGRATION.md
• https://blog.nrwl.io/ngrx-patterns-and-techniques-f46126e2b1e5
• https://blog.nrwl.io/using-ngrx-4-to-manage-state-in-angular-applications-64e7a1f84b7b
• https://medium.com/ngrx/ngrx-5-and-schematics-2d8cc3906506
• https://github.com/ngrx/platform/blob/master/docs/router-store/api.md

Excellent video on ngrx

• https://www.youtube.com/watch?v=N_UQx8dPPkc&list=PLW2eQOsUPlWJRfWGOi9gZdc3rE4Fke0Wv

Pipeable operators

• https://blog.hackages.io/rxjs-5-5-piping-all-the-things-9d469d1b3f44
• https://blog.angularindepth.com/rxjs-understanding-lettable-operators-fe74dda186d3

Real World App - Part 18: Revisiting ngrx was originally published in Real World Full Stack on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is Part 17 of our Real World App series. For other parts, see links at the bottom of this post or go to https://blog.realworldfullstack.io/. The app is currently under development and can be viewed at https://bitwiser.io and the github repo can be found here.

Recap

In the previous post, we migrated our data from Firebase realtime database (rtdb) to firestore.

In this part, we’ll add a couple of features to the application. The bulk upload feature will allow select users upload question/answer content in bulk instead of doing it one at a time. We’ll also add an initial version of user profile settings.

Bulk upload feature

The bulk upload feature will allow users (with appropriate role) to upload a csv file with Question and Answers in a batch. The file will get saved into the firebase storage and the data will be imported into the unpublished questions. The admin will then be able to verify them, approve/reject or send it back to be revised.

The instructions to upload are shown below and the material stepper control will be used for uploading the file. We’re restricting the file to have only one category at a time. If a user wants to upload Qs for multiple categories, they must do it one file at a time.

User Profile feature

The user profile allows the user their basic profile info. The user profile will also need firebase storage to store avatar images of the user.

Cloud storage

Cloud Storage stores your files in a Google Cloud Storage bucket, making them accessible through both Firebase and Google Cloud. This allows you the flexibility to upload and download files from mobile clients via the Firebase SDKs, and do server-side processing such as image filtering or video transcoding using Google Cloud Platform.

Firebase Cloud storage security rules

The firebase storage rules works similar to the firestore database. Refer to the documentation for an understanding the security rules - https://firebase.google.com/docs/storage/security/

We’ll start initially by allowing all authenticated users to read/write, but later setup so the users can only see their files -

service firebase.storage {
  match /b/{bucket}/o {
    match /{allPaths=**} {
      allow read, write: if request.auth != null;
    }
  }
}

For final rules for this part see here

Upgrading angularfire2

As always, we will upgrade our components to the latest versions.

NOTE: The support for firebase storage sdk was introduced in version 5.0.0-rc.6

This part has large amount of code change, but a most of it is wiring up our components, store and services that we’ve seen enough times in this series. The complete source for this part is here.

We’ll simply focus on code related to storage and file uploads.

Make sure to include the “AngularFireStorageModule” in our core.module.ts

The following function in question.service.ts uploads the file to the storage. The file is created under the users uid node, so we can use the security rules appropriately). It also has a unique id as part of it’s name, that will be be used to refer it back from the questions saved in the firestore db.

//question.service.ts

...
saveBulkQuestions(bulkUpload: BulkUpload) {
    const dbQuestions: Array<any> = [];
    const bulkUploadFileInfo = bulkUpload.bulkUploadFileInfo;
    const questions = bulkUpload.questions;

const bulkUploadId = this.db.createId();
    // store file in file storage
    // Not written any code monitor progress or error
    this.storage.upload(`bulk_upload/${bulkUploadFileInfo.created_uid}/${bulkUploadId}-${bulkUpload.file.name}`, bulkUpload.file);
    for (const question of questions) {
      if (question !== null) {
        question.bulkUploadId = bulkUploadId;
        const dbQuestion = Object.assign({}, question); // object to be saved
        dbQuestion.id = this.db.createId();
        // Do we really need to copy answer object as well?
        dbQuestion.answers = dbQuestion.answers.map((obj) => { return Object.assign({}, obj) });
        dbQuestions.push(dbQuestion);
      }
    }
    this.addBulkUpload(bulkUploadFileInfo, dbQuestions, bulkUploadId);
  }

...

bulk upload component

The bulk-upload.component incorporates the logic for parsing out the csv, validating it and displaying the questions back to the user before submitting it. Partial code below -

//bulk-upload.component.ts

...

onFileChange(event) {
    const reader = new FileReader();
    this.fileParseError = false;
    if (event.target.files && event.target.files.length > 0) {
      const file = this.file = event.target.files[0];
      this.uploadFormGroup.get('csvFile').setValue(file);
      reader.readAsText(file);
      reader.onload = () => {
        this.bulkUploadFileInfo = new BulkUploadFileInfo;
        this.bulkUploadFileInfo.fileName = file['name'];
        this.generateQuestions(reader.result);
      };
    }

}

generateQuestions(csvString: string): void {
    this.questionValidationError = false;
    this.fileParseError = false;
    this.fileParseErrorMessage = '';

const parseOptions = {
      'columns': columns => {
        const validColumns = ['Question', 'Option 1', 'Option 2', 'Option 3', 'Option 4', 'Answer Index', 'Tag 1', 'Tag 2', 'Tag 3',
          'Tag 4', 'Tag 5', 'Tag 6', 'Tag 7', 'Tag 8', 'Tag 9'];
        if (validColumns.join(',') === columns.join(',')) {
          return columns;
        } else {
          this.fileParseError = true;
          this.fileParseErrorMessage = 'File format is not correct, must be in CSV format, must not have missing or wrong column order';
          return '';
        }
      },
      'skip_empty_lines': true
    };

parse(csvString, parseOptions,
      (err, output) => {
        if (output !== undefined && output !== '') {
          this.questions =
            output.map(element => {
              const question: Question = new Question();
              question.questionText = element['Question'];
              question.answers = [
                { 'id': 1, 'answerText': element['Option 1'], correct: false },
                { 'id': 2, 'answerText': element['Option 2'], correct: false },
                { 'id': 3, 'answerText': element['Option 3'], correct: false },
                { 'id': 4, 'answerText': element['Option 4'], correct: false }
              ]

if (question.answers[element['Answer Index'] - 1] !== undefined) {
                question.answers[element['Answer Index'] - 1].correct = true;
              }

// add primary tag to question tag list
              question.tags = [this.uploadFormGroup.get('tagControl').value];

for (let i = 1; i < 10; i++) {
                if (element['Tag ' + i] && element['Tag ' + i] !== '') {
                  // check for duplicate tags
                  if (question.tags.indexOf(element['Tag ' + i].trim()) === -1) {
                    question.tags.push(element['Tag ' + i].trim());
                  }
                }
              }

question.published = false;
              question.explanation = 'status - not approved';
              question.status = QuestionStatus.PENDING;
              question.created_uid = this.user.userId;

if (!question.questionText || question.questionText.trim() === '') {
                this.questionValidationError = true;
                question.validationErrorMessages.push('Missing Question');
              } else if (question.answers[0].answerText.trim() === '' || question.answers[1].answerText.trim() === '' ||
                question.answers[2].answerText.trim() === '' || question.answers[3].answerText.trim() === '') {
                this.questionValidationError = true;
                question.validationErrorMessages.push('Missing Question Answer Options');
              } else if (question.answers.filter(a => a.correct).length !== 1) {
                this.questionValidationError = true;
                question.validationErrorMessages.push('Must have exactly one correct answer');
              } else if (question.answers.filter(a => !a.answerText || a.answerText.trim() === '').length > 0) {
                this.questionValidationError = true;
                question.validationErrorMessages.push('Missing Answer');
              } else if (question.tags.length < 3) {
                this.questionValidationError = true;
                question.validationErrorMessages.push('Atleast 3 tags required');
              }

return question;
            });
          this.bulkUploadFileInfo.uploaded = this.questions.length;
        }
      });
  }

private prepareUpload(): any {
    const input = new FormData();
    input.append('category', this.uploadFormGroup.get('category').value);
    input.append('tag', this.uploadFormGroup.get('tagControl').value);
    input.append('csvFile', this.uploadFormGroup.get('csvFile').value);
    return input;
  }

onUploadSubmit() {
    // validate
    if (!this.uploadFormGroup.valid || this.bulkUploadFileInfo === undefined) {
      return;

} else {
      const formModel = this.prepareUpload();

const dbQuestions: Array<Question> = [];

for (const question of this.questions) {
        this.bulkUploadFileInfo.categoryId = this.uploadFormGroup.get('category').value;
        this.bulkUploadFileInfo.primaryTag = this.uploadFormGroup.get('tagControl').value;
        question.categoryIds = [this.uploadFormGroup.get('category').value];
        question.createdOn = new Date();
        dbQuestions.push(question);
      }
      this.bulkUploadFileInfo.created_uid = this.user.userId;
      this.bulkUploadFileInfo.date = new Date().getTime();
      this.parsedQuestions = dbQuestions;
    }
  }

...

Profile settings

For uploading the avatar for the user profile, we’ll use the ngrx-image-cropper library. This will allow for cropping and adjusting the aspect ration of the uploaded image.

As with bulk upload, we will use the firebase storage to store user images on the cloud storage.

See profile-settings.component.ts for complete code.

Storage rules

We need to adjust our rules so that the bulk uploaded files can be created by an authenticated user but restrict read only to the creator. On the other hand, the profile avatar image can be read by everyone.

The admins will have permissions to everything. Here are the rules -

service firebase.storage {

function isAuthenticated() {
    return request.auth != null;
  }
 // Add uid of admin user 
 // This is temp solution for time being till fetching role from user document does not work
  function isAdmin() {
   return isAuthenticated() && 
       (request.auth.uid == '');
  }

function isUserAuthenticated(userId) {
    return isAuthenticated() && request.auth.uid == userId;
  }
  
  match /b/{bucket}/o {
    match /templates {
      // Anyone can view any template file
      match /{allFiles=**} {
        allow read;
      }
    }
    match /questions {
      // questions files, only authenticated user (for now)
      match /{allFiles=**} {
        allow read, write: if request.auth != null;
      }
    }
     match /bulk_upload/{userId} {
      // bulk upload files, only authenticated user allowed to write
      // only user is allowed to write user folder
      // request.resource != null, ensure that delete operation is not allowed
      // resource == null, ensure that update/overwrite operation on file is not allowed
      match /{allFiles=**} {
        allow write: if isUserAuthenticated(userId)
                      && request.resource != null
                      && resource == null;
        allow read:  if isUserAuthenticated(userId)
               || isAdmin();                      
      }
    }
    
    match /profile/{userId} {
      // user profile, only authenticated user allowed to write
      // only user is allowed to write user profile
      // request.resource != null, ensure that delete operation is not allowed
      // resource == null, ensure that update/overwrite operation on file is not allowed      
      match /avatar {
        match /{allFiles=**} {
          allow write: if isUserAuthenticated(userId)
                        && request.resource != null
                        && resource == null;  
          allow read:  if isAuthenticated(); 
          }
      }
      match /original {
        match /{allFiles=**} {
          allow write: if isUserAuthenticated(userId)
                        && request.resource != null
                        && resource == null;           
          }
      }
    }    
  }
}

Note the use of hard-coded admin uids for now. We’ll change this over the next parts to use the admin role instead of the uids.

Complete source code for this part here

In this part, we added much needed features of bulk upload and user profile using firebase storage for uploaded files. It’s missing the role assignment and some UI changes that we’ll cover in the next part of the series.

Next Up

Revisiting ngrx and cleanup UI for bulk upload

If you enjoyed this article please recommend and share and feel free to provide your feedback on the comments section.

Real World App Series (quick links) :

Series summary and index: Part X

• 0: From zero to cli-ng, 1: Not another todo list!
• 2: It’s a Material World!, 3: Form Formation
• 4: State of my SPA, 5: Light my fire
• 6: 3Rs — Roles, Rules & Routes, 6.1: Upgrading to 4.0.0-rc.2
• 7: Split my lazy loaded code, 8: Just Ahead of in Time
• Parts 9, 9.1 and 9.2 : Unit Testing
• 11: Gameplay with Angular; 12: Cloud Functions for Firebase
• 13: Elasticsearch on google cloud; 14: Faceted Search
• 15: Material UI Design; 16: Firebase to firestore

References

• https://firebase.google.com/docs/storage/
• https://firebase.google.com/docs/storage/security/
• https://github.com/angular/angularfire2/blob/master/docs/storage/storage.md
• https://firebase.google.com/docs/functions/gcp-storage-events
• https://www.npmjs.com/package/ngx-image-cropper

Real World App - Part 17: Cloud storage with Firebase and Angular was originally published in Real World Full Stack on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is Part 16 of our Real World App series. For other parts, see links at the bottom of this post or go to https://blog.realworldfullstack.io/. The app is currently under development and can be viewed at https://bitwiser.io and the github repo can be found here.

Recap

In the previous post, we started incorporating the Material UI design in our app. In this part of the series, we will migrate our data from Firebase realtime database (rtdb) to firestore.

Why Firestore?

Limitations of Firebase realtime database

• Data organized as single JSON tree - the entire data structure is organized in a single JSON tree as opposed to breaking it down into documents and collections.
• Limited querying capabilities - this is by far the biggest issue I have with rtdb. rtdb querying was so limited, that for pretty much any querying I had to rely on the Elasticsearch engine.
• Scalability - while this was a non-issue so far, scaling beyond a certain point requires sharding data across multiple databases.

What is firestore?

Cloud Firestore is a flexible, scalable database for mobile, web, and server development from Firebase and Google Cloud Platform.

As Firestore is part of the firebase platform, it offers seamless integration with other Firebase and Google Cloud Platform products, including Cloud Functions.

Although, still in beta, firestore solves some of the issues with rtdb. In addition to above, it has better support for batch writes & transactions, automatic query indexing and sharding, offline support for web clients and better security provisions. For complete comparison, check out -

Choose a Database: Cloud Firestore or Realtime Database | Firebase Realtime Database

Upgrade Angular, Angular Material, angularfire2 to v5

As always, we start by upgrading the project to the latest versions. Upgrading Angular and Angular Material (rc0) to v5, cli to 1.5 and typescript to 2.6.1 had no breaking changes to our project.

Refer to change logs for the new features and breaking changes -

• https://github.com/angular/angular/blob/master/CHANGELOG.md
• https://github.com/angular/material2/blob/master/CHANGELOG.md

However, angularfire2 upgrade did require changes as indicated here - https://github.com/angular/angularfire2/blob/master/docs/version-5-upgrade.md

The first change involved using the valuechanges() observable on the AngularFireList -

//category.service.ts

getCategories(): Observable<Category[]> {    
  return this.db.list('/categories').valueChanges();  
}

The other change was the way the $key and value gets accessed -

//question.service.ts

  getUserQuestions(user: User): Observable<Question[]> {
    return this.db.list('/users/' + user.userId + '/questions').snapshotChanges()
      .mergeMap((qids) => {
        return Observable.forkJoin(
          qids.map((qid) => this.db.object('/questions/' + qid.payload.val() + '/' + qid.payload.key).snapshotChanges()
            .take(1)
            .map(action => {
              const $key = action.payload.key;
              const data = { $key, ...action.payload.val() };
              return data;
            })
            .map(q => Question.getViewModelFromDb(q))))
    });
  }

For complete changes check out the commit. Note: I did not bother to optimize these functions as they would need to be modified again for using firestore anyway. Just wanted to make sure we didn’t break anything in the meantime.

Migrating data from rtdb to firestore

In firestore, the data is organized in documents and collections. Refer - https://firebase.google.com/docs/firestore/data-model

The migration gives me an opportunity to reorganize the data model a bit. I’ve moved the tag array under lists (we can add more lists here if needed). I’ve also separated out questions and unpublished_questions into their own top-level collections. Even though both these collections use Question as their underlying document model, their usage on different views, search, indexing and security is distinct enough to keep them in separate collections.

I also removed the question and game ids from the user hierarchy as this information was redundant and only helped us with querying as the rtdb querying was very limited. Hopefully, with firestore we wouldn’t need this, even though it can always be added as this data can be derived from the properties in the question and game documents.

Migration using cloud functions

As Firebase cloud functions gives us access to both - rtdb and firestore, I leveraged this to write migration scripts for migrating data. Since firestore supports batch writes, I’ve used the batch object to commit data. Also note that for questions and games, we’ll batch our data in chunks of 100 documents.

Here is a sample script for migrating questions -

//firestore-migration.ts

export class FirestoreMigration {
  constructor(private admin: any) {}

...

  migrateQuestions(sourceList, destinationCollection) {
    return new Promise<number>((resolve, reject) => {
      let admin = this.admin;
      let questions: Question[] = [];
      let qRef = admin.database().ref(sourceList);
      qRef.once("value", function(qs) {
        qs.forEach(q => {
          //console.log(c.key);
          console.log(q.val());
          let question: Question = q.val();
          question.id = q.key;
          questions.push(question);
        })
    
        firestoreBatchWrite(destinationCollection, questions, "id", 0, admin.firestore()).then(l => {
          resolve(l);
        });
      });
    });
  }

}

const BATCH_SIZE = 100;
function firestoreBatchWrite(collection: string, dataItems: any[], idField: string,
    start: number, firestore: any): Promise<number> {
  let arr = dataItems.slice(start, start + BATCH_SIZE);

  if (arr.length === 0) {
    return Promise.resolve(dataItems.length);
  }

  let batch = firestore.batch();
  arr.forEach (item => {
    let doc = firestore.doc(collection + "/" + item[idField]);
    console.log(doc);
    batch.set(doc, item);
  });
  console.log("Commiting questions batch: " + start);
  return batch.commit().then(() => {
    return firestoreBatchWrite(collection, dataItems, idField, start + BATCH_SIZE, firestore);
  });

}

Refer to the firestore documentation for their api -

• https://firebase.google.com/docs/reference/js/firebase.firestore
• https://firebase.google.com/docs/reference/functions/functions.firestore

Firestore security rules

The firestore security rules control authorization for direct access to from mobile and web clients to firestore data. (Note these rules do not impact server-side or cloud function access).

Firestore security rules have similarities to rtdb security rules, even though they are syntactically different and have subtle differences like rules do not automatically cascade, unless widlcards are used. For a complete guide refer to - https://firebase.google.com/docs/firestore/security/get-started

service cloud.firestore {
  match /databases/{database}/documents {

match /users/{userId} {
          allow read: if request.auth != null && request.auth.uid == userId;
    }
    
  match /{document=**} {
      allow read, write: if request.auth != null && get(/databases/$(database)/documents/users/$(request.auth.uid)).data.roles.admin;
    }
    
    match /categories {
      match /{document=**} {
        allow read;
      }
    }

match /lists {
      match /{document=**} {
        allow read;
      }
    }
    
    match /questions {
      match /{document=**} {
        allow read: if request.auth != null && request.auth.uid == resource.data.created_uid;
      }
    }
    
    match /unpublished_questions {
      match /{document=**} {
        allow read, update: if request.auth != null && request.auth.uid == resource.data.created_uid;
        allow create: if request.auth != null && request.auth.uid == request.resource.data.created_uid;
      }
    }
    
    match /games {
      match /{document=**} {
        allow read, update: if request.auth != null && (request.auth.uid == resource.data.playerId_0 || request.auth.uid == resource.data.playerId_1);
        allow create: if request.auth != null && request.auth.uid == request.resource.data.playerId_0;
      }
    }
    
  }
}

The rules are pretty self-explanatory once you go thru the docs. The following notes explain the rules above -

• To determine if the user has admin role, we need to go thru the document path /databases/$(database)/documents/users/$(request.auth.uid) and get the roles. We must first grant access to this path for the data to be evaluated. The first section “match /users/{userId} { …” allows read access to users collection and the userId’s path for the authenticated user.
• admin role can read/write all data
• published questions can be read only by the creator. They cannot be modified
• unpublished questions can be read/updated by the creator. The creators user id must match created_uid
• games can be read only by one of the players, while it needs to be created by the first player for the game

Modify Angular code to replace rtdb with firestore

We will replace the AngularFireDatabaseModule with AngularFirestoreModule. The category and tag services are pretty simple to convert.

The question and game service are interesting. Let’s cover the changes in those -

//question.service.ts

...

  getUserQuestions(user: User, published: boolean): Observable<Question[]> {
    let collection = (published) ? "questions" : "unpublished_questions";
    return this.db.collection(`/${collection}`, ref => ref.where('created_uid', '==', user.userId))
        .valueChanges()
        .map(qs => qs.map(q => Question.getViewModelFromDb(q)));
  }

  saveQuestion(question: Question) {
    let dbQuestion = Object.assign({}, question); //object to be saved

    let questionId = this.db.createId();
    dbQuestion.id = questionId;

    //Use the set method of the doc instead of the add method on the collection, so the id field of the data matches the id of the document
    this.db.doc('/unpublished_questions/' + questionId).set(dbQuestion).then(ref => {
        this.store.dispatch(this.questionActions.addQuestionSuccess());
    });
  }

  approveQuestion(question: Question) {
    let dbQuestion = Object.assign({}, question); //object to be saved

    let questionId = dbQuestion.id;
    dbQuestion.status = QuestionStatus.APPROVED;

//Transaction to remove from unpublished and add to published questions collection    
    this.db.firestore.runTransaction(transaction => {
      return transaction.get(this.db.doc('/unpublished_questions/' + questionId).ref).then(doc =>
       transaction.set(this.db.doc('/questions/' + questionId).ref, dbQuestion).delete(doc.ref)
      );
    })
  }
...

The getUserQuestions above uses the where clause [where(‘created_uid’, ‘==’, user.userId))] to get the questions for the user.

The saveQuestion uses “this.db.createId()” to first get the id for the question. It then uses this as the id of the document to be created. Note the use of Object.assign to convert the instance of a Typescript class to a pure object that can be persisted in the database.

The approveQuestion makes uses of a transaction to create a new document in questions collection and delete it from the unpublished_questions collection.

The game service makes use of the update method as below -

//game.service.ts

...

  addPlayerQnAToGame(game: Game, playerQnA: PlayerQnA) {
    game.playerQnAs.push(playerQnA);
    let dbGame = game.getDbModel();
    
    this.db.doc('/games/' + game.gameId).update(dbGame);
  }
...

}

...

The rest of the service method uses code similar to one of the above scenarios. For code changes to all the services, check out the github repo commit.

onWrite Cloud functions event

The onWrite cloud functions event needs to listen to the firestore instead of the rtdb database -

//db.ts

...

exports.onQuestionWrite = functions.firestore.document('/questions/{questionId}').onWrite(event => {
    const data = event.data.data();
    if (data) {
      //add or update
      ESUtils.createOrUpdateIndex(ESUtils.QUESTIONS_INDEX, data.categoryIds["0"], data, event.params.questionId);
    }
    else {
      //delete
      ESUtils.removeIndex(ESUtils.QUESTIONS_INDEX, event.params.questionId);
    }
});

And finally, we would change the reindex method in our cloud functions app.ts to use the firestore’s questions collection.

Complete source code for this part here

In this part, we migrated our data from Firebase realtime database to Cloud Firestore. I chose to do this sooner rather than later as we’re gonna keep expanding our reliance on the database for it’s rich features and delaying it would have meant only more work to be done later.

Next Up

I keep pushing the development of the core functionality of bulk question upload, dashboard features and complete game play mechanics. Next few parts will focus on building out these functional core of the app.

If you enjoyed this article please recommend and share and feel free to provide your feedback on the comments section.

Real World App Series (quick links) :

Series summary and index: Part X

• 0: From zero to cli-ng, 1: Not another todo list!
• 2: It’s a Material World!, 3: Form Formation
• 4: State of my SPA, 5: Light my fire
• 6: 3Rs — Roles, Rules & Routes, 6.1: Upgrading to 4.0.0-rc.2
• 7: Split my lazy loaded code, 8: Just Ahead of in Time
• Parts 9, 9.1 and 9.2 : Unit Testing
• 11: Gameplay with Angular; 12: Cloud Functions for Firebase
• 13: Elasticsearch on google cloud; 14: Faceted Search
• 15: Material UI Design

References

Changelogs

• https://github.com/angular/angular/blob/master/CHANGELOG.md
• https://github.com/angular/material2/blob/master/CHANGELOG.md
• https://github.com/angular/angularfire2/blob/master/docs/version-5-upgrade.md

Firestore Overview

• https://firebase.google.com/docs/firestore/rtdb-vs-firestore
• https://firebase.google.com/docs/firestore/firestore-for-rtdb
• https://firebase.google.com/docs/firestore/data-model
• https://firebase.google.com/docs/firestore/security/get-started

Firestore API Reference

• https://firebase.google.com/docs/reference/js/firebase.firestore
• https://firebase.google.com/docs/reference/functions/functions.firestore

Other Posts

• https://blog.cloudboost.io/angular-5-with-firebases-cloud-firestore-87b5d6c13ff
• https://blog.angular.io/improved-querying-and-offline-data-with-angularfirestore-dab8e20a4a06

Real World App - Part 16: From Firebase to Firestore was originally published in Real World Full Stack on Medium, where people are continuing the conversation by highlighting and responding to this story.