npm dev dependency not installed

When running npm install, the dev dependencies are usually also get installed. However today, we have a build runs inside a docker container which requires a test result to be generated and we cannot get test run. Turns out the Karma packages in the dev dependencies are not installed.

The reason is the container in which npm is run has a NODE_ENV environment var set to production so npm will pick that up and skip the dev dependencies installation.

Other scenarios dev will not be installed is

1. the npm config production value is set to true. use below command to check.

npm config get production

2. npm is run with --production flag.

 

So to solve this, we just unset that var and set it back after npm install.


unset NODE_ENV

npm install

export NODE_ENV=production

Advertisements

exclude file from karma coverage report

We have some files that is not so testable (like animation related, or things like polyfills) to be excluded from the karma coverage report. With angular-cli looks like it is straightforward that we can include the below config:

 "test": {
    "codeCoverage": {
      "exclude": [
        "src/polyfills.ts",
        "**/test.ts"
      ]
    },
    "karma": {
      "config": "./karma.conf.js"
    }
  }

However since we are not using cli , we need to tweak ourselves. Found the PR that implements the above feature. Looks like we just need to push the files to be excluded to the webpack config’s module rules for istanbul-instrumenter-loader`. Something like:

module:{
 rule: [
  ...
        {
        enforce: 'post',
        test: /\.(js|ts)$/,
        loader: 'istanbul-instrumenter-loader',
        include: helpers.root('src'),
        exclude: [
          /\.(e2e|spec)\.ts$/,
          /node_modules/,
          // add files needs to be excluded from coverage report below
          `${helpers.root('src')}/services/svg-animations.service.ts`
        ]
      },
  ...
 ]
}

The helpers.root is just a help function get get the proper path.

function root(args) {
  const path = require('path');
  args = Array.prototype.slice.call(arguments, 0);
  const ROOT = path.resolve(__dirname, '..');
  return path.join.apply(path, [ROOT].concat(args));
}

In the coverageReporter of karma.conf.js, we can set a lot of useful things like threshold, dir(normalized with subdir), etc…

execute async call sequentially

fix number of calls

Today, i need to figure out a way to execute a bunch of http call sequentially. The problem first looks simple if we have known number of calls so that we can easily put them in callback one by one or chain the promise.

call1(()=>{
  call2(()=>{
    call3(...);
  })
});

call1()
 .then(() => {
  return call2();
 }).then(()=>{
  return call3();
 })...

number of calls varies

However if what we get is an array with any number of calls, it becomes tricky. the above aways obviously does not work.

promise holder with IIFE

One way is to define a promise out of the for loop, and assign the promise returned by the previous promise to it.

let itemsToProcess = [...]
let promiseHolder = processItem(itemsToProcess[0]);
for (let i = 1; i < itemsToProcess.length; i++) {
    (function() {
        let thisItem = itemsToProcess[i];
        let getNewPromise = () => processItem(thisItem);
        promiseHolder = promiseHolder.then(res => {
            onReponse(res);
            return getNewPromise();
        });
    })(i)
}

Or use reduce function to rescue:

let itemsToProcess = [...]
itemsToProcess.reduce((lastPromise, item) => 
 lastPromise.then(() => 
  processItem(item)
 ), Promise.resolve()
)

async and await

a more elegant way is to use the async and await inside for loop which will auto block the execution and wait.

for (let item of itemsToProcess) {
    let rs = await processItem(browser, item);
    onReponse(rs);
}

Yes, the code is this simple. The introduction of async/await from node7.x+ is really useful. await suspends the current function evaluation, including all control structures.

Caveat with foreach/map

One caveat is be very careful use async in the forEach/map loop, it would not work as expected if what you want is serial.

itemsToProcess.forEach(async (item)=> {
    let rs = await processItem(browser, item);
    onReponse(rs);
})

Rather than waiting for execution one by one, all the async call will fire at the same time. forEach expects synchronous function. Think about it this way: an async function returns a promise and it’s the caller’s responsibility to do something with that promise. However, Array#forEach doesn’t do anything with the return value of the callback, it simply ignores it. All it does is call the callback for each element. The Promise.all([...myPromiseArray]) will behave similar because promise is hot that the time it is created, the xhr/event is already fired.

one way is to create a forEachAsync to help but basically use the same for loop underlying.

Array.prototype.forEachAsync = async function(cb){
    for(let x of this){
        await cb(x);
    }
}

A good video about async/await

angular change detection notes

Was reading some angular change detection related articles trying to understand it. some note.

Angular can detect when component data changes, and then automatically re-render the view to reflect that change.

NgZone

ngZone is an angular verison of zone.js, which patches the browser async calls like user event(click/keyup etc), settimeout/interval, XHR.  It is similar to the AOP we have in Spring that proxies are created by framework to do before/after custom logic around the original method call. The importance of this angular patch these brower-API to trigger the change-detection. Previously in ng1, we have all those special directives like ng-click, $timeout etc to make sure after all the custom logic is done, we all the angularjs $apply() to run the digest cycle to do the dirty check and update the view if necessary. Now in ng2+, all these special stuff are gone because of the usage of Zone which enables us to fire change detection after any of these browser event are done from the main stack. Here is a good article explaining zone in angular.

Basically, The short version is, that somewhere in Angular’s source code, there’s this thing called ApplicationRef, which listens to NgZones onStable event. Whenever this event is fired, it executes a tick() function which essentially performs change detection.

  tick() {
    this.changeDetectorRefs
      .forEach((ref) =&amp;gt; ref.detectChanges());
  }

change detection flow

So now we know how CD is triggered, now time for how it is executed.

change detector classes are created on the fly by angular for each component.

From the top of the component(view) tree, we start the CD.

The main logic responsible for running change detection for a view resides in checkAndUpdateView function. Most of its functionality performs operations on child component views. This function is called recursivelyfor each component starting from the host component. It means that a child component becomes parent component on the next call as a recursive tree unfolds.

When this function triggered for a particular view it does the following operations in the specified order:

  1. sets ViewState.firstCheck to true if a view is checked for the first time and to false if it was already checked before
  2. checks and updates input properties on a child component/directive instance
  3. updates child view change detection state (part of change detection strategy implementation)
  4. runs change detection for the embedded views (repeats the steps in the list)
  5. calls OnChanges lifecycle hook on a child component if bindings changed
  6. calls OnInit and ngDoCheck on a child component (OnInit is called only during first check)
  7. updates ContentChildren query list on a child view component instance
  8. calls AfterContentInit and AfterContentChecked lifecycle hooks on child component instance (AfterContentInit is called only during first check)
  9. updates DOM interpolations for the current view if properties on current view component instance changed
  10. runs change detection for a child view (repeats the steps in this list)
  11. updates ViewChildren query list on the current view component instance
  12. calls AfterViewInit and AfterViewChecked lifecycle hooks on child component instance (AfterViewInit is called only during first check)
  13. disables checks for the current view (part of change detection strategy implementation)

Some lifecycle hooks are called before the DOM update (3,4,5) and some after (9). So if you have the following components hierarchy: A -> B -> C, here is the order of hooks calls and bindings updates:

A: AfterContentInit
A: AfterContentChecked
A: Update bindings
    B: AfterContentInit
    B: AfterContentChecked
    B: Update bindings
        C: AfterContentInit
        C: AfterContentChecked
        C: Update bindings
        C: AfterViewInit
        C: AfterViewChecked
    B: AfterViewInit
    B: AfterViewChecked
A: AfterViewInit
A: AfterViewChecked

check on reference

By default, Angular Change Detection works by checking if the value of template expressions have changed. This is done for all components. In other word, Angular does not do deep object comparison to detect changes, it only takes into account properties used by the template.

Performance

Ng2+ gets rid of the ng1 way of doing dirty check which would result in multiple rounds of check. Now we only have 1 round. If we change the fields in the life cycle hooks like ngAfterViewChecked, we will get xxx has changed after it was checked. This error message is only thrown if we are running Angular in development mode. In production mode, the error would not be thrown and the issue would remain undetected.

trigger CD manually

There could be special occasions where we do want to turn off change detection. Imagine a situation where a lot of data arrives from the backend via a websocket. We might want to update a certain part of the UI only once every 5 seconds. To do so, we start by injecting the change detector into the component:

constructor(private ref: ChangeDetectorRef) {
    ref.detach();
    setInterval(() =&amp;gt; {
      this.ref.detectChanges();
    }, 5000);
  }

As we can see, we just detach the change detector, which effectively turns off change detection. Then we simply trigger it manually every 5 seconds by calling detectChanges().

 

Some references:

  1. How does Angular Change Detection Really Work ?
  2. ANGULAR CHANGE DETECTION EXPLAINED
  3. Everything you need to know about change detection in Angular

some other good article in angularInDepth:

Exploring Angular DOM manipulation techniques using ViewContainerRef

The mechanics of DOM updates in Angular

difference between detechChanges and markForCheck

Angular Rxjs Error Handling flow

In Angular 2+, http are all RX based. so the error flow is quite different from what we used to have in angular 1.x’s promise based way.

consider the code

this.http.get(‘someurl’)
.map((res: Response) => {
return res.json();
})
.catch((err) => {
console.log(error);
Observable.throw(‘my custom err msg’);
})
.subscribe((res)=>{
console.log(res);
}, (err) => {
console.log(err);
}, ()=> {
console.log(‘complete’);
})

So the basic flow is if anything exception happened, the error handler function in the catch block will be called first. That function should return an Observable since it is a chain. Then the (err) function will be invoked. In the context of an HTTP request in Angular2, a response with a status code different than 2xx is considered as an error.

Note, if we do not use Observable.throw() in catch but return some random result, it would probably result in funky result. In the ​subscribeToResult function, there are multiple type checks. For example, if we return an array, if would be treated valid and be passed to the first handler rather than the error handler. If we return a random object like the error object, we will get TypeError: unknown type returned, error and trigger the 2nd handler(error handler).

Another note is if angular will return some odd error object with status: 0 when invoking some non-exist cross origin endpoints. This is because Whenever you make an api call to a non existing route, the browser sends a preflight request (a) . If the backends error handle strategy is to return an http response with 404 code and the error details in the body, the browser returns a generic error with http status code 0 without the actual server’s payload. The second call is never made because there is no sense in doing it since the preflight went wrong. In the end the error thrown have 0 status without any information about the real cause of the error.

dependencies resolve difference between yarn and npm

Today we found an interesting behavior difference between yarn and npm when resolving/placing dependency on node_module folder. It happens when i was help one of my teammates setting up a local debugging server which has a require on Library A. In our package.json, we do not have an explicit declaration on A. We have 2 depedencies B and C both have A as their dependency but on different version. B -> A1 and C-> A2.

For me(using yarn), when I do yarn install, the newer one A2 is placed on the top level node_module directory and C does not have A2 in its own node_module. And B has an A1 in its node_module.

So if we do a npm install on the project, the behavior is opposite. the older  A1 is placed on the top level and B does not have A1 in its node_module folder. And C has an A2 in its node_module directory to fulfill the dependency.

It breaks because the local server we setup is using some new APIs from A2 [let A = require(‘A’)], so my local will work but my colleague’s will not since she has the A1 in node_module. Minor thing but really reminds us to check the consistency between different build tool. The solution is straightforward: declare the dependency on the package.json directly to get the desired version.

Also tried the new tool called pnpm , though declared to be super fast with links rather than copy files around, and has non-flat layout. seems not working well with direct github dependency. So gave up.

include scss in angular styleUrls

In ng2+, we can include css in the styleUrls on the component level to separate styling into file. In one of our recent project, we have a bunch of scss files to migrate, so we would like to include scss file to avoid major rewrite. We also have some global scss files which will be bundled by webpack withExtractTextPlugin  .

So we basically need 2 rules for scss:

 {
        test: /(main|index).scss$/,
        loader: ExtractTextPlugin.extract({
          fallbackLoader: "style-loader",
          loader: "css-loader!sass-loader",
        }),
      },
      {
        test: /\.scss$/,
        loader: ["raw-loader", "sass-loader",
          {
            loader: 'sass-resources-loader',
            options: {
              resources: ['./styles/global/_common.scss', './styles/global/_mixins.scss']
            },
          }
        ],
        // main scss will be handled by css/sass loader then ExtractTextPlugin then to the bundle css file.
        exclude: [helpers.root('styles/main.scss'), helpers.root('node_modules/header-footer/styles/index.scss')]
      },

If we use css-loader rather than raw loader for other scss files, then we get error message: Error: Uncaught (in promise): Expected 'styles' to be an array of strings..

The sass-resources-loader loader is helpful because it would make the shared commons/mixins available for all components so that we do not have to import them in each component to make the scss compilable.