base64 in shell

Used to go to some site when in need of en/de-code from/to base64. Turns out we can do that from the shell directly.

Encode

echo -n 'My TextTo Encode' | base64

The -n is to prevent the new line of the echo from being included in the result(by default echo outputs a trailing newline).

Decode

echo 'My Encoded TExt | base64 --decode.

we can also use -D(MacOS) or -d(Linux) instead of --decode for short.

Advertisements

MacOS bluetooth headset audio issue

Recently my bluetooth headset starts to act up when listening to music from MacOS. I initially thought it has died. Turns out it is an OS problem.

Basically if i use it as both input and output device, the sound quality is awful. So one solution is go to the sound setting and manually select input device internal microphone. Then looks like the audio will be back. However if you re-connect the device, it defaults to the bluetooth device as input again. quite annoying.

Another possible solution I found is to adjust the setting from the bluetooth explorer which can be downloaded from the Apple developer site‘s Hardware IO Tools for Xcode section. The downloaded the dmg is a bundle of various debugging tools. Open Bluetooth Explorer and go to Tools -> Audio Options -> force use aptX, and check the box. This will also improve the sound quality.

macos system launch config

MacOS Config options

MacOS boot service is mainly configured via three ways:
1.  the system preferences -> Users & Groups -> Login items
2. /System/Library/StartupItems and /Library/StartupItems/
3. launchd system initialization process configuration.

The first two are relatively simple, we will focus mainly on the third more complex launchd configuration optimization.
launchd is a key process for initializing the system environment under Mac OS. Similar to Linux under init, rc.

MacOS Boot Flow

Let’s take a look at the MacOS startup flow:
1, mac firmware activation, initialize hardware, load BootX boot.
2, BootX load kernel and kernel extension (kext).
3, the kernel starts the launchd process.
4, launchd Start the service daemon by following the plist configuration in the /System/Library/LaunchAgents,  System/Library/LaunchDaemons, ?/Library/LaunchDaemons, ?Library/LaunchAgents, ~/Library/LaunchAgents.

The five launch DIRs

After going thru the Mac OS startup flow, we can see that the plist files in the below five directories  ?/System/Library/LaunchAgents, ?/System/Library/LaunchDaemons, ?/Library/LaunchDaemons, ?Library/LaunchAgents?, ~/Library/LaunchAgents. are key to the system optimization.

some basic concepts

difference between/System/Library and /Library and ~/Library directory?

/System/Library directory is stored in Apple’s own software development.
The /Library directory is the third-party software that the system administrator holds.
~/Library/ is the user’s own third-party software.

What is the difference between LaunchDaemons and LaunchAgents?

LaunchDaemons is the service (daemon) that the user started before the login.
LaunchAgents is the service (daemon) that the user starts after landing.

SPEC

plist file format and the meaning of each field in the above mentioned five directories:

KEY DESCRIPTION REQUIRED
Label The name of the job yes
ProgramArguments Strings to pass to the program when it is executed yes
UserName The job will be run as the given user, who may not necessarily be the one who submitted it to launchd. no
inetdCompatibility Indicates that the daemon anticip to be run as if it were launched by? Inetd no
Program The path to your executable. This key can save the ProgramArguments key for flags and arguments. no
onDemand A? Boolean? Flag that defined if a job runs continuously or not no
RootDirectory The job will be? Chrooted? Into another directory no
ServiceIPC Any the daemon can speak IPC to launchd no
WatchPaths Allows launchd to start a job based on modifications at a file-system path no
QueueDirectories Similar to WatchPath, a queue will only watch an empty directory for new files no
StartInterval Used to schedule a job that runs on a repeating schedule. Specified as the number of seconds to wait between runs. no
StartCalendarInterval Job scheduling. The syntax is similar to cron. no
HardResourceLimits Controls restriction on the resources consumed by any job no
LowPriorityIO Tells the kernel that this task is of a low priority when doing file system I / O no
Sockets An array can be used to specify what socket the daemon will listen on for launch on demand no

Do not understand the above plist configuration? It is fine. Our optimization strategy is completely unload the service, so we do not care that much about plist in the configuration meaning.

Leverage launchctl

To start to optimize the disabled service, we need to use the Mac OS provides a tool instruction- launchctl
The launchctl directive sets a disable flag for the service. When launchd starts, it checks whether the service is disabled or not to determine whether the service needs to be enabled.

Method 1 to disable service

First find the disabled flag file /var/db/launchd.db/com.apple.launchd/overrides.plist to see if the service you want to disable has been disabled.
Some services have been disabled but are not listed in overrides.plist. At this point, you also need to check the service plist file Label field has been marked as Disable.

After confirming that the service is not disabled, we can disable the service by invoking the following command:
sudo launchctl unload plist-file-path

For example, I would like to disable spotlight :

sudo launchctl unload /System/Library/LaunchAgents/com.apple.Spotlight.plist

After disabling the service, restart Mac OS to take effect.

Method of banning service 2

a more effective and violent method (recommended)
Uninstall the service first
sudo launchctl unload /System/Library/LaunchAgents/com.apple.Spotlight.plist
And then the plist file mv to other directory backup. Reboot. Done.

I personally prefer this way to disable the service, so recommend it.

If you find that the service is disabled, the system or software is abnormal, you can restore the service by putting the file back to the original folder and run the following command:
sudo launchctl load plist-file-path

Note: Be very careful when disabling System-level service,. please get familiar what that system service does and do enough research before remove/disable it. Otherwise it may cause the system to fail to start. The safest thing to do is to stop it.

On the other hand for the user service, we can rest assured that we could disable, in case there are problems we just need to re-enable it.

Here is a list of my disabled services:
/Library/LaunchDaemons/com.google.keystone.daemon.plist? (Google Software Update)
/Library/LaunchAgents/com.google.keystone.root.agent? (Google Software Update)
~ / Library / LaunchAgents / com.google.keystone.agent.plist? (Google Software Update, users do not need to add the process of sudo)
~ / Library / LaunchAgents / com.apple.CSConfigDotMacCert-ken.wug \ @ me.com-SharedServices.Agent.plist (me.com’s shared services, I do not have)
/System/Library/LaunchDaemons/org.cups.cupsd.plist (printer)
/System/Library/LaunchDaemons/org.cups.cups-lpd.plist (printer)
/System/Library/LaunchDaemons/com.apple.blued.plist (bluetooth)
/System/Library/LaunchAgents/com.apple.AirPortBaseStationAgent.plist (apple wireless base station, i do not have this device)

I know the daemon (service) name, how to find the corresponding plist file?
Copy the process (service) name, and then to / System / Library / LaunchAgents,? / System / Library / LaunchDaemons,? / Library / LaunchDaemons,? Library / LaunchAgents, ~ / Library / LaunchAgents five directories, through the following command :

ll | grep process(service)-name such as
ll | grep blued
Found it in /System/Library/LaunchDaemons Next, follow the steps outlined above to disable the service

For McAfee

For React Native local development in iOS, i have to disable Mcafee daemon to free the 8081 port. For android, we can do the workaround by using 8088.

 cd /Library/LaunchDaemons
 sudo launchctl unload com.mcafee.agent.macmn.plist

angular change detection notes

Was reading some angular change detection related articles trying to understand it. some note.

Angular can detect when component data changes, and then automatically re-render the view to reflect that change.

NgZone

ngZone is an angular verison of zone.js, which patches the browser async calls like user event(click/keyup etc), settimeout/interval, XHR.  It is similar to the AOP we have in Spring that proxies are created by framework to do before/after custom logic around the original method call. The importance of this angular patch these brower-API to trigger the change-detection. Previously in ng1, we have all those special directives like ng-click, $timeout etc to make sure after all the custom logic is done, we all the angularjs $apply() to run the digest cycle to do the dirty check and update the view if necessary. Now in ng2+, all these special stuff are gone because of the usage of Zone which enables us to fire change detection after any of these browser event are done from the main stack. Here is a good article explaining zone in angular.

Basically, The short version is, that somewhere in Angular’s source code, there’s this thing called ApplicationRef, which listens to NgZones onStable event. Whenever this event is fired, it executes a tick() function which essentially performs change detection.

  tick() {
    this.changeDetectorRefs
      .forEach((ref) => ref.detectChanges());
  }

change detection flow

So now we know how CD is triggered, now time for how it is executed.

change detector classes are created on the fly by angular for each component.

From the top of the component(view) tree, we start the CD.

The main logic responsible for running change detection for a view resides in checkAndUpdateView function. Most of its functionality performs operations on child component views. This function is called recursivelyfor each component starting from the host component. It means that a child component becomes parent component on the next call as a recursive tree unfolds.

When this function triggered for a particular view it does the following operations in the specified order:

  1. sets ViewState.firstCheck to true if a view is checked for the first time and to false if it was already checked before
  2. checks and updates input properties on a child component/directive instance
  3. updates child view change detection state (part of change detection strategy implementation)
  4. runs change detection for the embedded views (repeats the steps in the list)
  5. calls OnChanges lifecycle hook on a child component if bindings changed
  6. calls OnInit and ngDoCheck on a child component (OnInit is called only during first check)
  7. updates ContentChildren query list on a child view component instance
  8. calls AfterContentInit and AfterContentChecked lifecycle hooks on child component instance (AfterContentInit is called only during first check)
  9. updates DOM interpolations for the current view if properties on current view component instance changed
  10. runs change detection for a child view (repeats the steps in this list)
  11. updates ViewChildren query list on the current view component instance
  12. calls AfterViewInit and AfterViewChecked lifecycle hooks on child component instance (AfterViewInit is called only during first check)
  13. disables checks for the current view (part of change detection strategy implementation)

Some lifecycle hooks are called before the DOM update (3,4,5) and some after (9). So if you have the following components hierarchy: A -> B -> C, here is the order of hooks calls and bindings updates:

A: AfterContentInit
A: AfterContentChecked
A: Update bindings
    B: AfterContentInit
    B: AfterContentChecked
    B: Update bindings
        C: AfterContentInit
        C: AfterContentChecked
        C: Update bindings
        C: AfterViewInit
        C: AfterViewChecked
    B: AfterViewInit
    B: AfterViewChecked
A: AfterViewInit
A: AfterViewChecked

check on reference

By default, Angular Change Detection works by checking if the value of template expressions have changed. This is done for all components. In other word, Angular does not do deep object comparison to detect changes, it only takes into account properties used by the template.

Performance

Ng2+ gets rid of the ng1 way of doing dirty check which would result in multiple rounds of check. Now we only have 1 round. If we change the fields in the life cycle hooks like ngAfterViewChecked, we will get xxx has changed after it was checked. This error message is only thrown if we are running Angular in development mode. In production mode, the error would not be thrown and the issue would remain undetected.

trigger CD manually

There could be special occasions where we do want to turn off change detection. Imagine a situation where a lot of data arrives from the backend via a websocket. We might want to update a certain part of the UI only once every 5 seconds. To do so, we start by injecting the change detector into the component:

constructor(private ref: ChangeDetectorRef) {
    ref.detach();
    setInterval(() => {
      this.ref.detectChanges();
    }, 5000);
  }

As we can see, we just detach the change detector, which effectively turns off change detection. Then we simply trigger it manually every 5 seconds by calling detectChanges().

 

Some references:

  1. How does Angular Change Detection Really Work ?
  2. ANGULAR CHANGE DETECTION EXPLAINED
  3. Everything you need to know about change detection in Angular

some other good article in angularInDepth:

Exploring Angular DOM manipulation techniques using ViewContainerRef

The mechanics of DOM updates in Angular

difference between detechChanges and markForCheck

Angular Rxjs Error Handling flow

In Angular 2+, http are all RX based. so the error flow is quite different from what we used to have in angular 1.x’s promise based way.

consider the code

this.http.get(‘someurl’)
.map((res: Response) => {
return res.json();
})
.catch((err) => {
console.log(error);
Observable.throw(‘my custom err msg’);
})
.subscribe((res)=>{
console.log(res);
}, (err) => {
console.log(err);
}, ()=> {
console.log(‘complete’);
})

So the basic flow is if anything exception happened, the error handler function in the catch block will be called first. That function should return an Observable since it is a chain. Then the (err) function will be invoked. In the context of an HTTP request in Angular2, a response with a status code different than 2xx is considered as an error.

Note, if we do not use Observable.throw() in catch but return some random result, it would probably result in funky result. In the ​subscribeToResult function, there are multiple type checks. For example, if we return an array, if would be treated valid and be passed to the first handler rather than the error handler. If we return a random object like the error object, we will get TypeError: unknown type returned, error and trigger the 2nd handler(error handler).

Another note is if angular will return some odd error object with status: 0 when invoking some non-exist cross origin endpoints. This is because Whenever you make an api call to a non existing route, the browser sends a preflight request (a) . If the backends error handle strategy is to return an http response with 404 code and the error details in the body, the browser returns a generic error with http status code 0 without the actual server’s payload. The second call is never made because there is no sense in doing it since the preflight went wrong. In the end the error thrown have 0 status without any information about the real cause of the error.

dependencies resolve difference between yarn and npm

Today we found an interesting behavior difference between yarn and npm when resolving/placing dependency on node_module folder. It happens when i was help one of my teammates setting up a local debugging server which has a require on Library A. In our package.json, we do not have an explicit declaration on A. We have 2 depedencies B and C both have A as their dependency but on different version. B -> A1 and C-> A2.

For me(using yarn), when I do yarn install, the newer one A2 is placed on the top level node_module directory and C does not have A2 in its own node_module. And B has an A1 in its node_module.

So if we do a npm install on the project, the behavior is opposite. the older  A1 is placed on the top level and B does not have A1 in its node_module folder. And C has an A2 in its node_module directory to fulfill the dependency.

It breaks because the local server we setup is using some new APIs from A2 [let A = require(‘A’)], so my local will work but my colleague’s will not since she has the A1 in node_module. Minor thing but really reminds us to check the consistency between different build tool. The solution is straightforward: declare the dependency on the package.json directly to get the desired version.

Also tried the new tool called pnpm , though declared to be super fast with links rather than copy files around, and has non-flat layout. seems not working well with direct github dependency. So gave up.

include scss in angular styleUrls

In ng2+, we can include css in the styleUrls on the component level to separate styling into file. In one of our recent project, we have a bunch of scss files to migrate, so we would like to include scss file to avoid major rewrite. We also have some global scss files which will be bundled by webpack withExtractTextPlugin  .

So we basically need 2 rules for scss:

 {
        test: /(main|index).scss$/,
        loader: ExtractTextPlugin.extract({
          fallbackLoader: "style-loader",
          loader: "css-loader!sass-loader",
        }),
      },
      {
        test: /\.scss$/,
        loader: ["raw-loader", "sass-loader",
          {
            loader: 'sass-resources-loader',
            options: {
              resources: ['./styles/global/_common.scss', './styles/global/_mixins.scss']
            },
          }
        ],
        // main scss will be handled by css/sass loader then ExtractTextPlugin then to the bundle css file.
        exclude: [helpers.root('styles/main.scss'), helpers.root('node_modules/header-footer/styles/index.scss')]
      },

If we use css-loader rather than raw loader for other scss files, then we get error message: Error: Uncaught (in promise): Expected 'styles' to be an array of strings..

The sass-resources-loader loader is helpful because it would make the shared commons/mixins available for all components so that we do not have to import them in each component to make the scss compilable.