React Native on a machine 8081 occupied

Today, as i was setting up a basic React native app on my company’s laptop, I cannot run the RN packager server which is default to port 8081​. I ran the lsof -i :8081 but get nothing. Turns out I have to run it with sudo because the app that uses this port is the Corporate Mcafee.

Obviously i do not want to uninstall Mcafee from the corporate laptop, I have to find a work around.

First thing to do is to run the packager server in a different port:

react-native start --port 8088

And now the packager server is up and running. However when i run react-native run-android , it still tells me JS server not recognized , so to get rid of this msg, i have to change some source code in the node module. The logic is in node_modules/react-native/local-cli/util/isPackagerRunning.js, and we just need to change the port in the fetch function from 8081 to the 8088 that we use above.

Now the android app should be able to be installed in the virtual device and run. However we still could not leverage the live reload capability of RN because if we double click r to reload, we get a red screen saying cannot connect to 10.0.2.2:8081. This is because when we use Genymotion virtual device, the code in node_modules/react-native/ReactAndroid/src/main/java/com/facebook/react/modules/systeminfo/AndroidInfoHelpers.java will return that url. So we need to cmd+M in the emulator, Go to Dev Settings → Debug server host for device, enter ‘localhost:8081’. 

This is to overwrite the emulator’s debug server. Now we have the last step which is to forward the request in our local VD’s 8081 port to our machine’s 8088 port which runs the packager server by doing:

adb reverse tcp:8081 tcp:8088

More about adb reverse.

Now you should be able to change your index.android.js and hit r twice or cmd+M -> Reload to reload the VD.

webpack custom plugin

Recently we work with a platform which need to use webpack to build some ng2/4 assets and also some custom steps to pull data from a headless cms(via gulp) and eventually render components. One problem here is we cannot do live reload/recompile that every time we make some change we have to run the npm command again to compile the resources.

To solve the issue, i decided to write a custom webpack plugin to make browser-sync and webpack work together.

The basic flow is 1. run webpack in watch mode so every time a resouce(ts/css/html) changes, webpack auto re-compile, 2. serve the resources via browser-sync, here browserSync just serve as a mini express server and provide browser reload capability. 3. a webpack plugin to start the browser-sync server and register the reload event when webpack compilation is done.

Plugin

The webpack api is pretty straightforward, it exposes a compile object from the plugin’s apply function. It represents the fully configured Webpack environment. This object is built once upon starting Webpack, and is configured with all operational settings including options, loaders, and plugins. When applying a plugin to the Webpack environment, the plugin will receive a reference to this compiler. Use the compiler to access the main Webpack environment.

const browserSync = require('browser-sync');

function WebpackAndromedaPlugin(options) {
  console.log(`WebpackAndromedaPlugin options: ${JSON.stringify(options, null, 2)}`);

  let browserSyncConfig = require('./lib/dev-server').getBrowserSyncConfig(options);
  browserSyncConfig.server.middleware.push(require('./lib/dev-server').makeServeOnDemandMiddleware(options));
  browserSync.init(browserSyncConfig);
}

WebpackAndromedaPlugin.prototype.apply = (compiler) => {

  compiler.plugin("compile", function (params) {
    console.log('--->>> andromeda started compiling');
  });

  compiler.plugin('after-emit', (_compilation, callback) => {
    console.log('--->>> files prepared');
    browserSync.reload();
    callback();
  });
}

as we can see above, we can register our callbacks with compiler.pulgin(), where webpack exposes different stages for us to interact.

Another important object is compilation, which  represents a single build of versioned assets. While running Webpack development middleware, a new compilation will be created each time a file change is detected, thus generating a new set of compiled assets. A compilation surfaces information about the present state of module resources, compiled assets, changed files, and watched dependencies. The compilation also provides many callback points at which a plugin may choose to perform custom actions.

For example, all the generated files will be in compilation.assets object.

webpack-dev-middleware

The webpack-dev-middleware is a nice little express middleware that serves the files emitted from webpack over a connect server. One good feature it has is serving files from memory since it uses a in memory file system, which exposes some simple methods to read/write/check-existence in its MemoryFileSystem.js.  The webpack-dev-middleware also exposes some hooks like close/waitUntilValid etc, unfortunately the callback that waitUntilValid registers will only be called once according to the compileDone function here. Anyway, it is still an efficient tool to serve webpack resources and very easy to integrate with the webpack nodejs APIs:

~function() {
  const options = require('./config');
  let  webpackMiddleware = require("webpack-dev-middleware");

  let webpack = require('webpack');
  let browserSyncConfig = getBrowserSyncConfig(options);
  browserSyncConfig.server.middleware.push(makeServeOnDemandMiddleware(options));
  const compiler = webpack(require('./webpack.dev'));
  compiler.plugin('done', ()=>browserSync.reload())
  let inMemoryServer = webpackMiddleware(compiler, {noInfo: true, publicPath:'/assets'});
  browserSyncConfig.server.middleware.push(inMemoryServer);
  browserSync.init(browserSyncConfig);
}();

 

webpack-dev-server

The webpack-dev-server is basically a wrapper over the above webpack-dev-middleware. it is good for simple resources serving since it does not expose much. I was trying to find a hook to it to intercept the resource it generates/serves but did not get a good solution. If you need more customization, it would be better to go with webpack-dev-middleware.

a very detailed webpack intro article

A deeper look at event loop (micro/macro tasks)

One common question

(function test() {
    setTimeout(function() {console.log(4)}, 0);
    new Promise(function executor(resolve) {
        console.log(1);
        for( var i=0 ; i<10000 ; i++ ) {
            i == 9999 && resolve();
        }
        console.log(2);
    }).then(function() {
        console.log(5);
    });
    console.log(3);
})()

So why the result is 1,2,3,5,4 rather than 1,2,3,4,5

If we look at the detail, looks like the async of setTimeout is different from the async of Promise.then, at least they are not in the same async queue.

The answer is here in the whatwg SPEC.

  • An event loop has one or more task queues.(task queue is macrotask queue)
  • Each event loop has a microtask queue.
  • task queue = macrotask queue != microtask queue
  • a task may be pushed into macrotask queue,or microtask queue
  • when a task is pushed into a queue(micro/macro),we mean preparing work is finished,so the task can be executed now.

And the event loop process model is as follows:

when call stack is empty,do the steps-

  1. select the oldest task(task A) in task queues
  2. if task A is null(means task queues is empty),jump to step 6
  3. set “currently running task” to “task A”
  4. run “task A”(means run the callback function)
  5. set “currently running task” to null,remove “task A”
  6. perform microtask queue
    • (a).select the oldest task(task x) in microtask queue
    • (b).if task x is null(means microtask queues is empty),jump to step (g)
    • (c).set “currently running task” to “task x”
    • (d).run “task x”
    • (e).set “currently running task” to null,remove “task x”
    • (f).select next oldest task in microtask queue,jump to step(b)
    • (g).finish microtask queue;
  7. jump to step 1.

a simplified process model is as follows:

  1. run the oldest task in macrotask queue,then remove it.
  2. run all available tasks in microtask queue,then remove them.
  3. next round:run next task in macrotask queue(jump step 2)

something to remember:

  1. when a task (in macrotask queue) is running,new events may be registered.So new tasks may be created.Below are two new created tasks:
    • promiseA.then()’s callback is a task
      • promiseA is resolved/rejected:  the task will be pushed into microtask queue in current round of event loop.
      • promiseA is pending:  the task will be pushed into microtask queue in the future round of event loop(may be next round)
    • setTimeout(callback,n)’s callback is a task,and will be pushed into macrotask queue,even n is 0;
  2. task in microtask queue will be run in the current round,while task in macrotask queue has to wait for next round of event loop.
  3. we all know callback of “click”,”scroll”,”ajax”,”setTimeout”… are tasks,however we should also remember js codes as a whole in script tag is a task(a macrotask) too.

 

In nodejs world:  setImmediate()is macro/task, and process.nextTick() is a micro/job

 

One good discussion in Chinese and blog.

regex dash – inside bracket []

We have a requirement to remove some illegal character in file name like slash etc… So I put on a regex in our code:

[^a-zA-Z0-9.-_]

I assume the above regex will match everything that is not number/char/dot/dash/underscore.

Turns out I was wrong! The - is special case inside [...] that is used for range. It should be in the beginning or in the last or escaped. Otherwise it will match all the character that is in between . and _ in ASCII character set. So in my case, the .-_ part will try to match characters from 46(.)-95(_) in ASCII char table.

The correct one should be putting it in the last  [^a-zA-Z0-9._-] . Or just escape it: [^a-zA-Z0-9.\-_]

print all data in paginated table/grid

direct Tabular data display

Recently our project has a page need to show tabular data from 30-6000 rows with 3-4 columns. At first, I thought this is pretty reasonable data to show in one page so I just throw the data into a ng-repeat table with my own implementation of filtering/sorting which is pretty straightforward in angular. Every time user select new category/type, fetch data from backend and replace the data in vm/$scope. Also with this implementation, it is quite easy to fulfill our another requirement which is export/print the page content. For export I just need to get the DOM content to the server and return as downloadable. For print, even easier, just call window.print() ,that’s it.

Performance issue with IE

Everything works fine until  our QA hits IE which is super slow when the data in the list is replaced from backend. Did some profiling in IE11, turns out the appendChild and removeChild calls are taking forever when it tries to clear the rows in the dom and put the new elements into dom. Also another slowness is from styleCalculation which it does for every column/row. Overall, IE takes 20s to render a page with 5000 rows and FF/safari/chrome need only 1-2 seconds. This forces us to abandon the straightforward way to use the more IE friendly way which is pagination with angular ui-grid. But this brings us to another problem which is print since data is now paginated and DOM only has 20 rows.

Server side render and client side print

What I eventually did is sending the model data back to server and do server side rendering and eventually send back to browser where an iFrame is created on the fly for printing. The pros of doing this is we have a lot of flexibility on content/layout by whatever manipulation/styling etc… The cons is we added more stuff to the stack and one more round trip comparing to the direct print.

server side

So on server side, when we get the REST call for print, we have a Thymeleaf template there for generating the html. I compared different java server side rendering engines like Velocity/Freemaker/Rythm etc, looks like Thymeleaf has the best Spring integration and most active development/release.

@Configuration
public class ThymeleafConfig
{
    @Autowired
    private Environment env;

    @Bean
    @Description("Thymeleaf template rendering HTML ")
    public ClassLoaderTemplateResolver exportTemplateResolver() {
        ClassLoaderTemplateResolver exportTemplateResolver = new ClassLoaderTemplateResolver();
        exportTemplateResolver.setPrefix("thymeleaf/");
        exportTemplateResolver.setSuffix(".html");
        exportTemplateResolver.setTemplateMode("HTML5");
        exportTemplateResolver.setCharacterEncoding(CharEncoding.UTF_8);
        exportTemplateResolver.setOrder(1);
        //for local development, we do not want template being cached so that we could do hot reload.
        if ("local".equals(env.getProperty("APP_ENV")))
        {
            exportTemplateResolver.setCacheable(false);
        }
        return exportTemplateResolver;
    }

    @Bean
    public SpringTemplateEngine templateEngine() {
        final SpringTemplateEngine engine = new SpringTemplateEngine();
        final Set<ITemplateResolver> templateResolvers = new HashSet<>();
        templateResolvers.add(exportTemplateResolver());
        engine.setTemplateResolvers(templateResolvers);
        return engine;
    }
}

With the engine we confined, we could used like:

            Context context = new Context();
            context.setVariable("firms", firms);
            context.setVariable("period", period);
            context.setVariable("rptName", rptName);
            context.setVariable("hasFirmId", hasFirmId);
            if (hasFirmId)
            {
                context.setVariable("firmIdType", FirmIdType.getFirmIdType(maybeFirmId).get());
            }

            return templateEngine.process("sroPrint", context);

Template with name sroPrint has some basic Theymleaf directives:

<html xmlns:th="http://www.thymeleaf.org">
<head>
<style>
    table thead tr th, table tbody tr td {
      border: 1px solid black;
      text-align: center;
    }
  </style>

</head>
<body>
<div>
<h4 th:text="${rptName}">report name</h4>
<div style="margin: 10px 0;"><b>Period:</b> <span th:text="${period}"></span>
<div>
  <h4 th:text="${rptName}">report name</h4>
  <div style="margin: 10px 0;"><b>Period:</b> <span th:text="${period}"></span></div>
  <table style="width: 100%; ">
    <thead>
    <tr>
      <th th:if="${hasFirmId}" th:text="${firmIdType}"></th>
      <th>crd #</th>
      <th>Firm Name</th>
    </tr>
    </thead>
    <tbody>
    <tr th:each="firm : ${firms}">
      <td th:if="${hasFirmId}" th:text="${firm.firmId}"></td>
      <td th:text="${firm.crdId}">CRD</td>
      <td th:text="${firm.firmName}">firm name</td>
    </tr>
    </tbody>
  </table>
</div>
</body>
</html>

client side

Now on the client side we need to consume the HTML string from the client side. The flow is we create an iFrame, write the html into it and call browser print on that iFrame and remove the element from DOM. The below implementation is inside the success callback of $http call for getting that dom string. It is in pure js without jQuery, with which it might be a bit more concise.


var printIFrame = document.createElement('iframe');
document.body.appendChild(printIFrame);
printIFrame.style.position = 'absolute';
printIFrame.style.top = '-9999px';
printIFrame.style.left = '-9999px';
var frameWindow = printIFrame.contentWindow || printIFrame.contentDocument || printIFrame;
var wdoc = frameWindow.document || frameWindow.contentDocument || frameWindow;
wdoc.write(res.data);
// tell browser write finished
wdoc.close();
$scope.$emit('UNLOAD');
// Fix for IE : Allow it to render the iframe
frameWindow.focus();
try {
    // Fix for IE11 - printng the whole page instead of the iframe content
    if (!frameWindow.document.execCommand('print', false, null)) {
        // document.execCommand returns false if it failed -http://stackoverflow.com/a/21336448/937891
        frameWindow.print();
    }
    // focus body as it is losing focus in iPad and content not getting printed
    document.body.focus();
}
catch (e) {
    frameWindow.print();
}
frameWindow.close();
setTimeout(function() {
    printIFrame.parentElement.removeChild(printIFrame);
}, 0);

PDF/XLS Export

For xls/pdf export, it is similar to the other POST that I have before. The only difference is the dom string was passed from client there. Here we generate the dom string in server side.

Fighting with browser popup block

Background

Recently in our project, we have a need of refactoring some old struct actions to rest based pages. This way we avoid multiple page navigation for our user so that all the stuff can be done in a single page.

One example is file download. Previously in the struts based app, if a page have 12 files. What user have to do is click the download link in the main page, if available, user will be taken to the download page where the real download link is, then download. if not available, user will be taken to a request page for confirmation and then once confirmed, to the download page to wait. So to download all the files, user have to constantly navigate between different pages with a lot of clicks which is kind of crazy. In the coming single page application, everything(request/confirm/download) is in the same page which is much better.

Issue

However, we hit one issue. When user click the download link, the same as the above flow, we first need to make an ajax call back to server to check, if not available, a modal will show up for confirming request. otherwise get the download id and open a new tab for download the stream. The problem comes from this point where the browser(chrome/FF, safari does not) will block the download tab from opening. Tried it both form submit and window open. What is really bad is in chrome the block notification is really not noticeable, which is a tiny icon on the upper-left where user can barely see.

check status

        this.requestDetail = function (requestObj, modalService) {
            that.checkDetailStatus(requestObj).then(
                function success(res) {
                    var status = res.data.status;
                    switch (status) {
                        case 'AVAIL_NOT_REQ':
                            that.createNewRequest(requestObj, modalService);
                            break;
                        case 'NO_DATA':
                            $.bootstrapGrowl('No data available!', {type: 'info'});
                            break;
                        case 'EXISTING_RPT':
                            that.downloadFile(res.data.requestId);
                            break;
                        case 'PENDING':
                            //add user to notify list then redirect
                            that.mapNotifyUser(res.data.requestId).then(
                                function success(res) {
                                    var DETAIL_RUN_INTERVAL = 3;
                                    var minute = DETAIL_RUN_INTERVAL - res.data.minute % DETAIL_RUN_INTERVAL;
                                    $.bootstrapGrowl('Your detail data file will be ready in ' + minute + ' minutes.', {type: 'info'});
                                });
                            break;
                        case 'ERROR':
                            $.bootstrapGrowl('Error Getting Detail data! Contact Admin or Wait for 24 hour to re-request.', {type: 'danger'});
                            break;
                        default:
                            $.bootstrapGrowl('Error Getting Detail data, Contact ADMIN', {type: 'danger'});
                    }
                },
                function error(err) {
                    console.log(err);
                    $.bootstrapGrowl('Network error or Server error!', {type: 'danger'});
                }
            );
        };

with form


        this.downloadFile = function (requestId) {
            //create a form which calls the download REST service on the fly
            var formElement = angular.element("
<form>");
            formElement.attr("action", "/scrc/rest/download/detail/requestId/" + requestId);
            formElement.attr("method", "get");
            formElement.attr("target", "_blank");
            // we need to attach iframe to the body before form could be attached to iframe(below) in ie8
            angular.element('body').append(formElement);
            //call the service
            formElement.submit();
        };

With window

        this.downloadFile = function (requestId) {
            $window.open('/scrc/rest/download/detail/requestId/' + requestId);
        };

Cause

Turns out the issue is: A browser will only open a tab/popup without the popup blocker warning if the command to open the tab/popup comes from a trusted event. That means the user has to actively click somewhere to open a popup.

In this case, the user performs a click so we have the trusted event. we do lose that trusted context, however, by performing the Ajax request. Our success handler does not have that event anymore.

Possible Solutions

  1. open the popup on click and manipulate it later when the callback fires

      var popup = $window.open('','_blank');
      popup.document.write('loading ...');
      ...
      inCallBack(){
        //existing:
        popup.location.href = '/scrc/rest/download/detail/requestId/' + res.data.requestId;
        // other:
        popup.close();
    
      }
    

    this will work but not elegant since it opens a tab and close instantly but still create a flash in browser that user could notice.

  2. you can require the user to click again some button to trigger the popup. This will work because we could update the link if existing then user click again, we init the download so popup is triggered by user directly. But still not quite user friendly

  3. Notify user to unblock our site.
    This is eventually what we do. So we detect on the client side if popup is blocked. If so, we ask user to unblock our site in setting. The reason we use this is the unblock/trust action is really a one time thing that browser will remember the behavior and will not bother user again.

            this.downloadFile = function (requestId) {
                var downloadWindow = $window.open('/scrc/rest/download/detail/requestId/' + requestId);
                if(!downloadWindow || downloadWindow.closed || typeof downloadWindow.closed=='undefined')
                {
                    $.bootstrapGrowl('Download Blocked!<br\> Please allow popup from our site in your browser setting!', {type: 'danger', delay: 8000});
                }
            };
    

csv file excel BOM for non-ascll characters

Sometimes you need to export a table to the Excel and other spreadsheets. CSV file saved in UTF-8 seems to be sufficiently universal format.

But there is problem with non-ASCII characters in Excel. Excel default encoding depends on the system. The workaround is to put three magical bytes to the file beginning. They are called BOM (Byte order mark) and say to the editor that file is encoded as UTF-8.

for Nodejs, it would be writing the BOM before writing anything else.

    var stream = fs.createWriteStream("YOUR.csv");
    stream.once('open', function(fd) {
        //excel requires utf-8 + BOM for csv file, otherwise chinese char could not be displayed
        stream.write(new Buffer('\xEF\xBB\xBF', 'binary'));
        stream.write('WHATEVER,OTHERCOLUMN\n');
        ...
        stream.end();
        console.log('file Saved');
    });