regex dash – inside bracket []

We have a requirement to remove some illegal character in file name like slash etc… So I put on a regex in our code:

[^a-zA-Z0-9.-_]

I assume the above regex will match everything that is not number/char/dot/dash/underscore.

Turns out I was wrong! The - is special case inside [...] that is used for range. It should be in the beginning or in the last or escaped. Otherwise it will match all the character that is in between . and _ in ASCII character set. So in my case, the .-_ part will try to match characters from 46(.)-95(_) in ASCII char table.

The correct one should be putting it in the last  [^a-zA-Z0-9._-] . Or just escape it: [^a-zA-Z0-9.\-_]

print all data in paginated table/grid

direct Tabular data display

Recently our project has a page need to show tabular data from 30-6000 rows with 3-4 columns. At first, I thought this is pretty reasonable data to show in one page so I just throw the data into a ng-repeat table with my own implementation of filtering/sorting which is pretty straightforward in angular. Every time user select new category/type, fetch data from backend and replace the data in vm/$scope. Also with this implementation, it is quite easy to fulfill our another requirement which is export/print the page content. For export I just need to get the DOM content to the server and return as downloadable. For print, even easier, just call window.print() ,that’s it.

Performance issue with IE

Everything works fine until  our QA hits IE which is super slow when the data in the list is replaced from backend. Did some profiling in IE11, turns out the appendChild and removeChild calls are taking forever when it tries to clear the rows in the dom and put the new elements into dom. Also another slowness is from styleCalculation which it does for every column/row. Overall, IE takes 20s to render a page with 5000 rows and FF/safari/chrome need only 1-2 seconds. This forces us to abandon the straightforward way to use the more IE friendly way which is pagination with angular ui-grid. But this brings us to another problem which is print since data is now paginated and DOM only has 20 rows.

Server side render and client side print

What I eventually did is sending the model data back to server and do server side rendering and eventually send back to browser where an iFrame is created on the fly for printing. The pros of doing this is we have a lot of flexibility on content/layout by whatever manipulation/styling etc… The cons is we added more stuff to the stack and one more round trip comparing to the direct print.

server side

So on server side, when we get the REST call for print, we have a Thymeleaf template there for generating the html. I compared different java server side rendering engines like Velocity/Freemaker/Rythm etc, looks like Thymeleaf has the best Spring integration and most active development/release.

@Configuration
public class ThymeleafConfig
{
    @Autowired
    private Environment env;

    @Bean
    @Description("Thymeleaf template rendering HTML ")
    public ClassLoaderTemplateResolver exportTemplateResolver() {
        ClassLoaderTemplateResolver exportTemplateResolver = new ClassLoaderTemplateResolver();
        exportTemplateResolver.setPrefix("thymeleaf/");
        exportTemplateResolver.setSuffix(".html");
        exportTemplateResolver.setTemplateMode("HTML5");
        exportTemplateResolver.setCharacterEncoding(CharEncoding.UTF_8);
        exportTemplateResolver.setOrder(1);
        //for local development, we do not want template being cached so that we could do hot reload.
        if ("local".equals(env.getProperty("APP_ENV")))
        {
            exportTemplateResolver.setCacheable(false);
        }
        return exportTemplateResolver;
    }

    @Bean
    public SpringTemplateEngine templateEngine() {
        final SpringTemplateEngine engine = new SpringTemplateEngine();
        final Set<ITemplateResolver> templateResolvers = new HashSet<>();
        templateResolvers.add(exportTemplateResolver());
        engine.setTemplateResolvers(templateResolvers);
        return engine;
    }
}

With the engine we confined, we could used like:

            Context context = new Context();
            context.setVariable("firms", firms);
            context.setVariable("period", period);
            context.setVariable("rptName", rptName);
            context.setVariable("hasFirmId", hasFirmId);
            if (hasFirmId)
            {
                context.setVariable("firmIdType", FirmIdType.getFirmIdType(maybeFirmId).get());
            }

            return templateEngine.process("sroPrint", context);

Template with name sroPrint has some basic Theymleaf directives:

<html xmlns:th="http://www.thymeleaf.org">
<head>
<style>
    table thead tr th, table tbody tr td {
      border: 1px solid black;
      text-align: center;
    }
  </style>

</head>
<body>
<div>
<h4 th:text="${rptName}">report name</h4>
<div style="margin: 10px 0;"><b>Period:</b> <span th:text="${period}"></span>
<div>
  <h4 th:text="${rptName}">report name</h4>
  <div style="margin: 10px 0;"><b>Period:</b> <span th:text="${period}"></span></div>
  <table style="width: 100%; ">
    <thead>
    <tr>
      <th th:if="${hasFirmId}" th:text="${firmIdType}"></th>
      <th>crd #</th>
      <th>Firm Name</th>
    </tr>
    </thead>
    <tbody>
    <tr th:each="firm : ${firms}">
      <td th:if="${hasFirmId}" th:text="${firm.firmId}"></td>
      <td th:text="${firm.crdId}">CRD</td>
      <td th:text="${firm.firmName}">firm name</td>
    </tr>
    </tbody>
  </table>
</div>
</body>
</html>

client side

Now on the client side we need to consume the HTML string from the client side. The flow is we create an iFrame, write the html into it and call browser print on that iFrame and remove the element from DOM. The below implementation is inside the success callback of $http call for getting that dom string. It is in pure js without jQuery, with which it might be a bit more concise.


var printIFrame = document.createElement('iframe');
document.body.appendChild(printIFrame);
printIFrame.style.position = 'absolute';
printIFrame.style.top = '-9999px';
printIFrame.style.left = '-9999px';
var frameWindow = printIFrame.contentWindow || printIFrame.contentDocument || printIFrame;
var wdoc = frameWindow.document || frameWindow.contentDocument || frameWindow;
wdoc.write(res.data);
// tell browser write finished
wdoc.close();
$scope.$emit('UNLOAD');
// Fix for IE : Allow it to render the iframe
frameWindow.focus();
try {
    // Fix for IE11 - printng the whole page instead of the iframe content
    if (!frameWindow.document.execCommand('print', false, null)) {
        // document.execCommand returns false if it failed -http://stackoverflow.com/a/21336448/937891
        frameWindow.print();
    }
    // focus body as it is losing focus in iPad and content not getting printed
    document.body.focus();
}
catch (e) {
    frameWindow.print();
}
frameWindow.close();
setTimeout(function() {
    printIFrame.parentElement.removeChild(printIFrame);
}, 0);

PDF/XLS Export

For xls/pdf export, it is similar to the other POST that I have before. The only difference is the dom string was passed from client there. Here we generate the dom string in server side.

Fighting with browser popup block

Background

Recently in our project, we have a need of refactoring some old struct actions to rest based pages. This way we avoid multiple page navigation for our user so that all the stuff can be done in a single page.

One example is file download. Previously in the struts based app, if a page have 12 files. What user have to do is click the download link in the main page, if available, user will be taken to the download page where the real download link is, then download. if not available, user will be taken to a request page for confirmation and then once confirmed, to the download page to wait. So to download all the files, user have to constantly navigate between different pages with a lot of clicks which is kind of crazy. In the coming single page application, everything(request/confirm/download) is in the same page which is much better.

Issue

However, we hit one issue. When user click the download link, the same as the above flow, we first need to make an ajax call back to server to check, if not available, a modal will show up for confirming request. otherwise get the download id and open a new tab for download the stream. The problem comes from this point where the browser(chrome/FF, safari does not) will block the download tab from opening. Tried it both form submit and window open. What is really bad is in chrome the block notification is really not noticeable, which is a tiny icon on the upper-left where user can barely see.

check status

        this.requestDetail = function (requestObj, modalService) {
            that.checkDetailStatus(requestObj).then(
                function success(res) {
                    var status = res.data.status;
                    switch (status) {
                        case 'AVAIL_NOT_REQ':
                            that.createNewRequest(requestObj, modalService);
                            break;
                        case 'NO_DATA':
                            $.bootstrapGrowl('No data available!', {type: 'info'});
                            break;
                        case 'EXISTING_RPT':
                            that.downloadFile(res.data.requestId);
                            break;
                        case 'PENDING':
                            //add user to notify list then redirect
                            that.mapNotifyUser(res.data.requestId).then(
                                function success(res) {
                                    var DETAIL_RUN_INTERVAL = 3;
                                    var minute = DETAIL_RUN_INTERVAL - res.data.minute % DETAIL_RUN_INTERVAL;
                                    $.bootstrapGrowl('Your detail data file will be ready in ' + minute + ' minutes.', {type: 'info'});
                                });
                            break;
                        case 'ERROR':
                            $.bootstrapGrowl('Error Getting Detail data! Contact Admin or Wait for 24 hour to re-request.', {type: 'danger'});
                            break;
                        default:
                            $.bootstrapGrowl('Error Getting Detail data, Contact ADMIN', {type: 'danger'});
                    }
                },
                function error(err) {
                    console.log(err);
                    $.bootstrapGrowl('Network error or Server error!', {type: 'danger'});
                }
            );
        };

with form


        this.downloadFile = function (requestId) {
            //create a form which calls the download REST service on the fly
            var formElement = angular.element("
<form>");
            formElement.attr("action", "/scrc/rest/download/detail/requestId/" + requestId);
            formElement.attr("method", "get");
            formElement.attr("target", "_blank");
            // we need to attach iframe to the body before form could be attached to iframe(below) in ie8
            angular.element('body').append(formElement);
            //call the service
            formElement.submit();
        };

With window

        this.downloadFile = function (requestId) {
            $window.open('/scrc/rest/download/detail/requestId/' + requestId);
        };

Cause

Turns out the issue is: A browser will only open a tab/popup without the popup blocker warning if the command to open the tab/popup comes from a trusted event. That means the user has to actively click somewhere to open a popup.

In this case, the user performs a click so we have the trusted event. we do lose that trusted context, however, by performing the Ajax request. Our success handler does not have that event anymore.

Possible Solutions

  1. open the popup on click and manipulate it later when the callback fires

      var popup = $window.open('','_blank');
      popup.document.write('loading ...');
      ...
      inCallBack(){
        //existing:
        popup.location.href = '/scrc/rest/download/detail/requestId/' + res.data.requestId;
        // other:
        popup.close();
    
      }
    

    this will work but not elegant since it opens a tab and close instantly but still create a flash in browser that user could notice.

  2. you can require the user to click again some button to trigger the popup. This will work because we could update the link if existing then user click again, we init the download so popup is triggered by user directly. But still not quite user friendly

  3. Notify user to unblock our site.
    This is eventually what we do. So we detect on the client side if popup is blocked. If so, we ask user to unblock our site in setting. The reason we use this is the unblock/trust action is really a one time thing that browser will remember the behavior and will not bother user again.

            this.downloadFile = function (requestId) {
                var downloadWindow = $window.open('/scrc/rest/download/detail/requestId/' + requestId);
                if(!downloadWindow || downloadWindow.closed || typeof downloadWindow.closed=='undefined')
                {
                    $.bootstrapGrowl('Download Blocked!<br\> Please allow popup from our site in your browser setting!', {type: 'danger', delay: 8000});
                }
            };
    

csv file excel BOM for non-ascll characters

Sometimes you need to export a table to the Excel and other spreadsheets. CSV file saved in UTF-8 seems to be sufficiently universal format.

But there is problem with non-ASCII characters in Excel. Excel default encoding depends on the system. The workaround is to put three magical bytes to the file beginning. They are called BOM (Byte order mark) and say to the editor that file is encoded as UTF-8.

for Nodejs, it would be writing the BOM before writing anything else.

    var stream = fs.createWriteStream("YOUR.csv");
    stream.once('open', function(fd) {
        //excel requires utf-8 + BOM for csv file, otherwise chinese char could not be displayed
        stream.write(new Buffer('\xEF\xBB\xBF', 'binary'));
        stream.write('WHATEVER,OTHERCOLUMN\n');
        ...
        stream.end();
        console.log('file Saved');
    });

understand CORS

My colleague told me that there is a chrome extension enables you do cross domain request for all sites. Was a bit surprised since my previous understanding was CORS is controlled from the server side with some control headers. So I decided to dig more to it. After reading wiki,  and some Chinese article, I think I know how that works.

Overview

One thing I found out is CORS is actually controlled by both server and client side, I mean CORS requires support from both browser and server. All the modern browser and IE 10+ are good to go. The whole process is handled by browser. So for USER, it is transparent. For DEVELOPER, the code is the same. Browser will add some header and sometimes add an extra request.

Two types of Request(Simple/Non-simple)

A simple cross-site request is one that meets all the following conditions:

  • The only allowed methods are:
    • GET
    • HEAD
    • POST
  • Apart from the headers set automatically by the user agent (e.g. Connection, User-Agent, etc.), the only headers which are allowed to be manually set are:
    • Accept
    • Accept-Language
    • Content-Language
    • Content-Type
  • The only allowed values for the Content-Type header are:
    • application/x-www-form-urlencoded
    • multipart/form-data
    • text/plain

For simple requests, browser will add origin  header to the request and see how server response. One caveat is, even if server does not allow, the response status code will probably still be 200. The error can be handled by the onError();

For non-simple requests, browser will send a preflight request by the OPTIONS method to the resource on the other domain to see whether it is allowed. According to the previous request definition, the typical xhr json content type(application/json) is non-simple request which will require a preflight.

chrome CORS extension

So I think how the chrome extension works is it would intercept all the cross site xhr requests.

For simple request, it after getting the response, it would add `Access-Control-Allow-Origin: *  to the header, so that the browser does not complain.

For non-simple request, it would directly return Access-Control-Allow-Origin: * for the preflight request so that browser will allow the subsequence ‘real’ request to be sent out. One thing I notice is that it will set the Origin to evil.com which is kind of funny.

 

js tilde IIFE

// Without superfluous operator, we need to surround the anonymous ‘scoping’ function in
// parenthesis to force it to be parsed as an expression instead of a *declaration*,
// which allows us to immediately function-call-pattern it.
;(function(){
   // ...
})()

// By inserting a superfluous operator, we can omit those parentheses,
// as the operator forces the parser to view the anonymous function as
// an expression *within* the statement, instead of as the
// statement itself, which saves us a character overall, as well as some ugliness:
;+function(){
   // ...
}()

// But, in all of the above examples, if one is depending on ASI, and
// doesn't needlessly scatter semicolons all over their code out of ignorance,
// a prepended semicolon is necessary to prevent snafus like the following:
var foo = 4
+function(){
   // ...
}()
// ... in which case, the variable `foo` would be set to a crazy
// addition / concatenation involving the (probably non-existent) *return value*
// of our anonymous ‘scoping’ function. Hence, our friend the bitflip:
var foo = 4
~function(){
   // ...
}()
// ... he solves all of our problems, by disnecessitating the prepended semicolon
// *and* the wrapping parentheses.

hoist for var, let, const, function, function*, class

I have been playing with ES6 for a while and I noticed that while variables declared with var are hoisted as expected…

console.log(typeof name); // undefined
var name = "John";

…variables declared with let or const seem to have some problems with hoisting:

console.log(typeof name); // ReferenceError
let name = "John";

and

console.log(typeof name); // ReferenceError
const name = "John";

 

these variables cannot be accessed before they are declared. However, it’s a bit more complicated than that.

Are variables declared with let or const not hoisted? What is really going on here?

All declarations (var, let, const, function, function*, class) are hoisted in JavaScript. This means that if a name is declared in a scope, in that scope the identifier will always reference that particular variable:

x = "global";
(function() {
    x; // not "global"

    var/let/… x;
}());
{
    x; // not "global"

    let/const/… x;
}

This is true both for function and block scopes1.

The difference between var/function/function* declarations and let/const/classdeclara­tions is the initialisation.
The former are initialised with undefined or the (generator) function right when the binding is created at the top of the scope. The lexically declared variables however stay uninitialised. This means that a ReferenceError exception is thrown when you try to access it. It will only get initialised when the let/const/class statement is evaluated, everything above that is called the temporal dead zone.

x = y = "global";
(function() {
    x; // undefined
    y; // Reference error: y is not defined

    var x = "local";
    let y = "local";
}());

Notice that a let y; statement initialises the variable with undefined like let y = undefined;would have.

Is there any difference between let and const in this matter?

No, they work the same as far as hoisting is regarded. The only difference between them is that a constant must be and can only be assigned in the initialiser part of the declaration (const one = 1;, both const one; and later reassignments like one = 2 are invalid).