Fighting with browser popup block

Background

Recently in our project, we have a need of refactoring some old struct actions to rest based pages. This way we avoid multiple page navigation for our user so that all the stuff can be done in a single page.

One example is file download. Previously in the struts based app, if a page have 12 files. What user have to do is click the download link in the main page, if available, user will be taken to the download page where the real download link is, then download. if not available, user will be taken to a request page for confirmation and then once confirmed, to the download page to wait. So to download all the files, user have to constantly navigate between different pages with a lot of clicks which is kind of crazy. In the coming single page application, everything(request/confirm/download) is in the same page which is much better.

Issue

However, we hit one issue. When user click the download link, the same as the above flow, we first need to make an ajax call back to server to check, if not available, a modal will show up for confirming request. otherwise get the download id and open a new tab for download the stream. The problem comes from this point where the browser(chrome/FF, safari does not) will block the download tab from opening. Tried it both form submit and window open. What is really bad is in chrome the block notification is really not noticeable, which is a tiny icon on the upper-left where user can barely see.

check status

        this.requestDetail = function (requestObj, modalService) {
            that.checkDetailStatus(requestObj).then(
                function success(res) {
                    var status = res.data.status;
                    switch (status) {
                        case 'AVAIL_NOT_REQ':
                            that.createNewRequest(requestObj, modalService);
                            break;
                        case 'NO_DATA':
                            $.bootstrapGrowl('No data available!', {type: 'info'});
                            break;
                        case 'EXISTING_RPT':
                            that.downloadFile(res.data.requestId);
                            break;
                        case 'PENDING':
                            //add user to notify list then redirect
                            that.mapNotifyUser(res.data.requestId).then(
                                function success(res) {
                                    var DETAIL_RUN_INTERVAL = 3;
                                    var minute = DETAIL_RUN_INTERVAL - res.data.minute % DETAIL_RUN_INTERVAL;
                                    $.bootstrapGrowl('Your detail data file will be ready in ' + minute + ' minutes.', {type: 'info'});
                                });
                            break;
                        case 'ERROR':
                            $.bootstrapGrowl('Error Getting Detail data! Contact Admin or Wait for 24 hour to re-request.', {type: 'danger'});
                            break;
                        default:
                            $.bootstrapGrowl('Error Getting Detail data, Contact ADMIN', {type: 'danger'});
                    }
                },
                function error(err) {
                    console.log(err);
                    $.bootstrapGrowl('Network error or Server error!', {type: 'danger'});
                }
            );
        };

with form


        this.downloadFile = function (requestId) {
            //create a form which calls the download REST service on the fly
            var formElement = angular.element("
<form>");
            formElement.attr("action", "/scrc/rest/download/detail/requestId/" + requestId);
            formElement.attr("method", "get");
            formElement.attr("target", "_blank");
            // we need to attach iframe to the body before form could be attached to iframe(below) in ie8
            angular.element('body').append(formElement);
            //call the service
            formElement.submit();
        };

With window

        this.downloadFile = function (requestId) {
            $window.open('/scrc/rest/download/detail/requestId/' + requestId);
        };

Cause

Turns out the issue is: A browser will only open a tab/popup without the popup blocker warning if the command to open the tab/popup comes from a trusted event. That means the user has to actively click somewhere to open a popup.

In this case, the user performs a click so we have the trusted event. we do lose that trusted context, however, by performing the Ajax request. Our success handler does not have that event anymore.

Possible Solutions

  1. open the popup on click and manipulate it later when the callback fires

      var popup = $window.open('','_blank');
      popup.document.write('loading ...');
      ...
      inCallBack(){
        //existing:
        popup.location.href = '/scrc/rest/download/detail/requestId/' + res.data.requestId;
        // other:
        popup.close();
    
      }
    

    this will work but not elegant since it opens a tab and close instantly but still create a flash in browser that user could notice.

  2. you can require the user to click again some button to trigger the popup. This will work because we could update the link if existing then user click again, we init the download so popup is triggered by user directly. But still not quite user friendly

  3. Notify user to unblock our site.
    This is eventually what we do. So we detect on the client side if popup is blocked. If so, we ask user to unblock our site in setting. The reason we use this is the unblock/trust action is really a one time thing that browser will remember the behavior and will not bother user again.

            this.downloadFile = function (requestId) {
                var downloadWindow = $window.open('/scrc/rest/download/detail/requestId/' + requestId);
                if(!downloadWindow || downloadWindow.closed || typeof downloadWindow.closed=='undefined')
                {
                    $.bootstrapGrowl('Download Blocked!<br\> Please allow popup from our site in your browser setting!', {type: 'danger', delay: 8000});
                }
            };
    

understand CORS

My colleague told me that there is a chrome extension enables you do cross domain request for all sites. Was a bit surprised since my previous understanding was CORS is controlled from the server side with some control headers. So I decided to dig more to it. After reading wiki,  and some Chinese article, I think I know how that works.

Overview

One thing I found out is CORS is actually controlled by both server and client side, I mean CORS requires support from both browser and server. All the modern browser and IE 10+ are good to go. The whole process is handled by browser. So for USER, it is transparent. For DEVELOPER, the code is the same. Browser will add some header and sometimes add an extra request.

Two types of Request(Simple/Non-simple)

A simple cross-site request is one that meets all the following conditions:

  • The only allowed methods are:
    • GET
    • HEAD
    • POST
  • Apart from the headers set automatically by the user agent (e.g. Connection, User-Agent, etc.), the only headers which are allowed to be manually set are:
    • Accept
    • Accept-Language
    • Content-Language
    • Content-Type
  • The only allowed values for the Content-Type header are:
    • application/x-www-form-urlencoded
    • multipart/form-data
    • text/plain

For simple requests, browser will add origin  header to the request and see how server response. One caveat is, even if server does not allow, the response status code will probably still be 200. The error can be handled by the onError();

For non-simple requests, browser will send a preflight request by the OPTIONS method to the resource on the other domain to see whether it is allowed. According to the previous request definition, the typical xhr json content type(application/json) is non-simple request which will require a preflight.

chrome CORS extension

So I think how the chrome extension works is it would intercept all the cross site xhr requests.

For simple request, it after getting the response, it would add `Access-Control-Allow-Origin: *  to the header, so that the browser does not complain.

For non-simple request, it would directly return Access-Control-Allow-Origin: * for the preflight request so that browser will allow the subsequence ‘real’ request to be sent out. One thing I notice is that it will set the Origin to evil.com which is kind of funny.

 

how browser render a page with html/css/js

Basic steps:

1. DNS get the IP. Then send http/s request to the server to fetch the page content.

2. On getting the content, download the resources(css/js/images) and try to render it. Note: the render process would not begin before css is all downloaded since it does not make sense to render something without konwing its styling.

3. Build the DOM for the given HTML and the CSS-DOM for the style sheets. Then put these two together and build the Render-tree which will be used to paint the real viewable page.

4. Once we have the render tree, browser starts to do LAYOUT which will calculate the dimension of the elements(width/height) as well as the position. The viewport(root element ‘HTML’) will be place in the (0,0) position. 0,0 here is the innerHeight/innerWidth.

5. After the Layout is done, browser start to PAINT the render tree and output all the elements to the viewport.

Anytime css/DOM changes, the re-Paint or re-Layout will be triggered so that user get the latest view. See the below Article for detail.

My another post explains the sequence of the execution when page load

This is a nice article explaining the basics about the rendering process:

This is the most detail article on how browser work. 中文翻译: 现代浏览器的工作原理

This video also explained the basic in 4 minutes.

This is a better video explaining how browser render a page

Below is the BEST video so far explaining this topic by CTO of Akamai:

gwt browser interaction

First, the user enters the URL of your application, which triggers the browser to
request the application’s HTML file. The HTML downloads a specifically named
nocache.js file—the so-called bootstrap file. The bootstrap code determines which specific permutation of your application in JavaScript is required and then requests it.
When the permutation is loaded, the bootstrap calls the compiled onModuleLoad method from the EntryPoint and the application starts.

gwt browser interaction

gwt browser interaction

The non-cache.js which is mentioned above, contains the cached html file name. if it it there, loaded it, otherwise, request from server:

You’ll have noticed in the deployable part of the application some strange filenames, such as 1D94FD19D212152F48C6F39017347DC8.cache.html. These are the JavaScript implementations of your application that the compiler creates and names. Within the bootstrap code of your application (also created by the compiler), the correct file will be requested.
By naming these files using an MD5 hashing algorithm, GWT ensures two things:
1. The browser can cache the file with no confusion. The next time you run the
application, the startup is even faster.

2. If you recompile, then the bootstrap code will force the browser to load the new version of your application (because it will be asking for a new filename). The
next time you run

It’s as simple as that.

Prevent nocache.js being cached

When a new release comes, we might deploy some new features so the xxx.cache.html content would change. However if the user’s browser still caches the nocache.js, then the new xxx.cache.html would not be able to be downloaded by user.

To prevent this, you can append a UUID to the nocache.js url so that every time the html/jsp is downloaded, it always get the nocache.js downloaded.

<%@ page import="java.util.UUID" %>
<script type="text/javascript" language="javascript" src="gwt_YOURAPP/gwt_YOURAPP.nocache.js?<%=UUID.randomUUID().toString()%>"></script>

What happens when you type a web page address in the address bar

  1. Your browser, if it doesn’t already know, will ask your OS’s DNS system what the address (IP address) of the host (“www.google.com,” for example) is. If your OS doesn’t know, it will query third-party DNS servers (those of your ISP, for example).
  2. Once an address is obtained, your web browser establishes a TCP/IP socket connection, typically on TCP port 80, with the web server at the IP address it resolved the host name to.
  3. Once your browser has established this connection, it sends an HTTP GET request to the web server for whatever resource was requested in your URL. For example, http://www.google.com/ would mean you’d send a ‘/’ request to whatever web server is at http://www.google.com.
  4. The web server will then, typically, respond to the request with an HTTP response, typically containing HTML. Your web browser downloads this response.
  5. Your web browser renders the HTML. It may need to send additional requests for any scripts, stylesheets, images, or other resources linked to in the HTML.

How session works

Basic ideas

When server creates a new session, it always adds a session identifier in the form of cookie. When web browser asks for a page or makes a request, the web browser always sends cookie which are created by the web server in the request. Therefore in the server side, web server checks for that cookie and find the corresponding session that is matched to the received cookie.

The session normally short-lived so the session cookie is not saved into disk. Session also has time out. When the time is out, the session is no longer exist in the server side. You can set time out of session in configuration file in the server.

Servlet handling

You can read the RFC describing Cookies and the related headers, Set-Cookie and Cookie to understand what they are.

You can go through Chapter 7 of the Servlet Specification if you want to understand in detail how Cookies and Sessions are related.

You first need to understand that HTTP is a stateless protocol. This means that each request that a client makes has no relation to any previous or future requests. However, as users, we very much want some state when interacting with a web application. A bank application, for example, only wants you to be able to see and manage your transactions. A music streaming website might want to recommend some good beats based on what you’ve already heard.

To achieve this, the Cookie and Session concepts were introduced. Cookies are key-value pairs, but with a specific format (see the links). Sessions are server-side entities that store information (in memory or persisted) that spans multiple requests/responses between the server and the client.

The Servlet HTTP session uses a cookie with the name JSESSIONID and a value that identifies the session.

The Servlet container keeps a map (YMMV) of HttpSession objects and these identifiers. When a client first makes a request, the server creates an HttpSession object with a unique identifier and stores it in its map. It then adds a Set-Cookie header in the response. It sets the cookie’s name to JSESSIONID and its value to the identifier it just created.

This is the most basic Cookie that a server uses. You can set any number of them with any information you wish. The Servlet API makes that a little simpler for you with the HttpServletResponse#addCookie(Cookie)method but you could do it yourself with the HttpServletResponse#addHeader(String, String) method.

The client receives these cookies and can store them somewhere, typically in a text file. When sending a new request to the server, it can use that cookie in the request’s Cookie header to notify the server that it might have done a previous request.

JSESSIONID :

A JSESSIONID cookie is created on the user’s computer each time a session is created with request.getSession(). Why ? Because each session created on server side has an ID. You can’t acces another user’s session, unless you don’t have the right ID. This ID is kept in JSESSIONID cookie, and allow the user to find his information. Look at this answer for more details !

When does a JSESSIONID is deleted ?

JSESSIONID doesn’t have an expiration date : it’s a session cookie. As all session cookies, it will be deleted when the broswer is closed. If you use the basic JSESSIONID mechanism, then the session will become unreachable after you close and re-open the browser, because JSESSIONID cookie is deleted.

browser handling

By default session tracking happens by cookiesWebServer sends the session id to the browser in the form of cookie. And, the browser send the cookie having session id for the subsequent requests.

How does the browser identifies which cookies to send for a link/request? It is based on the these parameters. If the request matches these paramters the browser sends that particular cookie:

  1. Domain: The domain name to which the request is made. Verify in your case if the domain name is same for two instances
  2. Path: If the path name is same. Web Server send the context root as the path , requests under same context root share cookies.
  3. Secure: Server sends if the given cookie is secure or not. Meaning, if the cookie can be sent on non-secure channel.

HERE is an example on how to get cookie.