upgrade emr3 to 4 and hive from 0.13 to 1.0.0

We have been using emr3.9.x and Hive 0.13 for a while and pretty satisfying with it even with some known bugs. Hive 0.13 is fine, but EMR3 comes with an old version of hadoop which has an annoying bug on concurrency reading which is: when multiple thread is doing hadoop fs -get on the same file…

bitten by system case sensitivity twice today

Today i was bitten by system case insensitivity twice: Maven test does not pick up Spring boot test profile properties git commit does not get the file name change. Spring boot profile based config In spring boot, we can have different resource files(.properties or .yml ) to hold properties. More over, we could have profile specific ones,…

Spring nested @Transactional rollback only

Recently we get some odd xxx Thread got an uncaught exception in Nagios alters. and the corresponding exception in log is : Turns out the reason is we have a nested method also marked @Transactional and some exception happened inside which causes Spring marked it as RollBackonly in the thread local TransactionStatus. So the possible solution is:…