large file from hive to rdbms(oracle)

Recently we have a requirement of dumping a sizable file(4+G) to oracle from s3. The file itself is hive-compatiable. so instead of downloading the file and generate sql for it, we decided to transfer the content using hive jdbc and persist in via jpa/hiberante. Hive On the hive side, one important thing is to make…

Optimize JPA one to many with fetch

 N+1 We all know the famous N+1 problem in the ORM world where the generated sql is not that optimized that we have to get the child collections with N number of separate sql query which leads to serious performance problem if there are many levels of objects tree. So we can fix the problem…

JPA hibernate @GeneratedValue with sequence on Oracle

When using Oracle, the JPA with hibernate implementation on the @GeneratedValue with sequence would be quite tricky. I am trying to make a summary here. The version used are JPA 2.1 and Hibernate 4.3.11.Final Default generator setting @GeneratedValue with strategy AUTO and default allocationSize(50) This will use the ‘org.hibernate.id.SequenceGenerator’ which will get a SEQ+1 value….

JPA SequenceGenerator with allocationSize 1 performance tuning

I had a blog last year about fixing the sequence number going wild by setting the allocationSize to 1. Overall it solves the inconsistency problem if you are using a sequence with ‘INCREMENT BY’ value 1 in database. Issue One problem comes up today is I am facing some performance issue with the above setting when I…

PESSIMISTIC_READ and PESSIMISTIC_WRITE

Readers–writer lock A database system is a highly concurrent environment, therefore many concurrency theory idioms apply to database access as well. Concurrent changes must be serialized to preserve data integrity, so most database systems use a two-phase locking strategy, even if it’s usually supplemented by a Multiversion concurrency control mechanism. Because a mutual exclusion locking…

Fix Hibernate JPA sequence generating odd id with allocationSize

I config my jpa entity class as usual using Intellij generated stub from DB schema with below config: However the id that is inserted into DB are way different(larger) than it should be. For example, id should be 10012 but it becomes 5223326. This odd number is actually getting from some other sequence. To fix…

memcached usage

memcached shares similar logic with Cassandra when locating the record, where the hash of the key is used to find which server the record could be. It could be used to cached frequent executed query(objects) or server-expensive html content we have to generate(jsp etc). More about the usage for java appliation is here: part1 for…

hibernate caching strategy

For databases, the following applies to all levels: the best database accesses are those that do not have to be carried out. For this, caches are used within the database and also in Hibernate and JPA. Caching is relatively complex and one of the most misunderstood concepts of Hibernate. There are three different caches: First…

batch / bulk insert/update in jpa/hibernate with flush and clear

For JPA use entity manager to do flush/clear For hibernate basically just switch the entity manager with hibernate session. When making new objects persistent flush() and then clear() the session regularly in order to control the size of the first-level cache. The suggest batch size is 20-50 by hibernate. However I found 1500 is good…

jpa performance over jdbc for large table

I have a table with about 80 million records. While I was doing a simple query using JPA with 2-3 predicates. It takes about 120s to get the result, comparing the 1s using JDBC. Notice, i am using exactly the same query that the jpa generates. This is somehow frustrating. To be honest, I have…

use oracle sysdate as hibernate Date

I have a column in DB with default value as sysdate. I’m looking for a way to get that default value inserted while I’m not giving anything to corresponding property on app side.  If I did not set any value to the field, i get null in the DB.  The reason why the date column…

Hibernate: Understanding Cascade Types

We want to understand what happens when we have CascadeType.REMOVE in a @ManyToOnerelationship. From the Hibernate Community Documentation 11.11. Transitive persistence It is quite cumbersome to save, delete, or reattach individual objects, especially if you deal with a graph of associated objects. A common case is a parent/child relationship. Consider the following example: If the…