Akamai CloudFront Certificate pinning

We recently took over a project that has some static angularJS code hosted in via standard Route 53 -> CloudFront -> S3. And domain CDN is using Akamai which has origin set to the R53 url. Akamai has a `Specific Certificates (pinning)` setting to pull certificate from a *.cloudfront.net. 

Over the weekend something odd happened, some of our users can access, some get Akamai error, some get 404. Turns out the reason is the CloudFront certificate expires and amazon signed a new one. However in our Akamai side, we still hold that old cert’s hash which is used for ssl validation. As a result the ssl connection cannot be established. To solve the issue we have to re-extract the hash for new certificate in Akamai console. And it worked in our QA. The time we are ready to apply this change  to prod, our prod suddenly worked… We believe Akamai also have some mechanism to periodically pull the site for new certificate chain, or the max-age of the old hash reached.

A final solution will be create a new cert with R53 domain name to avoid all these pin/expire issue.

Certificate Pinning is basically allow us to ignore the regular SSL certificate chain verification process and just/only trust the provided certs. For Akamai, it generates the SHA-1 fingerprint of the leaf cert in the chain and use that to compare on each SSL handshake.

This link show chrome/Symantec is working on phasing out Symantec issued certs( Certs signed with Dec 1st 2017 or later are still good).

Advertisements

EMR hive JDBC over SSL with ELB

Recently we need to setup a hive cluster consuming S3 objects so that we could run query from our java server(tomcat) via JDBC.

Several challenges:

  1. our java server is on prem(will move to aws in 2017) so we have to secure the channel to the ERM cluster in emr.
    Solution: use SSL cross the board.
  2. Every time the EMR cluster restarts, its ip changes. For the JDBC connection, we would like a constant value (DNS) name.
    Solution: use ELB to obtain a DNS name.
  3. EMR master needs to be attached to ELB every time it is created
    Solution: in the bootstrap script, we constantly pull the state of the EMR creation state and once it finishes, grab its IP and attach to ELB.

For the challenge 1

we need to install the certificate chain as well as the private key into ELB so that it could accept ssl connection from client side. In the client JDBC, we need to add ‘;ssl=true’ to the jdbc connection string to instruct client to init the connection with SSL. Another thing is to import the CA to the JRE’s lib/security cacerts so that when we do SSL handshake, the ELB’s certificate CA would be in the java client’s truststore.  For our test, we use a self-CA(finrarcselfca.crt is our self CA certificate):

sudo keytool -keystore cacerts -importcert -alias finrarcselfca -file finrarcselfca.crt

As for the connection between ELB and EMR master, we can just use a random self-signed keystore since ELB does not need to verify the ERM cluster.First generate a self-signed keystore using:

keytool -genkey -keyalg RSA -alias selfsigned -keystore self.keystore -storepass changeit -validity 360 -keysize 2048

and then add the below config to the hive-site.xml

    <property>
        <name>hive.server2.use.SSL</name>
        <value>true</value>
    </property>
    <property>
        <name>hive.server2.keystore.path</name>
        <value>/home/hadoop/hive/conf/hive2.keystore</value>
    </property>
    <property>
        <name>hive.server2.keystore.password</name>
        <value>changeit</value>
    </property>

Note: the password should match for the one when created and the one specifed in the hive site xml.

For the challenge 2

In the ELB listener, we can forward the SSL TCP 443 port to the SSL TCP 10000 port. This way, when our java client init the JDBC over SSL connection, the ELB could unwrap the message and then start another SSL connection with the EMR master via port 10000.

For the challenge 3

Previously We have following shell script to wait ERM startup and attach to ELB. The bash way is quick and dirty.
Now we are leverage nodejs with aws-js-sdk to maintain our cluster which is more robust and easier to maintain/understand.

if [ $exitStatus -eq 0 ]
then
   clusterIdLine=`echo $result| sed 's/{\|"\|:\|}\|ClusterId//g'`
   clusterId=`echo $clusterIdLine| awk '{$1=$1;print}'`
   Mail "Adhoc Query Cluster $clusterId started..."
else
   Mail "Error while creating Adhoc Query Cluster $result"
fi

sleep 30

while :
do
clusterState=`aws emr list-clusters --active |grep '"Id":\|"State":' | sed 'N;s/\n/ /' |grep "$clusterId" |sed 's/"\|,\|Name\|Id//g'|cut -d':' -f2|awk '{$1=$1;print}'`
echo $clusterId
echo $clusterState
if [ $clusterState = 'WAITING' ]
then
   break
fi
  echo 'Waiting for cluster to be created'
  sleep 30
done

masterInstanceID=`aws emr list-instances --cluster-id $clusterId --instance-group-types MASTER |grep '"Ec2InstanceId":' | sed 'N;s/\n/ /' |sed 's/"\|,\|Ec2InstanceId//g'|cut -d':' -f2|awk '{$1=$1;print}'`

echo $masterInstanceID

result=`aws elb register-instances-with-load-balancer --load-balancer-name $elbName --instances $masterInstanceID`

echo $result

Some reference

How to Create a Self Signed Certificate using Java Keytool

import private key to keystore

Cloudera hive config

How SSL works

Simple version

  1. Your web browser downloads the web server’s certificate, which contains the public key of the web server. This certificate is signed with the private key of a trusted certificate authority.
  2. Your web browser comes installed with the public keys of all of the major certificate authorities. It uses this public key to verify that the web server’s certificate was indeed signed by the trusted certificate authority.
  3. The certificate contains the domain name and/or ip address of the web server. Your web browser confirms that the address listed in the certificate is the one to which it has an open connection.
  4. Your web browser generates a shared symmetric key which will be used to encrypt the HTTP traffic on this connection; this is much more efficient than using public/private key encryption for everything. Your browser encrypts the symmetric key with the public key of the web server then sends it back, thus ensuring that only the web server can decrypt it, since only the web server has its private key.

Note that the certificate authority (CA) is essential to preventing man-in-the-middle attacks. However, even an unsigned certificate will prevent someone from passively listening in on your encrypted traffic, since they have no way to gain access to your shared symmetric key.

 复杂版本

SSL协议使用不对称加密技术实现会话双方之间信息的安全传递。可以实现信息传递的保密性、完整性,并且会话双方能鉴别对方身份。不同于常用的http协议,我们在与网站建立SSL安全连接时使用https协议,即采用https://ip:port/的方式来访问。当我们与一个网站建立https连接时,我们的浏览器与Web Server之间要经过一个握手的过程来完成身份鉴定与密钥交换,从而建立安全连接。具体过程如下:
用户浏览器将其SSL版本号、加密设置参数、与session有关的数据以及其它一些必要信息发送到服务器。
服务器将其SSL版本号、加密设置参数、与session有关的数据以及其它一些必要信息发送给浏览器,同时发给浏览器的还有服务器的证书。如果配置服务器的SSL需要验证用户身份,还要发出请求要求浏览器提供用户证书。
客户端检查服务器证书,如果检查失败,提示不能建立SSL连接。如果成功,那么继续。客户端浏览器为本次会话生成pre-master secret,并将其用服务器公钥加密后发送给服务器。如果服务器要求鉴别客户身份,客户端还要再对另外一些数据签名后并将其与客户端证书一起发送给服务器。
如果服务器要求鉴别客户身份,则检查签署客户证书的CA是否可信。如果不在信任列表中,结束本次会话。如果检查通过,服务器用自己的私钥解密收到的pre-master secret,并用它通过某些算法生成本次会话的master secret。
客户端与服务器均使用此master secret生成本次会话的会话密钥(对称密钥)。在双方SSL握手结束后传递任何消息均使用此会话密钥。这样做的主要原因是对称加密比非对称加密的运算量低一个数量级以上,能够显著提高双方会话时的运算速度。
客户端通知服务器此后发送的消息都使用这个会话密钥进行加密。并通知服务器客户端已经完成本次SSL握手。
服务器通知客户端此后发送的消息都使用这个会话密钥进行加密。并通知客户端服务器已经完成本次SSL握手。
本次握手过程结束,会话已经建立。双方使用同一个会话密钥分别对发送以及接受的信息进行加、解密。

更复杂版本

 Complete Version

To see a complete worked example of this process using Firefox connecting to amazon.com, seemoserware.com/2009/06/first-few-milliseconds-of-https.html

Certificate

cert = identity_info + public key + digit signature(CA signed identity-info)

my another post in Chinese about cert.

4 questions need to be address during the authentication:

  1. Has the Digital Certificate been issued/signed by a Trusted CA?
    in browser, already CA installed. In java, import others from `cacert` file
  2. Is the Certificate Expired – checks both the start and end dates
    make sure it is not expired
  3. Has the Certificate been revoked? (Could be OCSP or CRL check)
    either from a Certificate Revocation List (CRL), OR from a Online Certificate Status Protocol (OCSP).
  4. Has the client provided proof of possession?
    sign the message’s hash with its private key and send along with the cert so that the receiver could use the public key to decrypt the message hash and do a hash to the message and then compare.

Client certificate(2 way SSL)

Client also need cert from CA(same or another) as well as the private key. Server need the CA which signed the client cert in its cacert.

During the handshake, the client not only send the cert to the server, but also something signed by its private key. More specific: according to RFC5246 7.4.8.  Certificate Verify  in which client sign the handshake messages in the `Certificate Verify` message of the TLS handshake so that the server can verify it against the public key sent in the client certificate. Without this step, no client-certificate authentication would be taking place.

数字证书的原理与实现

数字证书概述

  数字证书是一段包含用户身份信息、用户公钥信息以及身份验证机构数字签名的数据。身份验证机构的数字签名可以确保证书信息的真实性,用户公钥信息可以保证数字信息传输的完整性,用户的数字签名可以保证数字信息的不可否认性。

  数字证书是各类终端实体和最终用户在网上进行信息交流及商务活动的身份证明,在电子交易的各个环节,交易的各方都需验证对方数字证书的有效性,从而解决相互间的信任问题。

  数字证书是一个经证书认证中心(CA)数字签名的包含公开密钥拥有者信息以及公开密钥的文件。认证中心(CA)作为权威的、可信赖的、公正的第三方机构,专门负责为各种认证需求提供数字证书服务。认证中心颁发的数字证书均遵循X.509 V3标准。X.509标准在编排公共密钥密码格式方面已被广为接受。X.509证书已应用于许多网络安全,其中包括IPSec(IP安全)、SSL、SET、S/MIME。

数字证书使用

  每一个用户有一个各不相同的名字,一个可信的证书认证中心(CA)给每个用户分配一个唯一的名字并签发一个包含名字和用户公开密钥的证书。

  如果甲想和乙通信,他首先必须从数据库中取得乙的证书,然后对它进行验证。如果他们使用相同的CA,事情就很简单。甲只需验证乙证书上CA的签名;如果他们使用不同的CA,问题就复杂了。甲必须从CA的树形结构底部开始,从底层CA往上层CA查询,一直追踪到同一个CA为止,找出共同的信任CA。

  证书可以存储在网络中的数据库中。用户可以利用网络彼此交换证书。当证书撤销后,它将从证书目录中删除,然而签发此证书的CA仍保留此证书的副本,以备日后解决可能引起的纠纷。

  如果用户的密钥或CA的密钥被破坏,从而导致证书的撤销。每一个CA必须保留一个已经撤销但还没有过期的证书废止列表(CRL)。当甲收到一个新证书时,首先应该从证书废止列表(CRL)中检查证书是否已经被撤销。

  现有持证人甲向持证人乙传送数字信息,为了保证信息传送的真实性、完整性和不可否认性,需要对要传送的信息进行数字加密和数字签名,其传送过程如下:

(1)甲准备好要传送的数字信息(明文)。

(2)甲对数字信息进行哈希(hash)运算,得到一个信息摘要。

(3)甲用自己的私钥(SK)对信息摘要进行加密得到甲的数字签名,并将其附在数字信息上。

(4)甲随机产生一个加密密钥(DES密钥),并用此密钥对要发送的信息进行加密,形成密文。

(5)甲用乙的公钥(PK)对刚才随机产生的加密密钥进行加密,将加密后的DES密钥连同密文一起传送给乙。

(6)乙收到甲传送过来的密文和加过密的DES密钥,先用自己的私钥(SK)对加密的DES密钥进行解密,得到DES密钥。

(7)乙然后用DES密钥对收到的密文进行解密,得到明文的数字信息,然后将DES密钥抛弃(即DES密钥作废)。

(8)乙用甲的公钥(PK)对甲的数字签名进行解密,得到信息摘要。 乙用相同的hash算法对收到的明文再进行一次hash运算,得到一个新的信息摘要。

(9)乙将收到的信息摘要和新产生的信息摘要进行比较,如果一致,说明收到的信息没有被修改过。

FROM HERE AND HERE

HERE is a good series of how to secure webservice