Oracle Cloud ubuntu instance connection refused

Background Oracle cloud is giving out always free instance/db/warehouse, so I decided to give a try for potential use of our church website. Application Applying a new account take a while, unlike AWS you can use it immediately after creation, this one seems to take some manual check so after 2 days of review, the…

Akamai Origin ERR_FWD_SSL_HANDSHAKE

Background We recently took a project from another team which has CICD pipeline setup that creates a CloudFront(CF) Distro each time and direct traffic for static content and api from there based on path. The static content is fronted via Akamai then route 53(r53) then CF. CF contains the r53 domains as cname, however without…

lambda@edge prototype

Recently I was doing a MVP for replacing a ELB/EC2/Docker based static site preview stack with a cloudfront/lambda/s3 based one. Background The purpose of this is to reduce the maintenance we has to do with the EC2 stack like regular AMI update. reduce the complexity of the stack  as the previous one involves building custom…

nginx reverse proxy S3 files

China access issue Recently some of our church site users reported that the sermon audio/video download feature does not work any more. We recently moved our large files from file system to s3. After some research looks like the aws s3 is blocked by the famous Chinese Great FireWall(GFW). Possible Solutions Moving files back to…

aws cli ProfileNotFound

I was trying to do some KMS encryption for some of our prod credentials with aws cli. After pulling down the temporary aws sts token for prod roles and run the aws –profile SOME_PROD_ROLE kms encrypt xxx, the  botocore.exceptions.ProfileNotFound: The config profile (SOME_DEV_ROLE) could not be found constantly pop up. I checked the ~/.aws/credentails file and make…

Serverless EMR Cluster monitoring with Lambda

Background One issue we typically have is our EMR cluster stops consuming hive queries due to the overload of the metastore loading/refreshing. This is partially caused by the usage of the shared-metastore which hosts many teams’ schema/tables inside our organization. When this happens in prod, we have to ask help from RIM to terminate our…

stateful firewall with inbound outbound traffic

Background I have worked as Devops for cloud migration in the recent 3 months without really writing much code. Even though being exposed to many AWS services like EMR/EC2/ASG(auto scaling group)/LC(launch config)/CF(cloud formation) etc.. with the need of setting up security groups(SG), i find myself still a bit confusing with inbound and outbound traffic rules….

Add Username password auth to Hive

In my previous post, we achieved end to end SSL encryption from client to ELB to the EMR master. Our next requirement is to add username password authentication. There are different ways in hive to do this: 1. LDAP, 2. PAM, 3. CUSTOM mode. After some evaluation we finally choosed the  CUSTOM mode way. We…

EMR hive JDBC over SSL with ELB

Recently we need to setup a hive cluster consuming S3 objects so that we could run query from our java server(tomcat) via JDBC. Several challenges: our java server is on prem(will move to aws in 2017) so we have to secure the channel to the ERM cluster in emr. Solution: use SSL cross the board….

partition key, composite key and clustering key in Cassandra

There is a lot of confusion around this, I will try to make it as simple as possible. The primary key is a general concept to indicate one or more columns used to retrieve data from a Table. The primary key may be SIMPLE create table stackoverflow ( key text PRIMARY KEY, data text );…