aws cli ProfileNotFound

I was trying to do some KMS encryption for some of our prod credentials with aws cli. After pulling down the temporary aws sts token for prod roles and run the aws –profile SOME_PROD_ROLE kms encrypt xxx, the  botocore.exceptions.ProfileNotFound: The config profile (SOME_DEV_ROLE) could not be found constantly pop up. I checked the ~/.aws/credentails file and make…

Serverless EMR Cluster monitoring with Lambda

Background One issue we typically have is our EMR cluster stops consuming hive queries due to the overload of the metastore loading/refreshing. This is partially caused by the usage of the shared-metastore which hosts many teams’ schema/tables inside our organization. When this happens in prod, we have to ask help from RIM to terminate our…