AWS Parallel Processing on AWS Lambda with Python using Multiprocessing If you are trying to use multiprocessing.Queue or multiprocessing.Pool on AWS Lambda, you are probably getting the exception:[Errno 38] Function not implemented: OSError sl = self._semlock = _multiprocessing.SemLock(kind, value,
AWS How to Setup a Serverless URL Shortener with API Gateway Lambda and DynamoDB on AWS Today we will set a Serverless URL Shortener using API Gateway, Lambda with Python and DynamoDB. Overview The service that we will be creating, will shorten URLs via our API which will create
AWS IAM Policy to Allow Team Wide and User Level Permissions on AWS Secrets Manager In this post we will simulate a scenario where a team would like to have access to create secrets under a team path name like /security-team/prod/* and /security-team/dev/* and allow all
AWS Investigating High Request Latencies on Amazon DynamoDB While testing DynamoDB for a specific use case I picked up at times where a GetItem will incur about 150ms in RequestLatency on the Max Statistic. This made me want to understand the
AWS Improving Performance from your Lambda Function from the use of Global Variables When using Lambda and DynamoDB, you can use global variables to gain performance when your data from DynamoDB does not get updated that often, and you would like to use caching to prevent
AWS Using IAM Authentication with Amazon Elasticsearch Service Today I will demonstrate how to allow access to Amazons Elasticsearch Service using IAM Authenticationi using AWS Signature Version4. Elasticsearch Service Authentication Support: When it comes to security, Amazons Elasticsearch Service supports three
DynamoDB Tutorial on DynamoDB using Bash and the AWS CLI Tools to Interact with a Music Dataset In this tutorial we will be using Amazons DynamoDB (DynamoDB Local) to host a sample dataset consisting of music data that I retrieved from the iTunes API, which we will be using the
AWS Using AWS SSM Parameter Store to Retrieve Secrets Encrypted by KMS using Python Today we will use Amazon Web Services SSM Service to store secrets in their Parameter Store which we will encyrpt using KMS. Then we will read the data from SSM and decrypt using
AWS Expanding the Size of your EBS Volume on AWS EC2 for Linux Resizing your EBS Volume on the fly, that is attached to your EC2 Linux instance, on Amazon Web Services. We want to resize our EBS Volume from 100GB to 1000GB and at the
AWS Using AWS SSM Parameter Store to Retrieve Secrets Encrypted by KMS using Python AWS Systems Manager Parameter Store provides secure storage for configuration data management and secrets management, which allows you to store sensitive iformation like passwords that you can encrypt with your KMS key. Today
AWS Authenticate to your AWS RDS MySQL Instance via IAM On Amazon Web Services with RDS for MySQL or Aurora with MySQL compatibility, you can authenticate to your Database instance or cluster using IAM for database authentication. The benefit of using this authentication
AWS API Gateway with Lambda using Python on AWS to Post info to Rocketchat Today we will setup a API Gateway which has a Lambda Function, written in Python which we will setup using the AWS CLI ( function() { if (window.CHITIKA === undefined) { window.CHITIKA = { 'units' : [] }; }; var unit
DevOps Deploy an EC2 Instance with Terraform Today we will test out Terraform and do a Basic Deployment by Launching an EC2 Instance on Amazon Web Services. What is Terraform: As described from their website: "Terraform enables you to
AWS Amazon Web Services Releases Elasticsearch VPC Support AWS have recently announced VPC Support for Amazons Elasticsearch Service, which is a big win! The official announcement: https://aws.amazon.com/about-aws/whats-new/2017/10/amazon-elasticsearch-service-announces-support-for-amazon-virtual-private-cloud-vpc/ Elasticsearch Private IP Based Authorization: This
AWS Your First Python Lambda Function on AWS Using the AWS CLI Tools Your first Lambda function on AWS with Python using the AWS CLI. Today we will use the AWS CLI Tools to create a Basic Lambda Function that will use the requests library to
AWS AWS - Using Python and Boto3 to get Information about Untagged EC2 resources Just a quick post on a Python script to scan through all your EC2 Instances in the Specified Region, and if there's no Tags associated to the resource, the script will print information
BigData Amazon EMR Performance Comparison dealing with Hadoops SmallFiles Problem Today I would like to have a dive into Job Performance with Hadoop, running on the Managed Hadoop Framework of Amazon Web Services, which is Elastic MapReduce (EMR). Hadoop does not deal well
AWS AWS IAM Users and Groups: Encrypt and Decrypt Data using KMS via the CLI In our previous post we went through the process on controlling access using the CLI for IAM, to Create a IAM Policy, Associating the Policy to a Group and Creating Users within the
AWS How to Control Access with AWS IAM for Users and Groups using the CLI for Granting S3 Access In this post, we will use Identity Access Management (IAM) on AWS to Control Access for our Groups and Users using the CLI, and as an example use-case, we will be allowing S3
AWS Using the AWS CLI Tools to interact with Amazons Athena Service In this Tutorial we will use the AWS CLI tools to Interact with Amazon Athena. What is Amazon Athena: Athena is a Serverless Query Service that allows you to analyze data in Amazon
Elasticsearch AWS: Access Kibana 5 behind ELB via Nginx Reverse Proxy on Custom DNS Update: [amazon-web-services-releases-elasticsearch-vpc-support/] (https://sysadmins.co.za/amazon-web-services-releases-elasticsearch-vpc-support/) In one of my previous posts: Secure Access to Kibana on AWS Elasticsearch Service, I walked you through on how to setup Basic HTTP Authentication to
Linux Unable to Launch EMR Clusters with Cronjobs The Issue: I was trying to Schedule the Launch of EMR Clusters using Cronjobs where I have noticed that it fails via cronjobs but passes via cli. The reasoning behind this in my
AWS AWS: Create a VPC and Launch EC2 Instance using the CLI What we will be doing Today: In this tutorial we will setup a VPC on AWS with the AWS CLI tools, and also Launching an EC2 Instance to our newly created VPC. This
AWS Interfacing Amazon DynamoDB with Python using Boto3 Introduction: In this Tutorial I will show you how to use the boto3 module in Python which is used to interface with Amazon Web Services (AWS). For other blogposts that I wrote on
AWS AWS: Create EMR Cluster with Java SDK Examples Today, providing some basic examples on creating a EMR Cluster and adding steps to the cluster with the AWS Java SDK. This tutorial will show how to create an EMR Cluster in eu-west-1