API Gateway with Lambda using Python on AWS to Post info to Rocketchat

Today we will setup a API Gateway which has a Lambda Function, written in Python which we will setup using the AWS CLI

What we will be doing:

We will setup an API Endpoint that we will use to post data that will interact with the Rocketchat API.

We will create an API using Amazons API Gateway, associate a Lambda Function written in Python, that will be invoked when submitting a POST request to the mentioned API, which will then send a message to our Rocketchat Server.

The Lambda Function:

Our function lambda_function.py:

import os
import requests
import json

def print_function(word):
    return word

def lambda_handler(event, context):
    print print_function('Function Executing')
    print("Event Passed to Handler: " + json.dumps(event))

    uri = os.environ['WEBOOK_URI']
    data = {}
    data = {
        "username": event['username'],
        "icon_emoji": ":python:",
        "attachments": [
            {
                "title": event['title'],
                "title_link": event['link'],
                "text": event['text'],
                "image_url": "https://i.snag.gy/nvtegp.jpg",
                "color":"#764FA5"
            }
        ]
    }

    r = requests.post(uri, json.dumps(data))
    return r.status_code

Because we are using a dependency that needs to be installed, prior to importing it, we need to zip up the dependencies, which we will upload with the function:

Instll the requests package, and specify that we would like to install it in our working directory:

Update: No need to install the requests library as you can import it from botocore as from botocore.vendored import requests

$ pip install requests -t .

Now we will zip up the dependencies:

$ zip -r rocketchatLambda.zip *

IAM Permissions:

We need to create a Trust Relationship, in this example my trust-relationship.json will look like the following:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Use the AWS CLI to create the IAM Role, which in this case will be named rocketchat-lambda-role and we are also specifying the path to our assume role policy document:

$ aws --profile aws iam create-role \
--role-name rocketchat-lambda-role \
--assume-role-policy-document file://trust-relationship.json

{
    "Role": {
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Action": "sts:AssumeRole",
                    "Principal": {
                        "Service": "lambda.amazonaws.com"
                    },
                    "Effect": "Allow",
                    "Sid": ""
                }
            ]
        },
        "RoleId": "AROAJ6ZHX7ULIDHGBBPDU",
        "CreateDate": "2017-10-17T18:41:57.946Z",
        "RoleName": "rocketchat-lambda-role",
        "Path": "/",
        "Arn": "arn:aws:iam::942270256823:role/rocketchat-lambda-role"
    }
}

Now that our IAM Role is created, we need to define our IAM Policy that we will associate to our Role, our policy document lambda-policy.json, will look like the following:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1508266010991",
      "Action": [
        "execute-api:Invoke"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:execute-api:*:*:*"
    },
    {
      "Sid": "Stmt1508266078275",
      "Action": [
        "logs:PutLogEvents",
        "logs:CreateLogGroup",
        "logs:CreateLogStream"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}

Now that we have our policy document, lets create the policy and then specify the path of our policy document:

$ aws --profile aws iam create-policy \
--policy-name lambda-rocketchat-policy \
--policy-document file://lambda-policy.json

{
    "Policy": {
        "PolicyName": "lambda-rocketchat-policy",
        "CreateDate": "2017-10-17T18:49:20.538Z",
        "AttachmentCount": 0,
        "IsAttachable": true,
        "PolicyId": "ANPAIGQYQ62NFFZZC74II",
        "DefaultVersionId": "v1",
        "Path": "/",
        "Arn": "arn:aws:iam::942270256823:policy/lambda-rocketchat-policy",
        "UpdateDate": "2017-10-17T18:49:20.538Z"
    }
}

]

Now that our policy is created, we received a policy arn, which we will need to attach the role policy to the role name:

$ aws --profile aws iam attach-role-policy \
--policy-arn arn:aws:iam::942270256823:policy/lambda-rocketchat-policy \
--role-name rocketchat-lambda-role

Creating the Lambda Function:

We will now create the function, specify the zipped package, handler name, runtime and environment variables.

The function uses an environment variable os.environ['WEBOOK_URI'] which we need to set in Lambda's environment variable setting, so our Function will consist of:

Region Name
Function Name
Role-ARN
Function Handler
Function Runtime
Environment Variable

$ aws --profile aws lambda create-function \
--region eu-west-1 --function-name LambdaRocketChat \
--zip-file fileb://rocketchatLambda.zip \
--role arn:aws:iam::123456789012:role/rocketchat-lambda-role \
--handler lambda_function.lambda_handler \
--runtime python2.7 \
--environment Variables={WEBOOK_URI=https://rocketchat.apps.ruanbekker.com/hooks/abcdefghijklmnopq}

{
    "TracingConfig": {
        "Mode": "PassThrough"
    },
    "CodeSha256": "Pxpnu3DtNXzF2GC6GzveDgFCwoGRBv3jhG0XQczSE1I=",
    "FunctionName": "LambdaRocketChat",
    "CodeSize": 1125212,
    "MemorySize": 128,
    "FunctionArn": "arn:aws:lambda:eu-west-1:942270256823:function:LambdaRocketChat",
    "Environment": {
        "Variables": {
            "WEBOOK_URI": "https://rocketchat.apps.ruanbekker.com/hooks/abcdefghijklmnopq"
        }
    },
    "Version": "$LATEST",
    "Role": "arn:aws:iam::123456789012:role/rocketchat-lambda-role",
    "Timeout": 3,
    "LastModified": "2017-10-17T19:04:26.745+0000",
    "Handler": "lambda_function.lambda_handler",
    "Runtime": "python2.7",
    "Description": ""
}

Create the API Gateway:

I will go through the steps on creating the API, Resource, Method, Integration Type, Stage and API Keys, via the AWS Management Console, and how you would do it via the AWS CLI.

From the AWS Management Console, use with the following steps:

1. Create API
2. Create Resource  (/resource)
3. Create method (post,put)
4. Select Integration Type => Lambda, Lambda Region, Select Lambda Function.
4. Create stage
5. API Keys, Create API Key
6. Create usage plan, add api stage, select stage, tick the check mark to add
7. Add API key to usage plan.
8. done

Via the AWS CLI:

Create the API:

$ aws --profile aws apigateway create-rest-api \
--name post2 \
--region eu-west-1
{
    "name": "post2",
    "id": "4w7rcuk4xi",
    "createdDate": 1508276787
}

Get the API Resource:

$ aws --profile aws apigateway get-resources \
--rest-api-id 4w7rcuk4xi \
--region eu-west-1
{
    "items": [
        {
            "path": "/",
            "id": "xxeg2zxhie"
        }
    ]
}

Create a HTTP POST Method on the API ID:

$ aws --profile aws apigateway put-method \
--rest-api-id 4w7rcuk4xi \
--resource-id xxeg2zxhie \
--http-method POST \
--authorization-type "NONE" \
--region eu-west-1
{
    "apiKeyRequired": false,
    "httpMethod": "POST",
    "authorizationType": "NONE"
}

Lambda Integration with our API Gateway

Create a Lambda Integration Type on the POST Method of our API:

$ aws --profile aws apigateway put-integration \
--rest-api-id 4w7rcuk4xi \
--resource-id xxeg2zxhie \
--http-method POST \
--type AWS \
--integration-http-method POST \
--uri arn:aws:apigateway:eu-west-1:lambda:path/2015-03-31/functions/arn:aws:lambda:eu-west-1:942270256823:function:LambdaRocketChat/invocations \
--region eu-west-1
{
    "httpMethod": "POST",
    "passthroughBehavior": "WHEN_NO_MATCH",
    "cacheKeyParameters": [],
    "type": "AWS",
    "uri": "arn:aws:apigateway:eu-west-1:lambda:path/2015-03-31/functions/arn:aws:lambda:eu-west-1:942270256823:function:LambdaRocketChat/invocations",
    "cacheNamespace": "xxeg2zxhie"
}

Grant permissions on Lambda to associate API Gateway to invoke the Lambda Function:

$ aws --profile aws lambda add-permission \
--region eu-west-1 \
--function-name LambdaRocketChat \
--statement-id StatementId3 \
--action lambda:InvokeFunction \
--principal apigateway.amazonaws.com \
--source-arn arn:aws:execute-api:eu-west-1:942270256823:4w7rcuk4xi/*/POST/
{
    "Statement": "{\"Sid\":\"StatementId3\",\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"apigateway.amazonaws.com\"},\"Action\":\"lambda:InvokeFunction\",\"Resource\":\"arn:aws:lambda:eu-west-1:942270256823:function:LambdaRocketChat\",\"Condition\":{\"ArnLike\":{\"AWS:SourceArn\":\"arn:aws:execute-api:eu-west-1:942270256823:4w7rcuk4xi/*/POST/\"}}}"
}

Add a Method Response to our POST Method Resource:

$ aws --profile aws apigateway put-method-response \
--rest-api-id 4w7rcuk4xi \
--resource-id xxeg2zxhie \
--http-method POST \
--status-code 200 \
--response-models '{"application/json": "Empty"}' \
--region eu-west-1
{
    "responseModels": {
        "application/json": "Empty"
    },
    "statusCode": "200"
}

Setup the PUT Integration Response:

$ aws --profile aws apigateway put-integration-response \
--rest-api-id 4w7rcuk4xi \
--resource-id xxeg2zxhie \
--http-method POST \
--status-code 200  \
--selection-pattern "" \
--response-templates '{"application/json": ""}' \
--region eu-west-1
{
    "statusCode": "200",
    "selectionPattern": "",
    "responseTemplates": {
        "application/json": null
    }
}

Create the Deployment Resource, which makes a specified Rest API Callable over the Internet:

$ aws --profile aws apigateway create-deployment \
--rest-api-id 4w7rcuk4xi \
--stage-name prod \
--stage-description 'prod stage' \
--description '10th deployment' \
--region eu-west-1         {
    "description": "10th deployment",
    "id": "385p7y",
    "createdDate": 1508283002
}

Test the POST Request to our API Gateway

The Data that we will use in our payload via our POST Request:

{
  "username": "AWS",
  "title": "Whats New on AWS",
  "link": "https://aws.amazon.com/new/",
  "text": "Info posted via API Gateway and Lambda!"
}

Making the POST Request:

$ curl -XPOST https://vs26nxxpyg.execute-api.eu-west-1.amazonaws.com/test/post -d '{"username": "AWS", "title": "Whats New on AWS", "link": "https://aws.amazon.com/new/", "text": "Info posted via API Gateway and Lambda!"}'
200

Heading over to Rocketchat:

Authentication:

As at this moment, we have an Open API, anyone can post data to our API Endpoint, next we will edit our API Endpoint to require a API Key.

Create the API Key:

$ aws --profile aws apigateway create-api-key \
--name rc \
--description "rocketchat api key" \
--enabled --stage-keys restApiId=4w7rcuk4xi,stageName=prod \
--region eu-west-1
{
    "description": "rocketchat api key",
    "enabled": true,
    "value": "ZNlob9iO1y3Ug5Hx9Qji5aefJZ8jgKE13PXj2wc5",
    "stageKeys": [
        "4w7rcuk4xi/prod"
    ],
    "lastUpdatedDate": 1508283803,
    "createdDate": 1508283803,
    "id": "e3d1r5doa0",
    "name": "rc"
}

Update the POST Method to Require our API Key:

$ aws --profile aws apigateway update-method \
--rest-api-id 4w7rcuk4xi \
--resource-id xxeg2zxhie \
--http-method POST \
--patch-operations op="replace",path="/apiKeyRequired",value="true" \
--region eu-west-1
{
    "apiKeyRequired": true,
    "httpMethod": "POST",
    "methodIntegration": {
        "integrationResponses": {
            "200": {
                "responseTemplates": {
                    "application/json": null
                },
                "selectionPattern": "",
                "statusCode": "200"
            }
        },
        "passthroughBehavior": "WHEN_NO_MATCH",
        "cacheKeyParameters": [],
        "uri": "arn:aws:apigateway:eu-west-1:lambda:path/2015-03-31/functions/arn:aws:lambda:eu-west-1:942270256823:function:LambdaRocketChat/invocations",
        "httpMethod": "POST",
        "cacheNamespace": "xxeg2zxhie",
        "type": "AWS"
    },
    "methodResponses": {
        "200": {
            "responseModels": {
                "application/json": "Empty"
            },
            "statusCode": "200"
        }
    },
    "authorizationType": "NONE"
}

Create a New Usage Plan:

$ aws --profile aws apigateway create-usage-plan \
--name "New Usage Plan" \
--description "A new usage plan" \
--api-stages apiId=4w7rcuk4xi,stage=prod \
--throttle burstLimit=10,rateLimit=5 \
--quota limit=500,offset=0,period=MONTH \
--region eu-west-1
{
    "throttle": {
        "rateLimit": 5.0,
        "burstLimit": 10
    },
    "description": "A new usage plan",
    "apiStages": [
        {
            "apiId": "4w7rcuk4xi",
            "stage": "prod"
        }
    ],
    "name": "New Usage Plan",
    "quota": {
        "limit": 500,
        "period": "MONTH",
        "offset": 0
    },
    "id": "u0nx0r"
}

]

Associate the API Key to the Usage Plan:

$ aws --profile aws apigateway create-usage-plan-key \
--usage-plan-id u0nx0r \
--key-type "API_KEY" \
--key-id e3d1r5doa0 \
--region eu-west-1 
{
    "type": "API_KEY",
    "id": "e3d1r5doa0",
    "name": "rc"
}

Deploy the API to make it callable for the users:

$ aws --profile aws apigateway create-deployment \
--rest-api-id 4w7rcuk4xi \
--stage-name prod \
--stage-description 'prod stage' \
--description '15th deployment' \
--region eu-west-1
{
    "description": "15th deployment",
    "id": "njwr89",
    "createdDate": 1508284890
}

Make the POST Request and Include The API Key in the Headers:

$ curl -XPOST -H "x-api-key: ZNlob9iO1y3Ug5Hx9Qji5aefJZ8jgKE13PXj2wc5" https://4w7rcuk4xi.execute-api.eu-west-1.amazonaws.com/prod/ -d '{"username": "AWS", "title": "Whats New on AWS", "link": "https://aws.amazon.com/new/", "text": "Info posted via API Gateway and Lambda!"}'
200

Heading over to Rocketchat:

API Gateway with Lambda using Python on AWS to Post info to Rocketchat

The Lambda Function:

IAM Permissions:

Creating the Lambda Function:

Create the API Gateway:

Lambda Integration with our API Gateway

Test the POST Request to our API Gateway

Authentication:

Comments

Read Next

Amazon EMR Performance Comparison dealing with Hadoops SmallFiles Problem

Convert CSV to JSON files with AWS Lambda and S3 Events

The Lambda Function:

IAM Permissions:

Creating the Lambda Function:

Create the API Gateway:

Lambda Integration with our API Gateway

Test the POST Request to our API Gateway

Authentication:

Comments

Subscribe to Sysadmins

Read Next

Tags