How to easily set up “serverless server” for IoT devices using Amazon AWS Lambda, Python, Flask & Zappa
Motivation:
Often times Internet of Things (IoT) projects require small proof of concept (PoC) prototypes that communicate with some kind of server. Every time setting up server might seem both daunting and a lot of work.
The focus of this tutorial is to show how easy it is to set up Amazon AWS Lambda functions that are capable of communicating with IoT device via RESTful web services.
Summary:
Since Amazon AWS introduced Lambda back in 2013, “serverless cloud computing” has taken off and has become popular subject among Software Developers.
In the following tutorial we will cover on how to set up functions (Amazon AWS Lambda services) in order to quickly test IoT prototype device without setting up an entire server.
We will use Flask — micro web framework, Zappa — server-less Python framework, and you guessed it — Amazon AWS Lambda.
Why Amazon AWS Lambda?
There are several points that make Amazon AWS Lambda and serverless computing appealing:
- Out of the box horizontal scaling of the resources
- Pay only for compute
- Don’t have to bother managing the server
- Free Tier of 1M requests/month and 400,000 GB-seconds of compute time
Before we begin…
Although serverless ecosystem is growing rapidly in popularity, it doesn’t mean that they are here to replace classic models. There are pros/cons to each side, and it all depends on your use case. Also, there is an ongoing debate among practitioners, and plenty of literature. So, please read before you jump in the ‘serverless bandwagon’
Outline:
- Set up Work Environment
- Create RESTful Web Services using Flask & Zappa
- REST API Testing
- Set up IoT device
- Summary
- Part 2
Set up Work Environment
Amazon AWS Account & Credentials
Before you begin working with AWS, I recommend you read about cloud computing and AWS
Once you familiarize yourself with AWS, you can create free Amazon AWS account.
If you have some time on your hand, you can learn about overall Amazon AWS services. Cloud Practitioner is a free certification provided by AWS.
There are several online sources and tutorials on how to set up Amazon AWS account:
NOTE
In IAM, if you are setting up new user and are defining permissions for your user, make sure user has access to the following AWS services:
- AWS Lambda
- AWS IAM
- AWS API Gateway
- AWS S3
- AWS CloudFormation
Create Python Working Environment
In the terminal window create new working directory, install Python virtual environment (Official Docs), and activate it:
# Create working directory
mkdir my-first-lambda-project
cd my-first-lambda-project# Update pip and install virtualenv
python3 -m pip install --user --upgrade pip
python3 -m pip install --user virtualenv# Create virtual environment and activate it
python3 -m venv <name_of_your_virtual_environment>
source <name_of_your_virtual_environment>/bin/activate
Install AWS CLI (Official Docs) in the virtual environment
pip3 install awscli --upgradeIn terminal type the following command to configure AWS CLI credentials
aws configureYou will be prompted to enter Access Key (ID), and Region and Output format. You can find more information and examples in the official documentation.
Install Flask and Zappa in your virtual environment:
pip install flask zappaCreate RESTful Web Services using Flask & Zappa
Navigate to your project directory, and create file app.py
cd my-first-lambda-projectOpen your favorite source-code editor and insert Python code:
# app.py
from flask import Flask
app = Flask(__name__)@app.route('/device/<device_id>')
def deviceId(deviceId):
return {'deviceId': deviceId}if __name__ == '__main__':
app.run()
Upload and Run Zappa on AWS Lambda
Initiate Zappa in the working directory. Note that:
- You only need to execute this command once
- Zappa autodetects if app is using Flask or Django
cd my-first-lambda-project
zappa initDeploy Zappa to AWS Server and create API URL. Note that dev means that our source code will be pushed to production stage
zappa deploy devMoving forward if you are going to make changes to source code, you can update it, instead of deploy it
zappa update devOutput URL from either deploy or update will give us with an endpoint, which is a link similar to example provided below:
https://XXXXXX.execute-api.REGION-XX-X.amazonaws.com/devAt this point, Zappa has:
- Zipped your source code, and sent it to AWS
- Connected it with AWS API Gateway to provide above mentioned link
REST API Testing
In order to check test our API calls we can trigger the function via 2 options.
- Option 1: By putting the link in the URL of any web browser or terminal
- Option 2: Use REST client and automate the API testing using CI/CD
Option 1
In the URL section of any web browser type:
https://XXXXXX.execute-api.REGION-XX-X.amazonaws.com/dev/device/deviceId-1Alternatively, in the terminal type:
curl https://XXXXXX.execute-api.REGION-XX-X.amazonaws.com/dev/device/deviceId-1The output in the both cases should be dictionary in the format:
{
"message": "deviceId-1"
}Option 2
I decided to use Restlet Client in order to automate the API testing. It is Google Chrome web browser extension, which you can add to your Chrome.
Note: The example discussed in this tutorial is simple GET HTTP method. If you are going to use HTTP methods (i.e. POST…) with body and/or headers in your requests, I recommend you read several articles about cURL and Python requests.
Alternatively, Restlet provides user friendly UI to work with if you are configuring HTTP methods with headers and/or body.
Set up IoT device
In the previous tutorial, I wrote extensively on how to set up and configure IoT device great for prototyping — Raspberry PI.
Follow the steps to set up Raspberry PI. Moving forward you place source code either to your device or local machine, and make API requests from AWS Lambda.
Request info from AWS Lambda
At this point, we have running AWS Lambda with an open API endpoint. In the previous step we tested API calls via Restlet and cURL. Now, let’s make API calls via Python. You can place Python code on either on your local machine on your IoT device.
Note: Before you proceed further, please make sure you install virtual environment.
Install simple HTTP library for Python either on your IoT device or local PC:
pip install requestsCreate file device_info.py and replaceURL variable with your API endpoint URL.
In the code snippet below, there are 2 methods: main() and get_device()
get_device()function takes deviceId as an argument and makes request to using an API endpointmain()function has hardcodeddeviceIdvariable and invokesget_device()function
import requests
import json
from requests.exceptions import HTTPErrorURL = 'https://XXXXXX.execute-api.REGION-XX-X.amazonaws.com/dev'
API_GET_DEVICE = 'device'
def get_device(deviceId):
"""Request device information given deviceId
Arguments:
deviceId {string} -- requested information
Returns:
responseJson [json] -- API request reponse in the JSON format
""" try:
url = '{}/{}/{}'.format(URL, API_GET_DEVICE, deviceId)
response = requests.get(url) responseJson = json.dumps(response.json()) response.raise_for_status() except HTTPError as http_err:
print('HTTP error occurred: {}'.format(http_err))
except Exception as err:
print('Other error occurred: {}'.format(err))
else:
return responseJsondef main(): deviceId = 'deviceId-1'
responseJson = get_device(deviceId)
print(responseJson)if __name__ == "__main__":
main()
In your terminal run the code: python device_info.py
The example above is simple HTTP request in Python with hardcoded value and non-complex structure. As you get more familiar with Python HTTP Requests, and AWS Lambda, you can add more features and create better structured functions where you should
- Catch errors
- Output in the logger
- Organize files
- Write comments and document everything
Summary
In this tutorial we wrote small RESTful web service using Flask and uploaded it to AWS Lambda via Zappa. In turn, Zappa used AWS API Gateway to create a publicly accessible API endpoint. Then, we wrote HTTP request Python code that can pull information using an API endpoint. You can place HTTP request files on any device: IoT (Raspberry PI), web browser, and/or server that is connected to the internet.
The summary of the process is illustrated below
Part 2
Stay tuned for Part 2 where I will show how to upload image to AWS S3 bucket using AWS Lambda as triggers. Also, tutorial will include bigger Flask example with all the HTTP methods (GET, POST, PUT, DELETE), and GitHub source code repo.
For now, you can check out GitHub with code.
