AWS Lambda Workshop

Run a Python script that processes and exports data from external sources to a Google Sheet on AWS Lambda, deployed via AWS ECR.

Workshop prerequisites

• Python installed on local machine
• Poetry installed on local machine (e.g. via pip)
• docker desktop installed on local machine
• Google Cloud access to create service account
• AWS command line with access to to ECR and Lambda

Workshop Steps

1. Initialize new package

mkdir lambda-workshop
cd lambda-workshop
poetry init
git init

2. Create/copy lambda_function.py, modules/sheet.py, modules/finance.py and .gitginore

3. Create Google Cloud project and service account, generate and download key as JSON file, add file to root folder as google_account.json

4. Create Google Sheet and add edit permission for service account

5. Update the SYMBOL and SHEET_ID environment variables

6. Add dependencies via Poetry

poetry add yahooquery pandas google-api-python-client

7. Test script locally

poetry run python -c "import lambda_function as lam; lam.lambda_handler()"

8. Create/copy Dockerfile

9. Create AWS ECR repository

10. Create AWS Lambda function with "lambda-test" execution role and 5 minutes timeout

11. Follow deployment steps below

App Readme

Setup

Create virtual environment and install dependencies with Poetry

poetry install

Run locally

poetry run python -c "import lambda_function as lam; lam.lambda_handler()"

Deployment (Docker)

1. Authenticate with ECR through AWS CLI

aws ecr get-login-password --region eu-west-1 | docker login --username AWS --password-stdin 880172922216.dkr.ecr.eu-west-1.amazonaws.com

2. Build docker image

docker build -t lambda-workshop . --platform linux/amd64

3. Tag image

docker tag lambda-workshop:latest 880172922216.dkr.ecr.eu-west-1.amazonaws.com/lambda-workshop:latest

4. Push image

docker push 880172922216.dkr.ecr.eu-west-1.amazonaws.com/lambda-workshop:latest

5. Update lambda function

aws lambda update-function-code --region eu-west-1 --function-name lambda-workshop --image-uri 880172922216.dkr.ecr.eu-west-1.amazonaws.com/lambda-workshop:latest

6. Set lambda environment variables (from locally sourced environment)

aws lambda update-function-configuration --region eu-west-1 --function-name lambda-workshop --environment "Variables={SYMBOL=$SYMBOL,SHEET_ID=$SHEET_ID}"

7. (Optional) Schedule cron job to trigger function execution