Problem-solving with lambda
A Pragmatic sysadmin’s perspective
My background is not predominantly programming I’ve spent most of my time as a system/infra/cloud/storage/networking engineer. Like most in my profession can tell you, we have 2 main tasks; build it & keep it online.
When you spend any time at a helpdesk type job and you’ll be well versed in solving a wide range of problems that customers face, that’s actually how this blog started. This makes you a creative and fast problem solver and if you’re lucky you’ll get enough edge cases you’ve never seen before to add to your arsenal of knowledge and keep yourself from going insane.
The takeaway from all of this is that you build muscle memory in fixing these problems.
Here’s my fix anything workflow:
- Start with the point that was noticed by the client or monitoring
- Figure out what delivered that thing that doesn’t work
- Follow the break as deep as you need to go
- Start fixing from there and follow all the way up
- Repeat until you can verify the noticed thing to be working again.
Recently I realized that if you remove the deep dive into the root cause of the problem and replace it with a need for a solution and drill down to the core functionality you end up at the same place in the middle. From this point you then fix or build your way back up to the user-visible layers, this strategy is how I now build most of my lambda solutions.
What is lambda?
AWS Lambda is an event-driven, serverless computing platform that aims to make it possible to deploy code in a smaller form factor and makes it possible to ship only a single function. The Lambda platform handles the interaction with the events and runs your code only when necessary, this makes it super simple to integrate with other components and it is only billed for the time your code is running. When your code is triggered it initializes a small container that runs your code, the items you have in memory that you initialize will stay available for a few minutes (as long as the container stays online) this enables you to make subsequent invocations faster. Lambda supports multiple languages I’ll be using python in this guide, for a full list of supported runtimes check out the lambda documentation.
Writing python lambda solutions
So let’s get to building some python code.
First up create a fresh virtualenv, you might want to use a Python virtualenv manager if you’re lazy like me use this: zsh-autoswitch-virtualenv This gives you a separate space for python packages only for this project.
1. virtualenv setup
# svanbroekhoven @ lynx in ~/git/tst [0:54:57]
$ mkvenv
Creating tst virtualenv
created virtual environment CPython3.8.3.final.0-64 in 147ms
creator CPython3Posix(dest=/home/svanbroekhoven/.virtualenvs/tst, clear=False, global=False)
seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/home/svanbroekhoven/.local/share/virtualenv/seed-app-data/v1.0.1)
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
Switching virtualenv: tst [🐍Python 3.8.3]
(tst)
# svanbroekhoven @ lynx in ~/git/tst [0:55:03]
$ cd ..
Deactivating: tst
# svanbroekhoven @ lynx in ~/git [1:53:20]
$ cd tst
Switching virtualenv: tst [🐍Python 3.8.3]
(tst)
# svanbroekhoven @ lynx in ~/git/tst [1:53:27]
$
As you can see in the example above, a virtualenv switcher helps if you’re working on multiple projects.
2. try to solve it once
Core problem: I need the nth Fibonacci number on many places where I don’t have much compute power or local storage to store enough of them.
Let’s open up a python shell or jupyter notebook and a web browser to try things and research possible solutions, in this example I’ll be using the example of a fibonacci number returning API. As a first project I would try to avoid complex dependencies outside of pure python, as a rough guide this means no Audio, Video and image-based projects. These libraries often require system-level dependencies which we can only install through building a lambda layer and that is a bit too involved for this guide. If you do want to dive into building layers check out this article in the lambda documentation.
The most basic Fibonacci function you can write looks like this, it has many problems but that leaves us some room for improvement in the future.
def fibonacci(n):
if n == 0:
return 0
elif n == 1:
return 1
else:
return fibonacci(n-1) + fibonacci(n-2)
for n in range(10 + 1):
print(str(n)+ "\t" + str(fibonacci(n)))
3. Build and tweak your main.py
After some tinkering our basic script and function it looks like the one above, now my favorite way to proceed from here in put it in a main.py script and run it interactive like this: python -i main.py
# svanbroekhoven @ lynx in ~ [14:28:01] C:127
$ cd git/proza/content/post/problem-solving-with-lambda
Switching virtualenv: problem-solving-with-lambda [Python 3.8.3]
(problem-solving-with-lambda)
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [14:28:16]
$ cat main.py
def fibonacci(n):
if n == 0:
return 0
elif n == 1:
return 1
else:
return fibonacci(n-1) + fibonacci(n-2)
for n in range(10 + 1):
print(str(n)+ "\t" + str(fibonacci(n)))
(problem-solving-with-lambda)
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [14:28:59]
$ python -i main.py
0 0
1 1
2 1
3 2
4 3
5 5
6 8
7 13
8 21
9 34
10 55
>>> dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'fibonacci', 'n']
>>> n
10
>>> fibonacci(n)
55
>>> fibonacci(20)
6765
>>>
This way we can play with what we created without the need to paste our code in the interactive shell, this way we can hack on it and incorporate our first function in some other loops to validate if it’ll fit our requirements. Below I demonstrate my experiments in the interactive shell to try and show how I use this process.
# svanbroekhoven @ lynx in ~ [14:28:01] C:127
$ cd git/proza/content/post/problem-solving-with-lambda
Switching virtualenv: problem-solving-with-lambda [Python 3.8.3]
(problem-solving-with-lambda)
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [14:28:16]
$ cat main.py
def fibonacci(n):
if n == 0:
return 0
elif n == 1:
return 1
else:
return fibonacci(n-1) + fibonacci(n-2)
for n in range(10 + 1):
print(str(n)+ "\t" + str(fibonacci(n)))
(problem-solving-with-lambda)
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [14:28:59]
$ python -i main.py
0 0
1 1
2 1
3 2
4 3
5 5
6 8
7 13
8 21
9 34
10 55
>>> dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'fibonacci', 'n']
>>> n
10
>>> fibonacci(n)
55
That was the fix once side over now let’s do it in a loop and display the input and output to the user.
You can read all the KeyboardInterrupt as the operation took too long, I killed it to keep working instead of waiting on it.
>>> for n in range(10, 200):
... fibonacci(n)
...
55
89
144
...
...
...
5702887
9227465
^C
KeyboardInterrupt
>>> for n in range(10, 200):
... print( str(n) + ": " + str(fibonacci(n)))
...
10: 55
11: 89
...
...
...
36: 14930352
37: 24157817
^C
KeyboardInterrupt
>>> import random
>>> random.randrange(10)
6
>>>
>>>
>>> for i in range(10):
... n = random.randrange(20)
... print( str(n) + ": " + str(fibonacci(n)))
...
3: 2
5: 5
5: 5
12: 144
15: 610
1: 1
1: 1
13: 233
4: 3
11: 89
>>> for i in range(10):
... n = random.randrange(200)
... print( str(n) + ": " + str(fibonacci(n)))
...
^C
KeyboardInterrupt
>>> from datetime import datetime, timedelta
>>> now = datetime.now()
>>> start = datetime.now()
>>> delta = datetime.now() - start
>>> delta
datetime.timedelta(seconds=37, microseconds=994986)
>>> print(delta)
0:00:37.994986
>>> for i in range(10):
... start = datetime.now()
... n = random.randrange(50)
... print( str(n) + ": " + str(fibonacci(n)) + " " + str(datetime.now() - start))
...
41: 165580141 0:00:43.840478
3: 2 0:00:00.000009
9: 34 0:00:00.000011
^C
KeyboardInterrupt
>>>
What I’ve accomplished this far is a simple loop that prints out 10 random fib numbers in a range from 0 to 50 and print time taken to complete it, but the results are really slow so I needed a built-in stopwatch to do some crude benchmarking of the function. I now start with placing some of my experiments back into the main.py with vim and make sure this “benchmark” auto starts as I run the file. I’ll add a lru_cache to the function to cache results to hide the slowness and make the output a bit more readable.
(problem-solving-with-lambda)
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [15:06:00]
$ vim main.py
(problem-solving-with-lambda)
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [15:06:06]
$ python -i main.py
38: 39088169 0:00:00.000024
14: 377 0:00:00.000002
10: 55 0:00:00.000002
26: 121393 0:00:00.000001
17: 1597 0:00:00.000002
29: 514229 0:00:00.000001
30: 832040 0:00:00.000001
46: 1836311903 0:00:00.000005
4: 3 0:00:00.000001
28: 317811 0:00:00.000001
>>>
(problem-solving-with-lambda)
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [15:06:23]
$ vim main.py
(problem-solving-with-lambda)
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [15:07:45]
$ python -i main.py
11 0:00:00.000005 89
4 0:00:00.000002 3
17 0:00:00.000001 1597
30 0:00:00.000001 832040
47 0:00:00.000001 2971215073
38 0:00:00.000001 39088169
35 0:00:00.000002 9227465
44 0:00:00.000001 701408733
2 0:00:00.000001 1
1 0:00:00.000001 1
>>>
(problem-solving-with-lambda)
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [15:08:05]
$ vim main.py
(problem-solving-with-lambda)
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [15:08:17]
$ python -i main.py
67 0:00:00.000007 44945570212853
20 0:00:00.000004 6765
72 0:00:00.000002 498454011879264
79 0:00:00.000001 14472334024676221
27 0:00:00.000002 196418
68 0:00:00.000002 72723460248141
33 0:00:00.000002 3524578
10 0:00:00.000002 55
17 0:00:00.000002 1597
90 0:00:00.000001 2880067194370816120
>>>
Well this looks kinda oke, let’s get rid of the auto run benchmark crap and add a function that triggers the benchmark with a range variable.
(problem-solving-with-lambda)
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [15:08:23]
$ vim main.py
(problem-solving-with-lambda)
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [15:09:33]
$ python -i main.py
>>> dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'datetime', 'fibonacci', 'lru_cache', 'random', 'range_test']
>>> range_test(100)
61 0:00:00.000066 2504730781961
50 0:00:00.000017 12586269025
61 0:00:00.000010 2504730781961
93 0:00:00.000010 12200160415121876738
2 0:00:00.000015 1
17 0:00:00.000011 1597
82 0:00:00.000011 61305790721611591
15 0:00:00.000011 610
65 0:00:00.000011 17167680177565
61 0:00:00.000006 2504730781961
>>> range_test(500)
262 0:00:00.000029 2542592393026885507715496646813780220945054040571721231
468 0:00:00.000016 28624020537229717283244863695841661789309459371299941322398704536965075971940465647510004531000816
424 0:00:00.000011 18250487254938611555074410636524312658020849229014745928158881386149462839394269270468043
237 0:00:00.000006 15156039800290547036315704478931467953361427680642
496 0:00:00.000006 20341574322680408081083829243820203612317308197211964554628215486203974898255803242740333222721700974747
402 0:00:00.000014 460835978753503578226215883073872246385764472086797082873203188542544616448248343576
286 0:00:00.000006 263621064469290555679241849789653324393054271110084140201023
387 0:00:00.000006 337856107814181089864066841370437948648180483483258553487263501935155470092051458
24 0:00:00.000005 46368
371 0:00:00.000005 153083904475345790698149223310665389766178449653686710164582374234640876900329
>>>
(problem-solving-with-lambda)
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [15:10:09]
$ cat main.py
from functools import lru_cache
from datetime import datetime
import random
@lru_cache(maxsize = None)
def fibonacci(n):
if n == 0:
return 0
elif n == 1:
return 1
else:
return fibonacci(n-1) + fibonacci(n-2)
def range_test(nMax):
for i in range(10):
start = datetime.now()
n = random.randrange(nMax)
print( str(n) + "\t" + str(datetime.now() - start) + " " + str(fibonacci(n)))
I discovered that it was way to slow and by adding a
lru_cache to the function it became a bit more responsive and usable for our purpose.
Also I’ve changed the old range test a new variable nMax version with random numbers and a time spent counter.
Now for a final cherry on top let’s add a
main, if you’re familiar with other programming languages this is probably what you’re used to but in python it is not strictly necessary.
if __name__ == "__main__":
# execute only if run as a script
main()
The code above will only be triggered if the script is executed directly and not when the file is imported. Let’s adapt it to our situation and run a simple benchmark when started directly as a script. The main snippet is always placed at the bottom of our script as all the functions need to placed in memory before we can call them in our main.
from functools import lru_cache
from datetime import datetime
import random
@lru_cache(maxsize = None)
def fibonacci(n):
if n == 0:
return 0
elif n == 1:
return 1
else:
return fibonacci(n-1) + fibonacci(n-2)
def range_test(nMax):
for i in range(10):
start = datetime.now()
n = random.randrange(nMax)
print( str(n) + "\t" + str(datetime.now() - start) + " " + str(fibonacci(n)))
if __name__ == "__main__":
# execute only if run as a script
range_test(100)
This wil give us the following result:
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [10:35:55]
$ python main.py
96 0:00:00.000005 51680708854858323072
58 0:00:00.000003 591286729879
53 0:00:00.000001 53316291173
0 0:00:00.000001 0
12 0:00:00.000001 144
45 0:00:00.000001 1134903170
32 0:00:00.000001 2178309
73 0:00:00.000001 806515533049393
97 0:00:00.000001 83621143489848422977
44 0:00:00.000001 701408733
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [10:35:59]
$
Now this isn’t very useful, let’s add some cli params to make it calculate a Fibonacci number.
if __name__ == "__main__":
# User input handling
import argparse
parser = argparse.ArgumentParser(description="Returns the n'th fibonacci number.")
parser.add_argument('-n', required=True, type=int, help="which number in the fibonacci sequence you want")
args = parser.parse_args()
# execute calculation
print(fibonacci(args.n))
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [10:44:07]
$ python main.py
usage: main.py [-h] -n N
main.py: error: the following arguments are required: -n
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [10:44:34] C:2
$ python main.py -n 20
6765
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [10:47:35]
$ python main.py -n 200
280571172992510140037611932413038677189525
That’s yielding us a lot more value and a pleasant help text if we forget to pass a -n #num, now let’s add our calculation part of the main block to a lambda handler function to give lambda a way to interact with it and add a
api gateway test event to test the
lambda_handler function.
I won’t go into details about this if you want to read more just follow the links below, for now we only care about the following location in the event: queryStringParameters
All data is passed through in JSON so we will need to import json at the top of our file.
I use the API Gateway implementation here, but there are many ways to trigger and each has it’s own event body please check the documentation to find out how to use AWS Lambda with other services.
def lambda_handler(event, context):
#print("Received event: " + json.dumps(event, indent=2))
return fibonacci(int(event['queryStringParameters']['n']))
# This represents an api gateway request to: https://sub.domain.tld/api/v1/fibonacci?n=100
test_event = json.loads("""{
"path": "/api/v1/fibonacci",
"httpMethod": "GET",
"queryStringParameters": {
"n": "100"
},
"pathParameters": {
"proxy": "/fibonacci"
},
"requestContext": {
"identity": {
"sourceIp": "127.0.0.1",
"userAgent": "Custom User Agent String"
},
"path": "/prod/fibonacci",
"protocol": "HTTP/1.1"
}
}""")
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [11:24:40]
$ python -i main.py -n 100
354224848179261915075
>>> lambda_handler(test_event, '')
354224848179261915075
>>> fibonacci(100)
354224848179261915075
>>>
# svanbroekhoven @ lynx in ~/git/proza/content/post/problem-solving-with-lambda on git:master x [11:25:21]
$
Now as a final step to complete our local lambda test we run our lambda handler from our main block with the test event, I also place the test event into the main block to reduce memory usage in lambda.
from functools import lru_cache
from datetime import datetime
import random
import json
@lru_cache(maxsize = None)
def fibonacci(n):
if n == 0:
return 0
elif n == 1:
return 1
else:
return fibonacci(n-1) + fibonacci(n-2)
def range_test(nMax):
for i in range(10):
start = datetime.now()
n = random.randrange(nMax)
print( str(n) + "\t" + str(datetime.now() - start) + " " + str(fibonacci(n)))
def lambda_handler(event, context):
#print("Received event: " + json.dumps(event, indent=2))
return fibonacci(int(event['queryStringParameters']['n']))
if __name__ == "__main__":
# User input handling
import argparse
parser = argparse.ArgumentParser(description="Returns the n'th fibonacci number.")
parser.add_argument('-n', required=True, type=int, help="which number in the fibonacci sequence you want")
args = parser.parse_args()
# execute calculation directly
# print(fibonacci(args.n))
# This represents an api gateway request to: https://sub.domain.tld/api/v1/fibonacci?n=100
test_event = json.loads("""{
"path": "/api/v1/fibonacci",
"httpMethod": "GET",
"queryStringParameters": {
"n": "100"
},
"pathParameters": {
"proxy": "/fibonacci"
},
"requestContext": {
"identity": {
"sourceIp": "127.0.0.1",
"userAgent": "Custom User Agent String"
},
"path": "/prod/fibonacci",
"protocol": "HTTP/1.1"
}
}""")
# insert aur desired n in the test_event
test_event['queryStringParameters']['n'] = args.n
# execute calculation with test event through lambda_handler function
print(lambda_handler(test_event, ''))
Packaging python lambda functions
To prepare our function to be able to run in Lambda we need to zip it together with it’s dependencies this sounds pretty easy, and guess what… it is! I use GitLab for remote git storage and their great CI/CD pipelines, my examples will be based around GitLab but you can use whatever you’re comfortable with.
There are several ways to deploy your function:
1. AWS Web Console
This is only usable if you have no external dependencies and small scripts, it is user friendly to setup but updating code will become a fragile human-driven process.
2. AWS Lambda CLI
This is often used in examples from AWS themselves, it is a lot better than copying and pasting in the web console but still not what I’d recommend for production use. The syntax is quite clear and the reference helps you to do everything you can think of with the lambda service.
# AWS Lambda CLI update code example
aws lambda update-function-code \
--function-name my-function \
--zip-file fileb://my-function.zip
3. AWS Cloudformation
This is my favorite way of deploying on AWS, it gives you full control over the recourses and makes expanding with other services like DynamoDB at a later point in time much easier since you’ve already build your infra as code pipeline. This is also the way we’ll be deploying the code in this tutorial.
4. AWS Serverless Application Model
From the brochure: Build serverless applications in simple and clean syntax
To me this wasn’t a super intuitive method and more confusing because it isn’t just plain vanilla cloudformation and multi-account deployment isn’t supported out of the box. I did make a multi-account hack on top of sam but I wouldn’t recommend it. If you’re still curious here is the AWS SAM Overview page and a blank python sam project
CI/CD Packaging and deployment with Gitlab and Cloudformation
Before we take it apart step by step, this is what our Gitlab pipeline and cfn.yml (cloudformation) will look like.
default:
image: python
stages:
- deployment
deployment:
only:
refs:
- master
stage: deployment
artifacts:
paths:
- fibonacciPackage.zip
expire_in: 1 week
script:
- apt update -qqq && apt install -qqq -y zip > /dev/null # pull in zip as it is not in the python default container
- pip install --quiet awscli # need this for the upload but not in the package
- pip install --quiet -r requirements.txt -t ./ # silently install requirements in the current dir
- chmod -R 755 .
- zip -r fibonacciPackage.zip . -x ".git*" --quiet
- export zipname="fibonacciPackage-$(date +%s).zip"
- aws s3 cp fibonacciPackage.zip s3://bucket/lambda-packages/fibonacci/${zipname}
- aws cloudformation deploy --template-file cfn.yml --capabilities CAPABILITY_IAM --stack-name fibonacci --no-fail-on-empty-changeset --parameter-overrides s3Bucket="bucketname" s3Key="lambda-packages/fibonacci/${zipname}"
AWSTemplateFormatVersion: '2010-09-09'
Description: fibonacci service
Parameters:
s3Bucket:
Type: String
Default: "bucket"
s3Key:
Type: String
Default: "lambda-packages/fibonacci/fibonacciPackage.zip"
Resources:
LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: root
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:*
Resource: arn:aws:logs:*:*:*
FibonacciFunction:
Type: AWS::Lambda::Function
Properties:
Handler: main.lambda_handler
Role: !GetAtt LambdaExecutionRole.Arn
Code:
S3Bucket: !Ref s3Bucket
S3Key: !Ref s3Key
Runtime: 'python3.8'
Timeout: 20
MemorySize: 128
Steps required for python lambda packaging
Basically there are 2 steps:
- install your requirements locally
- zip it
install requirements in the local directory
pip makes this really easy, you can install multiple packages with a requirements.txt file or a single one directly.
Without requirements.txt
pip install boto3 -t ./
pip install requests -t ./
pip install git+https://github.com/psf/requests.git -t ./
With requirements.txt
pip install -r requirements.txt -t ./
If you need python packages in your pipeline but don’t want to include them in you lambda package just use pip without the -t parameter. Also If you don’t like the amount of logs pip creates add a --quiet parameter to silence the noisy output.
pip install --quiet awscli
zip it
Now zip your project, change the permissions to prevent issues surrounding that and exclude things you won’t need in runtime.
chmod -R 755 . # makes sure there are no permission errors
zip -r lambdaPackage.zip . -x ".git*" --quiet
Upload to S3
Deployment to lambda is easiest if you can point to your deployment zip in S3, to do this you could take a couple of naming roads I’ve used all of them, and technically there isn’t a clear best way to do it in my opinion. You shouldn’t overwrite your old zip file when releasing a new version, this makes rollbacks only possible if you also rebuild your old version and that’s not the desired behavior.
Here are a few of my favorite zip identifiers:
- git short hash
git rev-parse --short HEAD - epoch date
date +%s - random string
openssl rand -hex 12 - sem ver of the app
git tag --points-at HEAD(untested) - build number > check your pipeline manual
For now, I’ll use epoch timestamped zip files, but you could replace the date +%s with whatever command you prefer to tag your lambda packages.
zip -r lambdaPackage.zip . -x ".git*" --quiet
export zipname="lambdaPackage-$(date +%s).zip"
aws s3 cp lambdaPackage.zip s3://bucket/lambda-packages/projectName/${zipname}
After those commands, you can use the env var zipname in the following deployment to point to your new zip file.
Deploy with cloudformation
A deep dive into cloudformation is beyond the scope here so I’ll just give you my default basic deploy command:
aws cloudformation deploy --template-file cfn.yml --capabilities CAPABILITY_IAM --stack-name fibonacci --no-fail-on-empty-changeset --parameter-overrides s3Bucket="bucketname" s3Key="lambda-packages/projectName/${zipname}"
Now you should be able to see your lambda in your AWS account, if not there are a couple of things that could have gone wrong. No panic at the bottom of this article I’ve listed some common problems you can have with this kind of setup and how to debug and fix them.
Testing, improving, and finalizing your lambda in aws
Now that your Lambda is ready to be used, have a go with the build-in testing in the web interface.
On the Lambda function page at the to you’ll find a bar with a test button next to it from the dropdown click Configure test event, here you can input the json from your script or try out different examples. This test event in the web interface is my preferred way of trying new things in lambda because I have all the logs and metrics right in front of me.
If you’re looking for a more hands-off automated test system approach have a look at the
aws lambda invoke cli reference.
From here, just repeat the following: tweak, commit, push, test, and once your results are to your likings connect it to your preferred trigger and start testing the whole stack. Keep in mind that triggers will need iam permissions to trigger your lambda.
Help it’s not working for me
Here’s a checklist to fix most of the common issues with automated deployments like this one.
aws command not found
pip install --quiet awscli
Are there AWS credentials attached to the pipeline?
aws sts get-caller-identity
Deployment / upload fails
Do your iam credentials connected to the pipeline allow you to use: cloudformation, lambda, logs, and S3? Did you select the correct bucket name and is that bucket already created, if not create it before you deploy.
OSError: no library called … was found
[ERROR] OSError: no library called "cairo" was found no library called "libcairo-2" was found cannot load library 'libcairo.so': libexpat.so.1: cannot open shared object file: No such file or directory cannot load library 'libcairo.2.dylib': libcairo.2.dylib: cannot open shared object file: No such file or directory....
You’re trying to call an os lib that is not installed, you can fix this by building a lambda layer that provides the necessary lib files. A lambda layer is another zip file that provides some files to the container runtime a good place to start when building a lambda layer is the AWS Lambda layers page in the lambda configuration documentation. If you want to have a look at the solution for the layer solution to the above problem, check out cloud-print-utils.
Fibonacci example project on gitlab
){kind=avatar}