Part 4: Add S3 event source¶
So far, we have been manually invoking the Lambda function ourselves in order to detect objects in the image and add the information to our database. However, we can automate this workflow using Lambda event sources so that the Lambda function is invoked every time an object is uploaded to the S3 bucket.
For this section, we will be doing the following:
Add Lambda event source for S3 object creation event¶
Change the Lambda function to be invoked whenever an object is uploaded to a S3 bucket via the on_s3_event decorator.
Instructions¶
In the
app.py
file, change thedetect_labels_on_image
signature to be namedhandle_object_created
that accepts a singleevent
parameter:def handle_object_created(event):
Update the decorator on
handle_object_created
to use theapp.on_s3_event
decorator instead and have the Lambda function be triggered whenever an object is created in the bucket specified by the environment variableMEDIA_BUCKET_NAME
:@app.on_s3_event(bucket=os.environ['MEDIA_BUCKET_NAME'], events=['s3:ObjectCreated:*']) def handle_object_created(event):
Add the tuple
_SUPPORTED_IMAGE_EXTENSTIONS
representing a list of supported image extensions:_SUPPORTED_IMAGE_EXTENSIONS = ( '.jpg', '.png', )
Update the
handle_object_created
function to use the newevent
argument of type S3Event and only do object detection and database additions on specific image file extensions:@app.on_s3_event(bucket=os.environ['MEDIA_BUCKET_NAME'], events=['s3:ObjectCreated:*']) def handle_object_created(event): if _is_image(event.key): _handle_created_image(bucket=event.bucket, key=event.key) def _is_image(key): return key.endswith(_SUPPORTED_IMAGE_EXTENSIONS) def _handle_created_image(bucket, key): labels = get_rekognition_client().get_image_labels(bucket=bucket, key=key) get_media_db().add_media_file(key, media_type=db.IMAGE_TYPE, labels=labels)
Validation¶
Ensure the contents of the
app.py
file is:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 | import os
import boto3
from chalice import Chalice
from chalicelib import db
from chalicelib import rekognition
app = Chalice(app_name='media-query')
_MEDIA_DB = None
_REKOGNITION_CLIENT = None
_SUPPORTED_IMAGE_EXTENSIONS = (
'.jpg',
'.png',
)
def get_media_db():
global _MEDIA_DB
if _MEDIA_DB is None:
_MEDIA_DB = db.DynamoMediaDB(
boto3.resource('dynamodb').Table(
os.environ['MEDIA_TABLE_NAME']))
return _MEDIA_DB
def get_rekognition_client():
global _REKOGNITION_CLIENT
if _REKOGNITION_CLIENT is None:
_REKOGNITION_CLIENT = rekognition.RekognitonClient(
boto3.client('rekognition'))
return _REKOGNITION_CLIENT
@app.on_s3_event(bucket=os.environ['MEDIA_BUCKET_NAME'],
events=['s3:ObjectCreated:*'])
def handle_object_created(event):
if _is_image(event.key):
_handle_created_image(bucket=event.bucket, key=event.key)
def _is_image(key):
return key.endswith(_SUPPORTED_IMAGE_EXTENSIONS)
def _handle_created_image(bucket, key):
labels = get_rekognition_client().get_image_labels(bucket=bucket, key=key)
get_media_db().add_media_file(key, media_type=db.IMAGE_TYPE, labels=labels)
|
Redeploy the Chalice application¶
Deploy the updated Chalice application.
Instructions¶
Run
chalice deploy
:$ chalice deploy Creating deployment package. Creating IAM role: media-query-dev-handle_object_created Creating lambda function: media-query-dev-handle_object_created Configuring S3 events in bucket media-query-mediabucket-fb8oddjbslv1 to function media-query-dev-handle_object_created Deleting function: arn:aws:lambda:us-west-2:123456789123:function:media-query-dev-detect_labels_on_image Deleting IAM role: media-query-dev-detect_labels_on_image Resources deployed: - Lambda ARN: arn:aws:lambda:us-west-2:123456789123:function:media-query-dev-handle_object_created
Validation¶
Upload the
othersample.jpg
image to the S3 bucket:$ aws s3 cp ../chalice-workshop/code/media-query/final/assets/othersample.jpg s3://$MEDIA_BUCKET_NAME
Use the
get-item
CLI command to ensure theothersample.jpg
data was automatically populated in the DynamoDB table:$ aws dynamodb get-item --table-name $MEDIA_TABLE_NAME \ --key '{"name": {"S": "othersample.jpg"}}' { "Item": { "name": { "S": "othersample.jpg" }, "labels": { "L": [ { "S": "Human" }, { "S": "People" }, { "S": "Person" }, { "S": "Phone Booth" }, { "S": "Bus" }, { "S": "Transportation" }, { "S": "Vehicle" }, { "S": "Man" }, { "S": "Face" }, { "S": "Leisure Activities" }, { "S": "Tourist" }, { "S": "Portrait" }, { "S": "Crowd" } ] }, "type": { "S": "image" } } }
If the item does not appear, try running the
get-item
command after waiting for ten seconds. Sometimes, it takes a little bit of time for the Lambda function to get triggered.