Part 3: Integrate with a DynamoDB table

Now that we have a Lambda function that can detect labels in an image, let’s integrate a DynamoDB table so we can query information across the various images stored in our bucket. So instead of returning the labels, the Chalice application will store the items in a DynamoDB table.

For this section, we will be doing the following to integrate the DynamoDB table:

Copy over boilerplate files

Copy over files needed for integrating the DynamoDB table into the application

Instructions

1. Using media-query as the current working directory, copy the db.py module into the chalicelib package:

   $ cp ../chalice-workshop/code/media-query/03-add-db/chalicelib/db.py chalicelib/
   

2. Using media-query as the current working directory, copy over an updated version of the resources.json file:

   $ cp ../chalice-workshop/code/media-query/03-add-db/resources.json .
   

Verification

1. Ensure the structure of the media-query directory includes the following files and directories:

   $ tree -a .
   ├── .chalice
   │   ├── config.json
   │   └── policy-dev.json
   ├── .gitignore
   ├── app.py
   ├── chalicelib
   │   ├── __init__.py
   │   ├── db.py
   │   └── rekognition.py
   ├── recordresources.py
   ├── requirements.txt
   └── resources.json
   

   Note there will be more files listed with tree assuming you already deployed the application once. However, the files listed from the tree output above are required.

2. Ensure the contents of the resources.json is now the following:

   $ cat resources.json
   {
     "Outputs": {
       "MediaBucketName": {
         "Value": {
           "Ref": "MediaBucket"
         }
       },
       "MediaTableName": {
         "Value": {
           "Ref": "MediaTable"
         }
       }
     },
     "Resources": {
       "MediaBucket": {
         "Type": "AWS::S3::Bucket"
       },
       "MediaTable": {
         "Properties": {
           "AttributeDefinitions": [
             {
               "AttributeName": "name",
               "AttributeType": "S"
             }
           ],
           "KeySchema": [
             {
               "AttributeName": "name",
               "KeyType": "HASH"
             }
           ],
           "ProvisionedThroughput": {
             "ReadCapacityUnits": 5,
             "WriteCapacityUnits": 5
           }
         },
         "Type": "AWS::DynamoDB::Table"
       }
     }
   }
   

Create a DynamoDB table

Create a DynamoDB table to store and query information about images in the S3 bucket.

Instructions

1. Use the AWS CLI and the resources.json CloudFormation template to redeploy the media-query CloudFormation stack and create a new DynamoDB

   $ aws cloudformation deploy --template-file resources.json --stack-name media-query
   

Verification

1. Retrieve and store the name of the DynamoDB table using the AWS CLI:

   $ MEDIA_TABLE_NAME=$(aws cloudformation describe-stacks --stack-name media-query --query "Stacks[0].Outputs[?OutputKey=='MediaTableName'].OutputValue" --output text)
   

2. Ensure the existence of the table using the describe-table CLI command:

   $ aws dynamodb describe-table --table-name $MEDIA_TABLE_NAME
   {
       "Table": {
           "AttributeDefinitions": [
               {
                   "AttributeName": "name",
                   "AttributeType": "S"
               }
           ],
           "TableName": "media-query-MediaTable-10QEPR0O8DOT4",
           "KeySchema": [
               {
                   "AttributeName": "name",
                   "KeyType": "HASH"
               }
           ],
           "TableStatus": "ACTIVE",
           "CreationDateTime": 1531769158.804,
           "ProvisionedThroughput": {
               "NumberOfDecreasesToday": 0,
               "ReadCapacityUnits": 5,
               "WriteCapacityUnits": 5
           },
           "TableSizeBytes": 0,
           "ItemCount": 0,
           "TableArn": "arn:aws:dynamodb:us-west-2:123456789123:table/media-query-MediaTable-10QEPR0O8DOT4",
           "TableId": "00eebe92-d59d-40a2-b5fa-32e16b571cdc"
       }
   }
   

Integrate the DynamoDB table

Integrate the newly created DynamoDB table into the Chalice application.

Instructions

1. Save the DynamoDB table name as an environment variable in the Chalice application by running the recordresources.py script:

   $ python recordresources.py --stack-name media-query
   

2. Import os and the chalicelib.db module in your app.py file:

import os

import boto3
from chalice import Chalice
from chalicelib import db
from chalicelib import rekognition

3. Add a helper function for instantiating a db.DynamoMediaDB class using the DynamoDB table name stored as the environment variable MEDIA_TABLE_NAME:

import os

import boto3
from chalice import Chalice
from chalicelib import db
from chalicelib import rekognition

app = Chalice(app_name='media-query')

_MEDIA_DB = None
_REKOGNITION_CLIENT = None


def get_media_db():
    global _MEDIA_DB
    if _MEDIA_DB is None:
        _MEDIA_DB = db.DynamoMediaDB(
            boto3.resource('dynamodb').Table(
                os.environ['MEDIA_TABLE_NAME']))
    return _MEDIA_DB

4. Update the detect_labels_on_image Lambda function to save the image along with the detected labels to the database:

   @app.lambda_function()
   def detect_labels_on_image(event, context):
       bucket = event['Bucket']
       key = event['Key']
       labels = get_rekognition_client().get_image_labels(bucket=bucket, key=key)
       get_media_db().add_media_file(key, media_type=db.IMAGE_TYPE, labels=labels)
   

Verification

1. Ensure the contents of the config.json contains environment variables for MEDIA_TABLE_NAME:

   $ cat .chalice/config.json
   {
     "version": "2.0",
     "app_name": "media-query",
     "stages": {
       "dev": {
         "api_gateway_stage": "api",
         "autogen_policy": false,
         "environment_variables": {
           "MEDIA_TABLE_NAME": "media-query-MediaTable-10QEPR0O8DOT4",
           "MEDIA_BUCKET_NAME": "media-query-mediabucket-fb8oddjbslv1"
         }
       }
     }
   }
   

   Note that the MEDIA_BUCKET_NAME will be present as well in the environment variables. It will be used in the next part of the tutorial.

2. Ensure the contents of the app.py file is:

import os

import boto3
from chalice import Chalice
from chalicelib import db
from chalicelib import rekognition

app = Chalice(app_name='media-query')

_MEDIA_DB = None
_REKOGNITION_CLIENT = None


def get_media_db():
    global _MEDIA_DB
    if _MEDIA_DB is None:
        _MEDIA_DB = db.DynamoMediaDB(
            boto3.resource('dynamodb').Table(
                os.environ['MEDIA_TABLE_NAME']))
    return _MEDIA_DB


def get_rekognition_client():
    global _REKOGNITION_CLIENT
    if _REKOGNITION_CLIENT is None:
        _REKOGNITION_CLIENT = rekognition.RekognitonClient(
            boto3.client('rekognition'))
    return _REKOGNITION_CLIENT


@app.lambda_function()
def detect_labels_on_image(event, context):
    bucket = event['Bucket']
    key = event['Key']
    labels = get_rekognition_client().get_image_labels(bucket=bucket, key=key)
    get_media_db().add_media_file(key, media_type=db.IMAGE_TYPE, labels=labels)

Redeploy the Chalice application

Deploy the updated Chalice application.

Instructions

1. Run chalice deploy:

   $ chalice deploy
   Creating deployment package.
   Updating policy for IAM role: media-query-dev-detect_labels_on_image
   Updating lambda function: media-query-dev-detect_labels_on_image
   Resources deployed:
     - Lambda ARN: arn:aws:lambda:us-west-2:123456789123:function:media-query-dev-detect_labels_on_image
   

Verification

1. Run chalice invoke with the sample-event.json on the updated detect_labels_on_image Lambda function:

   $ chalice invoke --name detect_labels_on_image < sample-event.json
   null
   

2. Use the get-item CLI command to ensure the sample.jpg data was populated in the DynamoDB table:

   $ aws dynamodb get-item --table-name $MEDIA_TABLE_NAME \
       --key '{"name": {"S": "sample.jpg"}}'
   {
       "Item": {
           "name": {
               "S": "sample.jpg"
           },
           "labels": {
               "L": [
                   {
                       "S": "Animal"
                   },
                   {
                       "S": "Canine"
                   },
                   {
                       "S": "Dog"
                   },
                   {
                       "S": "German Shepherd"
                   },
                   {
                       "S": "Mammal"
                   },
                   {
                       "S": "Pet"
                   },
                   {
                       "S": "Collie"
                   }
               ]
           },
           "type": {
               "S": "image"
           }
       }
   }