Part 6: Add REST API to query media files

So far we have been querying the image files stored in our table via the AWS CLI. However, it would be more helpful to have an API on-top of the table instead of having to query it directly with the AWS CLI. We will now use Amazon API Gateway integrations with Lambda to create an API for our application. This API will have two routes:

  • GET / - List all media items in the table. You can supply the query string parameters: startswith, media-type, and label to further filter the media items returned in the API call

  • GET /{name} - Retrieve the media item based on the name of the media item.

To create this API, we will perform the following steps:

Add route for listing media items

Add an API route GET / that lists all items in the table and allows users to query on startswith, media-type, and label.

Instructions

  1. In the app.py file, define the function list_media_files() that has the route GET / using the app.route decorator:

    @app.route('/')
    def list_media_files():
    
  2. Inside of the list_media_files() function, extract the query string parameters from the app.current_request object and query the database for the media files:

    @app.route('/')
    def list_media_files():
        params = {}
        if app.current_request.query_params:
            params = _extract_db_list_params(app.current_request.query_params)
        return get_media_db().list_media_files(**params)
    
    
    def _extract_db_list_params(query_params):
        valid_query_params = [
            'startswith',
            'media-type',
            'label'
        ]
        return {
            k.replace('-', '_'): v
            for k, v in query_params.items() if k in valid_query_params
        }
    

Verification

  1. Ensure the contents of the app.py file is:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
import os

import boto3
from chalice import Chalice
from chalicelib import db
from chalicelib import rekognition

app = Chalice(app_name='media-query')

_MEDIA_DB = None
_REKOGNITION_CLIENT = None
_SUPPORTED_IMAGE_EXTENSIONS = (
    '.jpg',
    '.png',
)


def get_media_db():
    global _MEDIA_DB
    if _MEDIA_DB is None:
        _MEDIA_DB = db.DynamoMediaDB(
            boto3.resource('dynamodb').Table(
                os.environ['MEDIA_TABLE_NAME']))
    return _MEDIA_DB


def get_rekognition_client():
    global _REKOGNITION_CLIENT
    if _REKOGNITION_CLIENT is None:
        _REKOGNITION_CLIENT = rekognition.RekognitonClient(
            boto3.client('rekognition'))
    return _REKOGNITION_CLIENT


@app.on_s3_event(bucket=os.environ['MEDIA_BUCKET_NAME'],
                 events=['s3:ObjectCreated:*'])
def handle_object_created(event):
    if _is_image(event.key):
        _handle_created_image(bucket=event.bucket, key=event.key)


@app.on_s3_event(bucket=os.environ['MEDIA_BUCKET_NAME'],
                 events=['s3:ObjectRemoved:*'])
def handle_object_removed(event):
    if _is_image(event.key):
        get_media_db().delete_media_file(event.key)


@app.route('/')
def list_media_files():
    params = {}
    if app.current_request.query_params:
        params = _extract_db_list_params(app.current_request.query_params)
    return get_media_db().list_media_files(**params)


def _extract_db_list_params(query_params):
    valid_query_params = [
        'startswith',
        'media-type',
        'label'
    ]
    return {
        k.replace('-', '_'): v
        for k, v in query_params.items() if k in valid_query_params
    }


def _is_image(key):
    return key.endswith(_SUPPORTED_IMAGE_EXTENSIONS)


def _handle_created_image(bucket, key):
    labels = get_rekognition_client().get_image_labels(bucket=bucket, key=key)
    get_media_db().add_media_file(key, media_type=db.IMAGE_TYPE, labels=labels)
  1. Install HTTPie to query the API:

    $ pip install httpie
    
  2. In a different terminal, run chalice local to run the API as a server locally:

    $ chalice local
    
  3. Use HTTPie to query the API for all images:

    $ http 127.0.0.1:8000/
    HTTP/1.1 200 OK
    Content-Length: 126
    Content-Type: application/json
    Date: Tue, 17 Jul 2018 13:59:35 GMT
    Server: BaseHTTP/0.6 Python/3.6.1
    
    [
        {
            "labels": [
                "Animal",
                "Canine",
                "Dog",
                "German Shepherd",
                "Mammal",
                "Pet",
                "Collie"
            ],
            "name": "sample.jpg",
            "type": "image"
        }
    ]
    
  4. Use HTTPie to query the API using the query string parameter label:

    $ http 127.0.0.1:8000/ label==Dog
    HTTP/1.1 200 OK
    Content-Length: 126
    Content-Type: application/json
    Date: Tue, 17 Jul 2018 14:01:22 GMT
    Server: BaseHTTP/0.6 Python/3.6.1
    
    [
        {
            "labels": [
                "Animal",
                "Canine",
                "Dog",
                "German Shepherd",
                "Mammal",
                "Pet",
                "Collie"
            ],
            "name": "sample.jpg",
            "type": "image"
        }
    ]
    $ http 127.0.0.1:8000/ label==Person
    HTTP/1.1 200 OK
    Content-Length: 2
    Content-Type: application/json
    Date: Tue, 17 Jul 2018 14:01:46 GMT
    Server: BaseHTTP/0.6 Python/3.6.1
    
    []
    

    Feel free to test out any of the other query string parameters as well.

Add route for retrieving a single media item

Add an API route GET /{name} that retrieves a single item in the table using the name of the item.

Instructions

  1. Import chalice.NotFoundError in the app.py file:

1
2
3
4
5
6
7
import os

import boto3
from chalice import Chalice
from chalice import NotFoundError
from chalicelib import db
from chalicelib import rekognition
  1. In the app.py file, define the function get_media_file() decorated by app.route('/{name}'):

    @app.route('/{name}')
    def get_media_file(name):
    
  2. Within the get_media_file() function, query the media item using the name parameter and raise a chalice.NotFoundError exception when the name does not exist in the database:

    @app.route('/{name}')
    def get_media_file(name):
        item = get_media_db().get_media_file(name)
        if item is None:
            raise NotFoundError('Media file (%s) not found' % name)
        return item
    

Verification

  1. Ensure the contents of the app.py file is:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
import os

import boto3
from chalice import Chalice
from chalice import NotFoundError
from chalicelib import db
from chalicelib import rekognition

app = Chalice(app_name='media-query')

_MEDIA_DB = None
_REKOGNITION_CLIENT = None
_SUPPORTED_IMAGE_EXTENSIONS = (
    '.jpg',
    '.png',
)


def get_media_db():
    global _MEDIA_DB
    if _MEDIA_DB is None:
        _MEDIA_DB = db.DynamoMediaDB(
            boto3.resource('dynamodb').Table(
                os.environ['MEDIA_TABLE_NAME']))
    return _MEDIA_DB


def get_rekognition_client():
    global _REKOGNITION_CLIENT
    if _REKOGNITION_CLIENT is None:
        _REKOGNITION_CLIENT = rekognition.RekognitonClient(
            boto3.client('rekognition'))
    return _REKOGNITION_CLIENT


@app.on_s3_event(bucket=os.environ['MEDIA_BUCKET_NAME'],
                 events=['s3:ObjectCreated:*'])
def handle_object_created(event):
    if _is_image(event.key):
        _handle_created_image(bucket=event.bucket, key=event.key)


@app.on_s3_event(bucket=os.environ['MEDIA_BUCKET_NAME'],
                 events=['s3:ObjectRemoved:*'])
def handle_object_removed(event):
    if _is_image(event.key):
        get_media_db().delete_media_file(event.key)


@app.route('/')
def list_media_files():
    params = {}
    if app.current_request.query_params:
        params = _extract_db_list_params(app.current_request.query_params)
    return get_media_db().list_media_files(**params)


@app.route('/{name}')
def get_media_file(name):
    item = get_media_db().get_media_file(name)
    if item is None:
        raise NotFoundError('Media file (%s) not found' % name)
    return item


def _extract_db_list_params(query_params):
    valid_query_params = [
        'startswith',
        'media-type',
        'label'
    ]
    return {
        k.replace('-', '_'): v
        for k, v in query_params.items() if k in valid_query_params
    }


def _is_image(key):
    return key.endswith(_SUPPORTED_IMAGE_EXTENSIONS)


def _handle_created_image(bucket, key):
    labels = get_rekognition_client().get_image_labels(bucket=bucket, key=key)
    get_media_db().add_media_file(key, media_type=db.IMAGE_TYPE, labels=labels)
  1. If the local server is not still running, run chalice local to restart the local API server:

    $ chalice local
    
  2. Use HTTPie to query the API for the sample.jpg image:

    $ http 127.0.0.1:8000/sample.jpg
    HTTP/1.1 200 OK
    Content-Length: 124
    Content-Type: application/json
    Date: Tue, 17 Jul 2018 14:09:01 GMT
    Server: BaseHTTP/0.6 Python/3.6.1
    
    {
        "labels": [
            "Animal",
            "Canine",
            "Dog",
            "German Shepherd",
            "Mammal",
            "Pet",
            "Collie"
        ],
        "name": "sample.jpg",
        "type": "image"
    }
    
  3. Use HTTPie to query the API for an image that does not exist:

    $ http 127.0.0.1:8000/noexists.jpg
    HTTP/1.1 404 Not Found
    Content-Length: 90
    Content-Type: application/json
    Date: Tue, 17 Jul 2018 14:09:34 GMT
    Server: BaseHTTP/0.6 Python/3.6.1
    
    {
        "Code": "NotFoundError",
        "Message": "NotFoundError: Media file (noexists.jpg) not found"
    }
    

Redeploy the Chalice application

Deploy the Chalice application based on the updates.

Instructions

  1. Run chalice deploy:

    $ chalice deploy
    Creating deployment package.
    Updating policy for IAM role: media-query-dev-handle_object_created
    Updating lambda function: media-query-dev-handle_object_created
    Configuring S3 events in bucket media-query-mediabucket-fb8oddjbslv1 to function media-query-dev-handle_object_created
    Updating policy for IAM role: media-query-dev-handle_object_removed
    Updating lambda function: media-query-dev-handle_object_removed
    Configuring S3 events in bucket media-query-mediabucket-fb8oddjbslv1 to function media-query-dev-handle_object_removed
    Creating IAM role: media-query-dev-api_handler
    Creating lambda function: media-query-dev
    Creating Rest API
    Resources deployed:
      - Lambda ARN: arn:aws:lambda:us-west-2:123456789123:function:media-query-dev-handle_object_created
      - Lambda ARN: arn:aws:lambda:us-west-2:123456789123:function:media-query-dev-handle_object_removed
      - Lambda ARN: arn:aws:lambda:us-west-2:123456789123:function:media-query-dev
      - Rest API URL: https://1lmxgj9bfl.execute-api.us-west-2.amazonaws.com/api/
    

Verification

  1. Reupload the othersample.jpg image using the CLI:

    $ aws s3 cp ../chalice-workshop/code/media-query/final/assets/othersample.jpg s3://$MEDIA_BUCKET_NAME
    
  2. Use HTTPie to query the deployed API for all media items:

    $ http $(chalice url)
    HTTP/1.1 200 OK
    Connection: keep-alive
    Content-Length: 126
    Content-Type: application/json
    Date: Tue, 17 Jul 2018 14:14:27 GMT
    Via: 1.1 a3c7cc30af6c8465e695a3c0d44793e0.cloudfront.net (CloudFront)
    X-Amz-Cf-Id: PAkgH2j5G2er_TZwyQOcwGahwNTR8dhEhrCUklcdDuuEBcKOYQ1-Ug==
    X-Amzn-Trace-Id: Root=1-5b4df9c1-89a47758a7a7989e47799a12;Sampled=0
    X-Cache: Miss from cloudfront
    x-amz-apigw-id: KLP2SFnTPHcFeqw=
    x-amzn-RequestId: b5e7488a-89cb-11e8-acbf-eda14961f501
    
    [
        {
            "labels": [
                "Human",
                "People",
                "Person",
                "Phone Booth",
                "Bus",
                "Transportation",
                "Vehicle",
                "Man",
                "Face",
                "Leisure Activities",
                "Tourist",
                "Portrait",
                "Crowd"
            ],
            "name": "othersample.jpg",
            "type": "image"
        },
        {
            "labels": [
                "Animal",
                "Canine",
                "Dog",
                "German Shepherd",
                "Mammal",
                "Pet",
                "Collie"
            ],
            "name": "sample.jpg",
            "type": "image"
        }
    ]
    

    Note chalice url just returns the URL of the remotely deployed API.

  3. Use HTTPie to test out a couple of the query string parameters:

    $ http $(chalice url) label=='Phone Booth'
    HTTP/1.1 200 OK
    Connection: keep-alive
    Content-Length: 207
    Content-Type: application/json
    Date: Sun, 22 Jul 2018 07:49:37 GMT
    Via: 1.1 75fd15ce5d9f38e4c444039a1548df96.cloudfront.net (CloudFront)
    X-Amz-Cf-Id: nYpeS8kk_lFklCA7wCkOI0NO1wabDI3jvs3UpHFlsJ-c0nvlXNrvJQ==
    X-Amzn-Trace-Id: Root=1-5b543710-8beb4000395cd60e5688841a;Sampled=0
    X-Cache: Miss from cloudfront
    x-amz-apigw-id: Ka2KpF0nvHcF1hg=
    x-amzn-RequestId: c7e9cabf-8d83-11e8-b109-5f2c96dac9da
    
    [
        {
            "labels": [
                "Human",
                "People",
                "Person",
                "Phone Booth",
                "Bus",
                "Transportation",
                "Vehicle",
                "Man",
                "Face",
                "Leisure Activities",
                "Tourist",
                "Portrait",
                "Crowd"
            ],
            "name": "othersample.jpg",
            "type": "image"
        }
    ]
    
    $ http $(chalice url) startswith==sample
    HTTP/1.1 200 OK
    Connection: keep-alive
    Content-Length: 126
    Content-Type: application/json
    Date: Sun, 22 Jul 2018 07:51:03 GMT
    Via: 1.1 53657f22d99084ad547a21392858391b.cloudfront.net (CloudFront)
    X-Amz-Cf-Id: TORlA6wdOff5n4xHUH9ftnXNxFrTmQsSFG18acx7iwKLA_NsUoUoCg==
    X-Amzn-Trace-Id: Root=1-5b543766-912f6e067cb58ddcb6a973de;Sampled=0
    X-Cache: Miss from cloudfront
    x-amz-apigw-id: Ka2YEGNvPHcF8SA=
    x-amzn-RequestId: fb25c9e7-8d83-11e8-898d-8da83b49132b
    
    [
        {
            "labels": [
                "Animal",
                "Canine",
                "Dog",
                "German Shepherd",
                "Mammal",
                "Pet",
                "Collie"
            ],
            "name": "sample.jpg",
            "type": "image"
        }
    ]
    
  4. Use HTTPie to query the deployed API for sample.jpg image:

    $ http $(chalice url)sample.jpg
    HTTP/1.1 200 OK
    Connection: keep-alive
    Content-Length: 124
    Content-Type: application/json
    Date: Tue, 17 Jul 2018 14:16:04 GMT
    Via: 1.1 7ca583dd6abc0b0f42b148142a75588a.cloudfront.net (CloudFront)
    X-Amz-Cf-Id: pzkZ0uZvk5e5W-ZV39v2zCCFAmmRJjDMJZ_I9GyDKhg6WEHotrMmnQ==
    X-Amzn-Trace-Id: Root=1-5b4dfa24-69d586d8e94fb75019b42f24;Sampled=0
    X-Cache: Miss from cloudfront
    x-amz-apigw-id: KLQFrF3svHcF32Q=
    x-amzn-RequestId: f0a6a6af-89cb-11e8-8420-e7ec8398ed6b
    
    {
        "labels": [
            "Animal",
            "Canine",
            "Dog",
            "German Shepherd",
            "Mammal",
            "Pet",
            "Collie"
        ],
        "name": "sample.jpg",
        "type": "image"
    }