Tutorial: Build an AI Assistant with API.ai and Amazon Lambda

Tech giants are betting big on conversational interfaces; Facebook acquired wit.ai, Google acquired api.ai, and Amazon announced Lex. These services all make it easy to parse user intention from natural language. In this tutorial, we’ll demonstrate how to connect a conversational interface with a third-party API. These steps will enable you to build rich experiences for the Google Home, Amazon Echo, Microsoft Cortana, Facebook chatbots, Slack Bots, or another AI assistant. For this example, we’ll build a Google Home action that tells us the height of Star Wars characters. We’ll use api.ai, Amazon Lambda and API Gateway, and the Star Wars API.

1. Create Your Conversational AI Assistant

  1. Sign up for api.ai. I used my Google Account.
  2. Click on “Create Agent.”
  3. Give your agent a name and description.
    Name: StarWars Description: A conversational assistant that can tell you the height of various star wars characters.
    Name: StarWars
    Description: A conversational assistant that can tell you the height of various star wars characters.
  4. Click “Save.”

2. Create A User Intent

An intent describes something a user is asking for. For example, a customer service agent may have a “returns” intent and a “tech support” intent. Our intent is for when the user asks about the height of a star wars character.

  1. Add a new intent by clicking the “+” next to “Intents.”
    api.ai add intents button
  2. Populate the intent.
    An animated GIF showing how to populate an intent of an ai assistant. Intent Name: Height Intent "How tall is Luke Skywalker?" "How short is Yoda?" "What is the height of Boba Fett?" "Tell me how big Chewbacca is."

    1. Intent Name: Height Intent
    2. Under User Says, we add different ways to express the intention that the user wants to know the height of a Star wars character:
      • “How tall is Luke Skywalker?”
      • “How short is Yoda?”
      • “What is the height of Boba Fett?”
      • “Tell me how big Chewbacca is.”
    3. Tag parameters. Notice that api.ai is automatically recognizing “Luke” and “Yoda” as a given-name and “Skywalker” as a last-name. Click on “Boba” and “Chewbacca” and identify them as given-name. Assign “Fett” as last-name.
  3. Test the intent in the right hand pane.
    Type “How tall is Yoda?”
    An animated gif demonstrating how to test an agent on api.ai. Type "How tall is Yoda?" The agent correctly identifies the height intent.
    Notice how the Height Intent is correctly identified, even though we didn’t train that exact wording. Also notice that “Yoda” has been correctly extracted as a given-name.
  4. Click “Save.”

At this point, we’ve successfully trained our agent to:

  • Recognize when users ask about a Star Wars character’s height, and
  • Extract the character’s name

Now, we’ll create a microservice to accept and process this information.

3. Create an Amazon IAM User

  1. Sign up for Amazon AWSI was able to use my existing Amazon account. You will have to enter a credit card (I used the one they had on file). Amazon bills based on usage, but you get 1 million free requests per month; this is more than enough to hobby-develop for free.
  2. Create an IAM user. AWS Identity and Access Management (IAM) enables you to securely control access to AWS services and resources. We’ll create an IAM user so we aren’t using our root Amazon account.
    1. Navigate to console.aws.amzon.com/iam
    2. Select “Users” from the left pane. Click “Add User.”
      amazon iam add user button
    3. Fill in the user info:
      User name: TestUser Access type: AWS Management Console access Console Password: Custom Password
      User name: TestUser
      Access type: AWS Management Console access
      Console Password: Custom Password
      Uncheck “Require password reset”
    4. Click “Next: Permissions”
    5. Click the checkboxes to add the following Permissions:
      • AWSLambdaFullAccess
        AWSLambdaFullAccess
      • AmazonAPIGatewayAdministrator
        AmazonAPIGatewayAdministrator
      • IAMFullAccess
        IAMFullAccess
    6. Click “Next: Review.” Your overview should look like this:IAM User Reivew
    7. Click “Create User”
  3. Sign in to your new IAM user.
    IAM TestUser Sign In

4. Create A Lambda Function

  1. Navigate to https://console.aws.amazon.com/lambda/home
  2. Click on “Create a Lambda function.”
    Amazon Lambda Create a Lambda function button
  3. Configure the Lambda function:
    an animated gif demonstrating how to create an AWS Lambda Function.

    1. Use the “hello-world” blueprint
    2. Under “Configure Triggers”, just press “Next.”
    3. Under “Configure Function”
      Name: starWarsBot
      Description: A middleman between api.ai and swapi.co
  4. Create a Handler and Role:
    an animated gif demonstrating how to create a role for a lambda function with simple microservice permissions
    Handler: index.handler
    Role: Create new role from templates
    Role Name: simpleMicroservice
    Template: Simple Microservice permissions
  5. Click “Next.”

5. Code Your Lambda Function

At this point we have a hosted node.js function that we can edit on the web. Now, we’ll change the code to accept our request from api.ai and query swapi.co.

  1. Replace the Lambda code with the following code:
    This code extracts the given-name and last-name from the inbound event and formats a web request to www.swapi.co/api/people/?search. It then extracts the height from the response and returns {"speech":" "} to api.ai.
  2. Press “Save.”
  3. Test Your Lambda Function.
    1. Back at api.ai, run a test query, and press “show JSON.”an animated gif that shows a user inputing a test query on api.ai, pressing "show JSON" and copying the JSON
    2. Copy the JSON
    3. Configure a Test Event for your Lambda
      an animated gif showing a user creating a test event to test an Amazon Lambda function

      1. Click Actions > Configure Test Event.
      2. Paste the JSON.
      3. Click “Save and Test.” Your test should succeed and you should see the output:

6. Create an Endpoint in API Gateway

At this point, we have:

  • An api.ai agent that collects user input and POSTs JSON.
  • A Lambda function that can consume that JSON, query swapi.co, and return a speech response.

Now, we need to enable the two pieces to communicate with each other.

  1. Navigate to http://console.aws.amazon.com/apigateway/home
  2. Create a New API:
    An animated gif showing how to create a new api on amazon api gateway. The api name is Starwars and the description is a microservice for finding information about star wars.
    API name: Starwars
    Description: A microservice for finding information about star wars
  3. Create an API endpoint:
    an animated gif demonstrating how to create an endpoint in Amazon API gateway. The user creates a resource "/height" and a POST method.
    Actions > Create Resource
    Resource Name: height
    Actions > Create Method > POST
    Integration Type: Lambda Function
    Lambda Region: **Must match your Lambda region**. (You should be able to figure out the region from the page url.)
    Lambda Function: starWarsBot
  4. Deploy your API:
    an animated gif demonstrating how to deploy an api on amazon api gateway. The user clicks actions, then deploy api, then creates a new state named "beta"
    Actions > Deploy API
    Deployment Stage: [New Stage]
    Stage Name: beta
    Click Deploy
  5. Copy the “invoke URL”

7. Use Your New API for Intent Fulfillment

  1. Back at api.ai, choose “Fulfillment” from the left pane.
    api.ai fulfillment button
  2. Enable the switch, paste the URL from your API, and complete the url to point at the “/height” endpoint.
    an animated gif demonstrating how to enable webhooks in api.ai
  3. Click “Save.”
  4. Navigate to our intent (Intents > Height Intent), expand the fulfillment section, and check the “Use webhook” box.
    an animated gif demonstrating how to use a webhook for fulfillment in api.ai
  5. In the “Try it now” field in the right pane, try another query.
    an animated gif demonstrating a user testing an ai assistant. The user types "how tall is yoda" and the agent responds "yoda is 66 centimeters tall"
    You should see the correct response: “Yoda is 66 centimeters tall.”

8. Test On The Google Home

  1. Navigate to “Integrations” from the left pane.
    api.ai integrations button
  2. Turn on “Actions on Google.”
    an animated gif demonstrating how to turn on the actions on google integration on api.ai
    Invocation name for testing: Star Wars
  3. Press “Authenticate.” This will have you sign in to your Google Account. You’ll want this to be the same as the Google Account that the Google Home is set up with.
  4. Press “Preview.” A Message should pop up on the bottom of the screen:a popup that says "Preview published. Try it out on the Google Home Web Simulator"
  5. Click on the link in the pop up. (Or here.)
  6. Test on the Web Simulator.
    an animated gif of a user testing a conversational ai assistant on the google web simulator.
    Note: You must first ask to talk to the Star Wars agent, or phrase your question with the agent’s name: (“Ask Star Wars how tall Yoda is.”)
  7. Test on the Google Home!
    Try out the agent on any Google Home set up with the same Google Account that you developed with. If the package preview runs out, you can refresh it for 24 hours on the Web Simulator by clicking Action Package > Preview for 24 hours.

Make Something Awesome!

Here at Raizlabs, we’ve had a lot of fun hacking on new conversational interactions. This tutorial should be enough to get you up and running on creating your own cool new services. If you have any questions, feel free to leave a comment below. If you’d like to learn more about any of the technologies discussed in this post, check out some of these in-depth tutorials:

Getting Started With api.ai
Build an API to Expose a Lambda Function
Build an API Gateway API step by step

Get In Touch

Raizlabs believes in improving lives through design and technology. We would love to work with you on your next project. Contact us!


Do you have a project in mind? We’d love to work with you. If you’d like an opportunity to work on projects with us, check out our Careers page. We’re hiring!

16 thoughts on “Tutorial: Build an AI Assistant with API.ai and Amazon Lambda”

  1. Great walkthrough! It helped me make an app so I can get predictions for my bus. Although after finishing it, I realize that I can only run a “preview” for 24 hours. I don’t have the time to develop this into a full-fledged application, so I might have to resort to making a job that runs every night and re-enables the preview. Very frustrating.

  2. Great walkthrough – very clear, thank you.

    Do you know how to use the ApiAiAssistant class with a Lambda request though? It looks like you initialise with a request and response object (as typical in an Express app), but not sure what the parallel initialisation parameters would be when receiving requests in through a exports.handler function.

  3. This was great. I appreciate the effort you put in to writing this up.

  4. Wow. You are awesome. Thanks so much for putting together this tutorial. It would have taken me hours upon hours to figure this out myself. Your tutorial was so easy to follow, many, many thanks!!!

  5. this great tutorial.. i have do your stepps..
    api.ai – lambda – apigateway.
    that works perfectly..

    thanks

  6. Hello, Its a good walkthrough, i am software developer , and i want to develop a desktop WPF application which is handle by Alexa echo. my concept is that,

    1. in my application there are three node , mode1, mode2 , mode3 and each node have own child and his functionality , so how can i handle those node and that related functionality ?

    2. how to handle and configure my alexa device with my c# application ?

    3. how to handle application node by voice command ?

    thanks! waitting for your reply

  7. I want information on Alexa device.can you please explain me about Alexa device.How it works and how to Configure this device.I am planning to develop addition functionality for my existing application which will use voice commands to perform functions so let me know asap.

  8. great tutorial helped me a lot ..thanks..

  9. The tutorial is great. Thanks for the same. The given example works fine. But when I try to tweak the same, to call yahoo weather api, it throws errors.

    I get ‘Error: getaddrinfo ENOTFOUND’ error. Am trying to test in lambda test console. Am trying to hit the yahoo weather api.

    When I try the url – https://query.yahooapis.com/v1/public/yql, in browser, it says mandatory query param excepted. Pls let me know how to fix it. I get similar errors with my other POST request as well.

    Complete Error:
    2017-06-22T20:48:03.241Z 15a9b228-578c-11e7-b00d-2b54bcc77ff8 Error: getaddrinfo ENOTFOUND https://query.yahooapis.com/v1/public/yql https://query.yahooapis.com/v1/public/yql:80
    at errnoException (dns.js:28:10)
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:76:26)
    END RequestId: 15a9b228-578c-11e7-b00d-2b54bcc77ff8

  10. I followed your guide, and the test from api gateway works but when I test it from API.AI, nothing goes through. How can I find out what is blocking it?

  11. You guys save my day……Thank you so much 🙂

  12. Awesome! This helped me better understanding on web hook and API gateway

  13. Aw, this was a really nice post. Taking a few minutes
    and actual effort to make a very good article… but what can I say… I put things off a lot and don’t manage to get anything done.

  14. I’m having a problem when I test the bot in Api.ai.

    It keeps telling me the height of Luke Skywalker no matter how many other characters I ask it about.

    I have followed this tutorials to the last T about 8 times now. any idea what the problem is??

  15. Thanks for the tutorial. It looks like AWS have changed their layout slightly and I’m not sure if there is something else I’m supposed to configure, however when I paste my JSON in to test on the Lambda function, I get:

    {
    “errorMessage”: “RequestId: 45eb5aa7-abbd-11e7-9c29-95aa773b2554 Process exited before completing request”
    }

    I’ve gone back and created everything again to make sure I haven’t missed a step, and I still get the same error. Has the process changed slightly from when this tutorial was created?

  16. Did they make a change to the sys.last-name entity? If I ask about Yoda, obviously there is no last name. However, if I ask about Darth Vader, Luke Skywalker or even Han Solo, the last-name field never populates (yeah, sys.given-name does). John Smith works, but John Doe does not. Some forum posts about this seem to indicate that sys.last-name matches against a set database of 20,000(?) names and won’t return the value if it doesn’t match.
    Btw, even tried some strange inquiries to test a theory. “Smith is how John.” returns “John” and “Smith” as given and last names respectively. “John Smith is in downtown.” gave “John” and “Smith” too. “Smith is how in John town.” only returned “Smith” with a blank given-name. Some other variations didn’t hit the correct intent, though.

Leave a Comment