Serverless Survey Application End-to-End on AWS with Python and Terraform

Introduction

I like this paradigm where you break your whole infrastructure into microservices, I thought it would be fun to create an app using Python and go full serverless on AWS.
Check out the full code here --> https://github.com/bobocuillere/Serverless-AWS-Project

The “serverless” approach lets you concentrate on what truly matters: creating the application logic and delivering features. With AWS’s serverless services, you get automatic scaling, high availability, and a pay-as-you-go billing model, all while AWS handles the heavy lifting behind the scenes.

For my survey web application—built entirely from scratch—I wanted an environment where I could iterate quickly, scale effortlessly.

Achieving this meant two things:

1. Go Serverless: Use AWS Lambda for the backend logic, Amazon S3 for hosting the frontend, Amazon Cognito for user authentication, Amazon DynamoDB for data storage, Amazon API Gateway for routing requests, and Amazon SNS for notifications.
2. Automate Everything with Infrastructure as Code (IaC): Employ Terraform to define all of these resources and their configurations in code. This means I can spin up or tear down the entire environment with a single command.

By the end of this article, you’ll see how these AWS components fit together to form a coherent, secure, and scalable serverless application.

Architecture Overview: A Survey App Without Servers

The entire goal is to have a browser-based survey frontend that never talks directly to a server you manage. Instead, it relies on AWS-managed services and logic running in AWS Lambda functions—both of which scale and operate without you ever provisioning a single VM.

We’ll look at the full request flow, authentication steps, data storage logic, and how secure isolation is maintained. We’ll also consider how each piece is wired together with Infrastructure as Code (IaC) and how the environment remains consistent across deployments.

Core AWS Services and Their Detailed Roles

Amazon Cognito (User Authentication & Identity Management)

• What It Does: Cognito handles sign-up, sign-in, and token issuance. Users provide their credentials through the frontend, and the frontend code exchanges those credentials with Cognito for a JWT token.
• Tokens & JWT Verification:
  Once a user logs in successfully, Cognito returns an ID token (JWT) and an access token. The frontend stores these tokens. On each subsequent API request, the token is included in the Authorization header. Backend Lambdas verify the token’s signature against Cognito’s JWKS (JSON Web Key Set) endpoints, ensuring requests are from authenticated users and haven’t been tampered with.

Amazon S3 (Frontend Hosting & Private Assets)
Public S3 Bucket (Frontend):

• The frontend assets (HTML, CSS) are hosted here. The bucket is configured for public-read access, so when the user opens the site URL, the browser downloads these static files directly. This simple setup eliminates the need for a web server.

Private S3 Bucket (Protected Assets):

• Certain assets should never be directly exposed on the public internet. The private S3 bucket is locked down with IAM policies. Only the frontend Lambda function, running with an IAM role that grants s3:GetObject, can retrieve these files. The browser never sees a direct link to these private files. For example, if a user wants to retrieve a special survey template, the frontend lambda sends an API request. If authorized, it fetches the needed object from the private bucket and returns the data to the user. The user never sees a public URL to that file, preserving confidentiality.
• Why Two Buckets: Separating public and private content enforces a clear security boundary. Public assets are global and cacheable, private assets require controlled, token-validated access via Lambda.

Amazon API Gateway

• Purpose: API Gateway provides a stable URL and a set of RESTful endpoints for frontend code to interact with the backend logic. For example:
• POST /flask/login for authenticating the user and retrieving a JWT.
• POST /survey/create for creating a new survey.
• GET /survey/responses?survey_id=XYZ to fetch existing responses.
• Integration Choices: Each route in API Gateway is integrated with the frontend Lambda, not the backend Lambda. This is a deliberate design choice.

AWS Lambda Functions (Backend and Frontend Logic in Python)

• Frontend Lambda:
  This Python-based Lambda is the “gatekeeper.” It handles user-facing logic that involves verifying JWT tokens and performing operations such as creating, fetching surveys by invoking the backend lambda.
• Backend Lambda:
  Another Python-based Lambda responsible for “core business logic.” It deals with survey creation, reading and writing responses to DynamoDB, and when needed, publishing messages to SNS for notifications.  The backend Lambda is never directly integrated with API Gateway. It is only called by the frontend Lambda, using the AWS SDK internally (e.g., boto3.client('lambda').invoke(...)). This design ensures that even if someone tried to craft a direct request to the backend functions, they’d fail—no public endpoint exists. The backend code remains protected behind the frontend Lambda’s logic.
• Why Two-Lambda Setup: Separating frontend from backend logic enforces a clean security boundary and layered architecture. The frontend Lambda is responsible for authentication and preliminary checks; the backend Lambda handles database operations and notifications.

Amazon DynamoDB (Primary Data Store)

• Why DynamoDB: It’s serverless, scales automatically, and doesn’t require capacity planning. Perfect for unpredictable workloads, like a survey that might suddenly go viral.
• Integration with Backend Lambda: The backend Lambda uses boto3 to interact with DynamoDB. For example, when creating a survey, it puts an item into the Surveys table. When fetching responses, it queries by survey_id.

Amazon SNS (Event Notifications)

• When Used: Suppose a survey is deleted. We might want to send an alert email to an admin or trigger a cleanup process. The backend Lambda publishes a message to an SNS topic.
• Why SNS: It decouples the event producer (backend Lambda) from event consumers (emails, other Lambdas, etc.). If we later want SMS alerts or a Slack integration, we just add a new SNS subscription—no need to modify the backend code.

Putting It All Together Step-by-Step

Let’s walk through the entire flow of the survey application’s architecture from start to finish, showing exactly how each part interacts with the others. This step-by-step approach will help clarify the roles of the frontend Lambda, backend Lambda, and all the AWS services in between.

1- User Opens the Survey App:

• The user navigates to the application’s URL. The frontend files (HTML, CSS, JavaScript) are served from a public Amazon S3 bucket.
• Because it’s a static site, the user’s browser directly fetches these assets from S3, no servers needed.

2 - User Authenticates with Cognito:

• In the browser, the user either signs up or logs in.
• The frontend JavaScript sends credentials (username/password) to the API endpoint (e.g., POST /flask/login).
• This request goes through Amazon API Gateway, which routes it to the frontend Lambda.
• The frontend Lambda calls Cognito using the AWS SDK to verify credentials. If correct, Cognito returns a JWT token.
• The frontend Lambda sends that JWT token back to the browser. Now the user’s browser stores this token for future requests.

3- User Creates or Manages a Survey:

• With a JWT token in hand, the user wants to create a new survey. The browser sends POST /survey/create with the JWT included in the Authorization header.
• Again, API Gateway receives this request and routes it to the frontend Lambda.
• The frontend Lambda checks the JWT token to ensure it’s valid and that the user is allowed to perform the action.
• If valid, the frontend Lambda then invokes the backend Lambda internally using the AWS SDK (e.g., boto3 for Python). There’s no direct public route to the backend Lambda; all calls must pass through the frontend Lambda first. This ensures strict control and an additional security layer.

4 - Backend Lambda Interacts with DynamoDB:

• Upon receiving the validated request from the frontend Lambda, the backend Lambda executes the logic for creating the survey.
• It constructs a DynamoDB put_item request to store the new survey record in the Surveys table.
• DynamoDB responds immediately, and the backend Lambda returns a success message (including survey_id) back to the frontend Lambda.

5 - User Fetches Survey Responses (Another Example Flow):

• Suppose the user requests GET /survey/responses?survey_id=XYZ to view all responses for a given survey.
• The browser again includes the JWT token.
• API Gateway routes the call to the frontend Lambda.
• Frontend Lambda validates the JWT token and checks if this user is indeed allowed to view the specified survey’s responses.
• If allowed, the frontend Lambda calls the backend Lambda, which queries the Responses table in DynamoDB.

6 - Working with Private Assets (On the private bucket):

• If a particular survey requires loading a private template file (stored in a private S3 bucket), the process is similar.
• The browser calls a special endpoint handled by the frontend Lambda.
• The frontend Lambda checks the JWT and ensures the user can access that template.
• If authorized, the frontend Lambda uses its IAM permissions to read the private S3 object directly (the browser never sees a private URL).

7 - Notifications via SNS:

• Suppose the user deletes a survey. The browser calls DELETE /survey?survey_id=XYZ with the JWT token.
• This request, like all others, passes through API Gateway to the frontend Lambda, which validates permissions.
• If authorized, the frontend Lambda calls the backend Lambda to perform the deletion.
• The backend Lambda deletes the survey record in DynamoDB.
• After successful deletion, the backend Lambda publish a message to an SNS topic, here I tested with an email.

Troubleshooting Common Issues

Throughout development, I encountered a few issues. Here are some highlights and how I solved them:

CORS Errors on the Frontend:

• Symptom: My frontend JavaScript called API Gateway endpoints but got CORS errors in the browser console.
• Solution: I updated the API Gateway integration in Terraform to return Access-Control-Allow-Origin and other CORS headers. This involved setting method.response.header.Access-Control-Allow-Origin to '*' and ensuring the OPTIONS method was configured. Re-applying Terraform resolved the issue.

Invalid JWT Token Errors in Lambda:

• Symptom: Even with a valid token, I got "Unauthorized" responses.
• Solution: I made sure the Lambda functions fetched Cognito’s JSON Web Key Set (JWKS) and validated tokens correctly. Adjusting my Python code to parse the token’s kid, retrieve the right public key, and verify signatures fixed the problem.

DynamoDB Throttling Under Heavy Load:

• Symptom: High-traffic tests caused some "ProvisionedThroughputExceeded" errors.
• Solution: Switching DynamoDB tables to On-Demand capacity mode in Terraform allowed automatic scaling with no manual provisioning. After this change and re-apply, throttling disappeared under normal test loads.

Permission Denied for Private S3 Bucket Access:

• Symptom: Backend Lambda failed to read private assets.
• Solution: I updated Terraform IAM policies to grant s3:GetObject on the private bucket’s ARN to the Lambda execution role. After a quick apply, the function could access files properly.

Conclusion

Building a serverless survey application on AWS—and automating every aspect of its infrastructure with Terraform—has shown just how far cloud computing and DevOps practices have come. Instead of setting up servers, manually creating users in an identity service, or worrying about scaling databases, we focused on writing clear configuration files and straightforward application code.

In essence, this approach transforms the way you build and run applications. It takes you from a world where operations can be slow, error-prone, and costly, to one where agility, reliability, and cost-efficiency are the natural byproducts of well-chosen architectural patterns and tools. Whether you’re working on a small hobby project, a startup’s MVP, or a complex enterprise system, the principles and workflows described here will help you embrace modern cloud-native development with confidence.