From API to Snowflake in 3 Steps (part 1)

In this 3-part blog series, we’ll walk through the process of extracting data from the Amplitude API using Python, uploading the output files to an Amazon S3 bucket, and finally loading that data into a Snowflake table.

Part One: Creating an S3 Bucket with the Right Permissions

Setting up the bucket early is a good practice—it allows us to test the connection before writing any Python code. If you happen to have access to a tool like Airbyte, that’s even better—it can streamline this entire process by automatically connecting to APIs and uploading to S3 (though it’s not quite enough for its own dedicated post).

Step 1: Create an IAM Policy

A policy in AWS is a JSON document that defines what actions are allowed or denied, and which resources (like S3 buckets) those rules apply to. We won't dive into writing custom policies here—but don't worry, tools like ChatGPT can generate them for you quickly.

1. Navigate to the policy tab under the IAM section and select 'Create Policy'
2. Select the JSON editor and copy the policy you wish to enforce
3. Review & create

For a policy to be effective, it must be attached to a principal (IAM user, group, role) or a resource (S3 Bucket)

Step 2: Creating an IAM User

This provides a unique identity within your AWS account; once a policy is attached, that user can then access AWS resources.

1. Create a new user through IAM and assign a sensible name
2. Under 'set permissions', attach policies directly and select the one you made in step 1

Here you can see any other services that apply to this policy, and this is where you access your ARN key (top right).

Step 3: Creating a Bucket

• Navigate to S3 > Create bucket.
• Provide a sensible bucket name & fill out the following:
  • Object Ownership - ACLs disabled (recommended)
  • Block Public Access - Block all public access
  • Bucket Versioning - Think about whether you need it
  • Default Encryption - Server-side encryption with AWS Key Management Service keys (SSE-KMS)
    •  Create a new KMS key
• Return to Amazon S3 tab and enter AWS KMS key ARN or choose the key from your AWS KMS keys
• Click Create bucket
• You are also able to create folders to increase clarity

👉 Next up: Check out the next post in this series to learn how to upload files to your S3 bucket using Python.