Most developers use boto3 to interact with AWS.

But have you ever wondered…

👉 What actually happens behind the scenes?
👉 How does AWS authenticate your API requests?

I always used boto3 without thinking much about it — until I tried calling AWS APIs directly.

In this blog, we’ll go one level deeper and:

🔥 Create an EC2 instance using raw HTTP requests with AWS Signature Version 4 (SigV4)

🧠 boto3 is Just a Wrapper

When you run:

ec2.run_instances(...)
Enter fullscreen mode Exit fullscreen mode

Behind the scenes, boto3:

  • Builds an HTTP request
  • Signs it using AWS Signature Version 4
  • Sends it to AWS APIs

In this blog, we’ll do all of that manually.

⚙️ What We’re Building

We’ll create a Python script that:

  • Uses requests
  • Implements AWS SigV4 authentication
  • Launches a real EC2 instance

No SDK. No shortcuts.

🔄 High-Level Flow 

Client → Canonical Request → String to Sign → Signature → AWS API → Response
Enter fullscreen mode Exit fullscreen mode

🔐 Understanding AWS Signature Version 4 (SigV4)

AWS secures every API request using SigV4. It ensures:

  • Authentication (who you are)
  • Integrity (request not tampered)

💻 Full Working Code 

import requests
import datetime
import hashlib
import hmac
import urllib.parse

# 🔐 AWS credentials
ACCESS_KEY = "YOUR_ACCESS_KEY"
SECRET_KEY = "YOUR_SECRET_KEY"

REGION = "ap-south-1"
SERVICE = "ec2"
HOST = f"ec2.{REGION}.amazonaws.com"
ENDPOINT = f"https://{HOST}/"

# 📦 EC2 parameters
params = {
    "Action": "RunInstances",
    "ImageId": "ami-0f5ee92e2d63afc18",
    "InstanceType": "t2.micro",
    "MinCount": "1",
    "MaxCount": "1",
    "Version": "2016-11-15"
}

# 🕒 Time
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
date_stamp = t.strftime('%Y%m%d')

# 🔹 Step 1: Canonical Query String
canonical_querystring = '&'.join(
    f"{urllib.parse.quote(k)}={urllib.parse.quote(v)}"
    for k, v in sorted(params.items())
)

# 🔹 Step 2: Canonical Request
canonical_headers = f"host:{HOST}\nx-amz-date:{amz_date}\n"
signed_headers = "host;x-amz-date"
payload_hash = hashlib.sha256(b"").hexdigest()

canonical_request = (
    "GET\n"
    "/\n"
    f"{canonical_querystring}\n"
    f"{canonical_headers}\n"
    f"{signed_headers}\n"
    f"{payload_hash}"
)

# 🔹 Step 3: String to Sign
algorithm = "AWS4-HMAC-SHA256"
credential_scope = f"{date_stamp}/{REGION}/{SERVICE}/aws4_request"

string_to_sign = (
    f"{algorithm}\n"
    f"{amz_date}\n"
    f"{credential_scope}\n"
    f"{hashlib.sha256(canonical_request.encode()).hexdigest()}"
)

# 🔹 Step 4: Signing Key
def sign(key, msg):
    return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

k_date = sign(("AWS4" + SECRET_KEY).encode(), date_stamp)
k_region = sign(k_date, REGION)
k_service = sign(k_region, SERVICE)
k_signing = sign(k_service, "aws4_request")

signature = hmac.new(
    k_signing,
    string_to_sign.encode(),
    hashlib.sha256
).hexdigest()

# 🔹 Step 5: Headers
authorization_header = (
    f"{algorithm} Credential={ACCESS_KEY}/{credential_scope}, "
    f"SignedHeaders={signed_headers}, Signature={signature}"
)

headers = {
    "x-amz-date": amz_date,
    "Authorization": authorization_header
}

# 🔹 Final request
request_url = ENDPOINT + "?" + canonical_querystring

print("👉 Calling:", request_url)

response = requests.get(request_url, headers=headers)

print("\nStatus Code:", response.status_code)
print("\nResponse:\n", response.text)
Enter fullscreen mode Exit fullscreen mode

🔍 Breaking Down the Code (with Snippets)

Let’s understand what’s happening step by step.

🔐 1. AWS Credentials & Config 

ACCESS_KEY = "YOUR_ACCESS_KEY"
SECRET_KEY = "YOUR_SECRET_KEY"

REGION = "ap-south-1"
SERVICE = "ec2"
HOST = f"ec2.{REGION}.amazonaws.com"
Enter fullscreen mode Exit fullscreen mode

👉 Defines credentials + region + service endpoint.

📦 2. EC2 Parameters 

params = {
    "Action": "RunInstances",
    "ImageId": "ami-0f5ee92e2d63afc18",
    "InstanceType": "t2.micro",
    "MinCount": "1",
    "MaxCount": "1",
    "Version": "2016-11-15"
}
Enter fullscreen mode Exit fullscreen mode

👉 This is the actual API request payload.

🕒 3. Timestamp 

t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
Enter fullscreen mode Exit fullscreen mode

👉 Required for AWS request validation.

🔹 4. Canonical Query String 

canonical_querystring = '&'.join(
    f"{urllib.parse.quote(k)}={urllib.parse.quote(v)}"
    for k, v in sorted(params.items())
)
Enter fullscreen mode Exit fullscreen mode

👉 Sort + encode parameters
👉 Critical step (most common failure point)

🔹 5. Canonical Request 

canonical_request = (
    "GET\n"
    "/\n"
    f"{canonical_querystring}\n"
    f"{canonical_headers}\n"
    f"{signed_headers}\n"
    f"{payload_hash}"
)
Enter fullscreen mode Exit fullscreen mode

👉 Exact request AWS validates internally.

🔹 6. String to Sign 

string_to_sign = (
    f"{algorithm}\n"
    f"{amz_date}\n"
    f"{credential_scope}\n"
    f"{hashlib.sha256(canonical_request.encode()).hexdigest()}"
)
Enter fullscreen mode Exit fullscreen mode

👉 This is what gets signed.

🔑 7. Signing Key 

k_date = sign(("AWS4" + SECRET_KEY).encode(), date_stamp)
k_region = sign(k_date, REGION)
k_service = sign(k_region, SERVICE)
k_signing = sign(k_service, "aws4_request")
Enter fullscreen mode Exit fullscreen mode

👉 Multi-step key derivation:

Secret → Date → Region → Service → aws4_request
Enter fullscreen mode Exit fullscreen mode

🔐 8. Signature 

signature = hmac.new(
    k_signing,
    string_to_sign.encode(),
    hashlib.sha256
).hexdigest()
Enter fullscreen mode Exit fullscreen mode

👉 Final cryptographic signature.

📬 9. Authorization Header 

headers = {
    "x-amz-date": amz_date,
    "Authorization": authorization_header
}
Enter fullscreen mode Exit fullscreen mode

👉 This header authenticates your request.

🌐 10. Sending Request 

response = requests.get(request_url, headers=headers)
Enter fullscreen mode Exit fullscreen mode

👉 Sends request → AWS validates → returns response.

🧠 Simple Analogy

Think of SigV4 like a sealed envelope:

  • Canonical request → message
  • Signing key → secret stamp
  • Signature → seal

AWS checks the seal before accepting it.

✅ Real Output

  • Status Code: 200
  • Instance created successfully
  • Instance ID: i-069a545b1814c6cec

⚠️ Common Errors

❌ SignatureDoesNotMatch

  • Wrong encoding
  • Params not sorted

I got SignatureDoesNotMatch for almost 30 minutes before realizing I wasn’t sorting parameters correctly.

❌ UnauthorizedOperation

  • Missing IAM permissions

❌ InvalidAMIID.NotFound

  • Wrong AMI

⚖️ boto3 vs requests

Feature boto3 requests + SigV4
Ease ✅ Easy ❌ Complex
Control ❌ Limited ✅ Full
Use Case Production Learning

🧠 Key Takeaway

After doing this, boto3 didn’t feel like magic anymore — just automation over HTTP.

🚨 Important Note

Use this approach for:

✅ Learning
✅ Debugging
✅ Deep understanding

❌ Not production

✍️ Final Thought

The hardest part wasn’t writing the code — it was getting the signature exactly right.