# VPC Endpoints Configuration for Private Subnet Deployment

## Overview

Since the Auto Clicker application is deployed in a **private subnet** (as confirmed in `vnc_guacamole_setup.md`), it requires VPC endpoints to access AWS services without routing traffic through the internet.

## Why VPC Endpoints Are Required

Resources in private subnets do not have direct internet access. To access AWS services, you have two options:

1. **VPC Endpoints (Recommended)** ✅
   - Private connectivity to AWS services
   - Traffic stays within AWS network
   - Lower latency
   - No data transfer charges for AWS-to-AWS traffic
   - More secure (no internet exposure)
   - Cost-effective for high-volume AWS service access

2. **NAT Gateway/Instance** ❌
   - Routes traffic through the internet
   - Less secure (traffic exposed to internet)
   - Higher data transfer costs
   - More expensive for AWS-to-AWS traffic

## AWS Services Used by This Application

Based on code analysis, the application uses the following AWS services:

1. **AWS Secrets Manager** - RDS database credentials
   - Used in: `mysql/config.py`
   - Service: `secretsmanager`

2. **AWS Systems Manager Parameter Store (SSM)** - RDS connection details
   - Used in: `mysql/config.py`
   - Service: `ssm`

3. **AWS S3** - Screenshot storage, file uploads, artifacts
   - Used in: `autoclicker/playback.py`, `autoclicker/direct_integration.py`
   - Service: `s3`

4. **AWS Rekognition** - Text detection and image analysis
   - Used in: `autoclicker/rekognition_tiler.py`, `autoclicker/xero_util/attach_files_flow.py`
   - Service: `rekognition`

## Required VPC Endpoints

### 1. S3 Gateway Endpoint (Free)

**Type:** Gateway Endpoint  
**Service:** `com.amazonaws.ap-southeast-2.s3`  
**Route Tables:** Add to all route tables for private subnets  
**Policy:** Default (full access) or custom policy

**Why Gateway Endpoint:**
- S3 supports Gateway Endpoints (free, no hourly charges)
- More cost-effective than Interface Endpoint for S3

**Configuration:**
```bash
# Via AWS Console:
# VPC → Endpoints → Create Endpoint
# Service: com.amazonaws.ap-southeast-2.s3
# VPC: Your VPC
# Route Tables: Select all route tables for private subnets
# Policy: Full access (or custom)
```

### 2. Secrets Manager Interface Endpoint

**Type:** Interface Endpoint  
**Service:** `com.amazonaws.ap-southeast-2.secretsmanager`  
**Subnets:** All private subnets where EC2 instances run  
**Security Groups:** Allow HTTPS (443) inbound from EC2 security group  
**DNS:** Enable private DNS

**Cost:** ~$7.20/month per endpoint + data processing charges

### 3. SSM Interface Endpoint

**Type:** Interface Endpoint  
**Service:** `com.amazonaws.ap-southeast-2.ssm`  
**Subnets:** All private subnets where EC2 instances run  
**Security Groups:** Allow HTTPS (443) inbound from EC2 security group  
**DNS:** Enable private DNS

**Cost:** ~$7.20/month per endpoint + data processing charges

### 4. Rekognition Interface Endpoint

**Type:** Interface Endpoint  
**Service:** `com.amazonaws.ap-southeast-2.rekognition`  
**Subnets:** All private subnets where EC2 instances run  
**Security Groups:** Allow HTTPS (443) inbound from EC2 security group  
**DNS:** Enable private DNS

**Cost:** ~$7.20/month per endpoint + data processing charges

## Step-by-Step Configuration

### Prerequisites

- VPC with private subnets
- EC2 instances in private subnets
- Security groups configured
- Route tables for private subnets

### Step 1: Create S3 Gateway Endpoint

1. Navigate to **VPC Console** → **Endpoints** → **Create Endpoint**
2. **Service category:** AWS services
3. **Service name:** Filter by `s3` → Select `com.amazonaws.ap-southeast-2.s3`
4. **VPC:** Select your VPC
5. **Route tables:** Select all route tables for private subnets
6. **Policy:** Full access (or create custom policy)
7. Click **Create endpoint**

### Step 2: Create Secrets Manager Interface Endpoint

1. Navigate to **VPC Console** → **Endpoints** → **Create Endpoint**
2. **Service category:** AWS services
3. **Service name:** Filter by `secretsmanager` → Select `com.amazonaws.ap-southeast-2.secretsmanager`
4. **VPC:** Select your VPC
5. **Subnets:** Select all private subnets
6. **Security group:** Create or select security group allowing:
   - Inbound: HTTPS (443) from EC2 security group
   - Outbound: All traffic
7. **Enable DNS name:** ✅ Enabled (recommended)
8. Click **Create endpoint**

### Step 3: Create SSM Interface Endpoint

1. Navigate to **VPC Console** → **Endpoints** → **Create Endpoint**
2. **Service category:** AWS services
3. **Service name:** Filter by `ssm` → Select `com.amazonaws.ap-southeast-2.ssm`
4. **VPC:** Select your VPC
5. **Subnets:** Select all private subnets
6. **Security group:** Same as Secrets Manager endpoint
7. **Enable DNS name:** ✅ Enabled (recommended)
8. Click **Create endpoint**

### Step 4: Create Rekognition Interface Endpoint

1. Navigate to **VPC Console** → **Endpoints** → **Create Endpoint**
2. **Service category:** AWS services
3. **Service name:** Filter by `rekognition` → Select `com.amazonaws.ap-southeast-2.rekognition`
4. **VPC:** Select your VPC
5. **Subnets:** Select all private subnets
6. **Security group:** Same as Secrets Manager endpoint
7. **Enable DNS name:** ✅ Enabled (recommended)
8. Click **Create endpoint**

## Security Group Configuration

Create a security group for VPC endpoints with the following rules:

**Inbound Rules:**
- Type: HTTPS
- Protocol: TCP
- Port: 443
- Source: EC2 security group (or specific private subnet CIDR)

**Outbound Rules:**
- Allow all traffic (default)

## Cost Estimation

### Monthly Costs (ap-southeast-2 region):

- **S3 Gateway Endpoint:** $0 (free)
- **Secrets Manager Interface Endpoint:** ~$7.20/month
- **SSM Interface Endpoint:** ~$7.20/month
- **Rekognition Interface Endpoint:** ~$7.20/month
- **Data Processing:** ~$0.01 per GB (minimal for API calls)

**Total:** ~$21.60/month + minimal data processing charges

### Cost Comparison with NAT Gateway:

- **NAT Gateway:** ~$32.40/month + data transfer charges ($0.045/GB)
- **VPC Endpoints:** ~$21.60/month + minimal data processing

**Savings:** ~$10.80/month + significant savings on data transfer

## Verification

### Automatic Usage (No Code Changes Required)

**Important:** Your existing code will automatically use VPC endpoints - no code changes needed!

When boto3 clients are created (e.g., `boto3.client('secretsmanager')`, `boto3.client('s3')`), they automatically route through VPC endpoints if:
- ✅ Code runs from within the VPC
- ✅ VPC endpoints exist and are in "Available" status
- ✅ Private DNS is enabled (default for Interface endpoints)

### Test from EC2 Instance

SSH into your EC2 instance and run these tests:

```bash
# Test Secrets Manager access (should use vpce-08f4f3080a6f4be59)
aws secretsmanager list-secrets --region ap-southeast-2

# Test SSM Parameter Store access (should use vpce-0351808d973bb0cc9)
aws ssm get-parameter --name /rdsDetails --region ap-southeast-2

# Test S3 access (should use vpce-0485d8de60c7f11aa gateway endpoint)
aws s3 ls --region ap-southeast-2

# Test Rekognition access (will use vpce-0685073298f7e4630 once available)
aws rekognition list-collections --region ap-southeast-2
```

### Verify Endpoint Configuration

```bash
# List all VPC endpoints and check their status
aws ec2 describe-vpc-endpoints --region ap-southeast-2

# Check specific endpoint details (replace with your endpoint ID)
aws ec2 describe-vpc-endpoints \
  --vpc-endpoint-ids vpce-08f4f3080a6f4be59 \
  --region ap-southeast-2

# Verify private DNS is enabled (should show "PrivateDnsEnabled: true")
aws ec2 describe-vpc-endpoints \
  --vpc-endpoint-ids vpce-08f4f3080a6f4be59 \
  --query 'VpcEndpoints[0].PrivateDnsEnabled' \
  --region ap-southeast-2
```

### Verify Traffic is Using Endpoints

You can verify traffic is going through endpoints by:
1. Checking CloudWatch metrics for endpoint data processing
2. Monitoring VPC Flow Logs (traffic should show endpoint IPs, not public IPs)
3. Testing connectivity without internet access (disable NAT Gateway temporarily)

## Troubleshooting

### Issue: Cannot access AWS services from EC2

**Check:**
1. VPC endpoints are created and in "available" state
2. Security groups allow HTTPS (443) from EC2
3. Route tables include endpoint routes (for Gateway Endpoint)
4. DNS resolution is working (for Interface Endpoints)
5. IAM role has necessary permissions

### Issue: DNS resolution fails

**Solution:**
- Ensure "Enable DNS name" is checked for Interface Endpoints
- Verify VPC DNS settings: `enableDnsHostnames` and `enableDnsSupport` are enabled

### Issue: Timeout errors

**Check:**
- Security group rules allow traffic
- Network ACLs don't block traffic
- Endpoint is in the same region as resources

## Best Practices

1. **Use Gateway Endpoints for S3** - Free and more efficient
2. **Enable Private DNS** - Simplifies endpoint usage
3. **Use Security Groups** - Restrict access to EC2 security group only
4. **Monitor Costs** - Track data processing charges
5. **Multi-AZ Deployment** - Deploy endpoints in multiple AZs for high availability
6. **Custom Endpoint Policies** - Restrict access to specific resources when possible

## Additional Considerations

### If Using Multiple Regions

If your application accesses AWS services in multiple regions, you'll need VPC endpoints for each region.

### If Using Additional AWS Services

If you add more AWS services (e.g., CloudWatch, KMS), create additional VPC endpoints as needed.

### Alternative: NAT Gateway

If you prefer using NAT Gateway instead:
- More expensive for AWS-to-AWS traffic
- Less secure (traffic goes through internet)
- Simpler setup (no endpoint configuration needed)
- Required if you need general internet access

## References

- [AWS VPC Endpoints Documentation](https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints.html)
- [VPC Endpoint Services](https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-access.html)
- [VPC Endpoint Pricing](https://aws.amazon.com/privatelink/pricing/)

---

**Last Updated:** 2025-01-XX  
**Region:** ap-southeast-2  
**Status:** Required for private subnet deployment