# Project plan — terraform_scripts

Evidence date: 2026-10-01.

Source repository: `git@bitbucket.org:overdrive/terraform_scripts.git`.

Inspected revision: `main` at `8af8aa6` (2026-10-01, Dirk van Rensburg, "update backup"). The scan-host path `~/projects/sonar_scan/repos/terraform_scripts` was not on this machine. The tree below was read from that git revision. Secret values are omitted. The phone number stored in the tfvars is omitted.

This plan describes what the repository contains. It does not decide what the product should become.

## Product

The repository README titles the work **Datafy AWS Infrastructure**. It is a Terraform project that declares one production environment for a client named `datafy` in AWS region `eu-west-1` (Europe, Ireland). The root module is `environments/production`. It builds a VPC and then a "datafy" stack: account baseline, three Ubuntu EC2 hosts, a Multi-AZ PostgreSQL database and RDS Proxy, two application S3 buckets, a public application load balancer, regional and CloudFront web ACLs, a CloudFront distribution for an existing web-portal bucket, and a placeholder Lambda named for ML image detection.

Provenance: **Documented** in `README.md` (short). The resource set is **Implemented** in `environments/production` and `stacks/datafy`.

There is no product requirements document in the source repository or in Silicon Ark before this set. A diagram file, `datafy project redeployment.jpg`, is titled "Datafy AWS Architecture Production Environment". It is **Documented** intent and disagrees with the code in several places recorded below.

Application source for Supabase, Water Wheel, Qwen, the web portal, and the detection function is not in this repository.

## Branch reality

| Line | Tip | Date | What it is |
|---|---|---|---|
| `main` (default; `origin/HEAD`) | `8af8aa6` | 2026-10-01 | Only branch. 46 commits from 2026-06-08 to 2026-10-01 |

No other local or remote branches were present at inspection.

History falls into three stretches:

- 2026-06-08 — Jaco Brits, initial commit.
- 2026-06-19 to 2026-07-10 — Mohamed Hoosain Ebrahim built the VPC, baseline, EC2-only layout (ECS and cache removed), S3, Lambda, RDS Proxy, CloudFront, WAF, and host-based ALB rules. Dirk van Rensburg changed instance sizes, target groups, security groups, CloudFront, and S3 permissions through 2026-07-08.
- 2026-07-31 — tanya: SSL certificate, instance, EBS volume, path, timeout, and host-header updates (`a0b9be3`).
- 2026-10-01 — Dirk van Rensburg: RDS backup tag exclusion (`e184981`), extra WAF allow rules (`85eee8f`), and a follow-up backup change (`8af8aa6`).

**Discrepancy:** the onboarding task recorded the tip as `a0b9be3` (2026-07-31, "host header update"). `main` has moved three commits past that snapshot. This set documents `8af8aa6`.

**Discrepancy:** the task-creation scan listed `sonar-project.properties` at the repository root. That path is not in `git ls-tree` and has no commit in `git log --all`. Provenance: **Unknown** — it may have been an untracked file on the scan host.

## Objectives (inferred from the implementation and the README, not from a written plan)

- Stand up a production VPC in `eu-west-1` for Datafy shared services.
- Run three private EC2 workloads (Supabase, Water Wheel, Qwen) behind one internet-facing load balancer, selected by Host header.
- Provide PostgreSQL with automated RDS backups, and AWS Backup for other resources tagged `env=backup`, while keeping the database itself off the AWS Backup selection.
- Publish `webportal.datafy.co.za` from an S3 bucket that this repository does not create.
- Turn on account security controls: Security Hub, GuardDuty, AWS Config, CloudTrail, password policy, public-access blocks, budgets, and contact records.
- Leave a Lambda shell for ML image detection whose real code is deployed outside this repository.

## Scope

### In scope of `main`

1. Terraform Cloud backend for organisation `Datafy-AWS-Deploy`, workspace `Production` (`environments/production/backend.tf`).
2. Production variable files `environments/production/common.auto.tfvars` and `production.auto.tfvars`.
3. Stacks `stacks/networking`, `stacks/baseline`, and `stacks/datafy`.
4. Reusable modules under `modules/`, including several that no stack calls (see Features).
5. Operator notes in `README.md` and `environments/production/README.md`.
6. Architecture diagram `datafy project redeployment.jpg`. `.gitignore` lists `.jpg`, but this file is still tracked.

### Out of scope / not in this repository

- Application code, container images, and CodeDeploy applications. User data only enables the CodeDeploy agent and the SSM agent.
- The web-portal bucket object contents. Terraform reads bucket `datafy-prod-webportal`; it does not create it.
- The detection Lambda body. Terraform uploads a Python placeholder and ignores later code changes.
- The ACM certificate `*.waterwheel.datafy.co.za`. Terraform looks up an already issued certificate.
- EC2 key pairs. Launch templates reference names that this repository does not create.
- A second environment. The README describes how to copy one. Only `environments/production` exists.
- Pipelines, tests, and SonarQube configuration in git.
- API Gateway, SES, and Step Functions. They appear on the diagram and have no resource in this tree.

## System architecture

```text
Internet
   |  HTTP 80 -> 301 to HTTPS
   |  HTTPS 443, Host header selects the target
   v
Public ALB (datafy-prod-alb-public) + regional WAFv2
   |-- waterwheel.datafy.co.za, *.waterwheel.datafy.co.za -> water-wheel EC2 :80
   |-- supabase.datafy.co.za -> supabase EC2 :8000
   |-- qwen.datafy.co.za -> qwen EC2 :80
   |-- any other host -> fixed 200 "default response"
   v
Private subnets (single NAT) in eu-west-1a / eu-west-1b
   |-- EC2 Ubuntu 22.04 (no Auto Scaling Group; create_asg = false)
   |-- Lambda datafy-prod-detection (placeholder code) in the lambda subnets
   |-- RDS Proxy (TLS required) -> PostgreSQL 15 Multi-AZ in database subnets
   |-- S3 gateway VPC endpoint

CloudFront (us-east-1 certificate and CLOUDFRONT WAF)
   origin: existing bucket datafy-prod-webportal
   alias: webportal.datafy.co.za
   S3 403/404 rewritten to /index.html

Account baseline (called from stacks/datafy)
   Security Hub, GuardDuty, Config, CloudTrail, budgets, AWS Backup, SNS alerts
```

State is remote in Terraform Cloud. AWS credentials are not stored in the repository. Provenance: **Implemented** for the resources above. Whether the last apply matches `8af8aa6` is **Unknown / Requires Confirmation**.

## Milestones visible in git

There is no written milestone plan. The commits themselves are the record:

| When | What landed |
|---|---|
| 2026-06-08 | Initial commit |
| 2026-06-19 | DNS names required for Route 53 / ACM; `common.auto.tfvars` so Terraform Cloud auto-loads shared values |
| 2026-06-21 | EC2-only security groups; S3 gateway endpoint; Lambda subnets; ML and Supabase buckets; detection Lambda; RDS Proxy; CloudFront and WAF; Security Hub FSBP and CIS 1.4 |
| 2026-06-22 | Missing EC2 instances, S3 name clashes, and ALB subnet availability-zone clash fixed; target groups; host routing for waterwheel, supabase, and qwen |
| 2026-06-23 to 2026-07-10 | NACLs, Secrets Manager on the EC2 role, volume and database size changes, port 8000, WAF body-size override, CloudFront index redirect, tags |
| 2026-07-31 | Public ALB `preserve_host_header = true` |
| 2026-10-01 | RDS excluded from the `env=backup` AWS Backup selection; regional WAF allow rules for pgAdmin and a storage upload path |

## Dependencies

| Dependency | Where | Notes |
|---|---|---|
| Terraform Cloud | `backend.tf` | Organisation `Datafy-AWS-Deploy`, workspace `Production` |
| hashicorp/aws | `environments/production/providers.tf` | `~> 6.0`. Stacks require `>= 5.0` |
| terraform-aws-modules/vpc/aws | networking stack | `~> 5.2.0` |
| terraform-aws-modules/security-group/aws | networking stack | `~> 5.1.2` |
| terraform-aws-modules/alb/aws | datafy stack | `~> 9.10.0` |
| terraform-aws-modules/rds/aws | RDS module | `6.9.0` |
| cloudposse/awsutils and awscc | account settings | EBS and snapshot public-access controls |
| Canonical AMI owner `099720109477` | `stacks/datafy/ec2.tf` | Ubuntu 22.04 Jammy, most recent |
| Existing ACM cert | `dns_and_certs.tf` | `*.waterwheel.datafy.co.za`, status `ISSUED` |
| Existing S3 bucket | `cdn_and_waf.tf` | `${client}-${stage}-webportal` |
| Existing EC2 key pairs | `modules/aws-so-ec2/main.tf` | Name pattern `${service_name}-${instance_name}-kp` |

## Current status

**Implemented** on `main`, as code. One production environment only. No open feature branch in this clone.

The in-repo READMEs are stale relative to the code:

- `README.md` says `stacks/datafy` is the shared-services application stack and that it calls `stacks/baseline`. The call is true (`stacks/datafy/baseline.tf`).
- `environments/production/README.md` says the environment calls `stacks/shared-services`. That stack directory does not exist.

`environments/common.tfvars` is not auto-loaded. Terraform Cloud loads `common.auto.tfvars` from the production directory. The two files disagree (database subnet group, budget subscriber list, SNS address).

## Known gaps requiring confirmation

- Whether Terraform Cloud workspace `Production` has applied `8af8aa6`.
- Whether the three extra public and private CIDRs that have no matching `*_subnet_names` entry still plan cleanly.
- Who owns the application releases that replace the Lambda placeholder and the EC2 user-data bootstrap.
- Whether API Gateway, SES, and Step Functions on the diagram are still desired or were dropped when the code moved to EC2 and CloudFront.
- The hardcoded EC2 instance ARN in the server IAM policy (see technical documentation).