# PRD — terraform_scripts

Evidence date: 2026-10-01.

Source: `git@bitbucket.org:overdrive/terraform_scripts.git`, `main` at `8af8aa6` (2026-10-01).

There is no prior PRD in the source repository or in Silicon Ark. Requirements below are recovered from the Terraform, the two READMEs, the architecture JPG, and git history. Provenance is labelled. Passwords, tokens, and the contact phone number are omitted.

## Product purpose

**Documented.** `README.md` describes the repository as Datafy AWS infrastructure, with a production environment, three stacks (networking, baseline, datafy), and internal modules. It tells an operator how to add an environment by copying a directory, changing `backend.tf`, and editing `.auto.tfvars`.

**Implemented.** `environments/production/main.tf` is the only root module. It instantiates `stacks/networking` and `stacks/datafy`. The datafy stack instantiates `stacks/baseline`. Variable values in `production.auto.tfvars` fix the client to `datafy`, the stage to `prod`, and the region to `eu-west-1`.

**Inferred.** The infrastructure exists to host three private HTTP services (Supabase, Water Wheel, Qwen) and a static web portal for Datafy, with a shared PostgreSQL database and an ML detection Lambda. The repository does not contain those applications.

**Unknown / Requires Confirmation:** which team is allowed to apply the Terraform Cloud workspace, and whether any environment other than production is planned.

## Users and roles

| Actor | Evidence | Status |
|---|---|---|
| Terraform operator | README "Adding an environment". Git authors: Jaco Brits, Mohamed Hoosain Ebrahim, Dirk van Rensburg, tanya | **Documented** as the person who copies an environment and edits tfvars. No IAM role for that person is declared here |
| AWS account alternate contacts | `common.auto.tfvars`: security, operations, and billing contacts are the same record (John Fourie, Managing Director, `john@datafy.co.za`) | **Implemented** via `aws_account_alternate_contact` when the email is non-empty |
| Budget and alert mailboxes | Addresses in `environments/production/common.auto.tfvars` | **Implemented** as SNS and Budget subscribers. See business rules |
| Internet client | Public ALB ingress `0.0.0.0/0` on 80 and 443. Diagram labels "Users, Mobile client" | **Implemented** for the ALB. The diagram's mobile client is **Documented** only |
| Workload on EC2 | Instance profile `datafy-prod-server-profile`, SSM core policy, Secrets Manager read of `rds!db-*` | **Implemented**. There is no per-application role split |
| Detection Lambda | Role `datafy-prod-lambda-role` | **Implemented** as infrastructure. The function body in this repo is a placeholder |

No human login, no application RBAC, and no customer tenant model are defined in this repository.

## Systems the product talks to

1. Terraform Cloud, organisation `Datafy-AWS-Deploy`, workspace `Production`.
2. AWS in `eu-west-1` for almost every resource, and `us-east-1` for the CloudFront ACM certificate and the CloudFront WAF (`aws.us-east-1` provider alias).
3. An existing S3 bucket `datafy-prod-webportal` (data source, not created here).
4. An existing issued ACM certificate `*.waterwheel.datafy.co.za`.
5. Canonical's public AMI catalogue for Ubuntu 22.04.
6. AWS-managed WAF rule groups, Security Hub standards (FSBP 1.0.0 and CIS 1.4.0), and the AWS Backup service role.

The diagram also draws Amazon API Gateway, Amazon SES, and AWS Step Functions. No `aws_api_gateway_*`, `aws_ses_*`, or `aws_sfn_*` resource exists in the tree. Provenance of the drawing: **Documented**. Provenance of those three services in code: **Not implemented**.

## Major workflows

### WF-1 — Apply production

**Trigger.** An operator runs Terraform against the production root. The backend block selects Terraform Cloud workspace `Production`.

**Inputs.** `common.auto.tfvars` and `production.auto.tfvars` in `environments/production`. `environments/common.tfvars` is a sibling file and is not auto-loaded.

**Behaviour.** Networking is created first. The datafy module receives VPC, subnet, and security-group outputs, then creates baseline, data, compute, edge, and DNS-related resources.

**Output.** AWS resources in the target account. `stacks/datafy/outputs.tf` is empty, so the root module publishes no outputs.

**Error conditions.** Missing key pairs, a missing web-portal bucket, or a missing waterwheel certificate will fail the plan or apply. Those dependencies are data sources or name references, not resources this module creates.

Provenance: **Implemented** for the module graph. The exact apply command and required Terraform version are **Unknown** (no `required_version`, no pipeline).

### WF-2 — Reach a private service by hostname

**Trigger.** A client sends HTTP or HTTPS to the public load balancer.

**Behaviour.**

- Port 80 redirects to 443 with status `HTTP_301`.
- Port 443 uses TLS 1.3 policy `ELBSecurityPolicy-TLS13-1-2-2021-06`, certificate `*.datafy.co.za` from `module.acm-alb`, plus the looked-up `*.waterwheel.datafy.co.za` certificate.
- Host `waterwheel.datafy.co.za` or `*.waterwheel.datafy.co.za` forwards to target group `water-wheel-supabase-server` (priority 10).
- Host `supabase.datafy.co.za` forwards to `datafy-prod-supabase-server` (priority 20).
- Host `qwen.datafy.co.za` forwards to `qwen3-supabase-server` (priority 30).
- Any other host gets a fixed response: plain text `default response`, status 200.
- `preserve_host_header` is true. Idle timeout is 180 seconds. Deletion protection is on. Access logs and connection logs go to the ALB access-logs bucket, prefix `alb`.

**Validation.** Listener rules match the Host header only. There is no path condition on these three rules.

Provenance: **Implemented** in `stacks/datafy/load_balancers.tf`.

### WF-3 — Serve the web portal

**Trigger.** A client requests `webportal.datafy.co.za`.

**Behaviour.** CloudFront distribution `datafy-prod-cdn` uses the existing S3 bucket as origin, default root object `index.html`, viewer protocol redirect to HTTPS, price class 200, HTTP/2 and HTTP/3, IPv6 off. S3 error 403 and 404 are rewritten to HTTP 200 and `/index.html`. A CloudFront-scope WAF is attached. The regional pgAdmin exceptions are not passed into this WAF.

Provenance: **Implemented** in `stacks/datafy/cdn_and_waf.tf` and `modules/aws-so-cdn`.

### WF-4 — Boot an EC2 host

**Trigger.** Terraform creates an instance because `create_asg` is false for every entry in `local.ec2`.

**Behaviour.** The module looks up the newest Ubuntu 22.04 AMI, builds a launch template (IMDSv2 required, gp3 encrypted volume, `delete_on_termination = false`), and creates one `aws_instance` per entry. User data enables `codedeploy-agent` and switches `amazon-ssm-agent` to the snap candidate channel, then starts it. The instance lifecycle ignores later launch-template changes.

**Output.** A running instance registered to an HTTP target group on the instance's port.

**Not in this workflow.** Installing Supabase, Water Wheel, or Qwen. No CodeDeploy application, deployment group, or revision is declared.

Provenance: **Implemented** in `modules/aws-so-ec2` and `modules/aws-so-ec2/user_data.txt`.

### WF-5 — Database access

**Trigger.** A client inside the VPC connects to PostgreSQL.

**Behaviour.** RDS Proxy `datafy-prod-rds-proxy` requires TLS, authenticates with the RDS-managed master secret (`iam_auth = DISABLED`), pools up to 90 percent of connections, and targets instance `datafy-prod-postgres`. The database is not publicly accessible, is Multi-AZ, uses gp3, and has deletion protection and a final snapshot.

**Who may connect.** Security groups allow the EC2 security group to reach the proxy and the database on 5432, and the Lambda security group to reach the proxy on 5432. Lambda is not granted a direct path to the database security group. The database does not accept the public ALB security group.

Provenance: **Implemented**.

### WF-6 — Detection Lambda

**Trigger.** Something invokes `datafy-prod-detection`. This repository does not declare an event source, API, or schedule.

**Behaviour.** Terraform creates the function with a generated zip whose handler returns HTTP 200 and the body `placeholder`, then ignores `filename` and `source_code_hash`. Runtime `python3.12`, memory 3008, timeout 900 seconds, X-Ray active, VPC-attached to private subnets index 6 and 7. Environment variable `ML_BUCKET_NAME` is `datafy-prod-ml-image-scan`. An alarm fires when errors exceed 5 in 60 seconds.

Provenance: **Implemented** for the shell. The real detector is **Unknown** and is expected to be deployed outside Terraform.

## Functional requirements

### FR-001 — Single production root

The only environment directory is `environments/production`. It must call networking and the datafy stack, and it must not embed resource blocks of its own.

Evidence: `environments/production/main.tf`. Implementation: **Implemented**. Confidence: High.

### FR-002 — Production identity

The production variable file sets region `eu-west-1`, client `datafy`, stage `prod`, VPC name `datafy-prod-vpc`, CIDR `10.1.0.0/16`, public DNS `datafy.co.za`, private DNS `internal.datafy.co.za`, and public alias `www`. Monthly budget limit is `1800` (USD, from the account-settings module default unit).

Evidence: `environments/production/production.auto.tfvars`, `modules/aws-account-settings`. Implementation: **Implemented**. Confidence: High.

The private zone module that would create `internal.datafy.co.za` is commented out in `stacks/datafy/dns_and_certs.tf`. The string is passed in and not used to create a zone. Provenance: **Implemented** as an unused variable. Creating that zone is **Not implemented**.

### FR-003 — Network shape

The VPC module creates public, private, and database subnets, one NAT gateway, DNS hostnames, and DNS support. A database subnet group is requested (`create_database_subnet_group = true` in the production auto tfvars). Flow logs and an S3 gateway endpoint are attached.

Named private subnets, in order: supabase a/b, waterwheel a/b, qwen a/b, lambda a/b. Named database subnets: db-a, db-b. Named public subnets: public-subnet-a, public-subnet-b.

Evidence: `stacks/networking/networking.tf`, `production.auto.tfvars`. Implementation: **Implemented**, with a list-length mismatch. See FR-016.

### FR-004 — Three compute hosts

| Local name | Type | Port | Volume | Subnet index | Health check |
|---|---|---|---|---|---|
| `datafy-prod-supabase-server` | `t3.xlarge` | 8000 | 100 GB | 0 | matcher `200,401`, path `/` |
| `water-wheel-supabase-server` | `c7i.2xlarge` | 80 | 60 GB | 1 | path `/.well-known/alb-health`, matcher `200` |
| `qwen3-supabase-server` | `g4dn.xlarge` | 80 | 50 GB | 1 | path `/`, matcher `200` |

All three set `create_asg = false`, desired capacity 1, and share `ec2_private_sg`. User data is the same file for all three.

Evidence: `stacks/datafy/locals.tf`. Implementation: **Implemented**. Confidence: High.

Qwen uses a GPU instance type. Nothing in this repo installs NVIDIA drivers or a model. Provenance: **Inferred** from the instance type and the name.

### FR-005 — Host-header routing

The public ALB routes the three hostnames in WF-2 and answers every other host with the fixed 200 response. The ALB is internet-facing (`internal = false`) even though variables `alb_internal`, `alb_internal_name`, and `alb_internal_preserve_host_header` exist and are passed into the stack. Those variables are not read by any resource.

Evidence: `stacks/datafy/load_balancers.tf`, `stacks/datafy/variables.tf`. Implementation: **Implemented** for the public listener. The internal ALB variables are **Not implemented**. Confidence: High.

### FR-006 — PostgreSQL

One database: name `datafy-prod-postgres`, engine PostgreSQL, family `postgres15`, engine version `15` (major only, so AWS may apply minor upgrades), class `db.t3.xlarge`, database name `appdb`, username `postgres`, port 5432, storage 100 GB growing to 500 GB, Multi-AZ, Performance Insights (7-day retention in the module), backup retention 7 days, backup window `00:00-01:00`, maintenance `Mon:01:00-Mon:02:00`, encrypted with a dedicated rotating KMS key, master password managed by RDS (`manage_master_user_password = true`).

Evidence: `stacks/datafy/locals.tf`, `modules/aws-so-rds/rds.tf`. Implementation: **Implemented**. Confidence: High.

### FR-007 — RDS Proxy

One proxy, engine family `POSTGRESQL`, TLS required, IAM auth disabled, idle timeout 1800 seconds, secret from the RDS master user secret, subnets = all database subnets, security group `rds-proxy-sg`.

Evidence: `stacks/datafy/locals.tf`, `modules/aws-so-rds-proxy/proxy.tf`. Implementation: **Implemented**. Confidence: High.

### FR-008 — Application buckets

Two buckets are created: `datafy-prod-ml-image-scan` and `datafy-prod-supabase-storage`. Versioning on, SSE-S3, public access blocked, `force_destroy = false`, bucket owner preferred, and a policy that denies non-TLS requests. The EC2 role may read and write the Supabase bucket. The Lambda role may read and write the ML bucket.

Evidence: `stacks/datafy/locals.tf`, `modules/aws-so-s3/s3.tf`, `stacks/datafy/ec2.tf`, `stacks/datafy/lambda.tf`. Implementation: **Implemented**. Confidence: High.

A third bucket, `datafy-prod-server-artifacts`, is created inside the EC2 module with `force_destroy = true`.

### FR-009 — Web portal CDN

CloudFront serves `webportal.datafy.co.za` from the pre-existing bucket `datafy-prod-webportal`, with the SPA error rewrite described in WF-3. Certificate is `*.datafy.co.za` issued in `us-east-1`.

Evidence: `stacks/datafy/cdn_and_waf.tf`. Implementation: **Implemented**, depending on a bucket this module does not create. Confidence: High.

### FR-010 — WAF

Two web ACLs:

- Regional `datafy-prod-waf`, associated with the public ALB. `pgadmin_host` is `pgadmin.waterwheel.datafy.co.za`, which turns on two allow rules and shifts managed-rule priorities.
- CloudFront-scope `datafy-prod-waf-cdn` in `us-east-1`, associated with the distribution. `pgadmin_host` is left empty, so the allow exceptions are off.

Default action is allow. Managed groups: Amazon IP reputation, common rule set, known bad inputs, Linux, SQLi, Unix. IP rate limit is 1000 requests per IP. `SizeRestrictions_BODY` is overridden to count. Several other common, Linux, and SQLi rules are explicitly set to block.

When the pgAdmin host is set, the common rule set skips `POST` URIs starting with `/storage/v1/object/`, and the SQLi rule set skips that host combined with URIs starting with `/sqleditor/`. A separate allow rule permits `POST` URIs starting with `/storage/v1/object/Reports/`.

Evidence: `modules/aws-so-waf/waf.tf`, `stacks/datafy/cdn_and_waf.tf`. Implementation: **Implemented**. Confidence: High.

### FR-011 — Account baseline

The datafy stack calls baseline, which enables Security Hub with FSBP and CIS 1.4, GuardDuty, AWS Config (delivery every six hours, global resource types, three conformance packs), a multi-region CloudTrail with log-file validation, and account settings.

Account settings include alternate contacts, an IAM password policy, S3 public-access block, default EBS encryption, block of new public AMI sharing, block of all public snapshot sharing, a monthly cost budget, an encrypted SNS topic for technical alerts, three AWS Backup plans selected by tag `env=backup`, and CloudWatch alarms for root usage and CloudTrail configuration changes.

Evidence: `stacks/baseline/baseline.tf`, `modules/aws-account-settings/settings.tf`. Implementation: **Implemented**. Confidence: High.

### FR-012 — Detection function shell

The Lambda described in WF-6 must exist with the placeholder zip and `ignore_changes` on the code attributes, so a later out-of-band publish is not reverted by Terraform.

Evidence: `modules/aws-so-lambda/lambda.tf`, `modules/aws-so-lambda/data.tf`. Implementation: **Implemented**. Confidence: High.

### FR-013 — Adding an environment

**Documented** in `README.md`: copy an environment directory, change the workspace name in `backend.tf`, and edit `.auto.tfvars`.

**Not implemented** beyond that instruction. No staging or poc directory exists. `stage` is described as `poc`, `staging`, or `prod`, and production sets `prod`.

### FR-014 — Certificates

`module.acm-alb` requests `*.${public_dns}` (`*.datafy.co.za`) in the workload region. `module.acm-cdn` requests the same name in `us-east-1`. Validation method comes from the SSL module variable (not overridden in the stack call). The waterwheel wildcard is not requested; it is selected with `data.aws_acm_certificate`.

Evidence: `modules/aws-so-ssl/acm.tf`, `stacks/datafy/dns_and_certs.tf`. Implementation: **Implemented**. Confidence: High.

DNS validation records and a public hosted zone are not created in the active code. The private zone is commented out. How ACM validation completes is **Unknown / Requires Confirmation**.

### FR-015 — Tags

Datafy-stack tags include `Environment=prod`, `Project=datafy`, `ManagedBy=terraform`, `Terraform=true`, `env=backup`, and `aws-apn-id`. Networking tags are the same set without `env`. The RDS module strips `env` before tagging the database so AWS Backup does not select it. EBS volumes on the launch template set `env=backup` explicitly.

Evidence: `stacks/datafy/locals.tf`, `stacks/networking/locals.tf`, `modules/aws-so-rds/rds.tf`, `modules/aws-so-ec2/main.tf`. Implementation: **Implemented**. The comment in the RDS module states the intent. Confidence: High.

### FR-016 — Subnet list lengths

`production.auto.tfvars` lists 3 public CIDRs and 9 private CIDRs. `stacks/networking/networking.tf` lists 2 public names and 8 private names. Availability zones are only `eu-west-1a` and `eu-west-1b`. The ALB uses the first two public subnets. Lambda uses private subnet indexes 6 and 7. The Lambda NACL attaches from index 6 through the end of the private subnet list.

Whether the VPC module accepts unequal name and CIDR lists was not executed in this investigation. Provenance: **Requires Confirmation**.

### FR-017 — Unused inputs

`peer_owner_id` and `peer_vpc_id` are variables on the environment and on the datafy stack. No peering resource reads them. No values are set in the auto tfvars (defaults are empty strings).

Provenance: **Implemented** as unused variables. VPC peering is **Not implemented**.

## Integrations

| System | Direction | Trigger | Data | Auth | Failure |
|---|---|---|---|---|---|
| Terraform Cloud | Both | Operator plan/apply | State, variables | Workspace credentials, not in git | Apply does not run. **Unknown** who is notified |
| AWS APIs | Outbound from Terraform | Apply | Resources listed above | Provider credentials outside git | Terraform error |
| ACM (existing waterwheel cert) | Read | Plan/apply | Certificate ARN | AWS credentials | Apply fails if no issued cert matches |
| S3 `datafy-prod-webportal` | Read | Plan/apply | Bucket domain | AWS credentials | Apply fails if the bucket is absent |
| SNS email | Outbound | Budget and alarms | Alert messages | AWS SNS subscription | Endpoint `software-alerts@overdrive.co.za` must confirm the subscription. Confirmation state is **Unknown** |
| Canonical AMI | Read | Plan | AMI id | Public | Apply fails if the filter matches nothing |

## Constraints

- AWS provider `~> 6.0` at the production root.
- RDS engine version is the major version `15` so Terraform does not pin a minor that AWS has already upgraded.
- Lambda code in this repository must stay a placeholder; real code is ignored after create.
- EC2 instances ignore launch-template drift after create.
- ALB and RDS have deletion protection. RDS does not skip the final snapshot.
- WAF log buckets expire objects after 365 days.
- No automated test and no Bitbucket pipeline are in git.

## Error and edge behaviour

- Unmatched HTTPS hosts receive HTTP 200 and the body `default response`, not a 404. Provenance: **Implemented**.
- Supabase target-group health treats HTTP 401 as healthy. Provenance: **Implemented**.
- Security group NACLs allow all protocols to and from `0.0.0.0/0`. Segmentation is by security group, not by NACL. Provenance: **Implemented**.
- Lambda has no ingress rules. It can initiate egress (`all-all`) and can reach RDS Proxy because the proxy allows the Lambda security group. Provenance: **Implemented**.
- The EC2 IAM statement `EC2MetadataSelfManage` names one instance ARN (`i-0e3ee87bcaac5d0fd` in account `926753675240`). It does not cover the three instances this module creates. Provenance: **Implemented**. Whether that instance is one of the three hosts is **Unknown**.
- `stacks/datafy/outputs.tf` is empty. Callers cannot read stack outputs from the root module. Provenance: **Implemented**.