# Project plan — datafy_rekognition

Evidence date: 2026-10-17.

Source repository: `git@bitbucket.org:overdrive/datafy_rekognition.git` (inspected over HTTPS at the same commit).

Inspected revision: `main` at `1d03653` (2024-06-03, Dirk, "Adding function to detect images"). Full SHA `1d036534c1e34c07588b5b04074146e1e25cd4d1`.

The scan-host path `~/projects/sonar_scan/repos/datafy_rekognition` was not on this machine. The tree below was read from that git revision.

This plan describes what the repository contains. It does not decide what the product should become.

There was no `silicon_ark/datafy_rekognition/` directory before this document set. Silicon Ark already has `datafy_web` and `datafy-supabase-functions`. Neither of those document sets names this repository. Whether they are one product is **Unknown / Requires Confirmation**.

## Product

The repository holds two bodies of work that are not wired together in code.

1. `training/` is a labelled image set of packaged goods. Folder names and Pascal VOC `<name>` values are Rhodes juice, Rhodes jam, and Rhodes Squish infant food. Provenance: **Implemented** (git tree and XML). The business purpose of the set is **Inferred** from those names. No document states a goal.
2. `DatafyRek/` is an AWS SAM application. On an S3 object-created event it starts an Amazon Rekognition Custom Labels model, calls `DetectCustomLabels`, writes the first label name to DynamoDB, and stops the model. Provenance: **Implemented** in `DatafyRek/aws_rekognition/app.py` and `DatafyRek/template.yaml`.
3. Root `readme.md` is a generic TensorFlow 2 object-detection walkthrough (LabelImg, TFRecord, Model Zoo, `model_main_tf2.py`). Provenance: **Documented**. `generate_tfrecord.py` and the training commands are text in the readme. They are not files in this repository.

No product name beyond the repository slug and the SAM description `DatafyRek` is stated. No customer, store chain, or success metric is written down.

## Branch reality

| Line | Tip | Date | What it is |
|---|---|---|---|
| `main` (default; `origin/HEAD`) | `1d03653` | 2024-06-03 | Only branch. 7 commits from 2024-05-28 to 2024-06-03 |

No tags. No other local or remote branches.

History, oldest first:

- 2024-05-28 — Jaco Brits (`jaco@overdrive.co.za`) added a stock Bitbucket `.gitignore` (`eaff27e`, "Initial commit").
- 2024-05-28 — Jaco added `sonar-project.properties` (`c87084a`) and `bitbucket-pipelines.yml` (`152517f`), both created online with Bitbucket.
- 2024-05-28 — Jaco added `training/` (`4124a18`, "update training"): 3,098 files, mostly JPEG images and Pascal VOC XML.
- 2024-05-28 — Jaco added `readme.md` (`b9477c5`, "update readme") and then changed Markdown fence labels (`9673edc`, "update scripts"). The fence edit does not change the commands.
- 2024-06-03 — Dirk (`dirk@overdrive.co.za`) added the SAM application under `DatafyRek/` (`1d03653`, "Adding function to detect images").

No commit after 2024-06-03 was on `origin/main` at inspection. Whether anyone deployed the stack after that date is **Unknown / Requires Confirmation**.

**Discrepancy:** the task-creation scan on 2026-09-28 listed `sonarqube_scan.bat` at the repository root. That path is not in git history. Provenance: **Unknown** — it may have been an untracked file on the scan host.

## Scope in the repository

Tracked layout:

| Path | What it is |
|---|---|
| `.gitignore` | Stock Bitbucket example (Node, Java, Python bytecode, logs, Maven, JetBrains, macOS, Windows, executables, large video and TIFF). |
| `readme.md` | TensorFlow object-detection notes. 265 lines. |
| `sonar-project.properties` | One line: `sonar.projectKey=overdrive_datafy_rekognition_1c209418-751f-4246-8ceb-88eee020eb9d`. |
| `bitbucket-pipelines.yml` | SonarQube scan on `main`, using the `maven:3.3.9` image and pipe `sonarsource/sonarqube-scan:2.0.1`. |
| `DatafyRek/` | SAM app: Lambda `app.py`, `template.yaml`, `samconfig.toml`, a hello-world API event, unit and integration tests that still import `hello_world`. |
| `training/` | 1,520 `.jpg`, 44 `.jpeg`, 1,530 Pascal VOC `.xml`, one YOLO-style `.txt`, `classes.txt`, one `.pptx`, one `.mhtml`. |

Out of scope of this repository, on the evidence: a web UI, a user account model, a Rekognition project or dataset upload script, a DynamoDB table definition, a PostgreSQL schema, a train/test split, and a deployed stack record.

`datafy_web` (separate Silicon Ark set, separate repository) POSTs to API Gateway `POST /analyze-image` in `eu-west-1`. This repository's template defines an S3 event, not that route, and no file here names `analyze-image` or that host. A shared backend is **Unknown / Requires Confirmation**.

## System architecture

```text
training/                         labelled images + Pascal VOC XML
readme.md                         TensorFlow 2 notes (not executed by this repo)

S3 bucket  ObjectCreated:*
    → Lambda DatafyRek/aws_rekognition/app.lambda_handler   (Python 3.9, timeout 900s)
        → Rekognition Custom Labels: start project version
        → Rekognition Custom Labels: DetectCustomLabels (MinConfidence 95)
        → DynamoDB PutItem (first label name + label count)
        → Rekognition Custom Labels: stop project version
```

The S3 bucket, the Rekognition project, the model version, and the DynamoDB table are named only as placeholders or environment variables. They are not created by `template.yaml`, except for an S3 bucket whose `BucketName` is the literal string `<Bucket-name>`.

Provenance: **Implemented** in `template.yaml` and `app.py`. The diagram's AWS account, region, and live ARNs are **Unknown**.

## Major components

| Component | Role | Evidence |
|---|---|---|
| Labelled image corpus | Shelf or pack photos with bounding boxes | `training/` |
| TensorFlow notes | How a reader might train a detector outside this repo | `readme.md` |
| SAM function | Start model, detect custom labels, write DynamoDB, stop model | `DatafyRek/aws_rekognition/app.py` |
| SAM template | Lambda, S3 bucket, S3 event, IAM managed policies, X-Ray | `DatafyRek/template.yaml` |
| SAM config | Stack name `DatafyRek`, `CAPABILITY_IAM`, confirm changeset | `DatafyRek/samconfig.toml` |
| Tests | Still the SAM hello-world samples | `DatafyRek/tests/` |
| Sonar pipeline | Scan `main` | `bitbucket-pipelines.yml` |

## Milestones visible in git

These are commit dates, not an agreed plan.

| When | What landed | Status in git |
|---|---|---|
| 2024-05-28 | Ignore file, Sonar project key, pipeline | On `main` |
| 2024-05-28 | Training images and XML | On `main` |
| 2024-05-28 | TensorFlow readme | On `main` |
| 2024-06-03 | Rekognition Lambda and SAM template | On `main` |
| After 2024-06-03 | Nothing further on `main` | No later commit |

The comment `# replace with postgress function` above `write_dynamodb` is the only in-repo note of a later storage change. Provenance: **Documented** as a comment. It is not a scheduled milestone.

## Implementation plan as written

The readme's order is: Python venv, TensorFlow / OpenCV / matplotlib, LabelImg in Pascal VOC, a `dataset/images` plus `dataset/annotations` layout, an inline `generate_tfrecord.py`, a Model Zoo config, 200,000 training steps, export, then OpenCV drawing when score is above 0.5.

The SAM app's order, in code, is different: S3 event, start Custom Labels, detect, DynamoDB, stop.

Neither document says which path is the one to operate. **Unknown / Requires Confirmation.**

## Known dependencies

| Dependency | Where | Pinned |
|---|---|---|
| Python 3.9 Lambda runtime | `template.yaml` | Runtime family only |
| `boto3`, `requests` | `DatafyRek/aws_rekognition/requirements.txt` | No versions. `requests` is not imported by `app.py` |
| `pytest`, `boto3`, `requests` | `DatafyRek/tests/requirements.txt` | No versions |
| AWS SAM CLI, Docker, Python 3 | `DatafyRek/README.md` | SAM sample text |
| Amazon Rekognition Custom Labels, S3, DynamoDB, IAM, X-Ray | `app.py`, `template.yaml` | Service APIs, not package versions |
| TensorFlow, OpenCV headless, matplotlib, LabelImg, pandas, Pillow, TensorFlow Models | `readme.md` commands and the pasted script | No versions |
| SonarQube pipe `sonarsource/sonarqube-scan:2.0.1` on image `maven:3.3.9` | `bitbucket-pipelines.yml` | Pipe version pinned. Maven image is not used by any Maven file in the repo |
| `SONAR_HOST_URL`, `SONAR_TOKEN` | Pipeline variables | Values not in git |

## Current project status

`main` is the whole product history. The training corpus and the Lambda were both committed. The Lambda template cannot be deployed as written until `<Bucket-name>` is replaced, and the function reads four environment variables the template never sets (`project_arn`, `model_arn`, `version_name`, `dynamo_table_name`). The unit test imports a `hello_world` package that is not in the tree.

No open branch, tag, or release was found. Operational status (deployed or not, model trained or not) is **Unknown / Requires Confirmation**.