# Technical documentation — datafy_rekognition Evidence date: 2026-10-17. Revision: `main` `1d036534c1e34c07588b5b04074146e1e25cd4d1` (2024-06-03). Remote: `https://bitbucket.org/overdrive/datafy_rekognition.git` (SSH remote `git@bitbucket.org:overdrive/datafy_rekognition.git`). ## Architecture Two trees in one repository. `training/` and `readme.md` are data and notes. `DatafyRek/` is a SAM application whose handler calls Rekognition and DynamoDB. No import or path in `app.py` reads `training/`. ```text datafy_rekognition/ main @ 1d03653 ├── .gitignore stock Bitbucket template, eaff27e ├── readme.md TensorFlow notes, b9477c5 / 9673edc ├── sonar-project.properties project key only, c87084a ├── bitbucket-pipelines.yml SonarQube on main, 152517f ├── training/ images, XML, pptx, mhtml, 4124a18 └── DatafyRek/ SAM app, 1d03653 ├── template.yaml ├── samconfig.toml ├── README.md unmodified SAM hello-world narrative ├── events/event.json API Gateway hello-world event ├── aws_rekognition/ │ ├── app.py lambda_handler │ └── requirements.txt boto3, requests └── tests/ ├── unit/test_handler.py imports hello_world ├── integration/test_api_gateway.py └── requirements.txt ``` There is no `Dockerfile`, no `package.json`, no `pom.xml`, and no infrastructure module besides `template.yaml`. ## Languages and frameworks | Item | Evidence | Provenance | |---|---|---| | Python 3.9 | `Runtime: python3.9` in `template.yaml` | Implemented | | AWS SAM / CloudFormation | `Transform: AWS::Serverless-2016-10-31` | Implemented | | boto3 | `requirements.txt` and `boto3.client` in `app.py` | Implemented. Version unpinned. Lambda's runtime also provides boto3; the file does not say which copy is intended | | requests | Both requirements files | Declared. Not imported by `app.py` or the unit test | | pytest | `tests/requirements.txt` and `import pytest` | Declared. Tests do not target `aws_rekognition.app` | | TensorFlow, OpenCV, matplotlib, LabelImg, pandas, Pillow | `readme.md` | Documented install lines and the pasted script. Not in a requirements file | `DatafyRek/aws_rekognition/__init__.py`, `DatafyRek/__init__.py`, and the test package `__init__.py` files are empty. ## Application structure ### Lambda handler `DatafyRek/aws_rekognition/app.py` (125 lines). Handler `app.lambda_handler`. | Function | AWS call | Notes | |---|---|---| | `start_model` | `rekognition.start_project_version`, waiter `project_version_running`, `describe_project_versions` | Exceptions printed, not raised | | `show_custom_labels` | `rekognition.detect_custom_labels` with `Image.S3Object` | Returns `(CustomLabels, len)` | | `stop_model` | `rekognition.stop_project_version` | Exceptions printed | | `write_dynamodb` | `dynamodb.put_item` | Low-level client, attribute values `S` and `N` | | `lambda_handler` | Orchestrates the four, then returns a 200 JSON body | `finally` always calls `stop_model` | Commented and unused: `from PIL import Image, ImageDraw, ExifTags, ImageColor, ImageFont`, and a call to `display_image`, which is not defined. Clients are `boto3.client('rekognition')` and `boto3.client('dynamodb')` with no `region_name`. Region follows the Lambda environment or the default boto3 chain. The committed region is **Unknown**. ### Template `DatafyRek/template.yaml`: | Setting | Value | |---|---| | Description | `DatafyRek` / `Sample SAM Template for DatafyRek` | | Globals function timeout | 900 | | Globals memory | 1028 | | Globals tracing | `Active` | | Globals logging | `LogFormat: JSON` | | Globals API | `TracingEnabled: true` (no API resource in the template) | | Function | `AWSRekognitionFunction`, `CodeUri: aws_rekognition/`, `Handler: app.lambda_handler` | | Policies | `AmazonS3FullAccess`, `AmazonRekognitionFullAccess` | | Architecture | `x86_64` | | Event | `AWSRekognitionEvent`, type `S3`, `s3:ObjectCreated:*`, bucket `SrcBucket` | | Bucket | `AWS::S3::Bucket`, `BucketName: ` | | Outputs | Function ARN and implicit role ARN `AWSRekognitionFunctionRole` | Not declared: environment variables, DynamoDB table, Rekognition project, dead-letter queue, reserved concurrency, VPC, or an explicit `AWS::Lambda::Permission` (SAM adds the S3 invoke permission when the event is expanded). The last point is SAM behaviour, **Inferred** from the `Type: S3` event. It was not deployed in this inspection. `samconfig.toml` sets `stack_name = "DatafyRek"`, `capabilities = "CAPABILITY_IAM"`, `confirm_changeset = true`, `resolve_s3 = true`, cached parallel build, lint on validate. It does not set `region`, `profile`, `s3_bucket`, or `parameter_overrides`. ### Sample event and tests `events/event.json` is the SAM hello-world API Gateway proxy event (`path` `/hello`, `httpMethod` `GET` at the top level and `POST` inside `requestContext`). It has no `Records`. `tests/unit/test_handler.py` does `from hello_world import app` and expects `data["message"] == "hello world"`. `tests/integration/test_api_gateway.py` reads `AWS_SAM_STACK_NAME`, calls `cloudformation.describe_stacks`, and requires output key `HelloWorldApi`. The template does not emit that key. The test then `GET`s the URL and expects `{"message": "hello world"}`. `DatafyRek/README.md` still documents `sam local invoke HelloWorldFunction`, `sam local start-api` on port 3000, `sam logs -n HelloWorldFunction --stack-name "datafyrek"`, and `sam delete --stack-name "datafyrek"`. The config file's stack name is `DatafyRek`, not `datafyrek`. ## Data ### Training tree File counts on `1d03653` (working tree matches the commit; no untracked source files): | Extension | Count | |---|---:| | `.xml` | 1,530 | | `.jpg` | 1,520 | | `.jpeg` | 44 | | `.txt` | 2 (`classes.txt` and one sidecar) | | `.pptx` | 1 | | `.mhtml` | 1 | Top directories and file counts: | Path | Files | |---|---:| | `training/100% JUICE` | 2,307 | | `training/INFANT FOOD` | 583 | | `training/JAM` | 191 | | `training/JAM TRAINING` | 16 | | `training/RFG CATALOGUE 2024.pptx` | 1 | Second level under juice: `YOLANDA 2L RHODES JUICE` (924), `MUSI 6X200ML RHODES JUICE` (756), `SHANNON 1L RHODES JUICE` (626), plus the MHTML file at `training/100% JUICE/6X200ML FRUIT MEDLEY - OneDrive.mhtml`. XML is Pascal VOC. A representative file (`.../MUSI 6X200ML APPLE/017543bf-fd8c-4c38-883e-5b8139b764be.xml`) has ``, ``, a Windows `` under `windows_v1.8.1/TRAIN/`, `` width 1200 height 1600 depth 3, `0`, and `` with ``, `Unspecified`, ``, ``, and `` in pixels. The `` element is the folder name at label time. It is not always the git directory name. Distinct `` strings and their file counts are listed in the evidence inventory used for this document (136 `YOLANDA JUICE APPLE & CRANBERRY 2L` down to 1 `RHODES APPLE JUICE 1L` and 1 `Downloads`). Label strings and box counts are in `prd.md` FR-002. Same-stem pairing: 1,512 images have a matching XML stem, 52 images do not, 18 XML files do not have a matching image. 16 of those 18 XML files are `training/JAM TRAINING/`. `classes.txt` line order: ```text dog person cat tv car meatballs marinara sauce tomato soup chicken noodle soup french onion soup chicken breast ribs pulled pork hamburger cavity rhodes_mixed_fruit_jam_900g rhodes_smooth_apricot_jam_900g ``` The sidecar `.txt` uses class id `16` on every line, which is the last name if numbering starts at 0. Provenance of that convention: **Inferred**. Image filenames are either UUIDs (`.jpg`) or `WhatsApp Image 2024-...` names (`.jpeg`). Dates visible in filenames include March and April 2024. That is filename text, not a complete capture log. ### DynamoDB item shape (written, table not defined) ```text key S S3 object key predicted S first CustomLabels Name label_count N stringified integer count ``` No TTL, no bucket attribute, no confidence, no box, no timestamp. ### Environment variables read by the handler | Name | Used for | |---|---| | `project_arn` | Waiter and `describe_project_versions` | | `model_arn` | `start_project_version`, `detect_custom_labels`, `stop_project_version` | | `version_name` | Waiter `VersionNames` | | `dynamo_table_name` | `PutItem` `TableName` | None are in `template.yaml`. How they are supplied in an account is **Unknown / Requires Confirmation**. ## External integrations See `prd.md` integrations table. Endpoint hosts are the AWS APIs for the client's region. No URL is hard-coded. Rekognition operations actually called: - `StartProjectVersion` - waiter `project_version_running` - `DescribeProjectVersions` - `DetectCustomLabels` - `StopProjectVersion` Not called: `CreateProject`, `CreateDataset`, `StartProjectVersion` training APIs such as `CreateProjectVersion` / train, `DetectLabels` (the non-custom API), or S3 `GetObject` from this function. Rekognition reads the object from S3 itself. ## Infrastructure and deployment Documented deploy path in `DatafyRek/README.md`: `sam build --use-container` then `sam deploy --guided`. `samconfig.toml` already contains non-guided defaults, including `confirm_changeset = true`, so a later `sam deploy` still asks before executing the change set. Provenance: **Implemented** config. Blockers visible in the template, without a deploy attempt: - `BucketName: ` is not a legal S3 name. - Missing environment variables cause `KeyError` on invoke. - No DynamoDB policy on the generated role. AWS account, region, stack status, and whether a stack named `DatafyRek` exists are **Unknown / Requires Confirmation**. This inspection did not call AWS. ## CI, quality, security `bitbucket-pipelines.yml`: - `image: maven:3.3.9` - `clone.depth: full` - Branch filter `'{main}'` only - Step name `SonarQube analysis` - Pipe `sonarsource/sonarqube-scan:2.0.1` - Variables `SONAR_HOST_URL` and `SONAR_TOKEN` (comments say repository or workspace variables; do not commit the token) - Cache `sonar: ~/.sonar` `sonar.projectKey=overdrive_datafy_rekognition_1c209418-751f-4246-8ceb-88eee020eb9d`. No `sonar.sources`, exclusions, or language setting. A full clone includes the training images. Whether Sonar indexes them is **Unknown**. `sonarqube_scan.bat` is not in the repository. The 2026-09-28 task scan listed it. Provenance of that file: **Unknown**. Security facts in the tree: - No `.env` file is tracked. `DatafyRek/.gitignore` ignores `.env`. - No embedded AWS keys were found in `app.py`, the template, or `samconfig.toml`. - The pipeline comment forbids a clear-text `SONAR_TOKEN`. - IAM policies are full S3 and full Rekognition (see business rules). - X-Ray tracing is on for the function. - The sample event contains example IPs (`127.0.0.1`), a sample account id `123456789012`, and a sample CloudFront id. Those are the public SAM sample, not this system's production identifiers. - Training XML `` values contain a local Windows user folder `C:/Users/User/...`. That username is the literal `User`. ## Development and testing No `Makefile`, no tox, no coverage config, no pre-commit config. Commands the SAM readme gives (they target the hello-world names): ```text sam build --use-container sam deploy --guided sam local invoke HelloWorldFunction --event events/event.json pip install -r tests/requirements.txt --user python -m pytest tests/unit -v AWS_SAM_STACK_NAME="datafyrek" python -m pytest tests/integration -v ``` The unit command cannot succeed on this tree: `hello_world` is not a package. That conclusion is from reading the import. Pytest was not executed during this documentation pass. The readme at the repository root is a workstation procedure (venv name `myenv`, `pip install tensorflow opencv-python-headless matplotlib`, `pip install labelImg`, clone TensorFlow models, `python generate_tfrecord.py`). None of those tools are pinned. ## Monitoring Template: AWS X-Ray active on the function, JSON Lambda logs. No alarms, no dashboard, no CloudWatch metric filter, and no application log sink beyond what Lambda writes with `print`. `start_model` and `stop_model` print status strings. The handler prints the label count and the output dict. `sam logs` is documented only for `HelloWorldFunction`. ## What this tree does not contain - A Rekognition project ARN or a trained model artifact. - A dataset manifest linking `training/` to Custom Labels. - PostgreSQL, despite the comment on `write_dynamodb`. - An HTTP API matching `datafy_web`'s `POST /analyze-image`. - Tests that exercise `lambda_handler` with an S3 event. - Any commit after 2024-06-03.