# Features — datafy-lambda-new

Evidence date: 2026-10-17. Source: `main` at `9519bea`.

Status words:

- **Current** — a resource in the default `template.yaml` for that folder (`sam deploy` uses `template.yaml` unless told otherwise).
- **Code present, not in the default template** — handler exists; `template.yaml` does not define it, or the resource is commented out.
- **Documented only** — described in a markdown file and not implemented as a handler here.
- **Stale documentation** — the file disagrees with the current template or handler.
- **Planned** — only if the repository says the work is future. The migration markdown files are plans for a move, not shipped features.

No product backlog or removed-feature register exists. "Removed" below means a template event or call that is commented out while the code remains.

## Current — datafyNew stack

Stack name in `lambda/datafy_new/samconfig.toml`: `datafyNew`. Template: `lambda/datafy_new/template.yaml`.

| Feature | What it does | Where |
|---|---|---|
| SJ import trigger | POST accepts a call, emails that import started, async-invokes Read Files, returns 202 | `trigger_read_files/app.py`, route `/trigger-read-files` |
| SJ spreadsheet import | Polls `sjreport_uploads`, fuzzy-matches columns, upserts rows, runs product and customer RPCs, always marks the upload processed | `read_files/app.py` |
| Post-import handoff | Requires week and record id, starts Create Tasks Bulk, marks the upload processed. In-function product-id SQL is commented out | `batch_update_sjreport/app.py` |
| Bulk task build | Pages SJ data, writes 5,000-row JSON to S3 under a timestamped prefix, starts Step Functions | `create_tasks_bulk/app.py` |
| Task insert workflow | Pass-through or S3 list, Map upsert into `task`, then finalize | `create_tasks_list_batches/`, `insert_tasks_batch/`, `create_tasks_finalize/`, `CreateTasksWorkflow` |
| In-process task build | Same upsert inside one Lambda. No API event. Import chain does not call it | `create_tasks/app.py` |
| Process queue | Customer-id RPC, archive, wipe and refill `store_processing_queue`, start backup | `start_process_queue/app.py`, `FLOWCHART.md` |
| SJ backup | CSV of old SJ rows to Storage `uploads/backups`, then clear | `backup_sjreport/app.py` |
| SJ purge | RPC `clear_old_sjreport_records`, then task backup | `sjreport_clear_old_data/app.py` |
| Task backup | CSV of old task rows, then clear | `backup_tasks/app.py` |
| Task purge | RPC `clear_old_task_records` | `task_clear_old_data/app.py` |
| Promo/KaChing import trigger | Start email, async invoke, HTTP 202 | `trigger_read_promos_kaching/app.py`, route `/trigger-read-promos-kaching` |
| Promo/KaChing import | One `.xlsx` from `promo_kaching_uploads` into the table named by the storage path | `read_promo_kaching/app.py` |
| Combined weekly report | Paged RPC, Excel in Storage `Reports`, row in `report_files`, cron-gated email | `create_combined_weekly_report/app.py`, route `/generate-combined-week-report`, cron 03:00 UTC |
| KaChing daily report | RPC to Excel in `Reports`, cron 03:00 UTC, customer id 1. No API | `create_kaching_daily_report/app.py` |
| Store issue refresh | Paged store RPC, issue counts, store update, totals. No trigger in the template | `update_store_issues/app.py` |
| Task status refresh | RPC `update_task_status_with_temp` for `week_start_date`. No trigger in the template | `update_task_issues/app.py` |
| Shared Python layer | requests, supabase, numpy, pandas, openpyxl, rapidfuzz | `lambda_layer/requirements.txt`, `CommonPythonLayer` |
| Operator email | SMTP via SSM Gmail parameters. Admin list and client list | `email_service.py` copies under the functions that send mail |
| Staging bucket | S3 `batches/` expire after 1 day | `CreateTasksStagingBucket` |
| Application Insights | Auto-configured for the stack | `ApplicationInsightsMonitoring` |

## Current — bedrock-analysis stack

Stack name: `datafy-bedrock-analysis`. Template: `lambda/bedrock_analysis/template.yaml`.

| Feature | What it does | Where |
|---|---|---|
| Production shelf analysis | POST `/analyze-image`. Claude Sonnet 4.6 profile, Supabase read/write, optional S3 artifacts, token log | `app/app.py` |
| Throttle fallback | One extra InvokeModel after primary backoff, using `BEDROCK_FALLBACK_INFERENCE_PROFILE_ARN` | `app/app.py`. Default parameter value is the same profile as primary |
| Dry-run test API | POST `/analyze-image-test`. Default model `qwen.qwen3-vl-235b-a22b` via Converse. No DB or S3 writes | `app/test_app.py` |
| Daily retry workflow | List incomplete tasks fixed today SAST with a shelf URL, Map the production analyser, paginate | `list_retry_tasks/app.py`, `BedrockRetryWorkflow` |
| Retry schedule | EventBridge 20:00 UTC daily when `EnableBedrockRetrySchedule=true` (default) | `BedrockRetryScheduleRule` |
| Batch workflow | Group the same window by `product_id`, one Claude call per chunk of up to 6 shelves | `list_bedrock_batch_groups/`, `bedrock_batch_analyze/`, `BedrockBatchWorkflow` |
| Past-day backfill | Execution input `run_date` (`YYYY-MM-DD`, SAST) is passed into the list Lambdas | Template Choice/Seed states; README |
| Artifacts | Shelf image, product image, prompt, response under `artifacts/{id}/{timestamp}/` when the bucket parameter is non-empty. Default name `datafy-bedrock-artifacts` | `save_artifacts_to_s3`, template condition `HasArtifactsBucket` |
| CORS | POST and OPTIONS, origin `*` | `BedrockAnalysisApi` |

## Current — processes stack

Stack name: `processes`.

| Feature | What it does | Where |
|---|---|---|
| VOC crop on XML upload | S3 `ObjectCreated` for `.xml` crops the sibling `.jpg` into label folders | `get_training_images/app.py` as logical id `ConvertTrainingImages` |
| Training bucket | SAM bucket referenced by that event | `TrainingS3Bucket` |

## Code present, not in the default template

| Feature | Where | Note |
|---|---|---|
| XML to Rekognition JSON | `processes/convert_xml_to_json/app.py` | Resource commented out in `processes/template.yaml` |
| Root SJ trigger without email | `lambda/trigger_read_files/app.py` | Expects `READ_FILES_ARN`. No template at repo root |
| Older SJ reader | `lambda/datafy_new/sj_read_files/app.py` | Not a resource in `template.yaml`. Calls `update_product_table` and a create-tasks invoke |
| Duplicate datafy template | `lambda/datafy_new/template copy.yaml` | Not what `sam deploy` uses by default |
| Rekognition dataset, train, and inference functions | `lambda/Rekognition/functions/*/app.py` | Defined in `template-copy.yaml`, not in `template.yaml` |
| Rekognition state machines | `lambda/Rekognition/statemachine/` | In the copy template |
| SMS auth-hook adapter | `docs/supabase-auth-hook-send-sms-lambda.md` | Documented handler sketch. No function directory |
| One-minute promo poll | Commented `Events` on `ReadPromosKaching` | Replaced by the HTTP trigger |
| Rekognition daily and weekly schedules | Commented in `template-copy.yaml` | README still says they run |

## Stale documentation

| Document | What it says | What the code or template says |
|---|---|---|
| `lambda/datafy_new/README.md` | Hello World sample, `sam logs -n HelloWorldFunction` | Stack is the Datafy import functions. A short weekly-report section was appended |
| `lambda/bedrock_analysis/README.md` opening | Claude 3 Vision | Template profile `eu.anthropic.claude-sonnet-4-6` |
| Same README, artifacts | Default is empty (disabled) | Parameter default is `datafy-bedrock-artifacts` |
| Same README, positions | left, center, or right | Changelog and prompt support center-left and center-right as well |
| `lambda/Rekognition/README.md` | `template.yaml` defines the functions and enabled schedules | `template.yaml` defines buckets, SSM parameters, and a role. Schedules in the copy are comments |
| Root `readme.md` | JS processors under `datafy_lambda\file_processing\` | Those files are not in this repository |
| Root `package.json` | Node dependencies including Azure MSAL | No application JavaScript in the tree |
| Onboarding inventory (2026-09-28) | `sonar-project.properties` at repo root | Not in the tree at `9519bea` |

## Changed features (from CHANGELOG, through 2026-04-23)

These are recorded as already changed on the tip that contains the changelog. They are not a separate unreleased branch.

| Area | Change |
|---|---|
| Read Files | Failed imports set `processed=true`. Duplicate product inserts (`23505`) are ignored. PostgREST read timeout 300s. Per-RPC try/except. Batch size 1000 |
| Create Tasks | Bulk path via S3 and Step Functions. Batch keys include a timestamp so reruns do not overwrite. `BatchUpdateSjreport` invokes `CreateTasksBulk` |
| Weekly report | Service-role key. Email only for cron or an explicit flag. Pagination 10,000 |
| Bedrock | Model moved to Sonnet 4.6. Exact product and size checks. Five horizontal positions. EXIF transpose. Token usage log. Retry and batch state machines. Qwen test route. Fallback profile parameter. Retry cron moved from 22:00 UTC to 20:00 UTC (22:00 SAST) |
| List retry | Task table only, `task_id` items, non-empty `image_url`, explicit `fixed_at IS NOT NULL` |

## Not a product feature

| Item | Note |
|---|---|
| `lambda/datafy_new/tests/` | SAM sample API Gateway event. Not a product test suite. Not executed for this document |
| `events/event.json` and `events/uploadEvent.json` | Local `sam local invoke` payloads |
| `datafy-rekognition.code-workspace` | VS Code workspace file pointing at a Rekognition folder. It does not deploy anything |
| `docs/scripts/` | Shell and Python helpers for a Supabase delta load and S3 key remap (`retry_failed_delta.sh`, `load_delta.sh`, `remap_storage_fast.py`). Operational scripts, not Lambda features |

## Planned

`lambda/datafy_new/migration.md` and `lambda/bedrock_analysis/migration.md` list what must change to deploy into a new AWS account and a new Supabase project (account id, hardcoded Lambda ARNs, VPC parameters, storage calls if files move to S3). They state that the files themselves contain no code changes. Execution status is **Unknown / Requires Confirmation**.

No other planned feature list was found.