# Project plan — theear_transcode

Evidence date: 2026-10-01.

Source repository: `git@bitbucket.org:overdrive/theear_transcode.git`.

Inspected revision: `master` at `fda49df` (2022-05-04, Jaco Brits, "more clean ups"). The scan-host path `~/projects/sonar_scan/repos/theear_transcode` was not on this machine. The tree below was read from that git revision. No credential values are stored in the repository; database settings are environment-variable names only.

This plan describes what the repository contains. It does not decide what the product should become.

## Product

There is no product requirements document in the source repository or in Silicon Ark before this set. The root `README.md` and `region_ireland/README.md` are the unmodified Anomaly Innovations Serverless Node.js Starter guide. They describe a generic `hello` function and a public demo at `https://z6pv80ao4l.execute-api.us-east-1.amazonaws.com/dev/hello`. That demo is not this service.

What the code implements is two Serverless Framework services for a system identified only by bucket and service names containing `the-ear`:

1. **Ireland** (`region_ireland/`, service `the-ear-production-transcode`, `eu-west-1`) starts an AWS Elastic Transcoder job when an `.mxf` object is created in `the-ear-prod-transcode`.
2. **South Africa** (`region_south_africa/`, service `theear-resources`, `af-south-1`) inserts a row into a MySQL table `resources` when an object of a known type is created in `the-ear-prod-resources`.

A loose sample key in `region_south_africa/Untitled-1.txt` names `Hein_Upright_Bass_Ep2_Harmonic_Minor_ac`. That is the only content clue in the repo. Provenance: **Inferred** that the media is lesson or music material. The product name, owner, and audience are **Unknown / Requires Confirmation**.

## Branch reality

| Line | Tip | Date | What it is |
|---|---|---|---|
| `master` (default; `origin/HEAD`) | `fda49df` | 2022-05-04 | Only branch. 5 commits, all by Jaco Brits |

No other local or remote branches were present at inspection.

| Commit | Date | What changed |
|---|---|---|
| `0a8a007` | 2021-08-15 | `.gitignore` only |
| `6a52440` | 2021-08-15 | Serverless starter, `transcode.js`, `create.js`, `updateuser.js`, mocks, Jest test |
| `ace0778` | 2021-08-25 | "Working version1". `global.js`. Snapshots under `old/` and `old2/`. Active `serverless.yml` and `transcode.js` rewritten |
| `8fdb690` | 2022-05-04 | Split into `region_ireland/` and `region_south_africa/`. Added `index.js`, `lambda_move_transcode_to_resources.js`, `sort_files.js`, `upload.js`, `policy/`, `test.html` |
| `fda49df` | 2022-05-04 | Moved the `old`, `old2`, and `tests` trees under `archived/` in both regions. No content edits |

**Discrepancy:** the task-creation scan listed `sonar-project.properties` and `sonarqube_scan.bat` at the repository root. Neither file is in git history. There is no `bitbucket-pipelines.yml`. Provenance: **Unknown** — they may have been untracked files on the scan host.

## Objectives (inferred from the implementation, not from a written plan)

- Turn an uploaded `.mxf` into an MP4, three HLS variants, an HLS playlist, and thumbnails, using a fixed Elastic Transcoder pipeline in Ireland.
- Record selected objects in `the-ear-prod-resources` as rows in MySQL `resources` (name, S3 URL, type, duration, created time).
- Keep a one-off HTTP backfill (`sort_files`) that lists a staging bucket and inserts rows that are not already present.
- Leave copy IAM policies in `policy/` for moving objects between the transcode bucket, the resources bucket, and `the-ear-prod-video-upload`.

## Scope

### In scope of `master` (wired in the current `serverless.yml` files)

1. Ireland function `transcode_process` on `s3:ObjectCreated:*` for `the-ear-prod-transcode`.
2. South Africa function `transcode` on `s3:ObjectCreated:*` for `the-ear-prod-resources`.
3. South Africa HTTP `GET /sort_files` (no authorizer in the template).
4. HTTP `POST /create` with `aws_iam` in both services. The handler is the Serverless Stack notes example (DynamoDB item with `content` and `attachment`), not a transcode API.
5. Shared libraries, API Gateway 4XX/5XX CORS responses, `package.json`, and `env.example`.

### Present in the tree but not registered as functions

- `region_ireland/index.js` — another Elastic Transcoder job creator.
- `region_ireland/lambda_move_transcode_to_resources.js` — S3 copy from `the-ear-prod-transcode` to `the-ear-prod-resources`.
- `region_ireland/transcode.js` — job creation plus MySQL insert; region in code is `af-south-1`.
- `region_ireland/sort_files.js` and `region_ireland/global.js` — copies of the South Africa files, not in the Ireland template.
- `region_south_africa/transcode_process.js` — commented out in `region_south_africa/serverless.yml`.
- `region_south_africa/upload.js` — reads an object's content type; a signed-upload helper in the same file is not exported as a function entry.
- `global.js`, `logging.js`, `handler.js` (`hello`), empty `libs/images-lib.js`.
- `policy/*.json` — not referenced by either `serverless.yml`.
- `mocks/` — user and permission request fixtures. No active handler reads them.
- `test.html` — a Bootstrap table titled "Saskia Test" that loads `https://jsonplaceholder.typicode.com/posts`.
- `region_*/archived/**` — previous copies, including `generic_transcode.js` (empty) and `updateuser.js`.

### Out of scope / not in this repository

- The Elastic Transcoder pipeline definition (input bucket, output bucket, role). Only pipeline ids are hardcoded.
- The MySQL schema, the host, and any application that reads `resources`.
- A user interface for upload or playback, other than the unrelated `test.html`.
- Deployment history, alarms, or a statement of which AWS account is live.
- Automated tests on the active path. The Jest file was moved under `archived/tests/` and still tests `hello`.

## System architecture

```text
Operator or upstream uploader
        |
        |  object created
        v
s3://the-ear-prod-transcode          region eu-west-1
        |
        |  s3:ObjectCreated:*
        v
Lambda  the-ear-production-transcode / transcode_process
        runtime nodejs12.x, 512 MB, timeout 60s, profile theear_prod
        |
        |  elastictranscoder:CreateJob
        |  pipeline 1635987395753-iukgee
        v
AWS Elastic Transcoder (eu-west-1)
        outputs under key prefix transcode/
        MP4 (preset 1351620000001-000001, Generic 1080p)
        HLS v3 2 Mb/s, 1 Mb/s, 600 kb/s
        playlist {name}.m3u8 and thumbnails
        |
        |  output bucket is set on the pipeline in AWS
        |  Unknown / Requires Confirmation
        v
s3://the-ear-prod-resources           region af-south-1
        |
        |  s3:ObjectCreated:*
        v
Lambda  theear-resources / transcode
        VPC (security group and two subnets from env)
        |
        |  INSERT INTO resources
        v
MySQL   host/db/user from env (DBHOST, DBNAME, DBUSERNAME, DBPASSWORD)

Unwired in serverless.yml, present as source:
  lambda_move_transcode_to_resources.js
        copies one key from the-ear-prod-transcode to the-ear-prod-resources

Separate HTTP, South Africa only:
  GET /sort_files  -> list s3://the-ear-staging-resources -> insert missing rows
  POST /create     -> DynamoDB put (notes starter), table name "loging"
```

The two S3 triggers are not connected in this repository. Joining them into one pipeline depends on the Elastic Transcoder output bucket, or on the copy function being deployed outside this template. That join is **Inferred** and **Requires Confirmation**.

## Major components

| Path | Role |
|---|---|
| `region_ireland/serverless.yml` | Production transcode service, Ireland |
| `region_ireland/transcode_process.js` | Wired job creator for `.mxf` |
| `region_south_africa/serverless.yml` | Production resources service, Cape Town, with VPC and MySQL env |
| `region_south_africa/transcode.js` | Wired catalog insert |
| `region_south_africa/sort_files.js` | Wired staging backfill |
| `region_*/create.js` | Wired notes endpoint from the starter |
| `policy/` | Standalone S3 copy policies |
| `region_*/archived/` | Superseded trees. Not deployed by the current templates |

## Milestones

There is no written milestone list. History implies:

1. **2021-08-15 — starter imported.** Notes `create`, DynamoDB logging helper, permission mocks, and a short `transcode.js`.
2. **2021-08-25 — first working transcode.** Active service still a single tree. `global.js` added. Prior trees kept as `old/` and `old2/`. Archived service name `theear-transcode`, stage `dev`, profile `unity`, bucket `the-ear-transcode-video`.
3. **2022-05-04 — regional split.** Ireland owns job creation against `the-ear-prod-transcode`. South Africa owns the `resources` insert against `the-ear-prod-resources`. Profile becomes `theear_prod`. Stage becomes `production`.
4. **2022-05-04 — archive move.** Historical trees relocated. No behaviour change. Last commit on the repository.

No later commit, tag, or pipeline run is in this repo. Whether the 2022 templates still match what is deployed is **Unknown / Requires Confirmation**.

## Implementation plan

No forward plan is written in the repository. The current templates are the implementation. Deploy, if repeated, is the Serverless Framework command from each region directory (`serverless deploy`), using AWS profile `theear_prod`, stage `production`.

Commented blocks that were not finished:

- HLS AES-128 (`HlsContentProtection`) in every job builder.
- Copy of an `.m3u8` from `the-ear-prod-resources` to `the-ear-prod-video-upload` inside `region_south_africa/transcode.js`.
- South Africa `transcode_process` function, aimed at `the-ear-staging-transcode` in the comment.
- Ireland VPC attachment.

These are commented source, not a backlog.

## Known dependencies

- Node.js 12 Lambda runtime (`nodejs12.x` in both current templates).
- Serverless Framework with `serverless-bundle` `^1.2.5`, `serverless-offline` `^5.3.3`, `serverless-dotenv-plugin` `^2.1.1`.
- `aws-sdk` `^2.706.0` (declared as a devDependency).
- `serverless-mysql` `^1.5.4` and `uuid` `^7.0.3`.
- AWS Elastic Transcoder in `eu-west-1` (pipelines `1635987395753-iukgee` on the wired Ireland path, `1634634986727-4ij4ha` in the unwired South Africa `transcode_process.js` and as a comment in Ireland `transcode.js`).
- S3 buckets named above.
- A MySQL database reachable from the South Africa VPC, table `resources` with columns `name`, `url`, `type`, `duration`, `created`.
- DynamoDB in `eu-west-1` for the notes `create` path and `logging.js`. Table name configured as `loging` on the South Africa service only.
- AWS account profile `theear_prod`. Archived templates used profile `unity`.

## Current project status

Dormant in git since 2022-05-04. One branch. No Bitbucket Pipelines file. No Sonar project file in git. README does not describe this product.

**Unknown / Requires Confirmation:**

- Whether either Serverless service is still deployed, and in which account.
- The Elastic Transcoder pipeline's output bucket and IAM role, and whether jobs still run.
- The MySQL host, schema owner, and which product reads `resources`.
- Whether `lambda_move_transcode_to_resources.js` or `index.js` was deployed by hand.
- Whether `POST /create` and `GET /sort_files` are called by anything.
- What `the-ear` refers to as a product, and who operates uploads.
- Why the task-creation scan saw Sonar files that are not in git.