# PRD — theear_transcode

Evidence date: 2026-10-01.

Source: `git@bitbucket.org:overdrive/theear_transcode.git`, `master` at `fda49df` (2022-05-04, Jaco Brits, "more clean ups").

There is no prior PRD in the source repository or in Silicon Ark. Requirements below are recovered from the implementation. Provenance is labelled. Environment files with secrets are gitignored (`.env`, `env.yml`). `env.example` contains one sample variable and no secrets.

## Product purpose

**Implemented.** When an `.mxf` object lands in S3 bucket `the-ear-prod-transcode` (Ireland, `eu-west-1`), a Lambda creates an AWS Elastic Transcoder job that writes an MP4, three HLS renditions, an HLS playlist, and thumbnails under the key prefix `transcode/`.

**Implemented.** When an object with a recognised extension lands in S3 bucket `the-ear-prod-resources` (Cape Town, `af-south-1`), a Lambda inserts one row into MySQL table `resources`.

**Inferred.** The two behaviours are meant to be one pipeline: transcode in Ireland, then catalog the outputs in South Africa. The repository does not set the pipeline output bucket, and the copy function that would bridge the buckets is not in `serverless.yml`. **Requires Confirmation.**

**Unknown / Requires Confirmation.** The business name, the audience, and whether this still runs. Bucket names say `the-ear`. One checked-in S3 URL refers to `Hein_Upright_Bass_Ep2_Harmonic_Minor_ac`.

The checked-in READMEs do not state this purpose. They are the Serverless Node.js Starter manual.

## Users and interactions

No login, role, or end-user screen for transcode exists in this repository.

| Actor | What they do | Evidence | Status |
|---|---|---|---|
| Uploader | Creates an object in an S3 bucket. Identity is not read from the event | S3 `ObjectCreated` handlers | **Inferred** |
| HTTP caller with AWS IAM | `POST /create` on either service | `serverless.yml` `authorizer: aws_iam` | **Implemented** |
| Anonymous HTTP caller | `GET /sort_files` on the South Africa service. No authorizer key | `region_south_africa/serverless.yml` | **Implemented** |
| Deploy operator | Serverless deploy with profile `theear_prod` | `provider.profile` | **Inferred** |

`create.js` comments say the user id is a Cognito Identity Pool id. The route authorizer is `aws_iam`, not a Cognito authorizer. **Requires Confirmation** whether any client calls this route.

`test.html` is a static page titled "Saskia Test". It loads public placeholder posts. It is not linked from the Serverless services.

## Systems the product talks to

1. Amazon S3. Buckets named in active or policy files: `the-ear-prod-transcode`, `the-ear-prod-resources`, `the-ear-prod-video-upload`, `the-ear-staging-resources`. Commented or archived names: `the-ear-staging-transcode`, `the-ear-transcode-video`, `the-ear-resources`.
2. AWS Elastic Transcoder in `eu-west-1`. Wired pipeline id `1635987395753-iukgee`.
3. MySQL through `serverless-mysql`, from the South Africa functions (and from unwired copies). Table `resources`.
4. Amazon DynamoDB `DocumentClient`. Table name `process.env.tableName`. The South Africa template sets that to `loging`. IAM is `arn:aws:dynamodb:eu-west-1:*:*`.
5. API Gateway, for `POST /create` and `GET /sort_files`, plus default 4XX and 5XX CORS responses in `resources/api-gateway-errors.yml`.

No email, queue, SNS publish, or third-party HTTP API is called. IAM allows `sns:List*` and `iam:List*`. Nothing in the handlers uses those actions.

## Major workflows

### WF-1 — Transcode an MXF

Trigger: `s3:ObjectCreated:*` on `the-ear-prod-transcode`, function `transcode_process` in `region_ireland/serverless.yml`. Existing bucket (`existing: true`).

For each record, if the key ends with `.mxf`:

1. Take the stem as the key with the first `.` and everything after it removed.
2. Take the file name as the last `/` segment of that stem.
3. Call `elastictranscoder.createJob` in `eu-west-1`.
4. Pipeline `1635987395753-iukgee`. `OutputKeyPrefix` `transcode/`.
5. Input key is the S3 key as received. Frame rate, resolution, aspect ratio, interlaced, and container are `auto`.
6. Outputs:
   - `{stem}/{file}.mp4` with thumbnail pattern `{stem}/{file}-thumbnail-{resolution}-{count}`, preset `1351620000001-000001` (comment: Generic 1080p).
   - `{stem}/{file}_2mb.ts`, segment duration `2`, preset `1351620000001-200010` (comment: HLS v3 2 Mb/s).
   - `{stem}/{file}_1mb.ts`, preset `1351620000001-200030` (comment: HLS v3 1 Mb/s).
   - `{stem}/{file}_600kb.ts`, preset `1351620000001-200040` (comment: HLS v3 600 kb/s).
7. One playlist, format `HLSv3`, name `{stem}/{file}.m3u8`, output keys the three `.ts` names.
8. On success, log the job id and call `waitFor('jobComplete')` without a promise or callback. The function returns `{ statusCode: 200, body: { transcode_job: "" } }` from inside `forEach`, which does not set the Lambda result.

Keys that do not end in `.mxf` are ignored.

Provenance: **Implemented** in `region_ireland/transcode_process.js`.

`region_ireland/index.js` builds the same job from `event.Records[0]` only, URI-decodes `+` to spaces, and exports `handler` rather than `main`. It is not in `serverless.yml`. Provenance: **Implemented** as source, **not wired**.

### WF-2 — Catalog an object in the resources bucket

Trigger: `s3:ObjectCreated:*` on `the-ear-prod-resources`, function `transcode` in `region_south_africa/serverless.yml`. Timeout 60 seconds. The function runs in the VPC declared on that service.

For each record, classify by lowercase extension:

| Key ends with | `type` stored | `url` stored |
|---|---|---|
| `.m3u8` | `m3u8` | `s3://the-ear-prod-resources/` + full key |
| `.mxf` | `mxf` | same, full key |
| `.pdf` | `pdf` | same |
| `.mid` | `midi` | same |
| `.png` | `png` | same |
| `.mp3` | `mp3` | same |
| `.mp4` | `mp4` | same |

`duration` is `"0"`. Other extensions produce no insert.

If `url` is longer than 2 characters, the function runs:

```text
INSERT INTO resources (name, url, type, duration, created)
VALUES ("{file}", "{url}", "{type}", "{duration}", "{UTC timestamp YYYY-MM-DD HH:MM:SS}")
```

`name` is the last path segment of the stem (text before the first `.`). There is no existence check. Errors are logged. The MySQL connection is closed after the loop is started, not after the inserts finish (`forEach` with async callbacks).

Provenance: **Implemented** in `region_south_africa/transcode.js`.

The `.m3u8` branch used to copy the object to `the-ear-prod-video-upload` using an `eu-west-1` client. That copy is commented out. Provenance: **not active**.

### WF-3 — Backfill the catalog from the staging bucket

Trigger: `GET /sort_files` on the South Africa API. CORS is `false`. No authorizer.

The handler lists objects in `the-ear-staging-resources` (not the prod resources bucket) with a single `listObjects` call, then for `.m3u8`, `.pdf`, `.mid`, `.png`, and `.mp3`:

1. `SELECT * FROM resources WHERE name="{file}" AND type="{type}"`.
2. If the result length is 0, `INSERT` the same columns as WF-2.

`.m3u8` URLs are built as `s3://the-ear-staging-resources/transcode/{stem}/{file}.m3u8`, not as the listed key. Other types use `s3://the-ear-staging-resources/` + the listed key. `duration` is `"0"`. `.mp4` and `.mxf` are not classified here.

Provenance: **Implemented** in `sort_files.js`, wired only in `region_south_africa/serverless.yml`.

### WF-4 — Create a note (starter endpoint)

Trigger: `POST /create` on both services, CORS enabled, authorizer `aws_iam`.

The body is JSON. The handler writes a DynamoDB item: `userId` from `event.requestContext.identity.cognitoIdentityId`, `noteId` UUID v1, `content`, `attachment`, `createdAt` epoch milliseconds. Table name is `process.env.tableName`. Success returns the item with CORS headers. Failure returns `{ status: false }` and HTTP 500.

The South Africa template sets `tableName` to `loging` and passes `DBHOST`, `DBUSERNAME`, `DBPASSWORD`, `DBNAME`. The Ireland template sets no environment block, so `tableName` is unset unless a local `.env` injects it. `create.js` does not use MySQL.

Provenance: **Implemented**. This is the starter notes handler. It does not create a transcode job or a `resources` row.

### WF-5 — Copy one transcoded object to the resources bucket

`region_ireland/lambda_move_transcode_to_resources.js` reads `event.Records[0]`, copies that key from `the-ear-prod-transcode` to `the-ear-prod-resources` with an `af-south-1` S3 client, and logs success or failure. It does not delete the source and does not invoke the callback. It is not in `serverless.yml`.

`policy/theear-lambda-transcode-resources.json` allows list on both buckets, get on `the-ear-prod-transcode/*`, put on `the-ear-prod-resources/*`. `policy/theear-lambda-resources-video-upload.json` allows the same pattern from `the-ear-prod-resources` to `the-ear-prod-video-upload`. Neither JSON file is referenced by Serverless.

Provenance: **Implemented** as files. Attachment to a role is **Unknown**.

## Functional requirements

### FR-001 — Start a transcode job for MXF

The Ireland service creates an Elastic Transcoder job when the object key ends with `.mxf`.

- Actor: S3 event on `the-ear-prod-transcode`.
- Inputs: S3 record key. Pipeline id and presets are constants.
- Behaviour: WF-1.
- Output: a created job. The Lambda response body does not contain the job id.
- Validation: extension suffix only. No size, content-type, or checksum check.
- Errors: `createJob` errors are logged. The `waitFor` call is not chained to the callback.
- Dependencies: Elastic Transcoder pipeline `1635987395753-iukgee` in `eu-west-1`.

Evidence: `region_ireland/transcode_process.js`, `region_ireland/serverless.yml`.

Implementation: **Implemented**. Confidence: **High** for the code path. **Unknown** whether the pipeline still exists.

### FR-002 — Write a resources row for a known object

The South Africa service inserts a `resources` row for the extensions in WF-2.

- Actor: S3 event on `the-ear-prod-resources`.
- Inputs: object key. Database settings from the environment.
- Behaviour: WF-2. Every matching event inserts. Duplicate keys are not suppressed.
- Output: a MySQL insert. The S3 invocation ignores the HTTP-shaped return from `handler-lib`.
- Validation: lowercase suffix. `url` must be longer than 2 characters, which the constructed URLs are.
- Errors: query failures are logged and do not fail the Lambda via `throw`.
- Dependencies: MySQL table `resources` and VPC network path.

Evidence: `region_south_africa/transcode.js`, `region_south_africa/serverless.yml`.

Implementation: **Implemented**. Confidence: **High** for the code. The live database is **Unknown**.

### FR-003 — Backfill from staging without duplicating name and type

`GET /sort_files` inserts a row only when `name` and `type` are not already in `resources`.

- Actor: HTTP GET. No authorizer in the template.
- Inputs: none from the query string. The bucket name is constant `the-ear-staging-resources`.
- Behaviour: WF-3.
- Output: inserts for missing pairs. The HTTP body is `JSON.stringify` of `undefined` when the handler returns nothing (`handler-lib`).
- Validation: same suffix checks. List is one page (`listObjects`, not `listObjectsV2`, no `ContinuationToken`).
- Errors: query errors are logged.
- Dependencies: MySQL and S3 list on the staging bucket.

Evidence: `region_south_africa/sort_files.js`.

Implementation: **Implemented**. Confidence: **High**.

### FR-004 — IAM-protected notes create

`POST /create` stores a DynamoDB note item as in WF-4.

Evidence: `region_*/create.js`, both `serverless.yml` files.

Implementation: **Implemented**. It is unrelated to transcode. Confidence: **High** that the route is declared. **Unknown** whether clients use it.

### FR-005 — Ignore non-MXF uploads on the transcode bucket

The wired Ireland handler does not transcode, copy, or catalog keys that do not end with `.mxf`.

Evidence: `region_ireland/transcode_process.js`.

Implementation: **Implemented**. Confidence: **High**.

### FR-006 — Do not encrypt HLS in this configuration

Every job builder contains an `HlsContentProtection` block (AES-128) that is commented out. Jobs created by this code do not set content protection.

Evidence: `transcode_process.js`, `index.js`, `region_ireland/transcode.js`.

Implementation: **Implemented** as the absence of protection. A decision to encrypt later is **not documented**.

## Constraints

- Lambda memory 512 MB. Transcode and resources handlers timeout at 60 seconds. `create` and `sort_files` use the provider default timeout (not set in the template).
- Runtime `nodejs12.x`. Stage name `production`. Region is per service, not per function.
- South Africa functions are in a VPC. Security group and subnet ids come from `SECURITYGROUPIDS`, `SUBNETIDS1`, `SUBNETIDS2`. Ireland VPC settings are commented out.
- Ireland `serverless.yml` does not grant `dynamodb:*` and does not set `tableName`, `databaseHost`, or the other database variables. South Africa does.
- South Africa IAM includes `elastictranscoder:*Job` and `*Preset` but not `elastictranscoder:Create*`. The wired South Africa handler does not call Elastic Transcoder. Ireland IAM includes `elastictranscoder:Create*` as well as `*Job`.
- Both services allow `s3:GetObject` and `s3:PutObject` on `arn:aws:s3:::*`.
- `package.individually: true`. Plugins: `serverless-bundle`, `serverless-offline`, `serverless-dotenv-plugin`. `useDotenv: true`.
- Filename stem is the substring before the first `.`, so extra dots in the name are dropped.
- Suffix checks are case-sensitive. `.MXF` does not match `.mxf`.
- `duration` is never read from the media. Catalog rows store `"0"` or, in the unwired Ireland `transcode.js` MXF path, an empty string.
- Inserts concatenate the file name and URL into SQL text.

## Behaviour that is easy to misread

- `region_ireland/transcode.js` still creates a job and inserts a row, including an `.m3u8` URL of the form `s3://the-ear-prod-resources/transcode/{stem}/{file}.m3u8`, and it sets the AWS region to `af-south-1`. It is not a function in the Ireland template. Elastic Transcoder regional support for `af-south-1` is not configured here.
- `region_south_africa/transcode_process.js` uses pipeline `1634634986727-4ij4ha`, points non-video URLs at `the-ear-staging-resources`, and has the MySQL inserts commented out. The function entry in `serverless.yml` is commented, with bucket `the-ear-staging-transcode`.
- `global.js` builds `SELECT`, `UPDATE`, or a bare `INSERT INTO` from JSON fields `a`, `c`, `t`, then appends `LEFT JOIN` and `WHERE` from `data.con` when `data.j.length()` or `data.w.length()` is large. `length` is not a function, and `con` is not in the comment's field list. The file is not a Serverless function. Provenance: **Implemented** source, **not wired**.
- `logging.js` writes the event, query, and result to DynamoDB. Only `global.js` calls it. The `put` callback takes a single argument.
- `upload.js` `getUploadURL` signs a `putObject` for `{random}.jpg` on `process.env.UploadBucket` and is not referenced. `UploadBucket` is not in `serverless.yml`. The exported `main` only logs the object's content type.
- `sort_files.js` does not reset `url` between keys. A later key that matches no extension can still insert using the previous key's URL. Provenance: **Inferred** from the variable scope.
- `forEach(async ...)` does not wait for queries or jobs. `mysql.end()` runs immediately after the loop is scheduled.
- `handler-lib` always returns HTTP 200 with `Access-Control-Allow-Origin: *` when the inner function resolves, including for S3 events.

## Integrations

| System | Direction | Trigger | Data | Auth | Failure |
|---|---|---|---|---|---|
| S3 `the-ear-prod-transcode` | In | Object created | Bucket, key | Lambda resource policy created by Serverless (not hand-written) | Non-`.mxf` ignored. Job errors logged |
| Elastic Transcoder | Out | `createJob` | Pipeline, input key, four outputs, playlist | Lambda role from `iamRoleStatements` | Logged. No retry in code |
| S3 `the-ear-prod-resources` | In | Object created | Key | Lambda role | Unknown extension: no insert. SQL error logged |
| MySQL | Out | After classification or during backfill | `resources` row | User and password from env | Logged, not thrown |
| DynamoDB | Out | `POST /create`, and `logging.js` if called | Note item or log item | Role on `eu-west-1` tables. Ireland template omits this statement | `create` returns HTTP 500 |
| API Gateway | In | POST `/create`, GET `/sort_files` | JSON body or empty | IAM on create only | Gateway 4XX/5XX include CORS `*` |
| S3 copy (unwired) | Out | Would be an S3 event | One key | Policy JSON allows get on source and put on destination | Logged |

Expected Elastic Transcoder response is the job object. The wired handler logs `data.Job.Id` and does not persist it.

## Acceptance notes for a reader

A tester can derive cases from FR-001 to FR-006 without a separate test plan in the repo. The checked-in Jest test only asserts that `handler.hello` returns status 200, and that file now lives under `archived/tests/`. It does not call the transcode or catalog handlers.