# Technical documentation — theear_transcode

Evidence date: 2026-10-01.

Source: `git@bitbucket.org:overdrive/theear_transcode.git`, `master` at `fda49df` (2022-05-04, Jaco Brits). Single branch `master`. No credential values are copied here.

## Architecture

Two Serverless Framework services, deployed separately, plus files that the templates do not deploy.

```text
region_ireland/          service the-ear-production-transcode
                         region eu-west-1, stage production, profile theear_prod
                         functions: create, transcode_process

region_south_africa/     service theear-resources
                         region af-south-1, stage production, profile theear_prod
                         VPC enabled
                         functions: create, sort_files, transcode

policy/                  IAM JSON, not referenced by either template
```

Elastic Transcoder is invoked only from code that sets region `eu-west-1` on the wired path (`region_ireland/transcode_process.js`). The resources catalog uses MySQL from `af-south-1` and does not call Elastic Transcoder.

The pipeline's output bucket is not in git. A copy function (`lambda_move_transcode_to_resources.js`) would copy a key between `the-ear-prod-transcode` and `the-ear-prod-resources`, but it is not a Serverless function.

## Repository layout

Active application code is the non-`archived` JavaScript under each region. Shared names (`create.js`, `global.js`, `handler.js`, `logging.js`, `sort_files.js`, `libs/*`, `resources/api-gateway-errors.yml`, `package.json`, `env.example`) are identical in both regions except `transcode_process.js`.

Files that exist on only one side:

| Only Ireland | Only South Africa |
|---|---|
| `README.md` (starter guide), `index.js`, `lambda_move_transcode_to_resources.js`, `transcode.js` (job + insert) | `transcode.js` (catalog only), `transcode_old.js`, `upload.js`, `Untitled-1.txt` |

Root files: `.gitignore`, `README.md` (same starter guide), `policy/`, `test.html`.

`region_*/archived/old` is the pre-split tree (service `theear-transcode`, stage `dev`, profile `unity`). `archived/old2` is an earlier resources service in `eu-west-1`. `archived/tests/handler.test.js` tests `hello`.

## Language, framework, runtime

- JavaScript. Mix of `import`/`export` and `require('aws-sdk')` inside handlers.
- Runtime in both current templates: `nodejs12.x`.
- Packaging: `serverless-bundle` `^1.2.5` (Webpack). `package.individually: true`.
- Local HTTP simulation plugin: `serverless-offline` `^5.3.3`. Not a running environment in this repo.
- `serverless-dotenv-plugin` `^2.1.1` and `useDotenv: true`.
- Dependencies: `serverless-mysql` `^1.5.4`, `uuid` `^7.0.3`. `aws-sdk` `^2.706.0` is a devDependency.
- `package.json` `name` is still `serverless-nodejs-starter`, repository URL still the Anomaly Innovations GitHub project, license MIT. That metadata describes the starter, not a published package from this repo.
- Test script: `serverless-bundle test`. The only Jest file is under `archived/tests/` and imports `../handler`, which does not resolve from that directory to the active `handler.js`.

No `engines` field, no Docker file, no `bitbucket-pipelines.yml`.

## Services and functions

### Ireland — `region_ireland/serverless.yml`

| Key | Value |
|---|---|
| `service` | `the-ear-production-transcode` |
| `provider.region` | `eu-west-1` |
| `provider.stage` | `production` |
| `provider.profile` | `theear_prod` |
| `provider.memorySize` | 512 |
| `custom.bucket` | `the-ear-prod-transcode` (the event uses the literal bucket name, not the custom value) |
| VPC | Commented out |

| Function | Handler | Trigger | Timeout |
|---|---|---|---|
| `create` | `create.main` | `POST /create`, CORS, `authorizer: aws_iam` | default |
| `transcode_process` | `transcode_process.main` | `s3:ObjectCreated:*` on `the-ear-prod-transcode`, `existing: true` | 60 |

IAM: `s3:GetObject` and `s3:PutObject` on `arn:aws:s3:::*`. On `*`: `elastictranscoder:Read*`, `List*`, `*Job`, `*Preset`, `Create*`, `s3:List*`, `iam:List*`, `sns:List*`. No DynamoDB statement. No `environment` block.

### South Africa — `region_south_africa/serverless.yml`

| Key | Value |
|---|---|
| `service` | `theear-resources` |
| `provider.region` | `af-south-1` |
| `provider.stage` | `production` |
| `provider.profile` | `theear_prod` |
| `provider.memorySize` | 512 |
| `custom.bucket` | `the-ear-prod-resources` |
| VPC | `SECURITYGROUPIDS`, `SUBNETIDS1`, `SUBNETIDS2` |

Environment:

| Name | Value |
|---|---|
| `tableName` | `loging` |
| `databaseHost` | `${env:DBHOST}` |
| `databaseUser` | `${env:DBUSERNAME}` |
| `databasePass` | `${env:DBPASSWORD}` |
| `databaseName` | `${env:DBNAME}` |

| Function | Handler | Trigger | Timeout |
|---|---|---|---|
| `create` | `create.main` | `POST /create`, CORS, `aws_iam` | default |
| `sort_files` | `sort_files.main` | `GET /sort_files`, `cors: false`, no authorizer | default |
| `transcode` | `transcode.main` | `s3:ObjectCreated:*` on `the-ear-prod-resources`, `existing: true` | 60 |

A `transcode_process` function is commented out and points at `the-ear-staging-transcode`.

IAM adds DynamoDB `DescribeTable`, `Query`, `Scan`, `GetItem`, `PutItem`, `UpdateItem`, `DeleteItem` on `arn:aws:dynamodb:eu-west-1:*:*`. Elastic Transcoder `Create*` is not in this template. `*Job` is.

## Handler behaviour (technical)

### `transcode_process.js` (Ireland, wired)

Constructs `AWS.ElasticTranscoder` with `region: 'eu-west-1'` and `apiVersion: '2012–09–25'`. The dashes in that version string are Unicode en dashes (U+2013), not ASCII hyphens. `index.js` uses ASCII `2012-09-25`.

`createJob` is callback-style. `context.succeed` is not used in this file. `elastictranscoder.waitFor('jobComplete', { Id: jobId })` is not given `.promise()` or a callback.

`event.Records.forEach(async ...)` does not propagate the returned status object to `handler-lib`.

### `transcode.js` (South Africa, wired)

`AWS.config.update({ region: 'af-south-1' })`. The Elastic Transcoder client line is commented out. MySQL config reads the four `database*` env vars. Insert SQL is string-concatenated. `mysql.end()` is awaited immediately after `forEach` schedules the work.

### `sort_files.js`

`s3.listObjects({ Bucket: 'the-ear-staging-resources' })` with no prefix, delimiter, or pagination. Same SQL style. Existence check is `rows.length == 0`.

### `create.js`

`AWS.DynamoDB.DocumentClient` with no explicit region (SDK default, which in Lambda is the function region). Ireland is `eu-west-1`. South Africa is `af-south-1`, while the IAM resource ARN is `eu-west-1`. `uuid.v1()` for `noteId`. Callback style, not `handler-lib`.

### `lambda_move_transcode_to_resources.js`

`AWS.S3` client for the copy is `af-south-1`. A second client is created for `eu-west-1` and not used. `copyObject` callback does not call the Lambda `callback`. `CopySource` is `encodeURI(bucket + "/" + key)`.

### `index.js`

CommonJS `exports.handler`. Same outputs as the wired job creator. Decodes `+` in the key to spaces. Does not check `.mxf`.

### `global.js` and `logging.js`

Not registered. `global.js` switches on `data.a` (`s`, `u`, `i`). Join and where clauses call `.length()` and read `data.con`. `logging.js` puts a UUID `logID`, stringified event, query, and result into `process.env.tableName`.

### `handler-lib.js`

Returns `{ statusCode, headers, body }` with CORS `*` and `JSON.stringify(body)`. Used by the transcode, catalog, sort, and global handlers. S3 does not require that shape.

### `libs/response-lib.js`

`success` / `failure` HTTP wrappers. Used by `global.js` only. `handler-lib` then stringifies that object again, so a wired `global` would double-encode. `global` is not wired.

### `libs/images-lib.js`

Empty file.

### `upload.js`

`main` calls `s3.getObject` and logs `ContentType`. `getUploadURL` is a sibling function. It references `s3`, which is local to `main`, and `process.env.UploadBucket`, which the template does not set.

## Data

### MySQL `resources`

Columns written by the insert:

| Column | Source |
|---|---|
| `name` | Last `/` segment of the key stem |
| `url` | `s3://` URI described in the business rules |
| `type` | Extension map |
| `duration` | `"0"` or `""` |
| `created` | UTC timestamp string |

No `CREATE TABLE`, migration, or index is in the repo. Host and database name are environment variables. Connection is `serverless-mysql`.

### DynamoDB

Table name string `loging` on the South Africa service. Item shapes:

- Notes: `userId`, `noteId`, `content`, `attachment`, `createdAt`.
- Log: `userID`, `logID`, `event`, `query`, `result`, `createdAt`.

No table definition (keys, indexes) is in the repo.

### S3 URL sample

`region_south_africa/Untitled-1.txt` contains the same URI twice:

```text
s3://the-ear-prod-resources/transcode/Hein_Upright_Bass_Ep2_Harmonic_Minor_ac/Hein_Upright_Bass_Ep2_Harmonic_Minor_ac_1mb.ts.m3u8
```

That is a note, not configuration.

## IAM policy files

`policy/theear-lambda-transcode-resources.json`

- List bucket and versions on `the-ear-prod-resources` and `the-ear-prod-transcode`.
- Get object and version on `the-ear-prod-transcode/*`.
- Put object on `the-ear-prod-resources/*`.

`policy/theear-lambda-resources-video-upload.json`

- List on `the-ear-prod-resources` and `the-ear-prod-video-upload`.
- Get on `the-ear-prod-resources/*`.
- Put on `the-ear-prod-video-upload/*`.

No `Principal`, no role name, no attachment. Whether either policy is on a live role is **Unknown**.

## Configuration and secrets

`env.example` documents renaming to `.env`. Its only variable is `SAMPLE_ENV_VAR`. `.gitignore` ignores `.env` and `env.yml`.

Names the South Africa template expects and that are not in `env.example`: `DBHOST`, `DBUSERNAME`, `DBPASSWORD`, `DBNAME`, `SECURITYGROUPIDS`, `SUBNETIDS1`, `SUBNETIDS2`.

`UploadBucket` is read only by the unused helper in `upload.js`.

No `.env` is in git.

## Auth

- S3 triggers: no application auth. Access is the bucket notification and the Lambda role.
- `POST /create`: API Gateway AWS_IAM authorizer.
- `GET /sort_files`: no authorizer in the template.
- No Cognito user pool, no JWT check, no session. Comments and mocks mention `cognitoIdentityId`.
- Deploy uses the shared AWS profile name `theear_prod`. Archived templates used `unity`.

## Background work, queues, storage, monitoring

- No SQS, SNS publish, EventBridge, or cron.
- The only async AWS job is Elastic Transcoder `createJob`. Completion is not persisted.
- Storage is S3 plus MySQL plus DynamoDB, as above.
- No CloudWatch alarm, X-Ray, or log-metric filter in the templates. Handlers use `console.log`. The archived Ireland `serverless.yml` had an explicit `logs:CreateLogGroup` statement. The current templates do not. Lambda's basic execution role is whatever Serverless adds by default; that default is not copied into `iamRoleStatements` here.
- `serverless.yml` does not set reserved concurrency, DLQ, or `onError`.

## Security notes from the implementation

These describe the code. They are not a pentest.

- Catalog and backfill SQL concatenate the object key into the statement.
- `global.js`, if it were deployed, would concatenate caller JSON into SQL.
- S3 object permissions in the Serverless role cover all buckets.
- `handler-lib` and the notes response allow every origin, with credentials allowed.
- HLS AES configuration is present only as comments.
- `sort_files` is an unauthenticated GET in the template and writes to MySQL.
- Pipeline ids are hardcoded. They are identifiers, not passwords.

## Development and testing

From the starter README, which matches the plugins in `package.json`:

- `npm install` in a region directory.
- `serverless invoke local --function <name>` and `serverless offline start` are documented for a `hello` function that is not deployed.
- `serverless deploy` and `serverless deploy function --function <name>` are the documented deploy commands. The README's example function name `hello` is not in the current templates.
- `npm test` runs `serverless-bundle test`.

No test exercises `transcode_process`, `transcode`, or `sort_files`. No fixture event for an S3 `ObjectCreated` record is checked in, other than user-permission mocks.

`mocks/getuser.json` includes a `requestContext.identity.cognitoIdentityId` and `pathParameters.id`. It is not referenced.

## Deployment

Not automated in this repository. Expected manual path, from the template and the starter README: Serverless Framework, AWS CLI profile `theear_prod`, once per region directory. `existing: true` on both S3 events means Serverless will not create the buckets.

There is no record in git of a successful deploy, a stack name output, or an API id.

Stage `production` is the Serverless stage string. It does not prove an environment is live.

## Related identifiers

| Kind | Value |
|---|---|
| Wired pipeline | `1635987395753-iukgee` |
| Other pipeline id in unwired South Africa process | `1634634986727-4ij4ha` |
| System preset comments | `1351620000001-000001`, `-200010`, `-200030`, `-200040` |
| Prod buckets | `the-ear-prod-transcode`, `the-ear-prod-resources`, `the-ear-prod-video-upload` |
| Staging bucket used by code | `the-ear-staging-resources` |
| Staging bucket only in a comment | `the-ear-staging-transcode` |
| DynamoDB table name | `loging` |

## Discrepancy with the onboarding scan

The issue's top-level listing included `sonar-project.properties` and `sonarqube_scan.bat`. A full `git ls-tree` of `fda49df` does not contain them, and `git log` has no blob with those names. **Unknown** whether they existed only on the scan host.