# Project plan — dms_archive (WCA Archive)

Evidence date: 2026-10-17.

Source repository: `git@bitbucket.org:overdrive/dms_archive.git`.

Inspected revision: `main` at `2bcf278` (2026-10-16, "update back to 50"). History below is the most recent 30 commits reachable from that revision. The full history was not cloned.

This plan describes what the repository contains. It does not decide what the product should become.

There was no `silicon_ark/dms_archive/` directory before this onboarding.

## Product

The repository is named **dms_archive**. `composer.json` describes it as **WCA Archive - Document Management System** and sets the homepage to `https://wineauthority.co.za`. `docs/README.md` and `docs/PROJECT_RULES.md` call it the **WCA Archive System**. The Bitbucket pipeline calls a deploy webhook at `https://wineauthority.co.za/deploy/trigger`.

What the letters WCA expand to is **Unknown / Requires Confirmation**. The hostname supports an inference that the product serves the organisation behind wineauthority.co.za. That expansion is **Inferred**, not stated in code.

The committed environment template sets `app.baseURL` to `http://localhost/dms/` and `database.default.database` to `extract`. Whether production uses that database name is **Unknown / Requires Confirmation**.

Provenance: names and homepage are **Documented** in `composer.json` and `docs/README.md`. The live host is **Implemented** as the webhook URL in `bitbucket-pipelines.yml`.

## Branch reality

Default branch inspected is `main`.

The task that created this onboarding recorded, on 2026-09-28, last commit `45627d3` (2026-04-02, "Removed footer buttons thats not needed") and a tree that included `app_old`, `docs_old`, `public_old`, and `htaccess_old`. Those directories are not on `main` at `2bcf278`. The April snapshot is stale. Recent work on `main` is vault folders, bulk-email campaign queue, mailbox, and document permissions.

| Commit | Date | Subject |
|---|---|---|
| `2bcf278` | 2026-10-16 | update back to 50 |
| `82f4726` | 2026-10-16 | change tick rate to 30 instead of 50 |
| `207ecde` | 2026-10-16 | clear mail attachement after each send so it does not build up and take up memory |
| `678f946` | 2026-10-16 | Queue ui |
| `8bbede0` | 2026-10-16 | add sending button for failed campaigns |
| `c4cf1fb` | 2026-10-16 | update cron interval estimate calc and changes to new campaign not being send out by mistake |
| `55ac428` | 2026-10-15 | added Campaign Queue and Cronjob batch sender |
| `ec1b940` | 2026-10-15 | campaign email logs table dont store entire email and attachements in the db |
| `ed1b000` | 2026-10-02 | Changes to the Recipient Import |
| `3453ac8` | 2026-09-30 | Campaigns attachment functionality |
| `4c64efb` | 2026-08-03 | Shared access restrictions |
| `180f4c1` | 2026-07-29 | fix(documents): enforce vault write ownership and scope admin categories |
| `b1cf9d9` | 2026-07-29 | docs: personal vault permissions feature |
| `5476ff8` | 2026-07-29 | feat(documents): grouped org vault folder picker on upload edit |
| `2151019` | 2026-07-29 | feat(documents): search scope toggles and vault share modal |
| `7dfb6dc` | 2026-07-29 | feat(documents): add unified vault folder sidebar |
| `c0ffb96` | 2026-07-29 | feat(documents): add vault folder CRUD endpoints |
| `eac6a4a` | 2026-07-29 | feat(documents): add vault RLS to DocumentModel |
| `727c0e5` | 2026-07-29 | fix(migrations): create vault scope indexes on document_categories |

Older history (document tables from 2025-01, bulk email from 2026-04, mailbox from 2026-05) is present in migrations. A complete milestone timeline before mid-2026 was not reconstructed from git.

## Scope in the repository

In scope of what is present on `main`:

- A CodeIgniter 4 application (`codeigniter4/framework` `^4.0`) for document storage, search, and administration.
- Session login, optional email two-factor codes, password reset, IP block list, and login history.
- Role and menu permissions. Seeded names include System Admin, Admin, Company Admin, Company User, System User, and Viewer. A Developer role is assumed by permission code but is not inserted by the role seeder. See [Current project status](#current-project-status).
- Document upload (PDF, JPG, PNG), OCR, folders, personal vault, viewer sharing with time limits, and audit logs.
- Search that combines database fields, extracted text, and a MySQL vector store (Neuron AI / OpenAI embeddings).
- Chat against documents, a separate ChatGPT page, and a RAG status screen.
- Bulk email: recipients, groups, GrapeJS templates, campaigns, a per-recipient queue, attachments, and open/click/unsubscribe tracking.
- Mailbox ticketing over IMAP (`php spark mailbox:poll`) with reply, compose, and close.
- System settings: cron actions, email templates and logs, holidays, ChatGPT settings, writable logs, sent queue.
- Server monitoring CLI `php spark monitor:system` and a monitoring screen.
- HTTP cron entry `GET cronJobs` and a token-protected deploy webhook.
- Developer screens: menu management and a CRUD generator.
- PHPUnit tests for vault permissions, documents, bulk email, mailbox, and deploy webhook.
- SonarQube and FTP deploy on the `main` branch via Bitbucket Pipelines, then a webhook to `wineauthority.co.za`.

Out of scope of the code that was found, or present only as routes or views:

- Controllers `Products`, `Orders`, `Company`, `FaultFinder`, `LabelStatus`, `LabelUserGroups`, `LabelEmailTemplates`, `GetXMLData`, `StoreApiController`, and `Organisations`. Routes name them. The classes are not in `app/Controllers`.
- `docs/overview.md` as a reliable description. It is a generic CodeIgniter sketch (PHP `^8.0`, "likely MySQL") and disagrees with `composer.json`.

## System architecture

```text
Browser
  → Apache or `php spark serve` (public/ is the web root)
      → CodeIgniter 4 (auto-route enabled)
          → Filters: session login, menu grant, CSRF on POST
          → documents* and mailbox: time-access filter
          → Controllers under app/Controllers
          → Models under app/Models
          → MySQL (template database name: extract)
          → Views (Admin-style layout, DataTables)

Document files
  → writable storage
  → OCR: pdftotext / pdftoppm or ImageMagick, then tesseract
  → embeddings via OpenAI, stored by app/Libraries/VectorStore/MySQLVectorStore.php
  → app/Neuron/DocumentRAG.php

Unauthenticated HTTP
  → login, password reset, 2FA confirm
  → email tracking pixels and clicks
  → GET cronJobs (weekday 08:00–23:00 window; production key check)
  → GET deploy/trigger and deploy/complete (shared secret + throttle)

CLI
  → php spark send:bulk-email-campaigns
  → php spark mailbox:poll
  → php spark monitor:system
  → php spark deploy:notify-log
```

## Major components

| Area | Where it lives |
|---|---|
| Login and 2FA | `app/Controllers/Welcome.php`, `app/Filters/Authentication.php` |
| Menu authorisation | `app/Filters/Authorization.php`, `user_access` tables |
| Documents and vault | `app/Controllers/Documents.php`, `app/Models/DocumentModel.php`, `app/Helpers/useraccess_helper.php`, `app/Helpers/vault_helper.php` |
| OCR and RAG | `app/Libraries/OcrLibrary.php`, `app/Libraries/PdfTextExtractor.php`, `app/Helpers/document_helper.php`, `app/Neuron/DocumentRAG.php` |
| Chat | `app/Controllers/Chat.php`, `app/Helpers/chat_memory_helper.php` |
| Bulk email | `app/Controllers/BulkEmail.php`, `app/Libraries/BulkEmailCampaignQueueService.php`, `app/Config/BulkEmail.php` |
| Mailbox | `app/Controllers/Mailbox.php`, `app/Commands/MailboxPoll.php` |
| Cron and mail queue | `app/Controllers/CronJobs.php`, `sent_queue` |
| Deploy | `app/Controllers/DeployWebhook.php`, `bitbucket-pipelines.yml` |
| Settings and users | `app/Controllers/SystemSettings.php`, `app/Controllers/Users.php`, `app/Controllers/SystemUsers.php` |

## Known dependencies

Runtime, from `composer.json`:

- PHP constraint `^8.4`
- `codeigniter4/framework` `^4.0`
- `aws/aws-sdk-php` `^3.254`
- `mpdf/mpdf` `^8.2.1`
- `phpoffice/phpspreadsheet` `^1.25`
- `orhanerday/open-ai` `^3.5`
- `neuron-core/neuron-ai` `^2.8`
- `smalot/pdfparser` `^2.12`
- `ezyang/htmlpurifier` `^4.17`
- `league/csv` `^9.16`
- PHP extension `simplexml`

Suggested, not required: `ext-fileinfo`, `ext-imap` (required in practice for `mailbox:poll`).

Host tools for upload OCR, documented in `docs/README.md`: Poppler (`pdftotext`, `pdftoppm`) or ImageMagick, and Tesseract. Optional env: `TESSERACT_PATH`, `OCR_EXTRA_PATH`.

External systems: MySQL, SMTP, OpenAI, IMAP mailbox, FTP deploy target, SonarQube. AWS SDK is a Composer requirement. Call sites were not fully traced. Africa's Talking and Salesforce helpers exist; whether they are called from a live workflow is **Unknown / Requires Confirmation**.

## PHP version conflict

| Source | Statement |
|---|---|
| `composer.json` | `"php": "^8.4"` |
| `docs/README.md` | "PHP version 8.3 is required" and a packages list of `php: ^8.3` |
| `bitbucket-pipelines.yml` | build image `php:8.3-cli` |
| `docs/overview.md` | "PHP Version: ^8.0 required" |

These disagree. Which version production runs is **Unknown / Requires Confirmation**.

## Milestones visible in the tree

These are implementation dates from migrations and commits, not an approved roadmap.

| When | What landed |
|---|---|
| 2022-11 | Users, roles, menus, sessions, organisations |
| 2023–2024 | Media files, cron, settings, holidays, email templates, sent queue, 2FA table, login history, blocked users |
| 2025-01 | Documents, types, categories, tags, user access times, audit logs, user-document assignments, chat logs |
| 2025-11 | ChatGPT settings, vector embeddings, chat memories, document contexts |
| 2026-04 | Bulk email tables |
| 2026-05 | Custom email tracking, mailbox tickets, document-type tabs, document year |
| 2026-07 | Vault scope on categories and vault folder shares |
| 2026-09 | Bulk email campaign queue table |
| 2026-10 | Queue worker batch size 50, queue UI, attachment memory fix |

No separate future roadmap file was found. `docs/superpowers/plans/2026-04-24-c1-html-builder-ai.md` is a design plan for an HTML builder. Whether that plan is finished is covered in `features.md`.

## Implementation plan (as the repository stands)

There is no project plan document inside `dms_archive` that sets future milestones. The operating loop that is implemented:

1. Develop on a feature branch. `docs/PROJECT_RULES.md` describes Git workflow conventions.
2. Push to `main`.
3. Bitbucket Pipelines on `main`: Composer install, SonarQube scan, FTP mirror of changed files, then `GET https://wineauthority.co.za/deploy/trigger?token=...`.
4. The app webhook starts a host deploy script and emails a finish notice (`deploy:notify-log` / `DeployWebhook::complete`).

SonarQube exclusions in `sonar-project.properties` omit most of `app/`, `tests/`, and `public/` from analysis. The scan step still runs. How much of the product Sonar actually reviews is limited by those exclusions.

## Current project status

**Implemented and recently changing:** document vault, folder permissions, bulk-email campaign queue (batch size 50, cron interval assumption 5 minutes in `app/Config/BulkEmail.php`), mailbox ingest, document upload limits (application check 100 MB, plus PHP `upload_max_filesize`).

**Inherited and not implemented in this tree:** product, order, label, company, fault-finder, and store-api routes. Views under `app/Views/edioverdrive/` remain. This matches the pattern in the sibling Silicon Ark project `wca_dms`, which is a different repository (`wca_dms`, brand Admin Overdrive). Whether `dms_archive` was forked from that admin shell is **Inferred** from shared controllers, seeds, and route names. It is **Unknown / Requires Confirmation**.

**Seeder defect (Implemented):** `app/Database/Seeds/UserRole.php` checks for the name `Developer` and, when that name is absent, inserts `System Admin`. On an empty table the same batch can insert `System Admin` twice and `Viewer` twice, and it never inserts a row named `Developer`. Permission helpers still special-case the name `Developer`.

**Security note for operators:** `CronJobs::SECURITY_KEY` is the placeholder string `add your security key here`. The key is checked only when `CI_ENVIRONMENT` is `production`. The deploy webhook expects `deployment.webhookToken`. The committed `env` file contains credential-shaped values. They are not copied into Silicon Ark. Rotating anything that was committed is a human action.

**Local path in the onboarding issue** (`~/projects/sonar_scan/repos/dms_archive`) was not present on the inspection machine. Evidence was taken from a clone of `git@bitbucket.org:overdrive/dms_archive.git` at `2bcf278`.