# Changelog

## [2025-08-08]
### Changed
- Refined large document processing prompt in `amplify/backend/function/openAIFileProcessing/src/buildPrompt.js`:
  - Replaced ambiguous wording "STRONG matches" with explicit matching threshold
  - Introduced dynamic confidence threshold (≥6 when filtering is enabled; ≥8 otherwise)
  - Renamed section from "MATCHING RULES" to "MATCHING GUIDELINES"
  - Added matching tolerances: amounts within 1% or absolute 5.00; dates within ±7 days
  - Clarified identifiers policy: exact or close variant when present on both sides; neutral if missing on one side
  - Added entity name normalization and abbreviations handling for similarity
  - Updated final CRITICAL line to reflect threshold-based inclusion

## [Latest] - 2024-12-19

### Fixed
- **Logs Page User Dropdown**: Fixed user dropdown not populating in logs page
  - **Root Cause**: Incorrect API response structure access - was using `res.rows` instead of `res.data.rows`
  - **Solution**: Updated `getUsers` function to properly access AWS Amplify wrapped response structure
  - **Improvements**: Added debug logging, better error handling, and enhanced dropdown display
  - **User Experience**: Dropdown now shows "Loading users..." during API call and displays full name with email
  - **Technical Details**: AWS Amplify API wrapper adds a `data` property to responses, requiring `res.data.rows` access pattern

### Changed
- **Logs Page Search Functionality**: Replaced separate search input field with integrated search functionality directly within the user email dropdown
  - **User Experience**: Users can now search for users by name or email directly in the dropdown
  - **Interface**: More streamlined interface with search integrated into the dropdown component
  - **Functionality**: Real-time filtering of user list as user types in the dropdown search field

### Added
- **Workpaper Logs Retrieval**: Added new "getLogs" action to manageFileProcessTable function
- **Logs Page Frontend**: Created new logs page with comprehensive filtering and Excel export capabilities
  - **New Page**: `src/pages/logsPage/` with index.jsx and details.jsx components
  - **Date Range Filtering**: Required from/to date inputs with validation
  - **Optional Filters**: User name and audit firm name filtering
  - **Excel Export**: Export functionality instead of on-screen display
  - **CSV Download**: Downloads logs as CSV file compatible with Excel
  - **File Naming**: Automatic filename with date range (e.g., `workpaper_logs_2024-12-19_to_2024-12-19.csv`)
  - **Navigation**: Added "Logs" link to both admin and user navigation menus
  - **Responsive Design**: Bootstrap-based layout matching existing page styling
  - **Error Handling**: Comprehensive error display and validation
  - **Loading States**: Spinner and disabled states during API calls and export
  - **Default Date Range**: Set to today only for better performance
  - **Export Button**: Green "Export to Excel" button with loading state
  - **New Function**: `handleGetLogs()` - Retrieves workpaper logs from the database with comprehensive filtering
  - **Required Parameters**: `fromDate` and `toDate` (ISO string format) for date range filtering
  - **Optional Filters**: `userName` and `auditFirmName` for additional filtering
  - **Excluded Fields**: `file_id` and `table_name` are excluded from results as requested
  - **Response Format**: Returns logs with metadata including filter information and total count
  - **Ordering**: Results are ordered by `created_at DESC` (most recent first)
  - **Validation**: Comprehensive date validation and format checking
  - **Error Handling**: Proper error responses with detailed logging

### Technical Details
- **Database Query**: Uses parameterized queries for security and performance
- **Filtering Logic**: Conditional WHERE clauses based on provided filter parameters
- **Date Validation**: Validates ISO date format and ensures fromDate ≤ toDate
- **Logging**: Comprehensive logging for debugging and monitoring
- **CORS Support**: Full CORS headers for cross-origin requests

### API Usage
```json
{
  "action": "getLogs",
  "fromDate": "2024-01-01T00:00:00.000Z",
  "toDate": "2024-12-31T23:59:59.999Z",
  "userName": "optional_user_filter",
  "auditFirmName": "optional_firm_filter"
}
```

### Security Fixes
- **Fixed ReDoS (Regular Expression Denial of Service) vulnerabilities**: Replaced all regex patterns vulnerable to catastrophic backtracking with secure, bounded alternatives
  - **Issue**: SonarQube identified 5 security hotspots in `src/pages/customMatchPage/utils/formatters.js` related to regex patterns vulnerable to ReDoS attacks
  - **Root Cause**: Unbounded quantifiers (`+`, `*`) and nested repetitions causing exponential/polynomial runtime complexity
  - **Fixed Patterns**:
    - **Date Format Patterns**: Replaced `/^dd?[-/.m]+[-/.y]+$/i` with `/^d{1,2}[-/.m]{1,2}[-/.y]{1,4}$/i` (bounded quantifiers)
    - **Currency Format Patterns**: Replaced `/^\[[^\]]*\$[^\]]*\]$/` with `/^\[[^\]]{0,50}\$[^\]]{0,50}\]$/` (bounded to 50 chars)
    - **Decimal Patterns**: Replaced `/^\d{1,3}(,\d{3})+\.\d{1,2}$/` with `/^\d{1,3}(,\d{3}){1,10}\.\d{1,2}$/` (bounded to 10 groups)
  - **Security Impact**: Reduced risk from Medium to Low, preventing potential DoS attacks
  - **Performance**: All patterns now have linear time complexity instead of exponential
  - **Documentation**: Added comprehensive ReDoS documentation to `NewKnowledgeBase.md`

### Code Quality Improvements
- **Fixed nested ternary operations**: Extracted nested ternary operations in logs page dropdown components
  - **Issue**: SonarQube code smell (javascript:S3358) - "Extract this nested ternary operation into an independent statement"
  - **Solution**: Created helper functions `getUserDropdownText()` and `getAuditFirmDropdownText()` to replace nested ternary operations
  - **Improvements**: Better code readability, maintainability, and reduced cognitive complexity
  - **Files Modified**: `src/pages/logsPage/details.jsx`
- **Fixed accessibility issue**: Removed problematic `role="status"` attribute from Spinner component
  - **Issue**: SonarQube accessibility code smell (javascript:S6819) - "Use `<output>` instead of the 'status' role to ensure accessibility across all devices"
  - **Solution**: Removed `role="status"` attribute and kept `aria-hidden="true"` for better accessibility
  - **Improvements**: Better accessibility compliance and cross-device compatibility
  - **Files Modified**: `src/pages/logsPage/details.jsx`
- **Reduced cognitive complexity**: Refactored `detectFieldTypeFromData` function to reduce cognitive complexity from 32 to under 15
  - Extracted helper functions: `checkExcelDateFormatting`, `checkExcelCurrencyFormatting`, `analyzeExcelCellTypes`, `analyzeFieldNamePatterns`, `analyzeDataPatterns`, `determineTypeFromDataPatterns`
  - Maintained exact same logic and behavior while improving code readability
  - Each helper function now has a single responsibility and low complexity
  - Improved maintainability and testability of the field type detection logic

### Technical Details
- **Security Hotspot Resolution**: Addressed SonarQube security hotspots (javascript:S5852)
- **Performance**: Improved regex performance by using more specific patterns
- **Maintainability**: Better code organization with single-responsibility functions
- **Testing**: Each helper function can now be tested independently

### Files Modified
- `src/pages/customMatchPage/utils/formatters.js` - Security fixes and refactoring
- `CHANGELOG.md` - Documentation updates
- `NewKnowledgeBase.md` - Knowledge base updates

### Added
- **True Hierarchical Export**: Implemented proper step-by-step matching chain export
  - Export now follows actual matching relationships from Step 1 through all subsequent steps
  - For each Step 1 record, finds all matches in Step 2, then Step 3, then Step 4, etc.
  - Builds complete matching chains recursively through all steps
  - Ensures related records appear together in the correct hierarchical order
  - Applies to both local Excel export and S3 export for ZIP files

### Fixed
- **Export Ordering Issues**: Reverted filename-based grouping that was causing incorrect record placement
  - Removed complex filename grouping logic that was interfering with step-to-step flow
  - Implemented true hierarchical matching that follows actual relationships
  - Fixed issue where "Unmatched" separators were appearing too frequently
  - Restored proper step 1→2→3→4 progression in export order
- **Excel Export Order**: Fixed issue where "Unmatched" section was appearing at the top instead of the bottom
- **Export Logic**: Improved the `buildAllExcelRows` function to:
  - Process Step 1 rows and their hierarchical matches first
  - Track processed rows to avoid duplicates
  - Add "Unmatched" section only at the end for truly unmatched records
  - Maintain proper chronological order within each hierarchical group

### Changed
- **Export Logic**: Completely redesigned `buildAllExcelRows` function with hierarchical matching
  - Now starts with Step 1 records and recursively finds all matches through the chain
  - Added `findCompleteMatchingChain()` to build full matching chains
  - Added `buildMatchingChain()` for recursive step-by-step matching
  - Added `findMatchesForRow()` to find matches for specific rows in specific steps
  - Added helper functions for record matching validation and row key generation
  - Maintains existing data structure compatibility
  - Updated both `ResultsTable.jsx` and `exportExcelToS3.js` for consistency

### Technical Details
- Enhanced duplicate detection using content-based row keys
- Implemented recursive matching chain traversal through all steps
- Added proper record matching validation using `match_table_db` data
- Maintained existing data structure compatibility
- Updated both `ResultsTable.jsx` and `exportExcelToS3.js` for consistency
- Added processed rows tracking to prevent duplicate records
- Fixed Excel styling to properly detect and style "Unmatched" separator rows
- Ensured unmatched records only appear in the final "Unmatched" section

## [Unreleased]

### Enhanced
- **Excel Field Type Detection**: Completely overhauled to TRUST EXCEL FORMATTING instead of guessing
  - **🚀 REVOLUTIONARY**: Now trusts Excel's formatting completely - if Excel formats a cell as Date, it's detected as Date (no more guessing!)
  - **🚀 REVOLUTIONARY**: Now trusts Excel's TEXT detection - if Excel marks as "Number Stored as Text", it's detected as TEXT
  - **🚀 REVOLUTIONARY**: Simplified detection logic - Excel formatting has HIGHEST PRIORITY, then Excel cell types, then minimal fallback
  - **🚀 REVOLUTIONARY**: Removed complex range checking and override logic - no more false positives from small numbers (100.1) being detected as dates
  - **🔥 CRITICAL FIX**: Made currency detection CONSERVATIVE - only numbers with decimal places (123.45, 100.5) are considered currency, not plain integers (100, 200)
  - **NEW**: Uses Excel's native cell types (`t` property: 'd'=Date, 's'=Text, 'n'=Number) as secondary detection method
  - **NEW**: Enhanced Format Pattern Matching - Added 20+ Excel date format patterns including format codes (14-22)
  - **REMOVED**: Complex date serial range checking that caused false positives
  - **REMOVED**: Override logic that second-guessed Excel's own formatting
  - **REMOVED**: Early detection system that tried to guess if numbers might be dates
  - **REMOVED**: Aggressive currency detection that considered any integer 50+ as currency
  - **PHILOSOPHY**: "If Excel says it's a date, it's a date. If Excel says it's text, it's text. If Excel says it's a number, it's a number. Don't overthink it."

### Technical Changes  
- **🚀 REVOLUTIONARY**: Completely rewrote `detectFieldTypeFromData()` to trust Excel formatting as highest priority
- **🚀 REVOLUTIONARY**: Added Excel TEXT type detection - when Excel marks cells as 's' (String), including "Number Stored as Text", they're detected as TEXT
- **🚀 REVOLUTIONARY**: Removed complex override logic, early detection, and range checking that caused false positives
- **🚀 REVOLUTIONARY**: Simplified detection to: Excel formatting → Excel cell types → Field name patterns → Basic fallback
- **🔥 CRITICAL**: Made `isCurrencyLikeNumber()` CONSERVATIVE - removed rule that considered integers 50+ as currency, now only decimal numbers (123.45, 100.5) are considered currency
- **NEW**: Enhanced Excel format pattern matching with 20+ patterns including format codes (14, 15, 16, 17, 18, 19, 20, 21, 22)
- **REMOVED**: `isExcelDateSerial()` range checking from detection logic (kept function for potential future use)
- **REMOVED**: Override logic that second-guessed Excel's native type detection
- **REMOVED**: Early detection system that tried to guess based on number ranges
- **REMOVED**: Aggressive currency detection that caused plain numbers to be misclassified
- **KEPT**: Modified `isCurrencyLikeNumber()` to exclude Excel date serials, preventing date/currency conflicts (still relevant)
- **SIMPLIFIED**: Detection now trusts Excel completely rather than trying to guess from data patterns
- Modified `formatters.js` with new functions: `isExcelDateSerial()`, `isCurrencyLikeNumber()`
- Updated `detectFieldTypeFromData()` to accept Excel cell objects and native types as third parameter
- Enhanced `parseExcelHeaders()` in StepCard.jsx to extract Excel cell type information (`cell.t` property)
- Added Excel native cell type analysis with statistical majority voting for type determination
- Added Excel format pattern recognition for dates and currency as fallback method
- Implemented multi-tier detection priority system with Excel native types as highest priority
- Added detailed debugging output showing value-by-value analysis for troubleshooting

### Added
- **Bulk User Import Feature**: Added ability to import multiple users via CSV file upload
  - CSV format: `company_name,fullname,email`
  - Automatic defaults: `role_id=3` (User), `active=1` (Active)
  - Company name lookup automatically converts to company ID
  - Real-time validation and preview before import
  - Progress tracking and detailed error reporting
  - CSV template download with sample data
  - Integrated into existing user management page with "BULK IMPORT" button
  - **NEW**: Cognito recovery mechanism for users created in Cognito but not in database
  - **NEW**: Duplicate user detection to prevent creating users that already exist in Cognito
  - **NEW**: Individual retry mechanism for failed database insertions

### Technical Changes
- Added `BulkImportUserModal` component (`src/components/tableBasic/users/bulkImportUser.jsx`)
- Enhanced `UserTable` component with bulk import functionality
- Added `/bulkaddusers` API endpoint for batch user processing
- Updated user management page to include bulk import modal
- Added company name validation against existing companies in database
- **NEW**: Added `checkUserExists()` function to prevent duplicate Cognito users
- **NEW**: Added `retryDatabaseInsert()` function for recovery of Cognito-only users
- **NEW**: Enhanced error tracking to distinguish between Cognito failures and database failures
- **NEW**: Added individual user insertion retry mechanism using `/adduser` endpoint

### Added
- ContactForm component for user inquiries
  - Created new modal-based contact form with Name, Email, and Message fields
  - Integrated with existing AWS Amplify API to send contact messages via `/contactmail` endpoint
  - Added form validation with user-friendly error messages
  - Implemented loading states and success notifications
  - Updated Welcome component to trigger contact form modal from "CONTACT US" button
  - Follows existing project patterns for styling and API integration
- Comprehensive OpenAI usage logging system
  - Created new `workpaper_logs` MySQL table to track all OpenAI API usage
  - Added logging for both document processing and matching operations
  - Captures user name, audit firm name, operation type, document details, token usage, and processing performance
  - Includes input tokens, output tokens, total tokens, model used, and processing time
  - Automatically logs both successful operations and errors
  - Enhanced OpenAI service to capture token usage from API responses
  - Added batch processing support for token aggregation across multiple API calls

### Changed
- Refactored data type detection utilities for better code organization
  - Moved `isDateValue`, `isDecimalValue`, `isNumericValue`, and `detectFieldTypeFromData` functions from StepCard.jsx to utils/formatters.js
  - Updated import statements to use centralized utility functions
  - Removed useCallback wrappers for utility functions to improve performance
  - Updated function dependency arrays to remove imported functions
  - Improved code reusability and maintainability by consolidating data type detection logic
  - Enhanced modularity by keeping utility functions separate from component logic

### Fixed
- Fixed React DOM validation error in ResultsTable component
  - Resolved "div cannot appear as child of tbody" error by moving tooltip rendering outside table structure
  - Modified RowTooltip component to only return tr elements inside tbody
  - Added tooltip state management to main ResultsTable component
  - Tooltip now renders outside table using fixed positioning to avoid DOM nesting violations
- Enhanced Excel field type detection with intelligent field name analysis
  - Modified parseExcelHeaders function to combine field name patterns with data analysis
  - Added detectFieldTypeFromData function that considers both field names and data patterns
  - Implemented smart field type detection for dates, currency, and identifiers based on field names
  - Added date field detection for fields named "date", "time", "created", "updated", etc.
  - Added currency field detection based on field names (debit, credit, balance, amount, total, etc.)
  - Added identifier field detection for text classification (invoice number, reference, account code, etc.)
  - Prioritized field name patterns over data patterns for well-known field types
  - Enhanced date pattern recognition to support text-based formats like "30 Nov 2022", "Nov 30, 2022"
  - Date fields (date, time, created, updated, etc.) are always classified as DATE regardless of data patterns
  - Currency fields (debit, credit, balance, etc.) are always classified as CURRENCY regardless of data patterns
  - Identifier fields (numbers, codes, references) are always classified as TEXT regardless of numeric content
  - Improved date detection to prevent numeric values from being misclassified as dates
  - Uses three-tier priority system: field name patterns → strong data patterns → fallback analysis
  - Samples up to 10 non-empty values per column for accurate type detection
- Fixed hyperlink styling in Excel exports
  - Migrated from `xlsx` to `xlsx-js-style` library for proper cell styling support
  - Implemented proper blue color and underline styling for hyperlinks in Excel files
  - Added trimming of filenames and folder paths to prevent broken links
  - Removed post-processing styling code in favor of direct cell styling during creation
- Fixed race conditions in localStorage access operations
  - Implemented a key-based queue system for synchronized access to localStorage
  - Added multiple key support for safe reading and writing to localStorage
  - Added support for four critical localStorage keys: 'custom_match_page', 'result_table_db', 'match_table_db', 'match_results'
  - Created safe utility functions for localStorage operations to prevent race conditions
- Fixed ResultsTable data display issues
  - Corrected double parsing bug in loadDataFromLocalStorage function
  - Fixed data handling when using safeGetItem utility
  - Added debug logging to track data loading process
- Fixed match_table_db data not showing correctly in results table
  - Enhanced record matching algorithm in ResultsDataTable.jsx
  - Added multi-strategy matching to find the correct corresponding records
  - Implemented value-only matching for fields when names don't match exactly
  - Added partial text matching as a fallback method
- Fixed incorrect row mapping for bank account transactions
  - Added specialized account number extraction and normalization
  - Implemented bank account number pattern recognition
  - Enhanced matching priority system for financial record identification
  - Added field synchronization to ensure consistent column display
- Fixed dynamic field handling in matching algorithm
  - Removed all hardcoded field assumptions and lists
  - Implemented pure data-driven field detection and matching
  - Added dynamic field uniqueness analysis for better matching
  - Prioritized matching based on statistical uniqueness of values
- Fixed complete field synchronization between matching records
  - Implemented bidirectional field copying for matched records
  - Added field name similarity detection for differently named fields
  - Applied exact value matching to identify related fields
  - Ensured complete field mirroring between matched steps
- **buildLargeDocumentPrompt Function**: Fixed large document processing prompt to properly extract all specified fields from images and match them with previous step records. The function now:
  - Uses the `fields` parameter to specify what fields to extract (was previously ignored)
  - Includes document type and options context
  - Uses dynamic field matching instead of hardcoded field names - compares against available fields in previous step records
  - Restructured to match the buildMatchingPrompt approach with clear two-phase process: EXTRACT → COMPARE → MATCH
  - Implements extremely strict matching criteria requiring ALL THREE criteria to be met: exact amount match, exact/close date match, and clear company name match
  - Prevents false matches by requiring triple verification (amount + date + company name)
  - Explicitly excludes random product names like "Samsung Galaxy", "Motorola V50", "Siemens S35J"
  - Adds "when in doubt, exclude" logic to prioritize quality over quantity
  - Adds professional auditor context for handling prepayments, partial payments, and installments
  - Maintains consistent formatting requirements across all extracted records

### Added
- Added **ZipWorkPapers** Lambda function for creating workpapers archive:
  - Creates zip files containing all user workpapers from S3 bucket
  - Maintains original folder structure (Step{number}-{description}/)
  - Automatically includes Results.xlsx file in the ZIP archive
  - Accessible via `/workpapers/{userId}` API endpoint
  - Supports pagination for large numbers of files
  - Includes comprehensive error handling and CORS support
  - Automatically cleans up previous zip files before creating new ones
  - Returns signed download URL for immediate file download
  - Added "Download ZIP" button in ResultsTable for easy access
- Added **Excel Export to S3** functionality:
  - Automatically generates Excel report when creating ZIP download
  - Uses same formatting and styling as manual Excel export
  - Saves Results.xlsx file to S3 alongside workpapers
  - Included in ZIP archive for comprehensive reporting
  - Added hyperlink columns linking to source files in Step folders
  - File links automatically point to Step{number}-{description}/{filename}
  - Enhanced filename detection with fallback to generic filenames
  - Ensures every row has a hyperlink (folder link if no file found)
- Added new utility file `safeLocalStorage.js` with functions:
  - `safeGetItem`: Safely read from localStorage with proper error handling
  - `safeSetItem`: Queue-based writing to localStorage to prevent race conditions
  - `safeUpdateItem`: Apply a transform function to update localStorage data
  - `safeRemoveItem`: Safely remove items from localStorage
  - `getPendingOperationsStats`: Debug function to monitor queue status
- Added `safeLocalStorage.js` to customFlowPage
  - Implemented the same robust storage management for customFlowPage
  - Ensured consistent localStorage handling across different pages
  - Fixed race conditions in customFlowPage localStorage operations
- Added account number processing utilities:
  - `normalizeAccountNumber`: Consistent formatting for account numbers
  - `extractAccountNumber`: Extract account numbers from various text formats
  - Detects common account number patterns (e.g., XXX-XXX-XXXX, XXXXXX-XX-XXXX)
  - Handles different separators and formats automatically
- Added dynamic field analysis system:
  - Statistical analysis of field value uniqueness
  - Automatic detection of potential key fields
  - Dynamic priority scoring based on value distribution
  - Cross-record value combination matching
- Added intelligent field ordering system:
  - Grouped related fields together for logical display
  - Applied semantic field type detection
  - Ordered columns consistently across steps
  - Used field name patterns to determine display order
- Added batch processing for large PDF documents in openAIFileProcessing function:
  - Implemented automatic batch processing when `isMultiRow = true` and `pdfPages > 3`
  - Processes documents in batches of 3 pages to handle large multi-page PDFs efficiently
  - Combines results from all batches into a single unified response
  - Added comprehensive progress logging for batch processing operations
  - Enhanced buildDocumentPrompt to include batch-specific context for AI processing
  - Added batch information tracking (total batches, pages processed, errors per batch)
  - Implemented parallel processing of batches for improved performance
  - Added error handling for individual batch failures while continuing with other batches

### Changed
- **Refactored ZipWorkPapers Lambda function** to reduce cognitive complexity:
  - Extracted helper functions to improve code maintainability
  - Created `createCorsResponse()` to handle CORS response creation
  - Created `listAllS3Objects()` to handle S3 object listing with pagination
  - Created `filterWorkpaperObjects()` to handle workpaper filtering logic
  - Created `createArchiveWithFiles()` to handle archive creation and file processing
  - Created `generateSignedDownloadUrl()` to handle signed URL generation
  - Created `clearExistingZipFile()` to handle existing zip file cleanup
  - Reduced cognitive complexity from 18 to under 15 allowed threshold
  - Maintained all original functionality and logic unchanged
- **Refactored getFilenameForStep function** in exportExcelToS3.js to reduce cognitive complexity:
  - Extracted helper functions to improve code maintainability and readability
  - Created `getStepResults()` to handle result table data retrieval
  - Created `getSingleFileFilename()` to handle single file case logic
  - Created `findMatchingRecord()` to handle record matching in file content
  - Created `getMultipleFilesFilename()` to handle multiple files with matching logic
  - Created `getFilenameFromKnownFields()` to handle filename extraction from known field names
  - Created `looksLikeFilename()` to validate filename-like values
  - Created `getFilenameFromFilenameFields()` to handle filename extraction from filename-like fields
  - Created `getFilenameFromCustomMatchData()` to handle final fallback to custom match data
  - Reduced cognitive complexity from 28 to under 15 allowed threshold
  - Maintained all original functionality and logic unchanged
- Updated `ResultsTable.jsx` to use safe localStorage access operations:
  - Fixed loadDataFromLocalStorage to handle race conditions
  - Improved error handling throughout localStorage operations
  - Added proper cleanup when components unmount
  - Enhanced cross-tab synchronization for localStorage changes
- Enhanced ResultsTable matching algorithm:
  - Implemented dynamic field detection and indexing
  - Improved match detection with value-based field matching
  - Added pure data-driven field matching without assumptions
  - Enhanced logging for match detection troubleshooting 
- Improved match_table_db data processing:
  - Added 4 matching strategies with progressive fallbacks
  - Fixed JSON parsing error when data was already parsed by safeGetItem
  - Enhanced debugging by logging the matching method used for each record
  - Optimized performance by stopping search after finding the first match
- Enhanced field synchronization between steps:
  - Added bidirectional field mapping between related steps
  - Ensured matched rows display all fields from both steps
  - Implemented consistent column alignment for matched data
  - Added detailed logging of field mappings between steps
- Moved to completely data-driven field handling:
  - Replaced hardcoded field lists with dynamic value frequency analysis
  - Implemented adaptable matching that works with any field structure
  - Added statistical scoring of field uniqueness for smarter matching
  - Prioritized matching based on actual data patterns in the dataset
- Improved table display organization:
  - Implemented semantic column ordering for related fields
  - Grouped fields by type (account, payee, amount, date, etc.)
  - Applied consistent column ordering across all steps
  - Enhanced visual alignment of related fields between steps 

### Added
- **Excel Export Refactoring**: Refactored Excel export functionality to eliminate code duplication
  - Created shared `createExcelWorkbook` function in `exportExcelToS3.js` that can be used for both local and S3 exports
  - Updated `exportExcelToS3` function to use the shared workbook creation logic
  - Modified `ResultsTable.jsx` to use the shared function for local Excel export with hyperlinks disabled
  - Removed duplicate Excel export helper functions from `ResultsTable.jsx`
  - Added `includeHyperlinks` parameter to control whether file hyperlinks are included in exports

### Changed
- **Excel Export Behavior**: 
  - Local Excel export now shows filenames as plain text instead of hyperlinks
  - S3 Excel export continues to include hyperlinks for ZIP file integration
  - Both exports use the same hierarchical ordering and styling logic

### Technical
- **Code Quality**: Eliminated ~200 lines of duplicate code between local and S3 Excel export functions
- **Maintainability**: Single source of truth for Excel export logic, making future updates easier
- **Consistency**: Both export methods now use identical data processing and formatting logic 

### Added
- **Matching Helper Functions Refactoring**: Moved matching-related functions to shared utility file
  - Created `src/pages/customMatchPage/utils/matchingHelpers.js` with all matching logic
  - Moved functions: `isMatchedRow`, `isMatchingPreviousRecord`, `isMatchingCurrentRecord`, `getRowKey`, `getMatchStatus`, `hasMatchesInNextStep`, `getPrimaryStep`, `isMatchingRow`, `removeMatchedDataKeys`, `generateUniqueKeysForResults`, `transformResultsTableDB`
  - Updated `ResultsTable.jsx` to import and use shared matching functions
  - Updated `exportExcelToS3.js` to import and use shared matching functions
  - Eliminated code duplication between table display and Excel export logic
- **CSV Export Service Refactoring**: Moved CSV export functionality to dedicated service file
  - Created `src/pages/customMatchPage/services/fileProcessing/exportCSV.js` with CSV export logic
  - Moved `exportToCSV` function from `ResultsTable.jsx` to dedicated service
  - Added `createCSVContent` function for generating CSV content with hierarchical ordering
  - Added `exportCSVToS3` function for S3 upload capability
  - Updated CSV export to use shared matching helpers for consistent data processing
  - Enhanced CSV export with hierarchical ordering (same logic as Excel export)

### Changed
- **Code Organization**: 
  - Single source of truth for all matching logic in `utils/matchingHelpers.js`
  - Consistent matching behavior across table display and Excel export
  - Improved maintainability by centralizing matching algorithms
- **Matching Logic Enhancement**: Improved data matching robustness
  - Added `compareValues()` helper function for flexible value comparison
  - Enhanced matching functions to handle different data types, case sensitivity, and whitespace
  - Fixed issues with null/undefined vs empty string comparisons
  - Improved matching accuracy for "demo company" and similar data inconsistencies
- **Hierarchical Ordering Fix**: Fixed issue with matched records appearing in unmatched section
  - Added `shouldIncludeInHierarchicalOrdering()` function to properly identify matched records
  - Updated hierarchical ordering logic to include rows that are matched according to `isMatchedRow`
  - Fixed issue where records with data across multiple steps were incorrectly placed in unmatched section
  - Applied fix consistently across table display, Excel export, and CSV export
- **Hierarchical Ordering Refactoring**: Eliminated code duplication across export functions
  - Added `applyHierarchicalOrdering()` helper function to centralize hierarchical ordering logic
  - Replaced ~100 lines of duplicate code in each export function with single function call
  - Improved maintainability by having single source of truth for hierarchical ordering
  - Applied refactoring to ResultsTable.jsx, exportExcelToS3.js, and exportCSV.js
- **Duplicate Detection Investigation**: Investigated duplicate record handling
  - Identified that records appearing at end are legitimate matched records, not duplicates
  - Reverted duplicate detection changes as they were removing valid records
  - **Fixed hierarchical ordering for multiple previous records**
  - Updated `isMatchingPreviousRecord()` to handle arrays of previous records
  - Updated `traverseMatches()` to process all previous records in a match
  - **Changed to sequential processing** to maintain proper grouping order
  - Now processes rows in order they appear instead of by step number
  - **Added grouping by matching criteria** to ensure related records are processed together
  - Added `getMatchingKey()` function to group records by date, amount, and company
  - **Added immediate matches processing** to group same-step matches together
  - Added `processRowWithMatches()` function to process row and all its immediate matches
  - **Fixed multiple current records matching** to handle cases where multiple invoices match same payment
  - Updated `traverseMatches()` to find ALL matching current records, not just the first one
  - Updated `findImmediateMatches()` to use match_table_db data for comprehensive matching
  - **Fixed cross-record matching** to search across all match records in localStorage
  - Changed from `find()` to `filter()` to search ALL match records instead of stopping at first match
  - Ensures matches from different records (Entry 21, Entry 26) are all found and grouped together
  - **Fixed separate previous record processing** to handle each previous record individually
  - Now collects all matching previous records first, then processes each one separately
  - Ensures each previous record finds ALL current records that match it across all entries
  - **Completely rewrote hierarchical ordering for all steps** to properly group related records
  - Changed from individual row processing to step-by-step group processing
  - Added `findAllMatchingRowsForStepRow()` function to find all matches across all steps
  - Now processes each step and groups all related records from other steps together
  - Ensures all "15 July" entries are grouped before "18 July" entries

### Technical
- **Code Quality**: Eliminated ~150 lines of duplicate matching logic code
- **Maintainability**: Updates to matching logic now only need to be made in one place
- **Consistency**: Both table display and Excel export use identical matching algorithms
- **Documentation**: Added comprehensive JSDoc comments for all matching helper functions 

### Added
- **Search Functionality for Logs Page**: Added comprehensive search functionality to the Workpaper Logs page
  - New search input field that searches across multiple log fields (user name, audit firm, document name, operation type, model used, error message, and ID)
  - Real-time filtering with search results counter
  - Search results summary card showing filtered vs total logs
  - Export functionality now respects search filters (exports only filtered results when search is active)
  - Clear filters button now also clears search term
  - Improved UI layout with better responsive design 

### Added
- Searchable dropdown functionality for Audit Firm filter in Logs Page
  - Replaced text input with searchable dropdown for audit firm selection
  - Integrated search functionality directly within the audit firm dropdown
  - Fetches audit firm data from getcompany API
  - Filters audit firms by name as user types
  - Maintains consistency with user email dropdown implementation

### Changed
- Logs Page Search Functionality
  - Replaced separate search input field with integrated search functionality directly within the user email dropdown
  - Removed client-side log filtering in favor of server-side filtering
  - Updated export functionality to use all logs instead of filtered logs