feat: Improve file display, timezone handling, and LOT management

Changes:
- Fix datetime serialization with UTC 'Z' suffix for correct timezone display
- Add PDF upload support with extension fallback for MIME detection
- Fix LOT add/remove by creating new list for SQLAlchemy JSON change detection
- Add file message components (FileMessage, ImageLightbox, UploadPreview)
- Add multi-file upload support with progress tracking
- Link uploaded files to chat messages via message_id
- Include file attachments in AI report generation
- Update specs for file-storage, realtime-messaging, and ai-report-generation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
egg
2025-12-08 12:39:15 +08:00
parent 599802b818
commit 44822a561a
36 changed files with 2252 additions and 156 deletions

View File

@@ -0,0 +1,73 @@
# Design: File Display and Timezone Improvements
## Context
Users upload files during incident discussions, but these files only appear in a separate drawer. This breaks the conversation flow and makes it hard to understand what was being discussed when a file was uploaded. The AI report generator also lacks this context.
## Goals
- Files appear inline in chat at the time they were uploaded
- Images show thumbnails with click-to-expand preview
- AI reports include file context with surrounding messages
- All timestamps display in GMT+8
## Non-Goals
- Drag-and-drop upload (future enhancement)
- Video file preview (out of scope)
- File editing/annotation
## Decisions
### Decision 1: Link files to messages via foreign key
- Add `message_id` column to `tr_room_files` table
- When file is uploaded, create an `image_ref` or `file_ref` message
- Store the message_id in the file record
- Allows bidirectional lookup
### Decision 2: Image thumbnail generation
- **Option A**: Generate thumbnails server-side on upload (MinIO)
- **Option B**: Use MinIO presigned URL with CSS resize on frontend
- **Chosen**: Option B - simpler, no additional storage needed, modern browsers handle resizing well
### Decision 3: File message format
```json
{
"message_type": "image_ref",
"content": "optional caption from user",
"message_metadata": {
"file_id": "uuid",
"filename": "defect.jpg",
"file_type": "image",
"mime_type": "image/jpeg",
"file_size": 2621440,
"thumbnail_url": "presigned-url",
"download_url": "presigned-url"
}
}
```
### Decision 4: Lightbox implementation
- Use a simple modal-based image viewer
- Support keyboard navigation (ESC to close)
- Show loading state while image loads
### Decision 5: AI Report file context
- When collecting data, include the message content and surrounding 2 messages for each file
- Format: `[附件: filename.ext] - 上傳者: display_name, 說明: "{message_content}"`
## Risks / Trade-offs
| Risk | Mitigation |
|------|------------|
| Large images slow down chat | Use lazy loading, thumbnails |
| Existing files have no message | Migration script to create messages for orphan files |
| Presigned URLs expire | Frontend refreshes URLs on load |
## Migration Plan
1. Add `message_id` column to `tr_room_files` (nullable)
2. Create migration script to generate messages for existing files
3. Update upload API to create message + file atomically
4. Deploy backend changes
5. Deploy frontend with inline file display
6. Verify existing files display correctly
## Open Questions
- Should file deletion also delete the associated message? (Proposed: Yes, soft delete both)

View File

@@ -0,0 +1,23 @@
# Change: Improve File Display in Chat and Fix Timezone Issues
## Why
Currently, uploaded files and images only appear in the "file drawer" sidebar, making it difficult to understand the context of when and why a file was uploaded. This also impacts AI report generation, as the AI cannot associate files with the conversation context. Additionally, some parts of the application still display times in GMT+0 instead of GMT+8.
## What Changes
- **BREAKING**: Message creation when uploading files - files now generate a linked message in chat
- Display uploaded files and images inline in the chat conversation
- Add image preview functionality with lightbox
- Show image thumbnails in chat messages (expandable to full size)
- Non-image files display with file type icons
- Update upload interface to show preview before sending
- Update AI report data collection to include file context (associated messages)
- Fix remaining GMT+0 timestamp displays to use GMT+8
## Impact
- Affected specs: file-storage, realtime-messaging, ai-report-generation
- Affected code:
- Backend: `app/modules/file_storage/router.py`, `app/modules/realtime/` (message creation on upload)
- Backend: `app/modules/report_generation/services/report_data_service.py`
- Frontend: `frontend/src/pages/RoomDetail.tsx`
- Frontend: New components `FileMessage.tsx`, `ImagePreview.tsx`
- Database: May need migration to add `message_id` FK to `tr_room_files`

View File

@@ -0,0 +1,95 @@
## MODIFIED Requirements
### Requirement: Report Data Collection
The system SHALL collect all relevant room data for AI processing, including messages, members, files with their conversation context, and room metadata.
#### Scenario: Collect complete room data for report generation
- **GIVEN** an incident room with ID `room-123` exists
- **AND** the room has 50 messages from 5 members
- **AND** the room has 3 uploaded files (2 images, 1 PDF)
- **WHEN** the report data service collects room data
- **THEN** the system SHALL return a structured data object containing:
- Room metadata (title, incident_type, severity, status, location, description, timestamps)
- All 50 messages sorted by created_at ascending
- All 5 members with their roles (owner, editor, viewer)
- All 3 files with metadata (filename, type, uploader, upload time) AND their associated message context
- **AND** messages SHALL include sender display name (not just user_id)
- **AND** file references in messages SHALL be annotated with surrounding context
#### Scenario: Include file context in report data
- **GIVEN** a file "defect_photo.jpg" was uploaded with the message "發現產品表面瑕疵"
- **AND** the previous message was "Line 3 溫度異常升高中"
- **AND** the next message was "已通知維修人員處理"
- **WHEN** report data is collected
- **THEN** the file entry SHALL include:
```json
{
"file_id": "...",
"filename": "defect_photo.jpg",
"uploader_display_name": "陳工程師",
"uploaded_at": "2025-12-08T14:30:00+08:00",
"caption": "發現產品表面瑕疵",
"context_before": "Line 3 溫度異常升高中",
"context_after": "已通知維修人員處理"
}
```
- **AND** the AI prompt SHALL format files as:
`[附件: defect_photo.jpg] - 上傳者: 陳工程師 (14:30), 說明: "發現產品表面瑕疵" (前文: "Line 3 溫度異常升高中")`
#### Scenario: Handle room with no messages
- **GIVEN** an incident room was just created with no messages
- **WHEN** report generation is requested
- **THEN** the system SHALL return an error indicating insufficient data for report generation
- **AND** the error message SHALL be "事件聊天室尚無訊息記錄,無法生成報告"
#### Scenario: Summarize large rooms exceeding message limit
- **GIVEN** an incident room has 500 messages spanning 5 days
- **AND** the REPORT_MAX_MESSAGES limit is 200
- **WHEN** report data is collected
- **THEN** the system SHALL keep the most recent 150 messages in full
- **AND** summarize older messages by day (e.g., "2025-12-01: 45 則訊息討論設備檢修")
- **AND** the total formatted content SHALL stay within token limits
### Requirement: Document Assembly
The system SHALL assemble professional .docx documents from AI-generated content with embedded images from MinIO and file context from conversations.
#### Scenario: Generate complete report document
- **GIVEN** DIFY has returned valid JSON report content
- **AND** the room has 2 image attachments in MinIO
- **WHEN** the docx assembly service creates the document
- **THEN** the system SHALL create a .docx file with:
- Report title: "生產線異常處理報告 - {room.title}"
- Generation metadata: 生成時間, 事件編號, 生成者
- Section 1: 事件摘要 (from AI summary.content)
- Section 2: 事件時間軸 (formatted table from AI timeline.events)
- Section 3: 參與人員 (formatted list from AI participants.members)
- Section 4: 處理過程 (from AI resolution_process.content)
- Section 5: 目前狀態 (from AI current_status)
- Section 6: 最終處置結果 (from AI final_resolution, if has_resolution=true)
- Section 7: 附件 (embedded images with captions + file list with context)
- **AND** images SHALL be embedded at appropriate size (max width 15cm)
- **AND** each image SHALL include its caption from the upload message
- **AND** document SHALL use professional formatting (標楷體 or similar)
#### Scenario: Handle missing images during assembly
- **GIVEN** a file reference exists in the database
- **BUT** the actual file is missing from MinIO
- **WHEN** the docx service attempts to embed the image
- **THEN** the system SHALL skip the missing image
- **AND** add a placeholder text: "[圖片無法載入: {filename}]"
- **AND** continue with document assembly
- **AND** log a warning with file_id and room_id
#### Scenario: Generate report for room without images
- **GIVEN** the room has no image attachments
- **WHEN** the docx assembly service creates the document
- **THEN** the system SHALL create a complete document without the embedded images section
- **AND** the attachments section SHALL show "本事件無附件檔案" if no files exist

View File

@@ -0,0 +1,105 @@
## MODIFIED Requirements
### Requirement: File Upload with Validation
The system SHALL accept multipart file uploads to incident rooms, validate file type and size, persist files to MinIO object storage with metadata tracking in PostgreSQL, AND create an associated message in the chat for context.
#### Scenario: Upload image to incident room
- **WHEN** a user with OWNER or EDITOR role uploads an image file via `POST /api/rooms/{room_id}/files`
```http
POST /api/rooms/room-123/files
Content-Type: multipart/form-data
Authorization: Bearer {jwt_token}
file: [binary data of defect.jpg, 2.5MB]
description: "Defect found on product batch A-45"
```
- **THEN** the system SHALL:
- Validate JWT token and extract user_id
- Verify user is member of room-123 with OWNER or EDITOR role
- Validate file MIME type is image/jpeg, image/png, or image/gif
- Validate file size <= 10MB
- Generate unique file_id (UUID)
- Upload file to MinIO bucket `task-reporter-files` at path `room-123/images/{file_id}.jpg`
- Create a message with `message_type=image_ref` containing file metadata
- Create database record in `room_files` table with `message_id` reference
- Return file metadata with presigned download URL (1-hour expiry) and message_id
#### Scenario: Upload document to incident room
- **WHEN** a user uploads a non-image file (PDF, log, etc.)
- **THEN** the system SHALL:
- Create a message with `message_type=file_ref`
- Store message_id in the file record
- Return file metadata with message_id
#### Scenario: Reject oversized file upload
- **WHEN** a user attempts to upload a 15MB PDF file
- **THEN** the system SHALL:
- Detect file size exceeds 20MB limit for documents
- Return 413 Payload Too Large error
- Include error message: "File size exceeds limit: 15MB > 20MB"
- NOT upload file to MinIO
- NOT create database record
- NOT create chat message
#### Scenario: Reject unauthorized file type
- **WHEN** a user attempts to upload an executable file (e.g., .exe, .sh, .bat)
- **THEN** the system SHALL:
- Detect MIME type not in whitelist
- Return 400 Bad Request error
- Include error message: "File type not allowed: application/x-msdownload"
- NOT upload file to MinIO
- NOT create database record
#### Scenario: Upload log file to incident room
- **WHEN** an engineer uploads a machine log file
```http
POST /api/rooms/room-456/files
Content-Type: multipart/form-data
file: [machine_error.log, 1.2MB]
description: "Equipment error log from 2025-11-17"
```
- **THEN** the system SHALL:
- Validate MIME type is text/plain
- Upload to MinIO at `room-456/logs/{file_id}.log`
- Store metadata with file_type='log'
- Create a message with message_type='file_ref'
- Return success response with file_id and message_id
## ADDED Requirements
### Requirement: File-Message Association
The system SHALL maintain a foreign key relationship between uploaded files and their associated chat messages, enabling contextual display of files in conversations.
#### Scenario: Query file with associated message
- **WHEN** a client requests file metadata via `GET /api/rooms/{room_id}/files/{file_id}`
- **THEN** the response SHALL include:
```json
{
"file_id": "550e8400-e29b-41d4-a716-446655440000",
"message_id": "msg-789",
"filename": "defect.jpg",
"file_type": "image",
"download_url": "...",
"uploaded_at": "2025-12-08T10:30:00+08:00"
}
```
#### Scenario: Delete file cascades to message
- **WHEN** a file is soft-deleted via `DELETE /api/rooms/{room_id}/files/{file_id}`
- **THEN** the system SHALL also soft-delete the associated message
- **AND** broadcast both `file_deleted` and `message_deleted` events
### Requirement: Image Thumbnail URLs
The system SHALL generate presigned URLs suitable for thumbnail display, allowing frontends to efficiently render image previews.
#### Scenario: File metadata includes thumbnail URL
- **WHEN** file metadata is returned for an image file
- **THEN** the response SHALL include a `thumbnail_url` field
- **AND** the URL SHALL be a presigned MinIO URL valid for 1 hour
- **AND** the frontend SHALL use CSS to constrain thumbnail display size
#### Scenario: Non-image files have no thumbnail
- **WHEN** file metadata is returned for a non-image file (PDF, log, etc.)
- **THEN** the response SHALL NOT include a `thumbnail_url` field
- **AND** the frontend SHALL display a file-type icon instead

View File

@@ -0,0 +1,133 @@
## MODIFIED Requirements
### Requirement: Message Types and Formatting
The system SHALL support various message types including text, image references, file references, and structured data for production incidents, with inline display of file attachments in the chat view.
#### Scenario: Text message with mentions
- **WHEN** a user sends a message with @mentions
```json
{
"content": "@maintenance_team Please check Line 3 immediately",
"mentions": ["maintenance_team@panjit.com.tw"]
}
```
- **THEN** the system SHALL parse and store mentions
- **AND** potentially trigger notifications to mentioned users
#### Scenario: Image reference message display
- **WHEN** a message with `message_type=image_ref` is rendered in the chat
- **THEN** the client SHALL display:
- A thumbnail of the image (max 300px width)
- The message content/caption below the image
- Sender name and timestamp
- A click-to-expand functionality
- **AND** clicking the thumbnail SHALL open a full-size preview lightbox
#### Scenario: File reference message display
- **WHEN** a message with `message_type=file_ref` is rendered in the chat
- **THEN** the client SHALL display:
- A file type icon (PDF, document, log, etc.)
- The filename
- File size in human-readable format
- A download button/link
- The message content/caption
- Sender name and timestamp
#### Scenario: Structured incident data
- **WHEN** reporting specific incident metrics
```json
{
"type": "message",
"message_type": "incident_data",
"content": {
"temperature": 85,
"pressure": 120,
"production_rate": 450,
"timestamp": "2025-11-17T10:15:00Z"
}
}
```
- **THEN** the system SHALL store structured data as JSON
- **AND** enable querying/filtering by specific fields later
### Requirement: GMT+8 Timezone Display
The frontend SHALL display all timestamps in GMT+8 (Asia/Taipei) timezone for consistent user experience across all browsers and all parts of the application.
#### Scenario: Message timestamp in GMT+8
- **WHEN** a message is displayed in the chat room
- **THEN** the timestamp SHALL be formatted in GMT+8 timezone
- **AND** use format "HH:mm" for today's messages
- **AND** use format "MM/DD HH:mm" for older messages
#### Scenario: Room list timestamps in GMT+8
- **WHEN** the room list is displayed
- **THEN** the "last updated" time SHALL be formatted in GMT+8 timezone
#### Scenario: File upload timestamp in GMT+8
- **WHEN** a file is displayed in chat or file drawer
- **THEN** the upload timestamp SHALL be formatted in GMT+8 timezone
#### Scenario: Report generation timestamp in GMT+8
- **WHEN** report metadata is displayed
- **THEN** the "generated at" timestamp SHALL be formatted in GMT+8 timezone
## ADDED Requirements
### Requirement: Image Preview Lightbox
The frontend SHALL provide a lightbox component for viewing full-size images from chat messages.
#### Scenario: Open image lightbox
- **WHEN** user clicks on an image thumbnail in the chat
- **THEN** a modal overlay SHALL appear
- **AND** the full-size image SHALL be displayed centered
- **AND** a loading indicator SHALL show while image loads
- **AND** the image SHALL be constrained to fit the viewport
#### Scenario: Close image lightbox
- **WHEN** the lightbox is open
- **THEN** user can close it by:
- Clicking the X button
- Pressing the ESC key
- Clicking outside the image
- **AND** focus SHALL return to the chat
#### Scenario: Image lightbox with download
- **WHEN** the lightbox is open
- **THEN** a download button SHALL be visible
- **AND** clicking it SHALL download the original file
### Requirement: File Type Icons
The frontend SHALL display appropriate icons for different file types in chat messages and file drawer.
#### Scenario: PDF file icon
- **WHEN** a PDF file is displayed
- **THEN** a PDF icon (red/document style) SHALL be shown
#### Scenario: Log/text file icon
- **WHEN** a .log or .txt file is displayed
- **THEN** a text file icon SHALL be shown
#### Scenario: Excel file icon
- **WHEN** an Excel file (.xlsx, .xls) is displayed
- **THEN** a spreadsheet icon (green) SHALL be shown
#### Scenario: Generic file icon
- **WHEN** a file with unknown type is displayed
- **THEN** a generic document icon SHALL be shown
### Requirement: Upload Preview
The frontend SHALL show a preview of the file being uploaded before the message is sent.
#### Scenario: Image upload preview
- **WHEN** user selects an image file for upload
- **THEN** a preview thumbnail SHALL be displayed in the input area
- **AND** user can add a caption/description
- **AND** user can cancel the upload before sending
- **AND** a send button confirms the upload
#### Scenario: File upload preview
- **WHEN** user selects a non-image file for upload
- **THEN** file info (name, size, type icon) SHALL be displayed
- **AND** user can add a description
- **AND** user can cancel or confirm

View File

@@ -0,0 +1,134 @@
# Tasks: Improve File Display and Timezone
## Phase 1: Database & Backend Foundation
### T-1.1: Database Migration
- [x] Add `message_id` column to `tr_room_files` table (nullable FK to `tr_messages`)
- [x] Create Alembic migration script
- [x] Test migration on dev database
### T-1.2: File Upload API Update
- [x] Modify `POST /api/rooms/{room_id}/files` to create associated message
- [x] Create message with `message_type=image_ref` or `file_ref`
- [x] Store message_id in file record
- [x] Return message_id in response
- [x] Update WebSocket broadcast to include message
### T-1.3: File Metadata API Update
- [x] Add `message_id` to file metadata response
- [x] Add `thumbnail_url` field for image files
- [x] Ensure timestamps are formatted in GMT+8
### T-1.4: File Deletion Cascade
- [x] Update DELETE endpoint to soft-delete associated message
- [x] Broadcast both `file_deleted` and `message_deleted` events
## Phase 2: Frontend File Display
### T-2.1: File Message Component
- [x] Create `FileMessage.tsx` component
- [x] Support `image_ref` type with thumbnail
- [x] Support `file_ref` type with icon
- [x] Display caption, sender, timestamp
- [x] Add download button
### T-2.2: File Type Icons
- [x] Create file icon mapping (PDF, Excel, text, generic)
- [x] Use appropriate colors (red for PDF, green for Excel, etc.)
- [x] Display file size in human-readable format
### T-2.3: Image Lightbox Component
- [x] Create `ImageLightbox.tsx` modal component
- [x] Support keyboard navigation (ESC to close)
- [x] Add loading state
- [x] Add download button
- [x] Constrain image to viewport
### T-2.4: Integrate File Messages in Chat
- [x] Update `RoomDetail.tsx` to render `FileMessage` for image_ref/file_ref
- [x] Lazy load images for performance
- [x] Handle click to open lightbox
## Phase 3: Upload Experience
### T-3.1: Upload Preview Component
- [x] Create `UploadPreview.tsx` component
- [x] Show thumbnail for images before upload
- [x] Show file info (name, size, icon) for documents
- [x] Add caption/description input field
- [x] Add cancel and send buttons
### T-3.2: Update ActionBar Upload Flow
- [x] Modify file selection to show preview instead of immediate upload
- [x] Integrate `UploadPreview` into input area
- [x] Handle upload progress in preview
- [x] Clear preview after successful upload
## Phase 4: AI Report Context
### T-4.1: Report Data Service Update
- [x] Modify `report_data_service.py` to fetch file context
- [x] Join files with their associated messages
- [x] Include surrounding messages (1 before, 1 after)
- [x] Format file entries with context for AI prompt
### T-4.2: Update AI Prompt
- [x] Modify prompt to explain file context format
- [x] Include file captions and context in prompt
### T-4.3: Update DOCX Assembly
- [x] Add captions below embedded images
- [x] Include file context in attachments section
## Phase 5: Timezone Fixes
### T-5.1: Audit Timezone Usage
- [x] Review all timestamp displays in frontend
- [x] Identify any remaining GMT+0 usages
- [x] Document locations needing fixes
### T-5.2: Apply GMT+8 Consistently
- [x] Update file drawer timestamps to use `formatDateTimeGMT8`
- [x] Update report metadata timestamps
- [x] Update any API responses returning timestamps
- [x] Verify backend returns ISO strings (UTC is fine, frontend converts)
## Phase 6: Migration & Testing
### T-6.1: Data Migration for Existing Files
- [x] Create script to generate messages for orphan files
- [x] Associate existing files with generated messages
- [x] Preserve original upload timestamps
### T-6.2: Testing
- [x] Test file upload creates message (API verified)
- [x] Test image thumbnail display in chat (component verified)
- [x] Test lightbox open/close/download (component verified)
- [x] Test non-image file display with icons (component verified)
- [x] Test upload preview flow (component verified)
- [x] Test AI report includes file context (code verified)
- [x] Test all timestamps in GMT+8 (code verified)
- [x] Build verification (frontend builds successfully)
## Implementation Summary
### Files Modified/Created:
**Backend:**
- `app/modules/file_storage/models.py` - Added `message_id` FK column
- `app/modules/file_storage/schemas.py` - Added `message_id`, `thumbnail_url` fields
- `app/modules/file_storage/services/file_service.py` - Updated upload/delete to create/cascade messages
- `app/modules/file_storage/router.py` - Updated broadcasts with message_id
- `app/modules/realtime/schemas.py` - Added `MessageDeletedBroadcast`, updated file broadcasts
- `app/modules/report_generation/services/report_data_service.py` - Added file context fetching
- `app/modules/report_generation/services/docx_service.py` - GMT+8 timestamps, captions
- `app/modules/report_generation/prompts.py` - GMT+8 timestamps, file context format
- `alembic/versions/a1b2c3d4e5f6_add_message_id_to_room_files.py` - Migration
- `scripts/migrate_orphan_files.py` - Data migration script
**Frontend:**
- `frontend/src/types/index.ts` - Updated types with message_id, thumbnail_url
- `frontend/src/components/chat/FileMessage.tsx` - New component for file messages
- `frontend/src/components/chat/ImageLightbox.tsx` - New lightbox component
- `frontend/src/components/chat/UploadPreview.tsx` - New upload preview component
- `frontend/src/pages/RoomDetail.tsx` - Integrated FileMessage, UploadPreview