- Add soft delete (deleted_at column) to preserve task records for statistics
- Implement cleanup service to delete old files while keeping DB records
- Add automatic cleanup scheduler (configurable interval, default 24h)
- Add admin endpoints: storage stats, cleanup trigger, scheduler status
- Update task service with admin views (include deleted/files_deleted)
- Add frontend storage management UI in admin dashboard
- Add i18n translations for storage management
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add TranslationLog model to track translation API usage per task
- Integrate Dify API actual price (total_price) into translation stats
- Display translation statistics in admin dashboard with per-task costs
- Remove unused Export and Settings pages to simplify frontend
- Add GET /api/v2/admin/translation-stats endpoint
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Force Office documents (PPTX, DOCX, XLSX) to use Direct track after
LibreOffice conversion, since converted PDFs always have extractable text
- Fix PDF generator to not exclude text in image regions for Direct track,
allowing text to render on top of background images (critical for PPT)
- Increase file_type column from VARCHAR(50) to VARCHAR(100) to support
long MIME types like PPTX
- Remove reference to non-existent total_images metadata attribute
This significantly improves processing time for Office documents
(from ~170s OCR to ~10s Direct) while preserving text quality.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add TableData.from_dict() and TableCell.from_dict() methods to convert
JSON table dicts to proper TableData objects during UnifiedDocument parsing.
Modified _json_to_document_element() to detect TABLE elements with dict
content containing 'cells' key and convert to TableData.
Note: This fix ensures table elements have proper to_html() method available
but the rendered output still needs investigation - tables may still render
incorrectly in OCR track PDFs.
Files changed:
- unified_document.py: Add from_dict() class methods
- pdf_generator_service.py: Convert table dicts during JSON parsing
- Add fix-ocr-track-table-rendering proposal
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add optional original_filename field to DocumentMetadata dataclass
to properly store the original filename when files are converted
(e.g., Office → PDF). This ensures the field is included in to_dict()
output for JSON serialization.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Added foundation for dual-track document processing:
1. UnifiedDocument Model (backend/app/models/unified_document.py)
- Common output format for both OCR and direct extraction
- Comprehensive element types (23+ types from PP-StructureV3)
- BoundingBox, StyleInfo, TableData structures
- Backward compatibility with legacy format
2. DocumentTypeDetector Service (backend/app/services/document_type_detector.py)
- Intelligent document type detection using python-magic
- PDF editability analysis using PyMuPDF
- Processing track recommendation with confidence scores
- Support for PDF, images, Office docs, and text files
3. DirectExtractionEngine Service (backend/app/services/direct_extraction_engine.py)
- Fast extraction from editable PDFs using PyMuPDF
- Preserves fonts, colors, and exact positioning
- Native and positional table detection
- Image extraction with coordinates
- Hyperlink and metadata extraction
4. Dependencies
- Added PyMuPDF>=1.23.0 for PDF extraction
- Added pdfplumber>=0.10.0 as fallback
- Added python-magic-bin>=0.4.14 for file detection
Next: Integrate with OCR service for complete dual-track processing
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Remove all V1 architecture components and promote V2 to primary:
- Delete all paddle_ocr_* table models (export, ocr, translation, user)
- Delete legacy routers (auth, export, ocr, translation)
- Delete legacy schemas and services
- Promote user_v2.py to user.py as primary user model
- Update all imports and dependencies to use V2 models only
- Update main.py version to 2.0.0
Database changes:
- Fix SQLAlchemy reserved word: rename audit_log.metadata to extra_data
- Add migration to drop all paddle_ocr_* tables
- Update alembic env to only import V2 models
Frontend fixes:
- Fix Select component exports in TaskHistoryPage.tsx
- Update to use simplified Select API with options prop
- Fix AxiosInstance TypeScript import syntax
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
