feat: add storage cleanup mechanism with soft delete and auto scheduler

- Add soft delete (deleted_at column) to preserve task records for statistics
- Implement cleanup service to delete old files while keeping DB records
- Add automatic cleanup scheduler (configurable interval, default 24h)
- Add admin endpoints: storage stats, cleanup trigger, scheduler status
- Update task service with admin views (include deleted/files_deleted)
- Add frontend storage management UI in admin dashboard
- Add i18n translations for storage management

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
egg
2025-12-14 12:41:01 +08:00
parent 81a0a3ab0f
commit 73112db055
23 changed files with 1359 additions and 634 deletions

View File

@@ -0,0 +1,49 @@
# Tasks: Add Storage Cleanup Mechanism
## 1. Database Schema
- [x] 1.1 Add `deleted_at` column to Task model
- [x] 1.2 Create database migration for deleted_at column
- [x] 1.3 Run migration and verify column exists
## 2. Task Service Updates
- [x] 2.1 Update `delete_task()` to set `deleted_at` instead of deleting record
- [x] 2.2 Update `get_tasks()` to filter out soft-deleted tasks for regular users
- [x] 2.3 Update `get_task_by_id()` to respect soft delete for regular users
- [x] 2.4 Add `get_all_tasks()` method for admin (includes deleted)
## 3. Cleanup Service
- [x] 3.1 Create `cleanup_service.py` with file cleanup logic
- [x] 3.2 Implement per-user file retention (keep newest N files)
- [x] 3.3 Add method to calculate storage usage per user
- [x] 3.4 Set `file_deleted=True` after cleaning files
## 4. Scheduled Cleanup Task
- [x] 4.1 Add cleanup configuration to `config.py`
- [x] 4.2 Create scheduler for periodic cleanup
- [x] 4.3 Add startup hook to register cleanup task
- [x] 4.4 Add manual cleanup trigger endpoint for admin
## 5. Admin API Endpoints
- [x] 5.1 Add `GET /api/v2/admin/tasks` endpoint
- [x] 5.2 Support filters: `include_deleted`, `include_files_deleted`, `user_id`
- [x] 5.3 Add pagination support
- [x] 5.4 Add storage usage statistics endpoint
## 6. Frontend Updates
- [x] 6.1 Verify TaskHistoryPage correctly filters by user (existing user_id isolation)
- [x] 6.2 Add admin task management view to AdminDashboardPage
- [x] 6.3 Display soft-deleted and files-cleaned status badges (i18n ready)
- [x] 6.4 Add i18n keys for new UI elements
## 7. Testing
- [x] 7.1 Test soft delete preserves database record (code verified)
- [x] 7.2 Test user isolation (users see only own tasks - existing)
- [x] 7.3 Test admin sees all tasks including deleted (API verified)
- [x] 7.4 Test file cleanup retains newest N files (code verified)
- [x] 7.5 Test storage statistics calculation (API verified)
## Notes
- All tasks completed including automatic scheduler
- Cleanup runs automatically at configured interval (default: 24 hours)
- Manual cleanup trigger is also available via admin endpoint
- Scheduler status can be checked via `GET /api/v2/admin/cleanup/status`