Commit Graph

35 Commits

Author SHA1 Message Date
egg
d8e91624d4 feat(qc-gate): add Package column to LOT detail table + archive 3 completed changes
Add PACKAGE_LEF as a dedicated `package` field in the QC-GATE API payload
and display it as a new column after LOT ID in LotTable.vue. Archive
qc-gate-lot-package-column, historical-query-slow-connection, and
msd-multifactor-backward-tracing changes with their delta specs synced
to main specs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 08:44:26 +08:00
egg
07ced80fb0 feat(admin-perf): full Vue SPA migration + slow-query/memory monitoring gaps
Remove Jinja2 template fallback (1249 lines) — /admin/performance now serves
Vue SPA exclusively via send_from_directory.

Backend:
- Add _SLOW_QUERY_WAITING counter with get_slow_query_waiting_count()
- Record slow-path latency in read_sql_df_slow/iter via record_query_latency()
- Extend metrics_history schema with slow_query_active, slow_query_waiting,
  worker_rss_bytes columns + ALTER TABLE migration for existing DBs
- Add cleanup_archive_logs() with configurable ARCHIVE_LOG_DIR/KEEP_COUNT
- Integrate archive cleanup into MetricsHistoryCollector 50-min cycle

Frontend:
- Add slow_query_active and slow_query_waiting StatCards to connection pool
- Add slow_query_active trend line to pool trend chart
- Add Worker memory (RSS MB) trend chart with preprocessing
- Update modernization gate check path to frontend style.css

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 09:48:54 +08:00
egg
dbe0da057c feat(trace-pipeline): memory triage, async job queue, and NDJSON streaming
Three proposals addressing the 2026-02-25 trace pipeline OOM crash (114K CIDs):

1. trace-events-memory-triage: fetchmany iterator (read_sql_df_slow_iter),
   admission control (50K CID limit for non-MSD), cache skip for large queries,
   early memory release with gc.collect()

2. trace-async-job-queue: RQ-based async jobs for queries >20K CIDs,
   separate worker process with isolated memory, frontend polling via
   useTraceProgress composable, systemd service + deploy scripts

3. trace-streaming-response: chunked Redis storage (TRACE_STREAM_BATCH_SIZE=5000),
   NDJSON stream endpoint (GET /api/trace/job/<id>/stream), frontend
   ReadableStream consumer for progressive rendering, backward-compatible
   with legacy single-key storage

All three proposals archived. 1101 tests pass, frontend builds clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 21:01:27 +08:00
egg
cbb943dfe5 feat(trace-pool-isolation): migrate event_fetcher/lineage_engine to slow connections + fix 51 test failures
Trace pipeline pool isolation:
- Switch event_fetcher and lineage_engine to read_sql_df_slow (non-pooled)
- Reduce EVENT_FETCHER_MAX_WORKERS 4→2, TRACE_EVENTS_MAX_WORKERS 4→2
- Add 60s timeout per batch query, cache skip for CID>10K
- Early del raw_domain_results + gc.collect() for large queries
- Increase DB_SLOW_MAX_CONCURRENT: base 3→5, dev 2→3, prod 3→5

Test fixes (51 pre-existing failures → 0):
- reject_history: WORKFLOW CSV header, strict bool validation, pareto mock path
- portal shell: remove non-existent /tmtt-defect route from tests
- conftest: add --run-stress option to skip stress/load tests by default
- migration tests: skipif baseline directory missing
- performance test: update Vite asset assertion
- wip hold: add firstname/waferdesc mock params
- template integration: add /reject-history canonical route

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 16:13:19 +08:00
egg
71c8102de6 feat: dataset cache for hold/resource history + slow connection migration
Two changes combined:

1. historical-query-slow-connection: Migrate all historical query pages
   to read_sql_df_slow with semaphore concurrency control (max 3),
   raise DB slow timeout to 300s, gunicorn timeout to 360s, and
   unify frontend timeouts to 360s for all historical pages.

2. hold-resource-history-dataset-cache: Convert hold-history and
   resource-history from multi-query to single-query + dataset cache
   pattern (L1 ProcessLevelCache + L2 Redis parquet/base64, TTL=900s).
   Replace old GET endpoints with POST /query + GET /view two-phase
   API. Frontend auto-retries on 410 cache_expired.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 13:15:02 +08:00
egg
cd061e0cfd test(msd): add unit tests for _build_detail_table structured output and CSV flatten
Covers task 14.3: verify UPSTREAM_MACHINES/UPSTREAM_MATERIALS list format,
WAFER_ROOT field, multi-reason row expansion, machine deduplication,
and CSV export flatten logic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 11:28:28 +08:00
egg
f14591c7dc feat(mid-section-defect): full-line bidirectional defect trace center with dual query mode
Transform /mid-section-defect from TMTT-only backward analysis into a full-line
bidirectional defect traceability center supporting all detection stations.

Key changes:
- Parameterized station detection: any workcenter group as detection station
- Bidirectional tracing: backward (upstream attribution) + forward (downstream reject rates)
- Dual query mode: date range OR LOT/工單/WAFER container-based seed resolution
- Multi-select filters for upstream station, equipment model (RESOURCEFAMILYNAME), and loss reasons
- Progressive 3-stage trace pipeline (seed-resolve → lineage → events) with streaming UI
- Equipment model lookup via resource cache instead of SPECNAME
- Session caching, auto-refresh, searchable MultiSelect with fuzzy matching
- Remove legacy tmtt-defect module (fully superseded)
- Archive openspec change artifacts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 16:16:33 +08:00
egg
e5d7700b36 harden released pages and archive openspec change 2026-02-23 17:48:32 +08:00
egg
5d570ca7a2 feat(admin-performance): Vue 3 SPA dashboard with metrics history trending
Rebuild /admin/performance from Jinja2 to Vue 3 SPA with ECharts, adding
cache telemetry infrastructure, connection pool monitoring, and SQLite-backed
historical metrics collection with trend chart visualization.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 09:18:10 +08:00
egg
1c46f5eb69 chore(openspec): archive query-tool change and commit remaining updates 2026-02-23 07:10:51 +08:00
egg
9890586191 feat(query-tool): align lineage model and tighten timeline mapping 2026-02-22 17:36:47 +08:00
egg
5e5cc487ac feat: archive filter strategy change and optimize reject-history filters 2026-02-22 12:50:05 +08:00
egg
7bf9e33cd5 feat: polish reject history UI and enhance WIP filter interactions 2026-02-22 11:54:51 +08:00
egg
05d907ac72 feat(reject-history): ship report page and archive openspec change 2026-02-13 20:35:52 +08:00
egg
248cbc25e0 fix(query-tool): batch detail loading, UX polish, and docs cleanup
- Fix multi-WO display: auto-select all tree roots after resolve so detail
  panel loads data for every work order, not just the first seed CID
- Disable scroll-wheel zoom on lineage tree (roam: 'move') to prevent
  accidental layout jumps while preserving drag-pan
- Add batch API endpoints (get_lot_history_batch, get_lot_associations_batch)
  to avoid N parallel requests hitting rate limits
- Remove redundant Split sub-tab from LOT detail (tree already shows splits)
- Rename 退貨 → 報廢 to match actual reject/scrap data semantics
- Hide internal ID columns (CONTAINERID, EQUIPMENTID, RESOURCEID) from
  history table display
- Add timeline scroll container and time range header for long timelines
- Remove obsolete migration and architecture docs no longer needed

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 17:42:11 +08:00
egg
5b358d71c1 feat(query-tool): rewrite frontend with ECharts tree, multi-select, and modular composables
Replace the monolithic useQueryToolData composable and nested Vue component tree
with a modular architecture: useLotResolve, useLotLineage, useLotDetail, and
useEquipmentQuery. Introduce ECharts TreeChart (LR orthogonal layout) for lot
lineage visualization with multi-select support, subtree expansion, zoom/pan,
and serial number normalization. Add unified LineageEngine backend with split
descendant traversal and leaf serial number queries. Archive the query-tool-rewrite
openspec change and sync delta specs to main.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 15:25:00 +08:00
egg
5a47bc87d8 fix(sql): remove colon prefix from SQL comments to prevent bind param errors, archive trace-progressive-ui
SQLAlchemy text() parses :param patterns in SQL comments as bind
parameters. When EventFetcher replaces the WHERE clause via string
substitution, orphaned :container_id in comments causes
"A value is required for bind parameter 'container_id'" errors.

Changes:
- Remove colon prefix from parameter names in SQL comments for
  lot_history, lot_rejects, lot_holds, lot_materials
- Archive trace-progressive-ui change (22/22 tasks complete)
- Sync delta specs to main: add trace-staged-api, progressive-trace-ux,
  merge api-safety-hygiene (+2 requirements)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 16:53:54 +08:00
egg
519f8ae2f4 feat(lineage): unified LineageEngine, EventFetcher, and progressive trace API
Introduce a unified Seed→Lineage→Event pipeline replacing per-page Python
BFS with Oracle CONNECT BY NOCYCLE queries, add staged /api/trace/*
endpoints with rate limiting and L2 Redis caching, and wire progressive
frontend loading via useTraceProgress composable.

Key changes:
- Add LineageEngine (split ancestors / merge sources / full genealogy)
  with QueryBuilder bind-param safety and batched IN clauses
- Add EventFetcher with 6-domain support and L2 Redis cache
- Add trace_routes Blueprint (seed-resolve, lineage, events) with
  profile dispatch, rate limiting, and Redis TTL=300s caching
- Refactor query_tool_service to use LineageEngine and QueryBuilder,
  removing raw string interpolation (SQL injection fix)
- Add rate limits and resolve cache to query_tool_routes
- Integrate useTraceProgress into mid-section-defect with skeleton
  placeholders and fade-in transitions
- Add lineageCache and on-demand lot lineage to query-tool
- Add TraceProgressBar shared component
- Remove legacy query-tool.js static script (3k lines)
- Fix MatrixTable package column truncation (.slice(0,15) removed)
- Archive unified-lineage-engine change, add trace-progressive-ui specs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 16:30:24 +08:00
egg
c38b5f646a feat(modernization): promote deferred routes to in-scope and unify page header styles
Promote /tables, /excel-query, /query-tool, /mid-section-defect from
deferred to full shell-governed in-scope routes with canonical redirects,
content contracts, governance artifacts, and updated CI gates.

Unify all page header gradients to #667eea → #764ba2 and h1 font-size
to 24px for visual consistency across all dashboard pages. Remove
Native Route-View dev annotations from job-query, excel-query, and
query-tool headers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 13:20:06 +08:00
egg
7cb0985b12 feat(modernization): full architecture blueprint with hardening follow-up
Implement phased modernization infrastructure for transitioning from
multi-page legacy routing to SPA portal-shell architecture, plus
post-delivery hardening fixes for policy loading, fallback consistency,
and governance drift detection.

Key changes:
- Add route contract enrichment with scope/visibility/compatibility policies
- Canonical 302 redirects from legacy direct-entry to /portal-shell/ routes
- Asset readiness enforcement and runtime fallback retirement for in-scope routes
- Shared feature-flag helpers (env > config > default) replacing duplicated _to_bool
- Defensive copy for lru_cached policy payloads preventing mutation corruption
- Unified retired-fallback response helper across app and blueprint routes
- Frontend/backend route-contract cross-validation in governance gates
- Shell CSS token fallback values for routes rendered outside shell scope
- Local-safe .env.example defaults with production recommendation comments
- Legacy contract fallback warning logging and single-hop redirect optimization

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 11:26:02 +08:00
egg
35d83d424c feat(shell): fluid layout with collapsible sidebar drawer + fix query-tool MultiSelect
Convert portal shell from block-centered (max-width 1600px) layout to full-viewport
fluid flexbox with collapsible sidebar: desktop push-mode (240px → 0), mobile overlay
drawer with backdrop. Rename .content → .shell-content to avoid CSS collision with
page-level classes. Override page-level max-width constraints when embedded in shell.

Also replace native <select multiple> in query-tool with shared MultiSelect component
for equipment and workcenter group filters, matching resource-status/history UX.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 18:04:55 +08:00
egg
1e7f8f4498 feat: finalize no-iframe portal shell route-view migration 2026-02-11 17:07:50 +08:00
egg
ccab10bee8 feat: finalize portal no-iframe migration baseline and archive change 2026-02-11 13:25:03 +08:00
egg
e2ce75b004 fix(hold): dedup equipment cache, fix portal iframe, improve Hold dashboards
- Equipment cache: add freshness gate so only 1 Oracle query per 5-min cycle
  across 4 gunicorn workers; sync worker waits before first refresh
- Portal: add frame-busting to prevent recursive iframe nesting
- Hold Overview: remove redundant TreeMap, add Product & Future Hold Comment
  columns to LotTable
- Hold History: switch list.sql JOIN from DW_MES_LOT_V (WIP snapshot) to
  DW_MES_CONTAINER (historical master) for reliable Product data; add
  Future Hold Comment column; fix comment truncation with hover tooltip
- Page status: reorganize drawer groupings

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 09:01:02 +08:00
egg
be22571421 feat(wip): preserve filters between Overview and Detail with thundering-herd fix
URL is now single source of truth for filter state (workorder, lotid,
package, type, status) across WIP Overview and Detail pages. Drill-down
carries all filters + status; back button dynamically reflects Detail
changes. Backend Detail API now supports pj_type filter parameter.

Harden concurrency: add pagehide abort for MPA navigation, double-check
locking on Redis JSON parse and snapshot build to prevent thread pool
saturation during rapid page switching. Fix watchdog setsid and PID
discovery. Fix test_realtime_equipment_cache RUNCARDLOTID field mismatch.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 07:22:48 +08:00
egg
9a4e08810b feat(hold-history): add Hold 歷史績效 Dashboard with trend, pareto, duration, and detail views
New independent report page based on DWH.DW_MES_HOLDRELEASEHISTORY providing
historical hold/release performance analysis. Includes daily trend with Redis
caching, reason Pareto with click-to-filter, duration distribution with
click-to-filter, multi-select record type filter (new/on_hold/released),
workcenter-group mapping via memory cache, and server-side paginated detail
table. All 32 backend tests passing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 18:03:08 +08:00
egg
8225863a85 feat(hold-overview): add Hold Lot Overview page with TreeMap, Matrix, and cascade filtering
Provide managers with a dedicated page to analyze hold lots across all stations.
Extends existing service functions (get_hold_detail_summary, get_hold_detail_lots,
get_wip_matrix) with optional parameters for backward compatibility, adds one new
function (get_hold_overview_treemap), and registers the page in the portal navigation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 13:02:24 +08:00
egg
af59031f95 feat(mid-section-defect): harden with distributed lock, rate limit, filter separation, abort, SQL classification and tests
Address 6 code review findings (P0-P3): add Redis distributed lock to prevent
duplicate Oracle pipeline on cold cache, apply rate limiting to 3 high-cost
routes, separate UI filter state from committed query state, add AbortController
for request cancellation, push workcenter group classification into Oracle SQL
CASE WHEN, and add 18 route+service tests. Also add workcenter group selection
to job-query equipment selector and rename button to "查詢".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:32:14 +08:00
egg
720e190bc6 feat(resource): migrate resource-status and resource-history from Jinja2 to Vue 3 + Vite
Rewrite both resource pages (1,697 lines vanilla JS + 3,200 lines Jinja2 templates)
as Vue 3 SFC components. Extract resource-shared/ module with shared CSS, E10 status
constants, and HierarchyTable tree component. History page charts use vue-echarts,
Status page reuses useAutoRefresh composable with 5-minute interval.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 18:19:32 +08:00
egg
a2653b8139 feat(wip): migrate WIP trio pages from Jinja2 to Vue 3 + Vite
Migrate /wip-overview, /wip-detail, and /hold-detail (1,941 lines vanilla JS)
to Vue 3 SFC architecture. Extract shared CSS/constants/components to
wip-shared/. Switch Pareto charts to vue-echarts with autoresize. Replace
Jinja2 template injection with frontend URL params + constant classification
for Hold Detail. Add 10-min auto-refresh + AbortController to Hold Detail.
Remove three Jinja2 templates, update Flask routes to send_from_directory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 16:39:20 +08:00
egg
dcbf6dcf1f feat(tables): migrate /tables page from Jinja2 to Vue 3 + Vite
Rewrite 237-line vanilla JS + Jinja2 template into Vue 3 SFC components
(App.vue, TableCatalog.vue, DataViewer.vue, useTableData composable).
Establishes apiPost POST request pattern for pure Vite pages. Removes
templates/index.html, updates Vite entry to HTML, and Flask route to
send_from_directory. Includes sql_fragments WHERE_CLAUSE escaping fix,
updated integration tests, and OpenSpec artifact archive.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 14:52:14 +08:00
egg
bf7285fb51 feat(qc-gate): add QC-GATE real-time LOT status report as first pure Vue 3 + Vite page
Introduce QC-GATE station monitoring with stacked bar chart and filterable LOT table,
using Vue 3 SFC + ECharts via npm. Establishes the pure Vite page architecture pattern
(no Jinja2) for future page migration. Also removes stale design files and README.mdj.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 13:06:59 +08:00
egg
9b1d2edc52 feat(portal): implement dynamic drawer/page navigation management
Replace hardcoded sidebar drawer configuration with admin-manageable
dynamic system. Extend page_status.json with drawer definitions and
page assignments, add drawer CRUD API endpoints, render portal sidebar
via Jinja2 loops, and extend /admin/pages UI with drawer management.
Fix multi-worker cache invalidation via mtime-based staleness detection.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 11:34:04 +08:00
beabigegg
c8e225101e chore: finalize vite migration hardening and archive openspec changes 2026-02-08 20:03:36 +08:00
beabigegg
b56e80381b chore: reinitialize project with vite architecture 2026-02-08 08:30:48 +08:00