Back to Product

AI selection architecture document

AI Selection Architecture Document: Video Search Desktop Client

1. Introduction

Project: Video Search macOS Desktop Application
Objective: Enable offline video content retrieval using on-device OCR for English/Chinese text recognition with sub-second response times.

2. AI Requirements Analysis

Requirement Specification
OCR Accuracy ≥95% for English/Chinese (mixed text scenarios)
Processing Speed <1s per video frame (1080p resolution)
Offline Support Zero network dependencies
Resource Constraints Max 500MB RAM usage during indexing
Language Support Simplified Chinese + English (expandable)

3. Core AI Technology Selection

Component Technology Version Rationale
OCR Engine Apple Vision Framework macOS 13+ Native Metal-accelerated text recognition with CN/EN support
Video Processing AVFoundation + Core ML AVF 4.3, Core ML 7 Hardware-accelerated frame extraction
Indexing Engine SQLite FTS5 3.42+ In-memory full-text search with <50ms query latency
Language Models Apple's on-device ML models VisionKit 1.3 Pre-trained Chinese/English text detection

4. Architecture Overview

🔄 正在加载流程图...

graph TD A[Video Input] --> B[AVFoundation Frame Extraction] B --> C[Vision Framework OCR Processing] C --> D[Text Normalization] D --> E[SQLite FTS5 Indexing] E --> F[Query Interface] F --> G[Timestamped Results]

5. Implementation Steps

  1. Frame Sampling

    • Use AVAssetImageGenerator at 1fps intervals
    • Resolution scaling: 1920x1080 → 960x540 (50% reduction)
    • Color space: Grayscale conversion for OCR optimization

  2. OCR Pipeline 

    let request = VNRecognizeTextRequest()
request.recognitionLevel = .accurate
request.recognitionLanguages = ["zh-Hans", "en"]
request.usesLanguageCorrection = true

  3. Indexing Strategy

    • Create virtual FTS5 table:
      CREATE VIRTUAL TABLE video_index USING fts5(frame_time, text_content)
    • Implement incremental indexing with transaction batching

  4. Query Processing

    • Use BM25 ranking:
      SELECT frame_time FROM video_index WHERE text MATCH 'keyword' ORDER BY rank

6. Performance Optimization

  • GPU Acceleration: Enable Metal Performance Shaders for Vision framework
  • Memory Management:
    • Frame cache limit: 100 MB
    • SQLite WAL mode with PRAGMA synchronous=NORMAL
  • Concurrency:
    • Grand Central Dispatch queues (QoS: .userInitiated)
    • Parallel frame processing (4 threads max)

7. Security & Privacy Controls

Aspect Implementation
Data Storage AES-256 encryption for index database
Permissions Sandboxed access with user-granted entitlements
Data Lifecycle Automatic purge of OCR data upon video removal
Compliance GDPR-ready via on-device processing only

8. Scalability Extensions

  • Modular OCR Engine: Replaceable Core ML model container
  • Language Expansion: Plug-in architecture for additional languages via Apple's ML model zoo
  • Cloud Hybrid Mode (Optional): Secure enclave key management for encrypted cloud indexing

9. Benchmarks

Metric Baseline Target
Indexing Speed 5 min/hr video 3 min/hr video
Query Latency 200ms <80ms
OCR Accuracy (Chinese) 92.3% 96.1%
Memory Footprint 650MB 420MB

10. Validation Plan

  1. Unit Testing: XCTest cases for OCR accuracy thresholds
  2. Stress Testing: 500+ video corpus (10,000 hours total)
  3. User Testing: Precision/recall metrics with real-world queries

Approvals
Technical Lead: ___________________
Date: //2024

(Document length: 3,280 characters)