The file /root/.claude/plans/scalable-doodling-peach.md has been updated successfully.

# Plan: Enhance thread_view.php + Fix Main Dashboard Issues

## Context

The user found `thread_view.php` already has a working AI form with multi-provider support, conversation history (sessionStorage), saved responses, and quick view. They want the new AI features (agents, context builder, token estimation, debug logging) merged into this existing page. Additionally, the main dashboard has 3 bugs: slow loading, wrong contact names on threads, and a useless AI Thread Analysis section.

## 4 Tasks (in execution order)

---

### TASK 1: Hide AI Thread Analysis section (index.php + app.js)

**Problem:** The "AI Thread Analysis" section shows "Unknown" contacts, runs heavy N+1 queries, and auto-refreshes every 30 seconds — slowing the page.

**Files:**

| File | Change |
|------|--------|
| [public/index.php:136](public/index.php#L136) | Add `style="display:none"` to `#analysis-status-section` div |
| [public/assets/app.js:1589](public/assets/app.js#L1589) | Comment out `refreshAnalysisStatus()` call |
| [public/assets/app.js:1592](public/assets/app.js#L1592) | Comment out `setInterval(refreshAnalysisStatus, 30000)` |

---

### TASK 2: Speed up main page (app.js)

**Problem:** Threads and contacts load 20 records each — too many for initial page load.

**Files:**

| File | Change |
|------|--------|
| [public/assets/app.js:176](public/assets/app.js#L176) | `loadContacts` → change `limit: 20` to `limit: 10` |
| [public/assets/app.js:650](public/assets/app.js#L650) | `loadEmails` → change `limit: 20` to `limit: 10` |

---

### TASK 3: Fix thread contact name bug (QueryAdapter.php)

**Problem:** `getThreadsMongo()` line 609 uses `'$max' => '$contact_id'` which picks the numerically highest contact_id (always "Etienne Turner"). Same issue in `getThreadsSQL()` line 549-550 with `MAX(e.contact_id)` and `MAX(c.name)`.

**Fix approach:**

**MongoDB** ([src/QueryAdapter.php:569-650](src/QueryAdapter.php#L569)):
1. In the `$group` stage, collect all contacts instead of `$max`:
   - Replace `'contact_id' => ['$max' => '$contact_id']` with `'contacts' => ['$push' => ['cid' => '$contact_id', 'email' => '$from_email', 'dir' => '$direction']]`
2. After aggregation, post-process each thread in PHP:
   - Filter out contacts where `email === GMAIL_USER_EMAIL` (the user's own email)
   - Filter to only `direction === 'inbound'` contacts
   - Count occurrences of each remaining `contact_id`
   - Pick the most frequent one
   - If none found, fall back to the first non-user contact_id
3. Use the resolved contact_id for the `$lookup` (or do a separate contact name lookup after post-processing)

**SQL** ([src/QueryAdapter.php:514-567](src/QueryAdapter.php#L514)):
1. Replace `MAX(e.contact_id)` with a subquery that finds the most frequent inbound sender:
   ```sql
   (SELECT e2.contact_id FROM emails e2
    LEFT JOIN contacts c2 ON e2.contact_id = c2.id
    WHERE e2.thread_id = e.thread_id
    AND e2.direction = 'inbound'
    AND LOWER(e2.from_email) != ?
    GROUP BY e2.contact_id
    ORDER BY COUNT(*) DESC LIMIT 1) as contact_id
   ```
2. Similar subquery for contact_name.
3. Add `GMAIL_USER_EMAIL` as bind parameter.

---

### TASK 4: Enhance thread_view.php (the main task)

**Problem:** thread_view.php needs agents, cross-thread context builder, token estimation, and debug logging — while keeping existing multi-provider simultaneous sending.

**Current architecture (keep):**
- Multi-provider checkboxes → sends to `api_opus/api/query.php` simultaneously
- Conversation history in sessionStorage
- Saved responses in MongoDB `ai_query_responses`
- Quick view modal, load more responses
- Content processing modes (full/stripped/smart)

**New features to add:**

#### 4A. Full-width layout
- [thread_view.php:174](public/thread_view.php#L174): Change `.page-container { max-width: 1600px }` to `max-width: 100%; padding: 20px 30px;`
- [thread_view.php:213-217](public/thread_view.php#L213): Change `.main-grid` from `grid-template-columns: 1fr 400px` to `grid-template-columns: 1fr 500px` (wider AI panel)

#### 4B. Agent buttons with quick prompts
Add above the provider grid in the right column (after line 975):

```html
<!-- Agent Selector -->
<div class="ai-form-section">
    <label>AI Agent</label>
    <div class="agent-buttons" id="agent-buttons">
        <!-- Populated by JS -->
    </div>
    <div class="agent-quick-prompts" id="agent-quick-prompts" style="display:none;">
        <!-- Quick prompt buttons per agent -->
    </div>
</div>
```

**JS agent presets** (add as constants near top of `<script>` block):
```js
const AI_AGENTS = {
    general: { name: 'General', icon: '💬', systemPrompt: '...', quickPrompts: [...] },
    quote_analyst: { name: 'Quotes', icon: '💰', systemPrompt: '...', quickPrompts: [...] },
    deception_detector: { name: 'Deception', icon: '🔍', systemPrompt: '...', quickPrompts: [...] },
    contact_profiler: { name: 'Profiler', icon: '👤', systemPrompt: '...', quickPrompts: [...] },
    email_composer: { name: 'Composer', icon: '✍', systemPrompt: '...', quickPrompts: [...] }
};
```

When an agent is selected:
- Highlight the active agent button
- Show agent-specific quick prompts below the buttons
- Clicking a quick prompt fills the question textarea
- The agent's `systemPrompt` is prepended to the content sent to AI

#### 4C. Context builder (search + add threads from any contact)
Add a collapsible section between "Include Other Threads" and "Ask AI" cards:

```html
<!-- Context Builder -->
<div class="card">
    <div class="card-header">
        <h2>+ Add More Context</h2>
        <button class="btn btn-small" onclick="toggleContextBuilder()">Search</button>
    </div>
    <div id="context-builder" style="display:none;">
        <input type="text" id="context-search" placeholder="Search contacts or threads...">
        <div id="context-search-results"></div>
    </div>
    <div id="context-cart"></div>
</div>
```

**JS functions:**
- `toggleContextBuilder()` — show/hide search
- `searchContext(query)` — calls `api.php?action=contacts&search=...` AND `api.php?action=threads&search=...` in parallel
- Results show contacts (click to see their threads with "+Add" buttons) and threads (with "+Add" directly)
- `addThreadToContext(threadId, subject, emailCount)` — adds to `contextThreads[]` array
- `removeThreadFromContext(threadId)` — removes
- `updateContextCart()` — renders green summary: "3 additional threads, ~25 emails"
- When `submitToAI()` runs, fetches email content for each context thread (via `api.php?action=thread&thread_id=...`) and appends to the content

#### 4D. Token estimation
Add a live token counter near the submit button:

```html
<div id="token-estimate" style="font-size:0.85em; color:var(--text-secondary); margin-bottom:10px;">
    Estimated: ~0 tokens
</div>
```

**JS:**
- `MODEL_LIMITS` constant mapping model names to max context (e.g. `{'claude-sonnet-4-5-20250929': 200000, 'gpt-4o': 128000, ...}`)
- `estimateTokens()` — called on: agent change, context change, question typing, provider toggle
  - Calculates `Math.ceil(totalContentLength / 3.5)`
  - Shows warning if approaching any selected model's limit
  - Compares against the **smallest** selected model's limit (since multi-provider)
  - Display: "Estimated: ~12,500 tokens (OK for all selected models)" or "Warning: exceeds gpt-4o limit (128K)"
- In `submitToAI()`: if over limit, confirm before sending

#### 4E. System prompt injection per agent
In `submitToAI()`, prepend the selected agent's system prompt:

```js
let systemPrompt = '';
if (selectedAgent && AI_AGENTS[selectedAgent]) {
    systemPrompt = AI_AGENTS[selectedAgent].systemPrompt + '\n\n';
}
// Prepend to fullContent
fullContent = systemPrompt + fullContent;
```

This works because `api_opus/query.php` accepts `user_content` as one big string — the system prompt at the top guides the AI's behavior.

#### 4F. Debug file logging
**New API endpoint** in [public/api.php](public/api.php) — `log_ai_debug`:

```php
case 'log_ai_debug':
    $logDir = dirname(__DIR__) . '/logs';
    if (!is_dir($logDir)) mkdir($logDir, 0755, true);
    $logFile = $logDir . '/ai_debug.log';
    $logData = json_decode(file_get_contents('php://input'), true);
    $logEntry = [
        'timestamp' => date('Y-m-d H:i:s'),
        'thread_id' => $logData['thread_id'] ?? '',
        'agent' => $logData['agent'] ?? '',
        'providers' => $logData['providers'] ?? [],
        'content_length' => $logData['content_length'] ?? 0,
        'estimated_tokens' => $logData['estimated_tokens'] ?? 0,
        'question' => $logData['question'] ?? '',
        'type' => $logData['type'] ?? 'request',  // 'request' or 'response'
        'response_preview' => substr($logData['response'] ?? '', 0, 500),
        'full_content' => $logData['full_content'] ?? '' // full payload for debug
    ];
    file_put_contents($logFile, json_encode($logEntry) . "\n", FILE_APPEND | LOCK_EX);
    echo json_encode(['success' => true]);
    break;
```

**JS:** Call this endpoint in `submitToAI()`:
1. **Before** sending to AI: log the request (type: 'request', full_content, providers, tokens)
2. **After** receiving response: log the response (type: 'response', response preview, costs, timing)

Log file: `/var/www/mssql.danmarcrm.com/dev1/extension2v1/logs/ai_debug.log` — one JSON line per entry.

#### 4G. Contact memory injection (bonus)
If `primaryContactId` exists, fetch memories from `api.php?action=ai_contact_memories&contact_id=X` and inject as "KNOWN FACTS ABOUT [CONTACT]" section in the content sent to AI. Display memories as pills in the context area.

---

## Implementation Order

1. **TASK 1** — Hide AI Thread Analysis (3 line changes in index.php + app.js)
2. **TASK 2** — Speed up main page (2 line changes in app.js)
3. **TASK 3** — Fix thread contact name bug (QueryAdapter.php — MongoDB + SQL)
4. **TASK 4** — Enhance thread_view.php:
   - 4A: Full-width layout (CSS changes)
   - 4B: Agent buttons + quick prompts (HTML + JS)
   - 4C: Context builder (HTML + JS)
   - 4D: Token estimation (HTML + JS)
   - 4E: System prompt injection (JS modification to submitToAI)
   - 4F: Debug logging endpoint (api.php + JS calls)
   - 4G: Contact memory injection (JS)

## Files Modified (summary)

| File | Tasks |
|------|-------|
| [public/index.php](public/index.php) | T1: hide analysis section |
| [public/assets/app.js](public/assets/app.js) | T1: disable refresh, T2: reduce limits |
| [src/QueryAdapter.php](src/QueryAdapter.php) | T3: fix contact name bug |
| [public/api.php](public/api.php) | T4F: add `log_ai_debug` endpoint |
| [public/thread_view.php](public/thread_view.php) | T4: all thread_view enhancements |

## Verification

1. **Main dashboard:** Load page — should be fast, no analysis section visible, threads show correct contact names (not all "Etienne Turner")
2. **Thread view:** Open a thread — should be full-width, agent buttons visible, context builder expandable
3. **Agents:** Click "Quotes" agent → quick prompts appear → click one → fills question box → submit → AI response guided by quote analysis system prompt
4. **Context builder:** Click "Search" → type contact name → see their threads → click "+Add" → green cart updates → submit → AI sees all added threads
5. **Token counter:** Shows live estimate → add more context → count increases → shows warning if near limit
6. **Multi-provider:** Select Claude + GPT → submit → both respond in tabs (existing behavior preserved)
7. **Debug log:** Check `/var/www/mssql.danmarcrm.com/dev1/extension2v1/logs/ai_debug.log` → has request + response JSON lines
8. **Conversation continuity:** Ask a follow-up question → AI sees previous Q&A in context (existing behavior preserved)
9. **Saved responses:** Response auto-saves to MongoDB → appears in "Previous AI Analyses" (existing behavior preserved)

-# Plan: AI Context Builder with Agents & Memory

+# Plan: Enhance thread_view.php + Fix Main Dashboard Issues

-The web app's AI chat panel currently only supports asking AI about **one thread** or **one contact** at a time, with no persistence and no deduplication. We're building a full-featured AI chat system with:

+The user found `thread_view.php` already has a working AI form with multi-provider support, conversation history (sessionStorage), saved responses, and quick view. They want the new AI features (agents, context builder, token estimation, debug logging) merged into this existing page. Additionally, the main dashboard has 3 bugs: slow loading, wrong contact names on threads, and a useless AI Thread Analysis section.

-- **Multi-thread context**: Search contacts, select multiple threads as AI context

-- **Named AI agents**: Specialized presets (Quote Analyst, Deception Detector, Profiler, Composer)

-- **Contact memory**: Persistent facts about contacts that accumulate over time

-- **Chat session persistence**: Save/reload/continue conversations

-- **Smart token management**: Strip quoted email content, enforce context budgets

-| [public/api.php](public/api.php) | Multi-thread context, quoted content stripping, token budget, session CRUD, agent presets, contact memory endpoints |

-| [public/assets/app.js](public/assets/app.js) | Context builder, provider/model selector, agent selector, session management, memory UI |

-| [public/index.php](public/index.php) | Provider+model dropdowns, context builder HTML, agent selector, session list, memory panel |

-| [src/ThreadAnalyzer.php](src/ThreadAnalyzer.php) | Make `stripQuotedContent()` public static (line 1616) |

+### TASK 1: Hide AI Thread Analysis section (index.php + app.js)

-## Data Structures (MongoDB collections)

+**Problem:** The "AI Thread Analysis" section shows "Unknown" contacts, runs heavy N+1 queries, and auto-refreshes every 30 seconds — slowing the page.

-  "agent": "general|quote_analyst|deception_detector|contact_profiler|email_composer",

-  "context_thread_ids": ["tid1", "tid2"],

-    {"role": "user", "content": "...", "timestamp": "datetime"},

-    {"role": "assistant", "content": "...", "timestamp": "datetime", "model": "claude-sonnet-4-5"}

-  "contact_email": "stephanie@asendia.com",

-      "fact": "Always negotiates 20-30% below initial quote",

-      "session_id": "uuid (which session discovered this)",

+| [public/index.php:136](public/index.php#L136) | Add `style="display:none"` to `#analysis-status-section` div |

+| [public/assets/app.js:1589](public/assets/app.js#L1589) | Comment out `refreshAnalysisStatus()` call |

+| [public/assets/app.js:1592](public/assets/app.js#L1592) | Comment out `setInterval(refreshAnalysisStatus, 30000)` |

-Stored as constants in `app.js` and mirrored in `api.php`:

+### TASK 2: Speed up main page (app.js)

-| Agent | System Prompt Focus | Preferred Model | Auto-Memory |

-| **General Assistant** | Open-ended business intelligence | User's choice | No |

-| **Quote Analyst** | Pricing patterns, quote history, scope analysis, negotiation leverage | Claude Sonnet | Yes: extracts pricing facts |

-| **Deception Detector** | Inconsistencies, contradictions, tone shifts, pressure tactics, red flags | Claude Opus (best reasoning) | Yes: flags contradictions |

-| **Contact Profiler** | Communication style, psychological patterns, loyalty, relationship health | Claude Sonnet | Yes: saves profile traits |

-| **Email Composer** | Professional replies, negotiation-aware, tone-matching | GPT-4o | No |

+**Problem:** Threads and contacts load 20 records each — too many for initial page load.

-Each agent = `{id, name, icon, systemPrompt, preferredProvider, preferredModel, autoMemory: bool, quickPrompts: []}`

+| [public/assets/app.js:176](public/assets/app.js#L176) | `loadContacts` → change `limit: 20` to `limit: 10` |

+| [public/assets/app.js:650](public/assets/app.js#L650) | `loadEmails` → change `limit: 20` to `limit: 10` |

-### Step 1: Make `stripQuotedContent()` reusable — ThreadAnalyzer.php

-Change line 1616 from `private function` to `public static function`. The method is pure/stateless (no `$this` references), so this is safe.

+### TASK 3: Fix thread contact name bug (QueryAdapter.php)

+**Problem:** `getThreadsMongo()` line 609 uses `'$max' => '$contact_id'` which picks the numerically highest contact_id (always "Etienne Turner"). Same issue in `getThreadsSQL()` line 549-550 with `MAX(e.contact_id)` and `MAX(c.name)`.

-**2a. Modify `ai_chat` action (lines 2310-2491):**

-- `thread_ids` (array) alongside existing `thread_id`

-- `session_id` (optional – continue existing session)

+1. In the `$group` stage, collect all contacts instead of `$max`:

+   - Replace `'contact_id' => ['$max' => '$contact_id']` with `'contacts' => ['$push' => ['cid' => '$contact_id', 'email' => '$from_email', 'dir' => '$direction']]`

+2. After aggregation, post-process each thread in PHP:

+   - Filter out contacts where `email === GMAIL_USER_EMAIL` (the user's own email)

+   - Filter to only `direction === 'inbound'` contacts

+   - Count occurrences of each remaining `contact_id`

+   - If none found, fall back to the first non-user contact_id

+3. Use the resolved contact_id for the `$lookup` (or do a separate contact name lookup after post-processing)

-**2b. Strip quoted content from email bodies:**

-- Before `substr($body, 0, 800)` on line 2347, call `ThreadAnalyzer::stripQuotedContent($body)`

-- This removes `> quoted` text, "On X wrote:" patterns, "Original Message" dividers

-- Reduces a 5-email thread's context by 60-80% (no more duplicate content per reply)

+1. Replace `MAX(e.contact_id)` with a subquery that finds the most frequent inbound sender:

+   (SELECT e2.contact_id FROM emails e2

+    LEFT JOIN contacts c2 ON e2.contact_id = c2.id

+    ORDER BY COUNT(*) DESC LIMIT 1) as contact_id

+3. Add `GMAIL_USER_EMAIL` as bind parameter.

-- If `thread_ids` array provided, loop and fetch emails per thread

-- Label "THREAD 1 OF N", "THREAD 2 OF N"

-- Include per-thread analysis summaries if available

-- Single `thread_id` keeps existing behavior

-- Estimate tokens: `strlen($contextBlock) / 3.5`

-- If over budget: reduce per-email body proportionally, then omit oldest emails with note

-- Return `context_stats` in response: `{threads, emails, estimated_tokens, truncated}`

+### TASK 4: Enhance thread_view.php (the main task)

-**2e. Inject contact memory into context:**

-- If `contact_id` provided, fetch from `ai_contact_memory` collection

-- Append as `=== KNOWN FACTS ABOUT [CONTACT] ===` section in system prompt

-- This means AI already "knows" key facts without re-reading all emails

+**Problem:** thread_view.php needs agents, cross-thread context builder, token estimation, and debug logging — while keeping existing multi-provider simultaneous sending.

-- Map `agent` ID to full system prompt (defined in PHP array)

-- Override mode-based prompts when agent is specified

+- Multi-provider checkboxes → sends to `api_opus/api/query.php` simultaneously

+- Conversation history in sessionStorage

+- Saved responses in MongoDB `ai_query_responses`

+- Quick view modal, load more responses

+- Content processing modes (full/stripped/smart)

-**2g. Auto-extract memories from AI response:**

-- If agent has `autoMemory: true`, append instruction to system prompt:

-  `"If you discover important new facts about this contact (pricing patterns, contradictions, preferences), include them in a [MEMORY] tag at the end of your response."`

-- Parse AI response for `[MEMORY]...[/MEMORY]` blocks

-- Save extracted facts to `ai_contact_memory` automatically

-- Strip the `[MEMORY]` tags from the response shown to user

-**2h. Fix provider/model proxy format:**

-- Change `'models' => [$chatModel]` to `'models' => [$chatProvider => $chatModel]` (line 2431)

-- Use explicit `provider` field when available

+- [thread_view.php:174](public/thread_view.php#L174): Change `.page-container { max-width: 1600px }` to `max-width: 100%; padding: 20px 30px;`

+- [thread_view.php:213-217](public/thread_view.php#L213): Change `.main-grid` from `grid-template-columns: 1fr 400px` to `grid-template-columns: 1fr 500px` (wider AI panel)

-- `ai_chat_sessions` — GET — list recent sessions (limit 20)

-- `ai_chat_session` — GET — load specific session by `session_id`

-- `ai_chat_session_delete` — POST — delete a session

-- `ai_contact_memories` — GET — get memories for a contact

-- `ai_contact_memory_add` — POST — manually add a memory fact

-- `ai_contact_memory_delete` — POST — delete a memory fact

+#### 4B. Agent buttons with quick prompts

+Add above the provider grid in the right column (after line 975):

-- After successful AI response, upsert to `ai_chat_sessions`

-- Create new `session_id` if not provided, return it in response

-- Title auto-generated from first user message (first 60 chars)

+    <div class="agent-buttons" id="agent-buttons">

+    <div class="agent-quick-prompts" id="agent-quick-prompts" style="display:none;">

+        <!-- Quick prompt buttons per agent -->

+**JS agent presets** (add as constants near top of `<script>` block):

+    general: { name: 'General', icon: '💬', systemPrompt: '...', quickPrompts: [...] },

+    quote_analyst: { name: 'Quotes', icon: '💰', systemPrompt: '...', quickPrompts: [...] },

+    deception_detector: { name: 'Deception', icon: '🔍', systemPrompt: '...', quickPrompts: [...] },

+    contact_profiler: { name: 'Profiler', icon: '👤', systemPrompt: '...', quickPrompts: [...] },

+    email_composer: { name: 'Composer', icon: '✍', systemPrompt: '...', quickPrompts: [...] }

-**3a. Extend `aiChat` state (line 2672):**

+- Show agent-specific quick prompts below the buttons

+- Clicking a quick prompt fills the question textarea

+- The agent's `systemPrompt` is prepended to the content sent to AI

+#### 4C. Context builder (search + add threads from any contact)

+Add a collapsible section between "Include Other Threads" and "Ask AI" cards:

+        <button class="btn btn-small" onclick="toggleContextBuilder()">Search</button>

+    <div id="context-builder" style="display:none;">

+        <input type="text" id="context-search" placeholder="Search contacts or threads...">

-contextThreads: [], contextContact: null,

-**3b. Add constants (after line 2684):**

-- `PROVIDER_MODELS` / `PROVIDER_NAMES` — synced with Chrome extension (`contentScript.js` lines 18-67): 8 providers (Claude, OpenAI, Gemini, Grok, DeepSeek, Qwen, Mistral, Cohere)

-- `AI_AGENTS` — agent presets with `{id, name, icon, preferredProvider, preferredModel, quickPrompts[]}`

+- `toggleContextBuilder()` — show/hide search

+- `searchContext(query)` — calls `api.php?action=contacts&search=...` AND `api.php?action=threads&search=...` in parallel

+- Results show contacts (click to see their threads with "+Add" buttons) and threads (with "+Add" directly)

+- `addThreadToContext(threadId, subject, emailCount)` — adds to `contextThreads[]` array

+- `removeThreadFromContext(threadId)` — removes

+- `updateContextCart()` — renders green summary: "3 additional threads, ~25 emails"

+- When `submitToAI()` runs, fetches email content for each context thread (via `api.php?action=thread&thread_id=...`) and appends to the content

-**3c. Provider/model dropdown functions:**

-- `initAiProviderDropdown()` — populates provider select

-- `updateAiModelDropdown()` — populates model based on provider

+Add a live token counter near the submit button:

-- `selectAgent(agentId)` — sets agent, updates provider/model to agent's preferred defaults, updates quick prompts, updates context label

-- Agent buttons replace or augment the mode tabs

+<div id="token-estimate" style="font-size:0.85em; color:var(--text-secondary); margin-bottom:10px;">

-- `toggleContextBuilder()` — show/hide search area

-- `searchContextContacts()` — search contacts API, render results

-- `selectContextContact(id, name, email)` — load threads, show picker with "+Add" and "Add All"

-- `addThreadToContext()` / `removeThreadFromContext()` / `addAllContactThreads()` / `clearContextCart()`

-- `updateContextCart()` — render selected threads summary

+- `MODEL_LIMITS` constant mapping model names to max context (e.g. `{'claude-sonnet-4-5-20250929': 200000, 'gpt-4o': 128000, ...}`)

+- `estimateTokens()` — called on: agent change, context change, question typing, provider toggle

+  - Calculates `Math.ceil(totalContentLength / 3.5)`

+  - Shows warning if approaching any selected model's limit

+  - Compares against the **smallest** selected model's limit (since multi-provider)

+  - Display: "Estimated: ~12,500 tokens (OK for all selected models)" or "Warning: exceeds gpt-4o limit (128K)"

+- In `submitToAI()`: if over limit, confirm before sending

-**3f. Modify `sendAiChat()` (line 2756):**

-- Send `thread_ids`, `provider`, `session_id`, `agent` in payload

-- On response: store `session_id`, display `context_stats`, parse any auto-extracted memories

-- If `context_stats.truncated`, show subtle note: "Context was trimmed to fit model limits"

+#### 4E. System prompt injection per agent

+In `submitToAI()`, prepend the selected agent's system prompt:

-- `loadChatSessions()` — fetch + render recent sessions list

-- `loadChatSession(id)` — restore messages, context, agent, model

-- `startNewSession()` — clear chat, reset session_id

-- `deleteChatSession(id)` — delete from server

+if (selectedAgent && AI_AGENTS[selectedAgent]) {

+    systemPrompt = AI_AGENTS[selectedAgent].systemPrompt + '\n\n';

+fullContent = systemPrompt + fullContent;

-- When a contact is selected in context builder, load their memories

-- Display as small badges/pills below the context cart: "Always negotiates 20-30% below" ×

-- Manual "Add Memory" button for user-defined facts

+This works because `api_opus/query.php` accepts `user_content` as one big string — the system prompt at the top guides the AI's behavior.

-- Each agent has its own quick prompts (replaces the mode-based `AI_QUICK_PROMPTS`)

-- When `contextThreads.length > 0` and agent is selected, show agent-specific prompts

+**New API endpoint** in [public/api.php](public/api.php) — `log_ai_debug`:

+    $logDir = dirname(__DIR__) . '/logs';

+    if (!is_dir($logDir)) mkdir($logDir, 0755, true);

+    $logFile = $logDir . '/ai_debug.log';

+    $logData = json_decode(file_get_contents('php://input'), true);

+        'timestamp' => date('Y-m-d H:i:s'),

+        'thread_id' => $logData['thread_id'] ?? '',

+        'agent' => $logData['agent'] ?? '',

+        'providers' => $logData['providers'] ?? [],

+        'content_length' => $logData['content_length'] ?? 0,

+        'estimated_tokens' => $logData['estimated_tokens'] ?? 0,

+        'question' => $logData['question'] ?? '',

+        'type' => $logData['type'] ?? 'request',  // 'request' or 'response'

+        'response_preview' => substr($logData['response'] ?? '', 0, 500),

+        'full_content' => $logData['full_content'] ?? '' // full payload for debug

+    file_put_contents($logFile, json_encode($logEntry) . "\n", FILE_APPEND | LOCK_EX);

+    echo json_encode(['success' => true]);

-**4a. Replace static model dropdown (lines 538-561):**

-- Row 1: Provider dropdown + Model dropdown (populated by JS)

-- Row 2: Agent selector buttons (icons: 💬 General, 💰 Quote, 🔍 Deception, 👤 Profiler, ✍ Composer)

+**JS:** Call this endpoint in `submitToAI()`:

+1. **Before** sending to AI: log the request (type: 'request', full_content, providers, tokens)

+2. **After** receiving response: log the response (type: 'response', response preview, costs, timing)

-**4b. Chat header additions (line 531 area):**

-- "History" button (toggles session list overlay)

+Log file: `/var/www/mssql.danmarcrm.com/dev1/extension2v1/logs/ai_debug.log` — one JSON line per entry.

-**4c. Insert context builder section (between line 570 and 574):**

-- Collapsible context summary bar → expands to show search + thread picker + cart + contact memories

+#### 4G. Contact memory injection (bonus)

+If `primaryContactId` exists, fetch memories from `api.php?action=ai_contact_memories&contact_id=X` and inject as "KNOWN FACTS ABOUT [CONTACT]" section in the content sent to AI. Display memories as pills in the context area.

-- Lists recent sessions with title, agent icon, date, message count

-- Click to load, swipe/button to delete

-Since this is a large feature, implement in this order to have something testable at each step:

+1. **TASK 1** — Hide AI Thread Analysis (3 line changes in index.php + app.js)

+2. **TASK 2** — Speed up main page (2 line changes in app.js)

+3. **TASK 3** — Fix thread contact name bug (QueryAdapter.php — MongoDB + SQL)

+4. **TASK 4** — Enhance thread_view.php:

+   - 4A: Full-width layout (CSS changes)

+   - 4B: Agent buttons + quick prompts (HTML + JS)

+   - 4E: System prompt injection (JS modification to submitToAI)

+   - 4F: Debug logging endpoint (api.php + JS calls)

-1. **`stripQuotedContent()` → public static** (1 line change, instant win)

-2. **Backend multi-thread + token budget + stripped content** (api.php modifications to `ai_chat`)

-3. **Frontend context builder + provider/model selector** (search, add threads, send multi-thread)

-4. **Agent presets** (frontend selector + backend system prompts)

-5. **Chat session persistence** (backend CRUD + frontend save/load)

-6. **Contact memory** (backend extraction/storage + frontend display)

+| [public/index.php](public/index.php) | T1: hide analysis section |

+| [public/assets/app.js](public/assets/app.js) | T1: disable refresh, T2: reduce limits |

+| [src/QueryAdapter.php](src/QueryAdapter.php) | T3: fix contact name bug |

+| [public/api.php](public/api.php) | T4F: add `log_ai_debug` endpoint |

+| [public/thread_view.php](public/thread_view.php) | T4: all thread_view enhancements |

-1. Open web app → AI chat → verify provider/model dropdowns work

-2. Select "Quote Analyst" agent → verify model switches to Claude Sonnet, prompts change

-3. Search "stephanie" → click contact → see threads → "Add All" → verify green context label

-4. Ask "how much have we quoted?" → verify AI sees all thread content without duplicate quoted text

-5. Check response includes `context_stats` → verify token count is reasonable

-6. Ask follow-up "what should we quote for next project?" → verify conversation continues

-7. Refresh page → click History → verify session appears → click to reload → messages restored

-8. Select "Deception Detector" agent → add threads for david → ask about deception level

-9. Check `ai_contact_memory` collection → verify auto-extracted facts are saved

-10. Load a different session about same contact → verify memories appear as context

-11. Test with 30+ threads for one contact → verify token budget truncation works gracefully

-12. Test backward compat: open single thread → "Ask AI About This Thread" → still works

+1. **Main dashboard:** Load page — should be fast, no analysis section visible, threads show correct contact names (not all "Etienne Turner")

+2. **Thread view:** Open a thread — should be full-width, agent buttons visible, context builder expandable

+3. **Agents:** Click "Quotes" agent → quick prompts appear → click one → fills question box → submit → AI response guided by quote analysis system prompt

+4. **Context builder:** Click "Search" → type contact name → see their threads → click "+Add" → green cart updates → submit → AI sees all added threads

+5. **Token counter:** Shows live estimate → add more context → count increases → shows warning if near limit

+6. **Multi-provider:** Select Claude + GPT → submit → both respond in tabs (existing behavior preserved)

+7. **Debug log:** Check `/var/www/mssql.danmarcrm.com/dev1/extension2v1/logs/ai_debug.log` → has request + response JSON lines

+8. **Conversation continuity:** Ask a follow-up question → AI sees previous Q&A in context (existing behavior preserved)

+9. **Saved responses:** Response auto-saves to MongoDB → appears in "Previous AI Analyses" (existing behavior preserved)

# Plan: AI Context Builder with Agents & Memory

## Context

The web app's AI chat panel currently only supports asking AI about **one thread** or **one contact** at a time, with no persistence and no deduplication. We're building a full-featured AI chat system with:

- **Multi-thread context**: Search contacts, select multiple threads as AI context
- **Named AI agents**: Specialized presets (Quote Analyst, Deception Detector, Profiler, Composer)
- **Contact memory**: Persistent facts about contacts that accumulate over time
- **Chat session persistence**: Save/reload/continue conversations
- **Smart token management**: Strip quoted email content, enforce context budgets

## Files to Modify

| File | Changes |
|------|---------|
| [public/api.php](public/api.php) | Multi-thread context, quoted content stripping, token budget, session CRUD, agent presets, contact memory endpoints |
| [public/assets/app.js](public/assets/app.js) | Context builder, provider/model selector, agent selector, session management, memory UI |
| [public/index.php](public/index.php) | Provider+model dropdowns, context builder HTML, agent selector, session list, memory panel |
| [src/ThreadAnalyzer.php](src/ThreadAnalyzer.php) | Make `stripQuotedContent()` public static (line 1616) |

## Data Structures (MongoDB collections)

### `ai_chat_sessions`
```json
{
  "session_id": "uuid",
  "created_at": "datetime",
  "updated_at": "datetime",
  "title": "auto from first question",
  "agent": "general|quote_analyst|deception_detector|contact_profiler|email_composer",
  "provider": "claude",
  "model": "claude-sonnet-4-5-20250929",
  "context_thread_ids": ["tid1", "tid2"],
  "contact_id": 123,
  "contact_name": "Stephanie K",
  "messages": [
    {"role": "user", "content": "...", "timestamp": "datetime"},
    {"role": "assistant", "content": "...", "timestamp": "datetime", "model": "claude-sonnet-4-5"}
  ]
}
```

### `ai_contact_memory`
```json
{
  "contact_id": 123,
  "contact_email": "stephanie@asendia.com",
  "memories": [
    {
      "id": "uuid",
      "fact": "Always negotiates 20-30% below initial quote",
      "source": "auto|manual",
      "session_id": "uuid (which session discovered this)",
      "created_at": "datetime"
    }
  ],
  "updated_at": "datetime"
}
```

## Agent Presets

Stored as constants in `app.js` and mirrored in `api.php`:

| Agent | System Prompt Focus | Preferred Model | Auto-Memory |
|-------|-------------------|-----------------|-------------|
| **General Assistant** | Open-ended business intelligence | User's choice | No |
| **Quote Analyst** | Pricing patterns, quote history, scope analysis, negotiation leverage | Claude Sonnet | Yes: extracts pricing facts |
| **Deception Detector** | Inconsistencies, contradictions, tone shifts, pressure tactics, red flags | Claude Opus (best reasoning) | Yes: flags contradictions |
| **Contact Profiler** | Communication style, psychological patterns, loyalty, relationship health | Claude Sonnet | Yes: saves profile traits |
| **Email Composer** | Professional replies, negotiation-aware, tone-matching | GPT-4o | No |

Each agent = `{id, name, icon, systemPrompt, preferredProvider, preferredModel, autoMemory: bool, quickPrompts: []}`

## Implementation Steps

### Step 1: Make `stripQuotedContent()` reusable — ThreadAnalyzer.php

Change line 1616 from `private function` to `public static function`. The method is pure/stateless (no `$this` references), so this is safe.

### Step 2: Backend — api.php

**2a. Modify `ai_chat` action (lines 2310-2491):**

Parse new fields:
- `thread_ids` (array) alongside existing `thread_id`
- `provider` (explicit provider code)
- `session_id` (optional – continue existing session)
- `agent` (agent preset ID)

**2b. Strip quoted content from email bodies:**
- Before `substr($body, 0, 800)` on line 2347, call `ThreadAnalyzer::stripQuotedContent($body)`
- This removes `> quoted` text, "On X wrote:" patterns, "Original Message" dividers
- Reduces a 5-email thread's context by 60-80% (no more duplicate content per reply)

**2c. Multi-thread context loop:**
- If `thread_ids` array provided, loop and fetch emails per thread
- Label "THREAD 1 OF N", "THREAD 2 OF N"
- Include per-thread analysis summaries if available
- Single `thread_id` keeps existing behavior

**2d. Token budget management:**
- Model context limits: `{'gpt-4o': 128000, 'claude-sonnet-4-5': 200000, 'grok-4': 131000, ...}`
- Estimate tokens: `strlen($contextBlock) / 3.5`
- Reserve 30% for response + history
- If over budget: reduce per-email body proportionally, then omit oldest emails with note
- Return `context_stats` in response: `{threads, emails, estimated_tokens, truncated}`

**2e. Inject contact memory into context:**
- If `contact_id` provided, fetch from `ai_contact_memory` collection
- Append as `=== KNOWN FACTS ABOUT [CONTACT] ===` section in system prompt
- This means AI already "knows" key facts without re-reading all emails

**2f. Agent system prompt:**
- Map `agent` ID to full system prompt (defined in PHP array)
- Override mode-based prompts when agent is specified

**2g. Auto-extract memories from AI response:**
- If agent has `autoMemory: true`, append instruction to system prompt:
  `"If you discover important new facts about this contact (pricing patterns, contradictions, preferences), include them in a [MEMORY] tag at the end of your response."`
- Parse AI response for `[MEMORY]...[/MEMORY]` blocks
- Save extracted facts to `ai_contact_memory` automatically
- Strip the `[MEMORY]` tags from the response shown to user

**2h. Fix provider/model proxy format:**
- Change `'models' => [$chatModel]` to `'models' => [$chatProvider => $chatModel]` (line 2431)
- Use explicit `provider` field when available

**2i. New API actions:**
- `ai_chat_sessions` — GET — list recent sessions (limit 20)
- `ai_chat_session` — GET — load specific session by `session_id`
- `ai_chat_session_delete` — POST — delete a session
- `ai_contact_memories` — GET — get memories for a contact
- `ai_contact_memory_add` — POST — manually add a memory fact
- `ai_contact_memory_delete` — POST — delete a memory fact

**2j. Auto-save session:**
- After successful AI response, upsert to `ai_chat_sessions`
- Create new `session_id` if not provided, return it in response
- Title auto-generated from first user message (first 60 chars)

### Step 3: Frontend JS — app.js

**3a. Extend `aiChat` state (line 2672):**
```
contextThreads: [], contextContact: null,
sessionId: null, agent: 'general',
contactMemories: []
```

**3b. Add constants (after line 2684):**
- `PROVIDER_MODELS` / `PROVIDER_NAMES` — synced with Chrome extension (`contentScript.js` lines 18-67): 8 providers (Claude, OpenAI, Gemini, Grok, DeepSeek, Qwen, Mistral, Cohere)
- `AI_AGENTS` — agent presets with `{id, name, icon, preferredProvider, preferredModel, quickPrompts[]}`

**3c. Provider/model dropdown functions:**
- `initAiProviderDropdown()` — populates provider select
- `updateAiModelDropdown()` — populates model based on provider

**3d. Agent selector:**
- `selectAgent(agentId)` — sets agent, updates provider/model to agent's preferred defaults, updates quick prompts, updates context label
- Agent buttons replace or augment the mode tabs

**3e. Context builder functions:**
- `toggleContextBuilder()` — show/hide search area
- `searchContextContacts()` — search contacts API, render results
- `selectContextContact(id, name, email)` — load threads, show picker with "+Add" and "Add All"
- `addThreadToContext()` / `removeThreadFromContext()` / `addAllContactThreads()` / `clearContextCart()`
- `updateContextCart()` — render selected threads summary

**3f. Modify `sendAiChat()` (line 2756):**
- Send `thread_ids`, `provider`, `session_id`, `agent` in payload
- On response: store `session_id`, display `context_stats`, parse any auto-extracted memories
- If `context_stats.truncated`, show subtle note: "Context was trimmed to fit model limits"

**3g. Session management:**
- `loadChatSessions()` — fetch + render recent sessions list
- `loadChatSession(id)` — restore messages, context, agent, model
- `startNewSession()` — clear chat, reset session_id
- `deleteChatSession(id)` — delete from server

**3h. Contact memory UI:**
- When a contact is selected in context builder, load their memories
- Display as small badges/pills below the context cart: "Always negotiates 20-30% below" ×
- Manual "Add Memory" button for user-defined facts
- Delete button on each memory

**3i. Quick prompts:**
- Each agent has its own quick prompts (replaces the mode-based `AI_QUICK_PROMPTS`)
- When `contextThreads.length > 0` and agent is selected, show agent-specific prompts

### Step 4: HTML — index.php

**4a. Replace static model dropdown (lines 538-561):**
- Row 1: Provider dropdown + Model dropdown (populated by JS)
- Row 2: Agent selector buttons (icons: 💬 General, 💰 Quote, 🔍 Deception, 👤 Profiler, ✍ Composer)

**4b. Chat header additions (line 531 area):**
- "New Chat" button (resets session)
- "History" button (toggles session list overlay)

**4c. Insert context builder section (between line 570 and 574):**
- Collapsible context summary bar → expands to show search + thread picker + cart + contact memories

**4d. Session list overlay:**
- Shown when "History" clicked
- Lists recent sessions with title, agent icon, date, message count
- Click to load, swipe/button to delete

## Execution Order

Since this is a large feature, implement in this order to have something testable at each step:

1. **`stripQuotedContent()` → public static** (1 line change, instant win)
2. **Backend multi-thread + token budget + stripped content** (api.php modifications to `ai_chat`)
3. **Frontend context builder + provider/model selector** (search, add threads, send multi-thread)
4. **Agent presets** (frontend selector + backend system prompts)
5. **Chat session persistence** (backend CRUD + frontend save/load)
6. **Contact memory** (backend extraction/storage + frontend display)

## Verification

1. Open web app → AI chat → verify provider/model dropdowns work
2. Select "Quote Analyst" agent → verify model switches to Claude Sonnet, prompts change
3. Search "stephanie" → click contact → see threads → "Add All" → verify green context label
4. Ask "how much have we quoted?" → verify AI sees all thread content without duplicate quoted text
5. Check response includes `context_stats` → verify token count is reasonable
6. Ask follow-up "what should we quote for next project?" → verify conversation continues
7. Refresh page → click History → verify session appears → click to reload → messages restored
8. Select "Deception Detector" agent → add threads for david → ask about deception level
9. Check `ai_contact_memory` collection → verify auto-extracted facts are saved
10. Load a different session about same contact → verify memories appear as context
11. Test with 30+ threads for one contact → verify token budget truncation works gracefully
12. Test backward compat: open single thread → "Ask AI About This Thread" → still works


ExitPlanMode

# Plan: Enhance thread_view.php + Fix Main Dashboard Issues

## Context

The user found `thread_view.php` already has a working AI form with multi-provider support, conversation history (sessionStorage), saved responses, and quick view. They want the new AI features (agents, context builder, token estimation, debug logging) merged into this existing page. Additionally, the main dashboard has 3 bugs: slow loading, wrong contact names on threads, and a useless AI Thread Analysis section.

## 4 Tasks (in execution order)

---

### TASK 1: Hide AI Thread Analysis section (index.php + app.js)

**Problem:** The "AI Thread Analysis" section shows "Unknown" contacts, runs heavy N+1 queries, and auto-refreshes every 30 seconds — slowing the page.

**Files:**

| File | Change |
|------|--------|
| [public/index.php:136](public/index.php#L136) | Add `style="display:none"` to `#analysis-status-section` div |
| [public/assets/app.js:1589](public/assets/app.js#L1589) | Comment out `refreshAnalysisStatus()` call |
| [public/assets/app.js:1592](public/assets/app.js#L1592) | Comment out `setInterval(refreshAnalysisStatus, 30000)` |

---

### TASK 2: Speed up main page (app.js)

**Problem:** Threads and contacts load 20 records each — too many for initial page load.

**Files:**

| File | Change |
|------|--------|
| [public/assets/app.js:176](public/assets/app.js#L176) | `loadContacts` → change `limit: 20` to `limit: 10` |
| [public/assets/app.js:650](public/assets/app.js#L650) | `loadEmails` → change `limit: 20` to `limit: 10` |

---

### TASK 3: Fix thread contact name bug (QueryAdapter.php)

**Problem:** `getThreadsMongo()` line 609 uses `'$max' => '$contact_id'` which picks the numerically highest contact_id (always "Etienne Turner"). Same issue in `getThreadsSQL()` line 549-550 with `MAX(e.contact_id)` and `MAX(c.name)`.

**Fix approach:**

**MongoDB** ([src/QueryAdapter.php:569-650](src/QueryAdapter.php#L569)):
1. In the `$group` stage, collect all contacts instead of `$max`:
   - Replace `'contact_id' => ['$max' => '$contact_id']` with `'contacts' => ['$push' => ['cid' => '$contact_id', 'email' => '$from_email', 'dir' => '$direction']]`
2. After aggregation, post-process each thread in PHP:
   - Filter out contacts where `email === GMAIL_USER_EMAIL` (the user's own email)
   - Filter to only `direction === 'inbound'` contacts
   - Count occurrences of each remaining `contact_id`
   - Pick the most frequent one
   - If none found, fall back to the first non-user contact_id
3. Use the resolved contact_id for the `$lookup` (or do a separate contact name lookup after post-processing)

**SQL** ([src/QueryAdapter.php:514-567](src/QueryAdapter.php#L514)):
1. Replace `MAX(e.contact_id)` with a subquery that finds the most frequent inbound sender:
   ```sql
   (SELECT e2.contact_id FROM emails e2
    LEFT JOIN contacts c2 ON e2.contact_id = c2.id
    WHERE e2.thread_id = e.thread_id
    AND e2.direction = 'inbound'
    AND LOWER(e2.from_email) != ?
    GROUP BY e2.contact_id
    ORDER BY COUNT(*) DESC LIMIT 1) as contact_id
   ```
2. Similar subquery for contact_name.
3. Add `GMAIL_USER_EMAIL` as bind parameter.

---

### TASK 4: Enhance thread_view.php (the main task)

**Problem:** thread_view.php needs agents, cross-thread context builder, token estimation, and debug logging — while keeping existing multi-provider simultaneous sending.

**Current architecture (keep):**
- Multi-provider checkboxes → sends to `api_opus/api/query.php` simultaneously
- Conversation history in sessionStorage
- Saved responses in MongoDB `ai_query_responses`
- Quick view modal, load more responses
- Content processing modes (full/stripped/smart)

**New features to add:**

#### 4A. Full-width layout
- [thread_view.php:174](public/thread_view.php#L174): Change `.page-container { max-width: 1600px }` to `max-width: 100%; padding: 20px 30px;`
- [thread_view.php:213-217](public/thread_view.php#L213): Change `.main-grid` from `grid-template-columns: 1fr 400px` to `grid-template-columns: 1fr 500px` (wider AI panel)

#### 4B. Agent buttons with quick prompts
Add above the provider grid in the right column (after line 975):

```html
<!-- Agent Selector -->
<div class="ai-form-section">
    <label>AI Agent</label>
    <div class="agent-buttons" id="agent-buttons">
        <!-- Populated by JS -->
    </div>
    <div class="agent-quick-prompts" id="agent-quick-prompts" style="display:none;">
        <!-- Quick prompt buttons per agent -->
    </div>
</div>
```

**JS agent presets** (add as constants near top of `<script>` block):
```js
const AI_AGENTS = {
    general: { name: 'General', icon: '💬', systemPrompt: '...', quickPrompts: [...] },
    quote_analyst: { name: 'Quotes', icon: '💰', systemPrompt: '...', quickPrompts: [...] },
    deception_detector: { name: 'Deception', icon: '🔍', systemPrompt: '...', quickPrompts: [...] },
    contact_profiler: { name: 'Profiler', icon: '👤', systemPrompt: '...', quickPrompts: [...] },
    email_composer: { name: 'Composer', icon: '✍', systemPrompt: '...', quickPrompts: [...] }
};
```

When an agent is selected:
- Highlight the active agent button
- Show agent-specific quick prompts below the buttons
- Clicking a quick prompt fills the question textarea
- The agent's `systemPrompt` is prepended to the content sent to AI

#### 4C. Context builder (search + add threads from any contact)
Add a collapsible section between "Include Other Threads" and "Ask AI" cards:

```html
<!-- Context Builder -->
<div class="card">
    <div class="card-header">
        <h2>+ Add More Context</h2>
        <button class="btn btn-small" onclick="toggleContextBuilder()">Search</button>
    </div>
    <div id="context-builder" style="display:none;">
        <input type="text" id="context-search" placeholder="Search contacts or threads...">
        <div id="context-search-results"></div>
    </div>
    <div id="context-cart"></div>
</div>
```

**JS functions:**
- `toggleContextBuilder()` — show/hide search
- `searchContext(query)` — calls `api.php?action=contacts&search=...` AND `api.php?action=threads&search=...` in parallel
- Results show contacts (click to see their threads with "+Add" buttons) and threads (with "+Add" directly)
- `addThreadToContext(threadId, subject, emailCount)` — adds to `contextThreads[]` array
- `removeThreadFromContext(threadId)` — removes
- `updateContextCart()` — renders green summary: "3 additional threads, ~25 emails"
- When `submitToAI()` runs, fetches email content for each context thread (via `api.php?action=thread&thread_id=...`) and appends to the content

#### 4D. Token estimation
Add a live token counter near the submit button:

```html
<div id="token-estimate" style="font-size:0.85em; color:var(--text-secondary); margin-bottom:10px;">
    Estimated: ~0 tokens
</div>
```

**JS:**
- `MODEL_LIMITS` constant mapping model names to max context (e.g. `{'claude-sonnet-4-5-20250929': 200000, 'gpt-4o': 128000, ...}`)
- `estimateTokens()` — called on: agent change, context change, question typing, provider toggle
  - Calculates `Math.ceil(totalContentLength / 3.5)`
  - Shows warning if approaching any selected model's limit
  - Compares against the **smallest** selected model's limit (since multi-provider)
  - Display: "Estimated: ~12,500 tokens (OK for all selected models)" or "Warning: exceeds gpt-4o limit (128K)"
- In `submitToAI()`: if over limit, confirm before sending

#### 4E. System prompt injection per agent
In `submitToAI()`, prepend the selected agent's system prompt:

```js
let systemPrompt = '';
if (selectedAgent && AI_AGENTS[selectedAgent]) {
    systemPrompt = AI_AGENTS[selectedAgent].systemPrompt + '\n\n';
}
// Prepend to fullContent
fullContent = systemPrompt + fullContent;
```

This works because `api_opus/query.php` accepts `user_content` as one big string — the system prompt at the top guides the AI's behavior.

#### 4F. Debug file logging
**New API endpoint** in [public/api.php](public/api.php) — `log_ai_debug`:

```php
case 'log_ai_debug':
    $logDir = dirname(__DIR__) . '/logs';
    if (!is_dir($logDir)) mkdir($logDir, 0755, true);
    $logFile = $logDir . '/ai_debug.log';
    $logData = json_decode(file_get_contents('php://input'), true);
    $logEntry = [
        'timestamp' => date('Y-m-d H:i:s'),
        'thread_id' => $logData['thread_id'] ?? '',
        'agent' => $logData['agent'] ?? '',
        'providers' => $logData['providers'] ?? [],
        'content_length' => $logData['content_length'] ?? 0,
        'estimated_tokens' => $logData['estimated_tokens'] ?? 0,
        'question' => $logData['question'] ?? '',
        'type' => $logData['type'] ?? 'request',  // 'request' or 'response'
        'response_preview' => substr($logData['response'] ?? '', 0, 500),
        'full_content' => $logData['full_content'] ?? '' // full payload for debug
    ];
    file_put_contents($logFile, json_encode($logEntry) . "\n", FILE_APPEND | LOCK_EX);
    echo json_encode(['success' => true]);
    break;
```

**JS:** Call this endpoint in `submitToAI()`:
1. **Before** sending to AI: log the request (type: 'request', full_content, providers, tokens)
2. **After** receiving response: log the response (type: 'response', response preview, costs, timing)

Log file: `/var/www/mssql.danmarcrm.com/dev1/extension2v1/logs/ai_debug.log` — one JSON line per entry.

#### 4G. Contact memory injection (bonus)
If `primaryContactId` exists, fetch memories from `api.php?action=ai_contact_memories&contact_id=X` and inject as "KNOWN FACTS ABOUT [CONTACT]" section in the content sent to AI. Display memories as pills in the context area.

---

## Implementation Order

1. **TASK 1** — Hide AI Thread Analysis (3 line changes in index.php + app.js)
2. **TASK 2** — Speed up main page (2 line changes in app.js)
3. **TASK 3** — Fix thread contact name bug (QueryAdapter.php — MongoDB + SQL)
4. **TASK 4** — Enhance thread_view.php:
   - 4A: Full-width layout (CSS changes)
   - 4B: Agent buttons + quick prompts (HTML + JS)
   - 4C: Context builder (HTML + JS)
   - 4D: Token estimation (HTML + JS)
   - 4E: System prompt injection (JS modification to submitToAI)
   - 4F: Debug logging endpoint (api.php + JS calls)
   - 4G: Contact memory injection (JS)

## Files Modified (summary)

| File | Tasks |
|------|-------|
| [public/index.php](public/index.php) | T1: hide analysis section |
| [public/assets/app.js](public/assets/app.js) | T1: disable refresh, T2: reduce limits |
| [src/QueryAdapter.php](src/QueryAdapter.php) | T3: fix contact name bug |
| [public/api.php](public/api.php) | T4F: add `log_ai_debug` endpoint |
| [public/thread_view.php](public/thread_view.php) | T4: all thread_view enhancements |

## Verification

1. **Main dashboard:** Load page — should be fast, no analysis section visible, threads show correct contact names (not all "Etienne Turner")
2. **Thread view:** Open a thread — should be full-width, agent buttons visible, context builder expandable
3. **Agents:** Click "Quotes" agent → quick prompts appear → click one → fills question box → submit → AI response guided by quote analysis system prompt
4. **Context builder:** Click "Search" → type contact name → see their threads → click "+Add" → green cart updates → submit → AI sees all added threads
5. **Token counter:** Shows live estimate → add more context → count increases → shows warning if near limit
6. **Multi-provider:** Select Claude + GPT → submit → both respond in tabs (existing behavior preserved)
7. **Debug log:** Check `/var/www/mssql.danmarcrm.com/dev1/extension2v1/logs/ai_debug.log` → has request + response JSON lines
8. **Conversation continuity:** Ask a follow-up question → AI sees previous Q&A in context (existing behavior preserved)
9. **Saved responses:** Response auto-saves to MongoDB → appears in "Previous AI Analyses" (existing behavior preserved)


User has approved your plan. You can now start coding. Start with updating your todo list if applicable

Your plan has been saved to: /root/.claude/plans/scalable-doodling-peach.md
You can refer back to it if needed during implementation.

## Approved Plan:
# Plan: Enhance thread_view.php + Fix Main Dashboard Issues

## Context

The user found `thread_view.php` already has a working AI form with multi-provider support, conversation history (sessionStorage), saved responses, and quick view. They want the new AI features (agents, context builder, token estimation, debug logging) merged into this existing page. Additionally, the main dashboard has 3 bugs: slow loading, wrong contact names on threads, and a useless AI Thread Analysis section.

## 4 Tasks (in execution order)

---

### TASK 1: Hide AI Thread Analysis section (index.php + app.js)

**Problem:** The "AI Thread Analysis" section shows "Unknown" contacts, runs heavy N+1 queries, and auto-refreshes every 30 seconds — slowing the page.

**Files:**

| File | Change |
|------|--------|
| [public/index.php:136](public/index.php#L136) | Add `style="display:none"` to `#analysis-status-section` div |
| [public/assets/app.js:1589](public/assets/app.js#L1589) | Comment out `refreshAnalysisStatus()` call |
| [public/assets/app.js:1592](public/assets/app.js#L1592) | Comment out `setInterval(refreshAnalysisStatus, 30000)` |

---

### TASK 2: Speed up main page (app.js)

**Problem:** Threads and contacts load 20 records each — too many for initial page load.

**Files:**

| File | Change |
|------|--------|
| [public/assets/app.js:176](public/assets/app.js#L176) | `loadContacts` → change `limit: 20` to `limit: 10` |
| [public/assets/app.js:650](public/assets/app.js#L650) | `loadEmails` → change `limit: 20` to `limit: 10` |

---

### TASK 3: Fix thread contact name bug (QueryAdapter.php)

**Problem:** `getThreadsMongo()` line 609 uses `'$max' => '$contact_id'` which picks the numerically highest contact_id (always "Etienne Turner"). Same issue in `getThreadsSQL()` line 549-550 with `MAX(e.contact_id)` and `MAX(c.name)`.

**Fix approach:**

**MongoDB** ([src/QueryAdapter.php:569-650](src/QueryAdapter.php#L569)):
1. In the `$group` stage, collect all contacts instead of `$max`:
   - Replace `'contact_id' => ['$max' => '$contact_id']` with `'contacts' => ['$push' => ['cid' => '$contact_id', 'email' => '$from_email', 'dir' => '$direction']]`
2. After aggregation, post-process each thread in PHP:
   - Filter out contacts where `email === GMAIL_USER_EMAIL` (the user's own email)
   - Filter to only `direction === 'inbound'` contacts
   - Count occurrences of each remaining `contact_id`
   - Pick the most frequent one
   - If none found, fall back to the first non-user contact_id
3. Use the resolved contact_id for the `$lookup` (or do a separate contact name lookup after post-processing)

**SQL** ([src/QueryAdapter.php:514-567](src/QueryAdapter.php#L514)):
1. Replace `MAX(e.contact_id)` with a subquery that finds the most frequent inbound sender:
   ```sql
   (SELECT e2.contact_id FROM emails e2
    LEFT JOIN contacts c2 ON e2.contact_id = c2.id
    WHERE e2.thread_id = e.thread_id
    AND e2.direction = 'inbound'
    AND LOWER(e2.from_email) != ?
    GROUP BY e2.contact_id
    ORDER BY COUNT(*) DESC LIMIT 1) as contact_id
   ```
2. Similar subquery for contact_name.
3. Add `GMAIL_USER_EMAIL` as bind parameter.

---

### TASK 4: Enhance thread_view.php (the main task)

**Problem:** thread_view.php needs agents, cross-thread context builder, token estimation, and debug logging — while keeping existing multi-provider simultaneous sending.

**Current architecture (keep):**
- Multi-provider checkboxes → sends to `api_opus/api/query.php` simultaneously
- Conversation history in sessionStorage
- Saved responses in MongoDB `ai_query_responses`
- Quick view modal, load more responses
- Content processing modes (full/stripped/smart)

**New features to add:**

#### 4A. Full-width layout
- [thread_view.php:174](public/thread_view.php#L174): Change `.page-container { max-width: 1600px }` to `max-width: 100%; padding: 20px 30px;`
- [thread_view.php:213-217](public/thread_view.php#L213): Change `.main-grid` from `grid-template-columns: 1fr 400px` to `grid-template-columns: 1fr 500px` (wider AI panel)

#### 4B. Agent buttons with quick prompts
Add above the provider grid in the right column (after line 975):

```html
<!-- Agent Selector -->
<div class="ai-form-section">
    <label>AI Agent</label>
    <div class="agent-buttons" id="agent-buttons">
        <!-- Populated by JS -->
    </div>
    <div class="agent-quick-prompts" id="agent-quick-prompts" style="display:none;">
        <!-- Quick prompt buttons per agent -->
    </div>
</div>
```

**JS agent presets** (add as constants near top of `<script>` block):
```js
const AI_AGENTS = {
    general: { name: 'General', icon: '💬', systemPrompt: '...', quickPrompts: [...] },
    quote_analyst: { name: 'Quotes', icon: '💰', systemPrompt: '...', quickPrompts: [...] },
    deception_detector: { name: 'Deception', icon: '🔍', systemPrompt: '...', quickPrompts: [...] },
    contact_profiler: { name: 'Profiler', icon: '👤', systemPrompt: '...', quickPrompts: [...] },
    email_composer: { name: 'Composer', icon: '✍', systemPrompt: '...', quickPrompts: [...] }
};
```

When an agent is selected:
- Highlight the active agent button
- Show agent-specific quick prompts below the buttons
- Clicking a quick prompt fills the question textarea
- The agent's `systemPrompt` is prepended to the content sent to AI

#### 4C. Context builder (search + add threads from any contact)
Add a collapsible section between "Include Other Threads" and "Ask AI" cards:

```html
<!-- Context Builder -->
<div class="card">
    <div class="card-header">
        <h2>+ Add More Context</h2>
        <button class="btn btn-small" onclick="toggleContextBuilder()">Search</button>
    </div>
    <div id="context-builder" style="display:none;">
        <input type="text" id="context-search" placeholder="Search contacts or threads...">
        <div id="context-search-results"></div>
    </div>
    <div id="context-cart"></div>
</div>
```

**JS functions:**
- `toggleContextBuilder()` — show/hide search
- `searchContext(query)` — calls `api.php?action=contacts&search=...` AND `api.php?action=threads&search=...` in parallel
- Results show contacts (click to see their threads with "+Add" buttons) and threads (with "+Add" directly)
- `addThreadToContext(threadId, subject, emailCount)` — adds to `contextThreads[]` array
- `removeThreadFromContext(threadId)` — removes
- `updateContextCart()` — renders green summary: "3 additional threads, ~25 emails"
- When `submitToAI()` runs, fetches email content for each context thread (via `api.php?action=thread&thread_id=...`) and appends to the content

#### 4D. Token estimation
Add a live token counter near the submit button:

```html
<div id="token-estimate" style="font-size:0.85em; color:var(--text-secondary); margin-bottom:10px;">
    Estimated: ~0 tokens
</div>
```

**JS:**
- `MODEL_LIMITS` constant mapping model names to max context (e.g. `{'claude-sonnet-4-5-20250929': 200000, 'gpt-4o': 128000, ...}`)
- `estimateTokens()` — called on: agent change, context change, question typing, provider toggle
  - Calculates `Math.ceil(totalContentLength / 3.5)`
  - Shows warning if approaching any selected model's limit
  - Compares against the **smallest** selected model's limit (since multi-provider)
  - Display: "Estimated: ~12,500 tokens (OK for all selected models)" or "Warning: exceeds gpt-4o limit (128K)"
- In `submitToAI()`: if over limit, confirm before sending

#### 4E. System prompt injection per agent
In `submitToAI()`, prepend the selected agent's system prompt:

```js
let systemPrompt = '';
if (selectedAgent && AI_AGENTS[selectedAgent]) {
    systemPrompt = AI_AGENTS[selectedAgent].systemPrompt + '\n\n';
}
// Prepend to fullContent
fullContent = systemPrompt + fullContent;
```

This works because `api_opus/query.php` accepts `user_content` as one big string — the system prompt at the top guides the AI's behavior.

#### 4F. Debug file logging
**New API endpoint** in [public/api.php](public/api.php) — `log_ai_debug`:

```php
case 'log_ai_debug':
    $logDir = dirname(__DIR__) . '/logs';
    if (!is_dir($logDir)) mkdir($logDir, 0755, true);
    $logFile = $logDir . '/ai_debug.log';
    $logData = json_decode(file_get_contents('php://input'), true);
    $logEntry = [
        'timestamp' => date('Y-m-d H:i:s'),
        'thread_id' => $logData['thread_id'] ?? '',
        'agent' => $logData['agent'] ?? '',
        'providers' => $logData['providers'] ?? [],
        'content_length' => $logData['content_length'] ?? 0,
        'estimated_tokens' => $logData['estimated_tokens'] ?? 0,
        'question' => $logData['question'] ?? '',
        'type' => $logData['type'] ?? 'request',  // 'request' or 'response'
        'response_preview' => substr($logData['response'] ?? '', 0, 500),
        'full_content' => $logData['full_content'] ?? '' // full payload for debug
    ];
    file_put_contents($logFile, json_encode($logEntry) . "\n", FILE_APPEND | LOCK_EX);
    echo json_encode(['success' => true]);
    break;
```

**JS:** Call this endpoint in `submitToAI()`:
1. **Before** sending to AI: log the request (type: 'request', full_content, providers, tokens)
2. **After** receiving response: log the response (type: 'response', response preview, costs, timing)

Log file: `/var/www/mssql.danmarcrm.com/dev1/extension2v1/logs/ai_debug.log` — one JSON line per entry.

#### 4G. Contact memory injection (bonus)
If `primaryContactId` exists, fetch memories from `api.php?action=ai_contact_memories&contact_id=X` and inject as "KNOWN FACTS ABOUT [CONTACT]" section in the content sent to AI. Display memories as pills in the context area.

---

## Implementation Order

1. **TASK 1** — Hide AI Thread Analysis (3 line changes in index.php + app.js)
2. **TASK 2** — Speed up main page (2 line changes in app.js)
3. **TASK 3** — Fix thread contact name bug (QueryAdapter.php — MongoDB + SQL)
4. **TASK 4** — Enhance thread_view.php:
   - 4A: Full-width layout (CSS changes)
   - 4B: Agent buttons + quick prompts (HTML + JS)
   - 4C: Context builder (HTML + JS)
   - 4D: Token estimation (HTML + JS)
   - 4E: System prompt injection (JS modification to submitToAI)
   - 4F: Debug logging endpoint (api.php + JS calls)
   - 4G: Contact memory injection (JS)

## Files Modified (summary)

| File | Tasks |
|------|-------|
| [public/index.php](public/index.php) | T1: hide analysis section |
| [public/assets/app.js](public/assets/app.js) | T1: disable refresh, T2: reduce limits |
| [src/QueryAdapter.php](src/QueryAdapter.php) | T3: fix contact name bug |
| [public/api.php](public/api.php) | T4F: add `log_ai_debug` endpoint |
| [public/thread_view.php](public/thread_view.php) | T4: all thread_view enhancements |

## Verification

1. **Main dashboard:** Load page — should be fast, no analysis section visible, threads show correct contact names (not all "Etienne Turner")
2. **Thread view:** Open a thread — should be full-width, agent buttons visible, context builder expandable
3. **Agents:** Click "Quotes" agent → quick prompts appear → click one → fills question box → submit → AI response guided by quote analysis system prompt
4. **Context builder:** Click "Search" → type contact name → see their threads → click "+Add" → green cart updates → submit → AI sees all added threads
5. **Token counter:** Shows live estimate → add more context → count increases → shows warning if near limit
6. **Multi-provider:** Select Claude + GPT → submit → both respond in tabs (existing behavior preserved)
7. **Debug log:** Check `/var/www/mssql.danmarcrm.com/dev1/extension2v1/logs/ai_debug.log` → has request + response JSON lines
8. **Conversation continuity:** Ask a follow-up question → AI sees previous Q&A in context (existing behavior preserved)
9. **Saved responses:** Response auto-saves to MongoDB → appears in "Previous AI Analyses" (existing behavior preserved)


The plan is approved. Let me start implementing. Let me update the todo list and begin with the tasks in order.


Plan approved. Let me start implementing all 4 tasks.


TodoWrite

TASK 1: Hide AI Thread Analysis section (index.php + app.js)

TASK 2: Speed up main page - reduce thread/contact limits

TASK 3: Fix thread contact name bug in QueryAdapter.php

TASK 4A: Full-width layout for thread_view.php

TASK 4B: Agent buttons with quick prompts

TASK 4C: Context builder (search + add threads)

Building context search and thread picker

TASK 4D: Token estimation display

TASK 4E: System prompt injection per agent

TASK 4F: Debug file logging endpoint + JS calls

TASK 4G: Contact memory injection

Verify PHP syntax and test readiness


Todos have been modified successfully. Ensure that you continue to use the todo list to track your progress. Please proceed with the current tasks if applicable

TASK 1: Hide AI Thread Analysis section (index.php + app.js)

TASK 2: Speed up main page - reduce thread/contact limits

TASK 3: Fix thread contact name bug in QueryAdapter.php

TASK 4A: Full-width layout for thread_view.php

TASK 4B: Agent buttons with quick prompts

TASK 4C: Context builder (search + add threads)

Building context search and thread picker

TASK 4D: Token estimation display

TASK 4E: System prompt injection per agent

TASK 4F: Debug file logging endpoint + JS calls

TASK 4G: Contact memory injection

Verify PHP syntax and test readiness


Starting with TASK 1: Hide AI Thread Analysis section.


Edit

        <!-- Analysis Status Section -->
        <div class="analysis-status-section" id="analysis-status-section">

        <!-- Analysis Status Section (hidden - slow N+1 queries, shows Unknown contacts) -->
        <div class="analysis-status-section" id="analysis-status-section" style="display:none">


    // Load analysis status on page load
    refreshAnalysisStatus();

    // Auto-refresh analysis status every 30 seconds
    setInterval(refreshAnalysisStatus, 30000);

    // Load analysis status on page load (disabled - slow N+1 queries, shows Unknown)
    // refreshAnalysisStatus();

    // Auto-refresh analysis status every 30 seconds (disabled)
    // setInterval(refreshAnalysisStatus, 30000);


The file /var/www/mssql.danmarcrm.com/dev1/extension2v1/public/index.php has been updated successfully.

        <!-- Analysis Status Section -->
        <div class="analysis-status-section" id="analysis-status-section">

        <!-- Analysis Status Section (hidden - slow N+1 queries, shows Unknown contacts) -->
        <div class="analysis-status-section" id="analysis-status-section" style="display:none">

<?php
/**
 * Email Intelligence Dashboard
 *
 * Main web interface for viewing contacts, emails, and analysis results.
 *
 * CALLING MAP:
 * - Entry point: Web browser
 * - Uses: api.php for data (via AJAX)
 * - Uses: oauth_callback.php for Gmail connection
 *
 * @package EmailIntelligence
 */

// Load configuration
require_once dirname(__DIR__) . '/config/config.php';
require_once dirname(__DIR__) . '/vendor/autoload.php';

use EmailIntelligence\Database;
use EmailIntelligence\GmailClient;

// Start session for flash messages
session_start();

// If contact parameter is present, redirect to contact view
if (isset($_GET['contact']) && is_numeric($_GET['contact'])) {
    header('Location: contact_view.php?contact=' . (int)$_GET['contact']);
    exit;
}

// Get flash message if any
$flashMessage = $_SESSION['flash_message'] ?? null;
unset($_SESSION['flash_message']);

// Get initial status
try {
    $db = Database::getInstance();
    $syncStatus = $db->fetchOne("SELECT * FROM sync_status WHERE id = 1");
    $stats = [
        'emails' => $db->fetchColumn("SELECT COUNT(*) FROM emails"),
        'contacts' => $db->fetchColumn("SELECT COUNT(*) FROM contacts"),
        'analyzed' => $db->fetchColumn("SELECT COUNT(*) FROM emails WHERE is_analyzed = 1"),
        'pending' => $db->fetchColumn("SELECT COUNT(*) FROM emails WHERE is_analyzed = 0"),
    ];

    $gmail = new GmailClient();
    $gmailStatus = $gmail->getAuthStatus();
    $authUrl = $gmail->hasCredentials() ? $gmail->getAuthUrl() : null;

} catch (Exception $e) {
    $error = $e->getMessage();
    $stats = ['emails' => 0, 'contacts' => 0, 'analyzed' => 0, 'pending' => 0];
    $gmailStatus = ['isAuthenticated' => false];
    $syncStatus = null;
    $authUrl = null;
}
?>
<!DOCTYPE html>

    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Email Intelligence Dashboard</title>
    <link rel="stylesheet" href="assets/style.css">

Email Intelligence

AI-Powered Email Analysis for <?= htmlspecialchars(GMAIL_USER_EMAIL) ?>

        <?php if (isset($error)): ?>

            Error: <?= htmlspecialchars($error) ?>

        <?php endif; ?>

        <?php if ($flashMessage): ?>
        <div class="alert alert-<?= $flashMessage['type'] ?>">
            <?= htmlspecialchars($flashMessage['text']) ?>

        <?php endif; ?>

            <div class="status-card <?= $gmailStatus['isAuthenticated'] ? 'status-ok' : 'status-warning' ?>">

Gmail

                <?php if ($gmailStatus['isAuthenticated']): ?>

Connected

<?= htmlspecialchars($gmailStatus['userEmail']) ?>

                <?php elseif (!$gmailStatus['hasCredentials']): ?>

Not Configured

OAuth credentials missing

                <?php else: ?>

Not Connected

                    <?php if ($authUrl): ?>
                    <a href="<?= htmlspecialchars($authUrl) ?>" class="btn btn-primary">Connect Gmail
                    <?php endif; ?>
                <?php endif; ?>

Sync Status

<?= htmlspecialchars($syncStatus['status'] ?? 'unknown') ?>

                    Last: <?= $syncStatus['last_sync_at'] ? date('M j, H:i', strtotime($syncStatus['last_sync_at'])) : 'Never' ?>

                <?php if ($gmailStatus['isAuthenticated']): ?>
                <button class="btn btn-small" onclick="triggerSync()">Sync Now</button>
                <?php endif; ?>

Emails

<?= number_format($stats['emails']) ?>

                    <?= number_format($stats['analyzed']) ?> analyzed,
                    <?= number_format($stats['pending']) ?> pending

Contacts

<?= number_format($stats['contacts']) ?>

Clients & Suppliers

AI Thread Analysis

                    Threads
                    -

                    Analyzed
                    -

                    Pending
                    -

                    Progress
                    -

                <button class="btn btn-primary" onclick="triggerAnalysis()" id="btn-analyze">Analyze Threads</button>
                <button class="btn btn-small" onclick="refreshAnalysisStatus()">Refresh Status</button>

            <button class="tab-btn active" data-tab="contacts">Contacts</button>
            <button class="tab-btn" data-tab="emails">Threads</button>
            <button class="tab-btn" data-tab="products">🛒 Products</button>
            <button class="tab-btn" data-tab="gmail-accounts">📧 Gmail Accounts</button>
            <button class="tab-btn" data-tab="enrichment">🔬 Enrichment</button>
            <button class="tab-btn" data-tab="patterns">📊 Patterns</button>
            <button class="tab-btn" data-tab="business">💼 Business</button>
            <button class="tab-btn" data-tab="alerts">⚠ Alerts</button>
            <button class="tab-btn" data-tab="search">🔍 AI Search</button>
            <button class="tab-btn" data-tab="quotes">Quotes</button>
            <button class="tab-btn" data-tab="filters">Filters</button>
            <button class="tab-btn" data-tab="business-ideas">💡 Business Ideas</button>

                <input type="text" id="contact-search" placeholder="Search contacts..." class="search-input">
                <select id="contact-type-filter">
                    <option value="">All Types</option>
                    <option value="client">Clients</option>
                    <option value="supplier">Suppliers</option>
                    <option value="other">Other</option>
                </select>
                <button class="btn btn-small" onclick="loadContacts()">Refresh</button>

Loading contacts...

                <input type="text" id="email-search" placeholder="Search threads..." class="search-input">
                <button class="btn btn-small" onclick="loadEmails()">Refresh</button>

Click Refresh to load email threads

🛒 Product Search with AI

Search products from Amazon & AliExpress with advanced filters and AI recommendations

                        Search Query
                        <input type="text" id="quick-product-search" placeholder="e.g., LED strip lights COB waterproof" class="search-input" style="width: 100%;">

                        IP Rating
                        <select id="quick-filter-ip" class="search-input" style="width: 100%;">
                            <option value="">Any</option>
                            <option value="ip65">IP65</option>
                            <option value="ip67">IP67</option>
                            <option value="ip68">IP68</option>
                            <option value="ip6+">IP6+</option>
                        </select>

                        Length
                        <select id="quick-filter-length" class="search-input" style="width: 100%;">
                            <option value="">Any</option>
                            <option value="5m">5m</option>
                            <option value="10m">10m</option>
                            <option value="5m+">5m+</option>
                            <option value="10m+">10m+</option>
                        </select>

                        <button class="btn btn-primary" onclick="window.location.href='product_search.php'" style="background: linear-gradient(135deg, #f59e0b 0%, #d97706 100%); border: none;">
                            Open Full Search
                        </button>

🎯 Smart Filters

                        Filter by IP rating (IP65, IP 65, ip6, etc.), length (5m, 5 m, 10 meters), and more.
                        Handles all format variations automatically.

🤖 AI Recommendations

                        Send your product list to AI and get best picks categorized as Best Value, Top Rated, and Recommended.

➕ Custom Filters

                        Add custom filters for COB type, wattage, voltage, or any text search. Extensible filter system.

                    🔍 Open Product Search Dashboard

📧 Gmail Account Management

Manage multiple Gmail accounts for email synchronization

Add New Gmail Account

                        Email Address
                        <input type="email" id="new-account-email" placeholder="account@gmail.com"
                               class="search-input" style="width: 100%;">

                        Display Name (Optional)
                        <input type="text" id="new-account-name" placeholder="Account Name"
                               class="search-input" style="width: 100%;">

                        <button class="btn btn-primary" onclick="addGmailAccount()" style="width: 100%;">Add Account</button>

Total Accounts

-

Active

-

Authenticated

-

Total Emails

-

                <button class="btn btn-primary" onclick="syncAllAccounts()">🔄 Sync All Accounts</button>
                <button class="btn btn-small" onclick="loadGmailAccounts()">Refresh</button>

Loading Gmail accounts...

🔬 Data Enrichment

Enrich contacts and companies with data from Apollo.io and web scraping

0

Pending

0

Processing

0

Completed

0

Failed

🔍 Enrich Single Contact

                    <input type="text" id="enrich-single-email" placeholder="Enter email address..." style="width: 100%; padding: 10px; border: 1px solid #d1d5db; border-radius: 6px; margin-bottom: 10px; box-sizing: border-box;">
                    <button class="btn btn-primary" onclick="enrichSingleContact()" style="width: 100%; background: #10b981; border: none; padding: 10px; border-radius: 6px; color: white; cursor: pointer;">
                        Enrich Contact
                    </button>

🏢 Enrich Single Company

                    <input type="text" id="enrich-single-domain" placeholder="Enter company domain..." style="width: 100%; padding: 10px; border: 1px solid #d1d5db; border-radius: 6px; margin-bottom: 10px; box-sizing: border-box;">
                    <button class="btn btn-primary" onclick="enrichSingleCompany()" style="width: 100%; background: #3b82f6; border: none; padding: 10px; border-radius: 6px; color: white; cursor: pointer;">
                        Enrich Company
                    </button>

📦 Batch Enrichment

                    <button class="btn" onclick="enrichAllContacts()" style="background: #10b981; color: white; border: none; padding: 12px; border-radius: 6px; cursor: pointer;">
                        👥 Enrich All Contacts
                    </button>
                    <button class="btn" onclick="enrichAllCompanies()" style="background: #3b82f6; color: white; border: none; padding: 12px; border-radius: 6px; cursor: pointer;">
                        🏢 Enrich All Companies
                    </button>
                    <button class="btn" onclick="processEnrichmentQueue()" style="background: #8b5cf6; color: white; border: none; padding: 12px; border-radius: 6px; cursor: pointer;">
                        ⚡ Process Queue
                    </button>
                    <button class="btn" onclick="loadEnrichmentStatus()" style="background: #6b7280; color: white; border: none; padding: 12px; border-radius: 6px; cursor: pointer;">
                        🔄 Refresh Status
                    </button>

📋 Recent Enrichment Activity

Click Refresh Status to load logs...

📊 Behavioral Pattern Detection

AI-powered detection of risky patterns across all email communications

Loading pattern statistics...

💼 Business Opportunities

AI-identified business opportunities, cross-sell suggestions, and deal insights from email analysis

Total Opportunities

-

Active Quotes

-

Cross-sell Ideas

-

Avg Deal Value

-

Loading business opportunities...

⚠ Risk Alerts

Automated alerts for high-risk behavioral patterns detected in communications

                <select id="alert-severity-filter" style="padding: 10px; border-radius: 6px; border: 1px solid #e0e0e0;">
                    <option value="">All Severities</option>
                    <option value="critical">Critical</option>
                    <option value="high">High</option>
                    <option value="medium">Medium</option>
                    <option value="low">Low</option>
                </select>
                <select id="alert-status-filter" style="padding: 10px; border-radius: 6px; border: 1px solid #e0e0e0;">
                    <option value="unread">Unread</option>
                    <option value="all">All</option>
                    <option value="read">Read</option>
                </select>
                <button class="btn btn-small" onclick="loadAlerts()">Refresh</button>

Loading alerts...

🔍 Semantic Search

Search across all emails using natural language. Find patterns, behaviors, and similar situations.

                    <input type="text" id="semantic-search-input" placeholder="e.g., 'payment delays', 'lies about deadlines', 'scope creep'..."
                           style="flex: 1; min-width: 300px; padding: 12px; border-radius: 6px; border: none; font-size: 1em;">
                    <select id="search-type" style="padding: 12px; border-radius: 6px; border: none;">
                        <option value="">All Types</option>
                        <option value="email">Emails</option>
                        <option value="thread_summary">Thread Summaries</option>
                        <option value="pattern">Detected Patterns</option>
                    </select>
                    <button class="btn" onclick="semanticSearch()" style="background: white; color: #667eea; padding: 12px 24px;">Search</button>

                Example searches:

                    <button class="btn btn-small" onclick="quickSearch('payment delay excuses')">Payment delays</button>
                    <button class="btn btn-small" onclick="quickSearch('changed story inconsistency')">Story changes</button>
                    <button class="btn btn-small" onclick="quickSearch('price negotiation aggressive')">Price negotiation</button>
                    <button class="btn btn-small" onclick="quickSearch('blame shifting responsibility')">Blame shifting</button>
                    <button class="btn btn-small" onclick="quickSearch('scope creep additional work')">Scope creep</button>
                    <button class="btn btn-small" onclick="quickSearch('competitor mention shopping around')">Competitor mentions</button>

                Index Management:

                    <button class="btn btn-small" onclick="indexVectors('all')">Index All Emails</button>
                    <button class="btn btn-small" onclick="checkVectorStatus()">Check Status</button>

Enter a search query above to find relevant emails and patterns.

                <select id="quote-outcome-filter">
                    <option value="">All Outcomes</option>
                    <option value="pending">Pending</option>
                    <option value="accepted">Accepted</option>
                    <option value="rejected">Rejected</option>
                    <option value="negotiated">Negotiated</option>
                </select>
                <button class="btn btn-small" onclick="loadQuotes()">Refresh</button>

Loading quotes...

                <input type="text" id="new-filter" placeholder="Email or @domain.com" class="search-input">
                <input type="text" id="filter-description" placeholder="Description (optional)" class="search-input">
                <button class="btn btn-primary" onclick="addFilter()">Add Filter</button>

                Add email addresses or domains (starting with @) to filter which emails are synced.
                Only emails to/from these addresses will be downloaded.

Loading filters...

💡 Business Ideas Research

AI-powered research into business verticals, horizontals, and growth scenarios using web scraping and analysis

📊 Your Business Context

                    Click to set/update your financial goals (helps AI provide better recommendations)

                            Revenue Target
                            <input type="text" id="owner-revenue-target" placeholder="e.g., $500K/year, 2x current revenue" style="width: 100%; padding: 10px; border: 1px solid #d1d5db; border-radius: 6px; box-sizing: border-box;">

                            Investment Capacity
                            <input type="text" id="owner-investment-capacity" placeholder="e.g., $50K available for new ventures" style="width: 100%; padding: 10px; border: 1px solid #d1d5db; border-radius: 6px; box-sizing: border-box;">

                            Growth Objectives
                            <input type="text" id="owner-growth-objectives" placeholder="e.g., Expand to new markets, diversify products" style="width: 100%; padding: 10px; border: 1px solid #d1d5db; border-radius: 6px; box-sizing: border-box;">

                            Timeline
                            <input type="text" id="owner-timeline" placeholder="e.g., 12-18 months" style="width: 100%; padding: 10px; border: 1px solid #d1d5db; border-radius: 6px; box-sizing: border-box;">

                            Additional Notes
                            <textarea id="owner-notes" placeholder="Any other relevant information about your business situation..." style="width: 100%; padding: 10px; border: 1px solid #d1d5db; border-radius: 6px; min-height: 60px; box-sizing: border-box;"></textarea>

                            <button class="btn btn-primary" onclick="saveOwnerStatus()" style="background: #8b5cf6;">Save Business Context</button>

Start New Research Project

                    Research Query
                    <textarea id="business-ideas-query"
                        placeholder="e.g., Research 50 business verticals related to LED lighting and smart home technology. For each vertical, identify 20-30 horizontal market segments. Focus on B2B opportunities, emerging markets, and areas with high growth potential."
                        style="width: 100%; min-height: 100px; padding: 12px; border: 1px solid #d1d5db; border-radius: 6px; resize: vertical; box-sizing: border-box;"></textarea>

                    Advanced Options

                                Target Verticals
                                <input type="number" id="param-verticals" value="50" min="1" max="100" style="width: 100%; padding: 8px; border: 1px solid #d1d5db; border-radius: 4px; box-sizing: border-box;">

                                Horizontals per Vertical
                                <input type="number" id="param-horizontals" value="25" min="1" max="50" style="width: 100%; padding: 8px; border: 1px solid #d1d5db; border-radius: 4px; box-sizing: border-box;">

                                Research Depth
                                <select id="param-depth" style="width: 100%; padding: 8px; border: 1px solid #d1d5db; border-radius: 4px; box-sizing: border-box;">
                                    <option value="shallow">Shallow (Fast)</option>
                                    <option value="medium" selected>Medium</option>
                                    <option value="deep">Deep (Thorough)</option>
                                </select>

                                Scenario Depth
                                <select id="param-scenario-depth" style="width: 100%; padding: 8px; border: 1px solid #d1d5db; border-radius: 4px; box-sizing: border-box;">
                                    <option value="high_level" selected>High-Level Strategic</option>
                                    <option value="detailed">Detailed Action Plans</option>
                                    <option value="financial">Financial Projections</option>
                                </select>

                <button class="btn btn-primary" onclick="createBusinessIdeasProject()" style="background: linear-gradient(135deg, #8b5cf6 0%, #6366f1 100%); padding: 12px 24px;">
                    🚀 Start Research Project
                </button>

0

Pending

0

Running

0

Completed

0

Failed

0

Findings

                <button class="btn btn-primary" onclick="processBusinessIdeasQueue()" style="background: #8b5cf6;">
                    ⚡ Process Queue
                </button>
                <button class="btn btn-small" onclick="loadBusinessIdeasProjects()">🔄 Refresh</button>

Loading research projects...

                ×

                ×

                ×

                    AI

AI Assistant

General Chat

                    <button onclick="startNewSession()" title="New chat" style="background:rgba(255,255,255,0.15); border:none; color:#fff; border-radius:6px; padding:4px 8px; cursor:pointer; font-size:0.7em;">+ New</button>
                    <button onclick="loadChatSessions()" title="Chat history" style="background:rgba(255,255,255,0.15); border:none; color:#fff; border-radius:6px; padding:4px 8px; cursor:pointer; font-size:0.7em;">History</button>
                    <button onclick="clearAiChat()" title="Clear chat" style="background:rgba(255,255,255,0.15); border:none; color:#fff; border-radius:6px; padding:4px 8px; cursor:pointer; font-size:0.7em;">Clear</button>
                    <button onclick="toggleAiChat()" style="background:rgba(255,255,255,0.15); border:none; color:#fff; border-radius:6px; padding:4px 8px; cursor:pointer; font-size:1em;">X</button>

                    <select id="ai-chat-provider" onchange="updateAiModelDropdown()" style="width:100px; background:#1e1e2e; color:#e2e8f0; border:1px solid #3d3d5c; border-radius:6px; padding:3px 6px; font-size:0.72em; cursor:pointer;">

                    </select>
                    <select id="ai-chat-model" style="flex:1; background:#1e1e2e; color:#e2e8f0; border:1px solid #3d3d5c; border-radius:6px; padding:3px 6px; font-size:0.72em; cursor:pointer;">

                    </select>

                    <button class="ai-agent-btn" data-agent="general" onclick="selectAgent('general')" style="background:#6366f1; color:#fff; border:none; border-radius:5px; padding:3px 8px; font-size:0.68em; cursor:pointer;">General</button>
                    <button class="ai-agent-btn" data-agent="quote_analyst" onclick="selectAgent('quote_analyst')" style="background:#2d2d4e; color:#94a3b8; border:none; border-radius:5px; padding:3px 8px; font-size:0.68em; cursor:pointer;">Quotes</button>
                    <button class="ai-agent-btn" data-agent="deception_detector" onclick="selectAgent('deception_detector')" style="background:#2d2d4e; color:#94a3b8; border:none; border-radius:5px; padding:3px 8px; font-size:0.68em; cursor:pointer;">Deception</button>
                    <button class="ai-agent-btn" data-agent="contact_profiler" onclick="selectAgent('contact_profiler')" style="background:#2d2d4e; color:#94a3b8; border:none; border-radius:5px; padding:3px 8px; font-size:0.68em; cursor:pointer;">Profiler</button>
                    <button class="ai-agent-btn" data-agent="email_composer" onclick="selectAgent('email_composer')" style="background:#2d2d4e; color:#94a3b8; border:none; border-radius:5px; padding:3px 8px; font-size:0.68em; cursor:pointer;">Composer</button>

                    Search contacts/threads to add context...
                    +

                        <input id="ai-context-search-input" type="text" placeholder="Search contacts or threads by name, email, subject..."
                               style="flex:1; background:#252540; color:#e2e8f0; border:1px solid #3d3d5c; border-radius:6px; padding:5px 8px; font-size:0.72em; outline:none;"
                               onkeydown="if(event.key==='Enter')searchContextContacts()">
                        <button onclick="searchContextContacts()" style="background:#6366f1; color:#fff; border:none; border-radius:6px; padding:4px 10px; font-size:0.7em; cursor:pointer;">Search</button>

AI

Search a contact, add threads, select an agent, and ask away.

Or open a thread/contact first for quick context.

                    <textarea id="ai-chat-input" placeholder="Ask anything... (Enter to send, Shift+Enter for newline)" style="flex:1; background:#1e1e2e; color:#e2e8f0; border:1px solid #3d3d5c; border-radius:8px; padding:8px 10px; font-size:0.82em; resize:none; height:50px; font-family:inherit; outline:none;" onkeydown="handleAiChatKey(event)"></textarea>
                    <button onclick="sendAiChat()" id="ai-chat-send-btn" style="background:linear-gradient(135deg,#6366f1,#8b5cf6); color:#fff; border:none; border-radius:8px; padding:8px 14px; cursor:pointer; font-size:1.1em; height:50px; min-width:44px; transition:opacity 0.2s;">Send</button>

        <button id="ai-chat-fab" onclick="toggleAiChat()" title="AI Assistant" style="position:fixed; bottom:20px; right:20px; width:56px; height:56px; background:linear-gradient(135deg,#6366f1,#8b5cf6); color:#fff; border:none; border-radius:50%; font-size:1.5em; cursor:pointer; box-shadow:0 4px 20px rgba(99,102,241,0.5); z-index:9998; transition:transform 0.2s; display:flex; align-items:center; justify-content:center;" onmouseover="this.style.transform='scale(1.1)'" onmouseout="this.style.transform='scale(1)'">AI</button>

Processing...

-        <!-- Analysis Status Section -->

-        <div class="analysis-status-section" id="analysis-status-section">

+        <!-- Analysis Status Section (hidden - slow N+1 queries, shows Unknown contacts) -->

+        <div class="analysis-status-section" id="analysis-status-section" style="display:none">