Projects / Booking Monitor 03/2026

Booking Monitor 03/2026 active

Edit

Overview

booking-monitor-03-2026

# bMonV3_2 - Session Log (2026-03-07)

## Overview

Full setup session covering: report fixes, nightly fetch automation, and daily AI email digest system.

---

## 1. Report Pricing Fixes (`reportsV3/report_pricing_v1.php`)

### Tab Loading Fix
- **Problem**: Only the Occupancy Rate tab loaded; all other tabs returned 0 bytes via AJAX.
- **Root cause**: Deeply nested `if/endif` blocks (`$loadError` if/elseif/else inside `!$isAjaxRequest`) made AJAX splitting impossible.
- **Fix**: Changed `shouldRenderTab()` to render ALL tabs on initial page load (no lazy loading). AJAX still works for date filter changes.

### Top Navigation Menu
- Added `nav.php` include after `<body>` tag for consistent site navigation.

### Default Date Ranges Changed
- Check-in: today to **+3 days** (was +30 days)
- Query Date: past **3 days** (was 14 days)
```php
$defaultCiTo = date('Y-m-d', strtotime('+3 days'));
$defaultQdFrom = date('Y-m-d', strtotime($defaultQdTo . ' -3 days'));
```

---

## 2. Dashboard 500 Error Fix (`fetch_v2026/dashboard.php`)

- **Endpoint**: `?action=recent_failures&limit=50`
- **Error**: `$in needs an array` — PHP's `array_unique()` preserves keys, creating associative array.
- **Fix**: Wrapped with `array_values()`:
```php
$hotelIds = array_values(array_unique(array_column($failures, 'hotel_id')));
```

---

## 3. Nightly Fetch Automation

### Script: `fetch_v2026/nightlyFetch.sh`
- Schedules jobs via `jobScheduler_propsid.php` (today + 30 days)
- Checks browser pool health at `http://127.0.0.1:3100/health`
- Launches **9 parallel workers** via `jobWorker.sh`
- Waits for all workers, logs results

### Cron (2:00 AM daily):
```
0 2 * * * /var/www/__slpx2/www/mssql.danmarcrm.com/dev1/bMonV3_2/fetch_v2026/nightlyFetch.sh >> /var/www/__slpx2/www/mssql.danmarcrm.com/dev1/bMonV3_2/fetch_v2026/logs/cron_nightly.log 2>&1
```

### Worker count: 9 (default in nightlyFetch.sh)

---

## 4. Daily AI Email Digest

### Files Created

| File | Purpose |
|------|---------|
| `fetch_v2026/daily_digest.php` | Main script: gathers data, calls AI, sends email |
| `fetch_v2026/daily_digest_config.php` | All configuration (editable without touching main script) |

### Config Highlights (`daily_digest_config.php`)

```php
'email' => [
'to' => 'dan@danmarcrm.com',
'from' => 'booking_monitor@crm.place',
'from_name' => 'bMon Daily Digest',
'subject' => 'bMon Daily Report - {date}',
'sendgrid_api_key' => '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',
],
'ai' => [
'provider' => 'claude',
'model' => null, // uses default from configIndex.php
'max_tokens' => 4000,
],
'data' => [
'query_days_back' => 3, // past 3 days of fetch data
'checkin_days_ahead' => 7, // next 7 days of bookings
'market_top_n' => 30, // top 30 cheapest hotels in detail
],
```

### Configurable Data Sections (toggle on/off in config)
| Section | Description |
|---------|-------------|
| `zenith_detail` | 4 Zenith properties: per-room prices, availability evolution |
| `market_summary` | All hotels aggregated: occupancy, avg/min/max prices per checkin |
| `market_hotels` | Top 30 cheapest hotels with prices per checkin date |
| `market_occupancy` | Per-checkin: total hotels, available, sold-out % |
| `booking_detection` | Zenith rooms that changed availability (booking/cancellation) |
| `fetch_stats` | Fetch job success/failure counts |

### System & User Prompts
- Both fully editable in config file
- `system_prompt`: Revenue manager role, format instructions (Key Highlights, Urgent Actions, Pricing Recommendations, etc.)
- `user_prompt`: Template with `{data}`, `{date}`, `{query_days_back}`, `{checkin_days_ahead}` placeholders

### CLI Flags
```bash
php8.3 daily_digest.php # Full run: AI + email + MongoDB save
php8.3 daily_digest.php --dry-run # Data gathering only, no AI call
php8.3 daily_digest.php --no-email # AI call + MongoDB save, skip email
php8.3 daily_digest.php --stdout # Print AI analysis to terminal
```

### Cron (8:00 AM daily):
```
0 8 * * * /usr/bin/php8.3 /var/www/__slpx2/www/mssql.danmarcrm.com/dev1/bMonV3_2/fetch_v2026/daily_digest.php >> /var/www/__slpx2/www/mssql.danmarcrm.com/dev1/bMonV3_2/fetch_v2026/logs/cron_digest.log 2>&1
```

### MongoDB Storage
- Collection: `price_ai_analyses`
- Fields: `source`, `query_date`, `analysis`, `prompt`, `system_prompt`, `model`, `sections_included`, `data_stats`, `created_at`

---

## 5. Key Technical Discoveries

### MongoDB `room_details` Field Types
| Field | Type | Example | Notes |
|-------|------|---------|-------|
| `price` | STRING | `"€ 92"` | Display format, NOT numeric |
| `raw_price` | STRING | `"92.00"` | Numeric string |
| `avg_price_per_night_eur` | STRING | `"92.00"` | Best field for price — use `$convert` to double |
| `available_units` | INT | `2` | Correct availability field |
| `nr_left` | often NULL | | DO NOT USE for availability |
| `room_name` | STRING | `"No Availability"` | Filter these out (price=0 placeholders) |

### Price Conversion in MongoDB Aggregation
```php
'price_num' => ['$convert' => [
'input' => '$avg_price_per_night_eur',
'to' => 'double',
'onError' => 0,
'onNull' => 0
]]
```

### Two-Stage Aggregation Pattern (for market stats)
1. Stage 1: Group by `hotel_id + checkin` → get per-hotel min price
2. Stage 2: Group by `checkin` → get market avg/min/max across hotels

---

## 6. Full Crontab (as of 2026-03-07)

```cron
## bMonV3_2 booking fetch + daily digest
0 2 * * * /var/www/__slpx2/www/mssql.danmarcrm.com/dev1/bMonV3_2/fetch_v2026/nightlyFetch.sh >> .../logs/cron_nightly.log 2>&1
0 8 * * * /usr/bin/php8.3 /var/www/__slpx2/www/mssql.danmarcrm.com/dev1/bMonV3_2/fetch_v2026/daily_digest.php >> .../logs/cron_digest.log 2>&1
```

---

## 7. Key File Paths

| Path | Description |
|------|-------------|
| `fetch_v2026/daily_digest.php` | Daily AI digest main script |
| `fetch_v2026/daily_digest_config.php` | Digest configuration (prompts, email, sections) |
| `fetch_v2026/nightlyFetch.sh` | Nightly fetch orchestrator |
| `fetch_v2026/jobScheduler_propsid.php` | Creates fetch jobs from propsid table |
| `fetch_v2026/jobWorker.sh` | Individual worker (claims + processes jobs) |
| `fetch_v2026/getHotelAndRoomDetailsPool.php` | Scraper using browser pool |
| `fetch_v2026/dashboard.php` | Fetch monitoring dashboard |
| `fetch_v2026/config.php` | Fetch system configuration |
| `fetch_v2026/pool/` | Browser pool service (PM2) |
| `fetch_v2026/logs/` | All log files |
| `reportsV3/report_pricing_v1.php` | Pricing reports dashboard |
| `mongodb/MongoConnection.php` | MongoDB wrapper class |

---

## 8. Test Results (2026-03-07 21:03)

Full pipeline test (`--stdout`):
- **Data**: 30 hotels, 8 checkin dates, 204 hotels in propsid, 30,814 char prompt
- **AI model**: claude-sonnet-4-20250514
- **AI response**: 1,946 chars
- **MongoDB**: Saved to `price_ai_analyses`
- **Email**: Sent via SendGrid (HTTP 202) to dan@danmarcrm.com
- **Note**: Zenith showed no data because last fetch was 2026-02-28 (outside 3-day window). After tonight's 2 AM fetch, the 8 AM digest will include fresh Zenith data.

---

## 9. Infrastructure Notes

- **PHP**: 8.3 (`/usr/bin/php8.3`)
- **MongoDB**: Local, database `booking_analysis`, port 27017
- **Browser Pool**: PM2-managed, 20 browsers, port 3100, local only
- **Zenith Hotel IDs**: 13613044, 15081492, 15046719, 13121164
- **Server**: `/var/www/__slpx2/www/mssql.danmarcrm.com/dev1/bMonV3_2`
- **URL**: `https://bmonv3-2.danmarcrm.com`


// after compactation

# bMonV3_2 - Session Log (2026-03-07)

## Overview

Full setup session covering: report fixes, nightly fetch automation, and daily AI email digest system.

---

## 1. Report Pricing Fixes (`reportsV3/report_pricing_v1.php`)

### Tab Loading Fix
- **Problem**: Only the Occupancy Rate tab loaded; all other tabs returned 0 bytes via AJAX.
- **Root cause**: Deeply nested `if/endif` blocks (`$loadError` if/elseif/else inside `!$isAjaxRequest`) made AJAX splitting impossible.
- **Fix**: Changed `shouldRenderTab()` to render ALL tabs on initial page load (no lazy loading). AJAX still works for date filter changes.

### Top Navigation Menu
- Added `nav.php` include after `<body>` tag for consistent site navigation.

### Default Date Ranges Changed
- Check-in: today to **+3 days** (was +30 days)
- Query Date: past **3 days** (was 14 days)
```php
$defaultCiTo = date('Y-m-d', strtotime('+3 days'));
$defaultQdFrom = date('Y-m-d', strtotime($defaultQdTo . ' -3 days'));
```

---

## 2. Dashboard 500 Error Fix (`fetch_v2026/dashboard.php`)

- **Endpoint**: `?action=recent_failures&limit=50`
- **Error**: `$in needs an array` — PHP's `array_unique()` preserves keys, creating associative array.
- **Fix**: Wrapped with `array_values()`:
```php
$hotelIds = array_values(array_unique(array_column($failures, 'hotel_id')));
```

---

## 3. Nightly Fetch Automation

### Script: `fetch_v2026/nightlyFetch.sh`
- Schedules jobs via `jobScheduler_propsid.php` (today + 30 days)
- Checks browser pool health at `http://127.0.0.1:3100/health`
- Launches **9 parallel workers** via `jobWorker.sh`
- Waits for all workers, logs results

### Cron (2:00 AM daily):
```
0 2 * * * /var/www/__slpx2/www/mssql.danmarcrm.com/dev1/bMonV3_2/fetch_v2026/nightlyFetch.sh >> /var/www/__slpx2/www/mssql.danmarcrm.com/dev1/bMonV3_2/fetch_v2026/logs/cron_nightly.log 2>&1
```

### Worker count: 9 (default in nightlyFetch.sh)

---

## 4. Daily AI Email Digest

### Files Created

| File | Purpose |
|------|---------|
| `fetch_v2026/daily_digest.php` | Main script: gathers data, calls AI, sends email |
| `fetch_v2026/daily_digest_config.php` | All configuration (editable without touching main script) |

### Config Highlights (`daily_digest_config.php`)

```php
'email' => [
'to' => 'dan@danmarcrm.com',
'from' => 'booking_monitor@crm.place',
'from_name' => 'bMon Daily Digest',
'subject' => 'bMon Daily Report - {date}',
'sendgrid_api_key' => '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',
],
'ai' => [
'provider' => 'claude',
'model' => null, // uses default from configIndex.php
'max_tokens' => 4000,
],
'data' => [
'query_days_back' => 3, // past 3 days of fetch data
'checkin_days_ahead' => 7, // next 7 days of bookings
'market_top_n' => 30, // top 30 cheapest hotels in detail
],
```

### Configurable Data Sections (toggle on/off in config)
| Section | Description |
|---------|-------------|
| `zenith_detail` | 4 Zenith properties: per-room prices, availability evolution |
| `market_summary` | All hotels aggregated: occupancy, avg/min/max prices per checkin |
| `market_hotels` | Top 30 cheapest hotels with prices per checkin date |
| `market_occupancy` | Per-checkin: total hotels, available, sold-out % |
| `booking_detection` | Zenith rooms that changed availability (booking/cancellation) |
| `fetch_stats` | Fetch job success/failure counts |

### System & User Prompts
- Both fully editable in config file
- `system_prompt`: Revenue manager role, format instructions (Key Highlights, Urgent Actions, Pricing Recommendations, etc.)
- `user_prompt`: Template with `{data}`, `{date}`, `{query_days_back}`, `{checkin_days_ahead}` placeholders

### CLI Flags
```bash
php8.3 daily_digest.php # Full run: AI + email + MongoDB save
php8.3 daily_digest.php --dry-run # Data gathering only, no AI call
php8.3 daily_digest.php --no-email # AI call + MongoDB save, skip email
php8.3 daily_digest.php --stdout # Print AI analysis to terminal
```

### Cron (8:00 AM daily):
```
0 8 * * * /usr/bin/php8.3 /var/www/__slpx2/www/mssql.danmarcrm.com/dev1/bMonV3_2/fetch_v2026/daily_digest.php >> /var/www/__slpx2/www/mssql.danmarcrm.com/dev1/bMonV3_2/fetch_v2026/logs/cron_digest.log 2>&1
```

### MongoDB Storage
- Collection: `price_ai_analyses`
- Fields: `source`, `query_date`, `analysis`, `prompt`, `system_prompt`, `model`, `sections_included`, `data_stats`, `created_at`

---

## 5. Key Technical Discoveries

### MongoDB `room_details` Field Types
| Field | Type | Example | Notes |
|-------|------|---------|-------|
| `price` | STRING | `"€ 92"` | Display format, NOT numeric |
| `raw_price` | STRING | `"92.00"` | Numeric string |
| `avg_price_per_night_eur` | STRING | `"92.00"` | Best field for price — use `$convert` to double |
| `available_units` | INT | `2` | Correct availability field |
| `nr_left` | often NULL | | DO NOT USE for availability |
| `room_name` | STRING | `"No Availability"` | Filter these out (price=0 placeholders) |

### Price Conversion in MongoDB Aggregation
```php
'price_num' => ['$convert' => [
'input' => '$avg_price_per_night_eur',
'to' => 'double',
'onError' => 0,
'onNull' => 0
]]
```

### Two-Stage Aggregation Pattern (for market stats)
1. Stage 1: Group by `hotel_id + checkin` → get per-hotel min price
2. Stage 2: Group by `checkin` → get market avg/min/max across hotels

---

## 6. Full Crontab (as of 2026-03-07)

```cron
## bMonV3_2 booking fetch + daily digest
0 2 * * * /var/www/__slpx2/www/mssql.danmarcrm.com/dev1/bMonV3_2/fetch_v2026/nightlyFetch.sh >> .../logs/cron_nightly.log 2>&1
0 8 * * * /usr/bin/php8.3 /var/www/__slpx2/www/mssql.danmarcrm.com/dev1/bMonV3_2/fetch_v2026/daily_digest.php >> .../logs/cron_digest.log 2>&1
```

---

## 7. Key File Paths

| Path | Description |
|------|-------------|
| `fetch_v2026/daily_digest.php` | Daily AI digest main script |
| `fetch_v2026/daily_digest_config.php` | Digest configuration (prompts, email, sections) |
| `fetch_v2026/nightlyFetch.sh` | Nightly fetch orchestrator |
| `fetch_v2026/jobScheduler_propsid.php` | Creates fetch jobs from propsid table |
| `fetch_v2026/jobWorker.sh` | Individual worker (claims + processes jobs) |
| `fetch_v2026/getHotelAndRoomDetailsPool.php` | Scraper using browser pool |
| `fetch_v2026/dashboard.php` | Fetch monitoring dashboard |
| `fetch_v2026/config.php` | Fetch system configuration |
| `fetch_v2026/pool/` | Browser pool service (PM2) |
| `fetch_v2026/logs/` | All log files |
| `reportsV3/report_pricing_v1.php` | Pricing reports dashboard |
| `mongodb/MongoConnection.php` | MongoDB wrapper class |

---

## 8. Test Results (2026-03-07 21:03)

Full pipeline test (`--stdout`):
- **Data**: 30 hotels, 8 checkin dates, 204 hotels in propsid, 30,814 char prompt
- **AI model**: claude-sonnet-4-20250514
- **AI response**: 1,946 chars
- **MongoDB**: Saved to `price_ai_analyses`
- **Email**: Sent via SendGrid (HTTP 202) to dan@danmarcrm.com
- **Note**: Zenith showed no data because last fetch was 2026-02-28 (outside 3-day window). After tonight's 2 AM fetch, the 8 AM digest will include fresh Zenith data.

---

## 9. Infrastructure Notes

- **PHP**: 8.3 (`/usr/bin/php8.3`)
- **MongoDB**: Local, database `booking_analysis`, port 27017
- **Browser Pool**: PM2-managed, 20 browsers, port 3100, local only
- **Zenith Hotel IDs**: 13613044, 15081492, 15046719, 13121164
- **Server**: `/var/www/__slpx2/www/mssql.danmarcrm.com/dev1/bMonV3_2`
- **URL**: `https://bmonv3-2.danmarcrm.com`

Links

No links added.

Upload Conversation

Raw: saves entire content as one entry. Structured: parses User:/Assistant: lines. JSON: parses [{role,content}].
API: https://mssql.danmarcrm.com/dev1/dmcallv1/api/projects/booking-monitor-03-2026 — Returns full project JSON for AI model context.