costco-grocery-list/docs/migration/MIGRATION_GUIDE.md

# Multi-Household Architecture Migration Guide

## Pre-Migration Checklist

- [ ] **Backup Database**
  ```bash
  pg_dump -U your_user -d grocery_list > backup_$(date +%Y%m%d_%H%M%S).sql
  ```

- [ ] **Test on Staging First**
  - Copy production database to staging environment
  - Run migration on staging
  - Verify all data migrated correctly
  - Test application functionality

- [ ] **Review Migration Script**
  - Read through `multi_household_architecture.sql`
  - Understand each step
  - Note verification queries

- [ ] **Announce Maintenance Window**
  - Notify users of downtime
  - Schedule during low-usage period
  - Estimate 15-30 minutes for migration

## Running the Migration

### 1. Connect to Database

```bash
psql -U your_user -d grocery_list
```

### 2. Run Migration

```sql
\i backend/migrations/multi_household_architecture.sql
```

The script will:
1. ✅ Create 8 new tables
2. ✅ Create default "Main Household"
3. ✅ Create default "Costco" store
4. ✅ Migrate all users to household members
5. ✅ Extract items to master catalog
6. ✅ Migrate grocery_list → household_lists
7. ✅ Migrate classifications
8. ✅ Migrate history records
9. ✅ Update user system roles

### 3. Verify Migration

Run these queries inside psql:

```sql
-- Check household created
SELECT * FROM households;

-- Check all users migrated
SELECT u.username, u.role as system_role, hm.role as household_role
FROM users u
JOIN household_members hm ON u.id = hm.user_id
ORDER BY u.id;

-- Check item counts match
SELECT 
  (SELECT COUNT(DISTINCT item_name) FROM grocery_list) as old_unique_items,
  (SELECT COUNT(*) FROM items) as new_items;

-- Check list counts
SELECT 
  (SELECT COUNT(*) FROM grocery_list) as old_lists,
  (SELECT COUNT(*) FROM household_lists) as new_lists;

-- Check classification counts
SELECT 
  (SELECT COUNT(*) FROM item_classification) as old_classifications,
  (SELECT COUNT(*) FROM household_item_classifications) as new_classifications;

-- Check history counts
SELECT 
  (SELECT COUNT(*) FROM grocery_history) as old_history,
  (SELECT COUNT(*) FROM household_list_history) as new_history;

-- Verify no data loss - check if all old items have corresponding new records
SELECT gl.item_name
FROM grocery_list gl
LEFT JOIN items i ON LOWER(i.name) = LOWER(TRIM(gl.item_name))
LEFT JOIN household_lists hl ON hl.item_id = i.id
WHERE hl.id IS NULL;
-- Should return 0 rows

-- Check invite code
SELECT name, invite_code FROM households;
```

### 4. Test Application

- [ ] Users can log in
- [ ] Can view "Main Household" list
- [ ] Can add items
- [ ] Can mark items as bought
- [ ] History shows correctly
- [ ] Classifications preserved
- [ ] Images display correctly

## Post-Migration Cleanup

**Only after verifying everything works correctly:**

```sql
-- Drop old tables (CAREFUL - THIS IS IRREVERSIBLE)
DROP TABLE IF EXISTS grocery_history CASCADE;
DROP TABLE IF EXISTS item_classification CASCADE;
DROP TABLE IF EXISTS grocery_list CASCADE;
```

## Rollback Plan

### If Migration Fails

```sql
-- Inside psql during migration
ROLLBACK;

-- Then restore from backup
\q
psql -U your_user -d grocery_list < backup_YYYYMMDD_HHMMSS.sql
```

### If Issues Found After Migration

```bash
# Drop the database and restore
dropdb grocery_list
createdb grocery_list
psql -U your_user -d grocery_list < backup_YYYYMMDD_HHMMSS.sql
```

## Common Issues & Solutions

### Issue: Duplicate items in items table
**Cause**: Case-insensitive matching not working
**Solution**: Check item names for leading/trailing spaces

### Issue: Foreign key constraint errors
**Cause**: User or item references not found
**Solution**: Verify all users and items exist before migrating lists

### Issue: History not showing
**Cause**: household_list_id references incorrect
**Solution**: Check JOIN conditions in history migration

### Issue: Images not displaying
**Cause**: BYTEA encoding issues
**Solution**: Verify image_mime_type correctly migrated

## Migration Timeline

- **T-0**: Begin maintenance window
- **T+2min**: Backup complete
- **T+3min**: Start migration script
- **T+8min**: Migration complete (for ~1000 items)
- **T+10min**: Run verification queries
- **T+15min**: Test application functionality
- **T+20min**: If successful, announce completion
- **T+30min**: End maintenance window

## Data Integrity Checks

```sql
-- Ensure all users belong to at least one household
SELECT u.id, u.username
FROM users u
LEFT JOIN household_members hm ON u.id = hm.user_id
WHERE hm.id IS NULL;
-- Should return 0 rows

-- Ensure all household lists have valid items
SELECT hl.id
FROM household_lists hl
LEFT JOIN items i ON hl.item_id = i.id
WHERE i.id IS NULL;
-- Should return 0 rows

-- Ensure all history has valid list references
SELECT hlh.id
FROM household_list_history hlh
LEFT JOIN household_lists hl ON hlh.household_list_id = hl.id
WHERE hl.id IS NULL;
-- Should return 0 rows

-- Check for orphaned classifications
SELECT hic.id
FROM household_item_classifications hic
LEFT JOIN household_lists hl ON hic.item_id = hl.item_id 
  AND hic.household_id = hl.household_id 
  AND hic.store_id = hl.store_id
WHERE hl.id IS NULL;
-- Should return 0 rows (or classifications for removed items, which is ok)
```

## Success Criteria

✅ All tables created successfully
✅ All users migrated to "Main Household"
✅ Item count matches (unique items from old → new)
✅ List count matches (all grocery_list items → household_lists)
✅ Classification count matches
✅ History count matches
✅ No NULL foreign keys
✅ Application loads without errors
✅ Users can perform all CRUD operations
✅ Images display correctly
✅ Bought items still marked as bought
✅ Recently bought still shows correctly

## Next Steps After Migration

1. ✅ Update backend models (Sprint 2)
2. ✅ Update API routes
3. ✅ Update controllers
4. ✅ Test all endpoints
5. ✅ Update frontend contexts
6. ✅ Update UI components
7. ✅ Enable multi-household features

## Support & Troubleshooting

If issues arise:
1. Check PostgreSQL logs: `/var/log/postgresql/`
2. Check application logs
3. Restore from backup if needed
4. Review migration script for errors

## Monitoring Post-Migration

For the first 24 hours after migration:
- Monitor error logs
- Watch for performance issues
- Verify user activity normal
- Check for any data inconsistencies
- Be ready to rollback if critical issues found