costco-grocery-list/backend/migrations/MIGRATION_GUIDE.md

Multi-Household Architecture Migration Guide

Pre-Migration Checklist

• Backup Database

  pg_dump -U your_user -d grocery_list > backup_$(date +%Y%m%d_%H%M%S).sql
  

• Test on Staging First

  • Copy production database to staging environment
  • Run migration on staging
  • Verify all data migrated correctly
  • Test application functionality

• Review Migration Script

  • Read through multi_household_architecture.sql
  • Understand each step
  • Note verification queries

• Announce Maintenance Window

  • Notify users of downtime
  • Schedule during low-usage period
  • Estimate 15-30 minutes for migration

Running the Migration

1. Connect to Database

psql -U your_user -d grocery_list

2. Run Migration

\i backend/migrations/multi_household_architecture.sql

The script will:

1. ✅ Create 8 new tables
2. ✅ Create default "Main Household"
3. ✅ Create default "Costco" store
4. ✅ Migrate all users to household members
5. ✅ Extract items to master catalog
6. ✅ Migrate grocery_list → household_lists
7. ✅ Migrate classifications
8. ✅ Migrate history records
9. ✅ Update user system roles

3. Verify Migration

Run these queries inside psql:

-- Check household created
SELECT * FROM households;

-- Check all users migrated
SELECT u.username, u.role as system_role, hm.role as household_role
FROM users u
JOIN household_members hm ON u.id = hm.user_id
ORDER BY u.id;

-- Check item counts match
SELECT 
  (SELECT COUNT(DISTINCT item_name) FROM grocery_list) as old_unique_items,
  (SELECT COUNT(*) FROM items) as new_items;

-- Check list counts
SELECT 
  (SELECT COUNT(*) FROM grocery_list) as old_lists,
  (SELECT COUNT(*) FROM household_lists) as new_lists;

-- Check classification counts
SELECT 
  (SELECT COUNT(*) FROM item_classification) as old_classifications,
  (SELECT COUNT(*) FROM household_item_classifications) as new_classifications;

-- Check history counts
SELECT 
  (SELECT COUNT(*) FROM grocery_history) as old_history,
  (SELECT COUNT(*) FROM household_list_history) as new_history;

-- Verify no data loss - check if all old items have corresponding new records
SELECT gl.item_name
FROM grocery_list gl
LEFT JOIN items i ON LOWER(i.name) = LOWER(TRIM(gl.item_name))
LEFT JOIN household_lists hl ON hl.item_id = i.id
WHERE hl.id IS NULL;
-- Should return 0 rows

-- Check invite code
SELECT name, invite_code FROM households;

4. Test Application

• Users can log in
• Can view "Main Household" list
• Can add items
• Can mark items as bought
• History shows correctly
• Classifications preserved
• Images display correctly

Post-Migration Cleanup

Only after verifying everything works correctly:

-- Drop old tables (CAREFUL - THIS IS IRREVERSIBLE)
DROP TABLE IF EXISTS grocery_history CASCADE;
DROP TABLE IF EXISTS item_classification CASCADE;
DROP TABLE IF EXISTS grocery_list CASCADE;

Rollback Plan

If Migration Fails

-- Inside psql during migration
ROLLBACK;

-- Then restore from backup
\q
psql -U your_user -d grocery_list < backup_YYYYMMDD_HHMMSS.sql

If Issues Found After Migration

# Drop the database and restore
dropdb grocery_list
createdb grocery_list
psql -U your_user -d grocery_list < backup_YYYYMMDD_HHMMSS.sql

Common Issues & Solutions

Issue: Duplicate items in items table

Cause: Case-insensitive matching not working Solution: Check item names for leading/trailing spaces

Issue: Foreign key constraint errors

Cause: User or item references not found Solution: Verify all users and items exist before migrating lists

Issue: History not showing

Cause: household_list_id references incorrect Solution: Check JOIN conditions in history migration

Issue: Images not displaying

Cause: BYTEA encoding issues Solution: Verify image_mime_type correctly migrated

Migration Timeline

• T-0: Begin maintenance window
• T+2min: Backup complete
• T+3min: Start migration script
• T+8min: Migration complete (for ~1000 items)
• T+10min: Run verification queries
• T+15min: Test application functionality
• T+20min: If successful, announce completion
• T+30min: End maintenance window

Data Integrity Checks

-- Ensure all users belong to at least one household
SELECT u.id, u.username
FROM users u
LEFT JOIN household_members hm ON u.id = hm.user_id
WHERE hm.id IS NULL;
-- Should return 0 rows

-- Ensure all household lists have valid items
SELECT hl.id
FROM household_lists hl
LEFT JOIN items i ON hl.item_id = i.id
WHERE i.id IS NULL;
-- Should return 0 rows

-- Ensure all history has valid list references
SELECT hlh.id
FROM household_list_history hlh
LEFT JOIN household_lists hl ON hlh.household_list_id = hl.id
WHERE hl.id IS NULL;
-- Should return 0 rows

-- Check for orphaned classifications
SELECT hic.id
FROM household_item_classifications hic
LEFT JOIN household_lists hl ON hic.item_id = hl.item_id 
  AND hic.household_id = hl.household_id 
  AND hic.store_id = hl.store_id
WHERE hl.id IS NULL;
-- Should return 0 rows (or classifications for removed items, which is ok)

Success Criteria

✅ All tables created successfully ✅ All users migrated to "Main Household" ✅ Item count matches (unique items from old → new) ✅ List count matches (all grocery_list items → household_lists) ✅ Classification count matches ✅ History count matches ✅ No NULL foreign keys ✅ Application loads without errors ✅ Users can perform all CRUD operations ✅ Images display correctly ✅ Bought items still marked as bought ✅ Recently bought still shows correctly

Next Steps After Migration

1. ✅ Update backend models (Sprint 2)
2. ✅ Update API routes
3. ✅ Update controllers
4. ✅ Test all endpoints
5. ✅ Update frontend contexts
6. ✅ Update UI components
7. ✅ Enable multi-household features

Support & Troubleshooting

If issues arise:

1. Check PostgreSQL logs: /var/log/postgresql/
2. Check application logs
3. Restore from backup if needed
4. Review migration script for errors

Monitoring Post-Migration

For the first 24 hours after migration:

• Monitor error logs
• Watch for performance issues
• Verify user activity normal
• Check for any data inconsistencies
• Be ready to rollback if critical issues found