# Coaster World DATA — Preproduction and operations runbook

This runbook applies to `v0.4.0-dev.18` and later. It is designed to prove that a clean DATA server can be rebuilt and operated before any population lock is opened.

## Safety baseline

Keep all six construction locks closed during every preproduction rehearsal:

```env
DATA_POPULATION_ENABLED=false
DATA_PROVIDER_SCHEDULES_ENABLED=false
DATA_PROVIDER_COLLECTION_ENABLED=false
DATA_CANONICAL_AUTO_APPLY_ENABLED=false
DATA_AI_AGENTS_ENABLED=false
DATA_RESEARCH_EXECUTION_ENABLED=false
```

Never use `migrate:fresh` against the active DATA database. The dedicated install drill creates its own temporary target.

## 1. Clean deployment / rebuild from source

On a new host:

```text
1. Copy the application source.
2. Install the PHP/Composer dependencies required by composer.lock.
3. Copy .env.example to .env and fill only environment-specific values.
4. Generate APP_KEY.
5. Configure the MySQL/MariaDB database and a dedicated application account.
6. Keep every DATA construction lock false.
7. Run migrations and stable reference seeders.
8. Clear/cache Laravel configuration as appropriate for the host.
9. Start scheduler and queue worker processes.
10. Run integrity, resilience, snapshot/replica/restore and preproduction checks.
```

Core commands:

```bash
php artisan migrate --force
php artisan db:seed --force
php artisan optimize:clear
php artisan cw:data:integrity --deep
php artisan cw:data:resilience
php artisan cw:data:install:drill
php artisan cw:data:snapshot --label=preproduction
php artisan cw:data:snapshot:replicate latest
php artisan cw:data:restore:drill latest
php artisan cw:data:preflight
```

The isolated fresh-install rehearsal can be repeated at any time:

```bash
php artisan cw:data:install:drill
```

It creates a generated empty MySQL/MariaDB database (or private SQLite file), runs every migration, executes the stable reference seeders twice to prove idempotency, checks required platform tables/reference minima, then removes the temporary target. The active DATA database is never selected as the drill target.

## 2. Production-only preflight

After production `.env`, HTTPS, queue services and automated backup schedules are configured:

```bash
php artisan cw:data:preflight --production --strict
```

Production preflight requires at least:

- `APP_ENV=production`;
- `APP_DEBUG=false`;
- HTTPS `APP_URL`;
- no localhost / `.test` CORS origin;
- MySQL or MariaDB;
- database queue driver;
- fresh scheduler + queue-worker heartbeats;
- zero pending migrations;
- healthy integrity;
- verified snapshot and external replica;
- verified isolated restore drill;
- verified resilience suite;
- verified fresh-install drill;
- internal API token of at least 48 characters;
- snapshot, replica, integrity and health schedules enabled.

A production preflight report is stored privately under `storage/app/private` and never contains database passwords or API token values.

## 3. Runtime processes

For Laragon development:

```bat
tools\run_data_runtime_windows.bat
```

For production, run the scheduler and worker with the host's process/service manager. The functional equivalents are:

```bash
php artisan schedule:work
php artisan queue:work database --queue=maintenance,provider,ai-research,default --sleep=2 --tries=1 --timeout=180
```

After a deployment:

```bash
php artisan queue:restart
php artisan cw:runtime:status
php artisan cw:data:health
```

## 4. Crash / interrupted-worker recovery

Always inspect first:

```bash
php artisan cw:runtime:recover --dry-run
php artisan cw:runtime:status
```

Apply only after reviewing the dry-run:

```bash
php artisan cw:runtime:recover
php artisan queue:restart
php artisan cw:runtime:status
```

Windows helper:

```bat
tools\recover_data_runtime_windows.bat
tools\recover_data_runtime_windows.bat apply
```

Recovery closes only stale ingestion / AI executions according to the configured heartbeat timeout. It does not open DATA locks or force canonical writes.

## 5. Internal API token rotation without downtime

Generate a new token locally:

```bash
php artisan cw:api:token
```

Do not paste the token into tickets, chat screenshots or source control.

For a grace-period rotation:

1. Keep the old current token temporarily as `DATA_INTERNAL_API_PREVIOUS_TOKEN`.
2. Set the newly generated token as `DATA_INTERNAL_API_TOKEN`.
3. Run `php artisan optimize:clear` on DATA.
4. Update every authorized client to the new token.
5. Verify `/api/v1/internal/status` from each client.
6. Remove `DATA_INTERNAL_API_PREVIOUS_TOKEN`.
7. Run `php artisan optimize:clear` again.
8. Run `php artisan cw:data:preflight --production --strict`.

The previous token is accepted only while explicitly configured. Preflight reports a warning while the grace token remains active so it is not forgotten.

## 6. Database backup and restore validation

Before production population:

```bash
php artisan cw:data:snapshot --label=preproduction
php artisan cw:data:snapshot:verify latest
php artisan cw:data:snapshot:replicate latest
php artisan cw:data:restore:check latest
php artisan cw:data:restore:drill latest
```

The replica destination must be genuinely outside the primary host for production continuity (NAS, another server or equivalent protected storage).

## 7. Final deployment gate

Before declaring a host ready:

```bash
php artisan cw:data:install:drill
php artisan cw:data:resilience
php artisan test
php artisan cw:data:preflight --production --strict
```

Do not open any population/provider/AI/canonical lock as part of deployment. Lock opening belongs to the later **Ready for DATA** milestone and must be deliberate and progressive.

## Final Ready-for-DATA certification

After the fresh-install drill, resilience suite, restore drill and construction preflight are validated, run the final isolated end-to-end certification:

```bat
php artisan cw:data:ready:audit
```

Do not continue to a real provider pilot unless the command reports both `Result: VERIFIED` and `Pilot gate: READY`.

The first real pilot starts with provider staging only. Follow `docs/READY_FOR_DATA.md`; do not enable schedules, canonical writes or autonomous AI at the same time.