Test Whether Mobile Backup Restores Data Without Restoring Its Key

A user restores an application backup onto a new phone. The encrypted database returns, but its key does not. The app opens to an endless loading state because storage recovery and key recovery were designed as one event even though the operating system treats them differently.

The opposite failure is worse: data intended to stay device-bound becomes readable after restore because both ciphertext and key entered backup.

Do not infer this behavior from API names such as “secure storage.” Test the exact device, OS, backup mechanism, key attributes, and lifecycle transition.

Declare the security contract

Choose one policy per data class:

“Encrypted” does not answer whether data should migrate. Backup eligibility and key accessibility are separate controls.

Build a test envelope

device_source: physical-device-model
os_source: exact-version
device_target: physical-device-model
os_target: exact-version
app_version_before: 4.2.0
app_version_after: 4.2.0
backup_path: encrypted-computer-or-cloud
key_policy: device-bound-or-migratable
screen_lock: enabled
network_after_restore: offline

Record framework and secure-storage dependency versions if a wrapper is used. Wrappers can change defaults between releases.

Create three canaries:

1. backup-allowed ciphertext containing a recognizable random marker;
2. backup-excluded ciphertext with a different marker;
3. device-bound key used to decrypt the first marker.

Use random test data, not personal data or production credentials.

Lifecycle matrix

On iOS, verify the selected Keychain accessibility and migration class rather than assuming all Keychain items behave alike. On Android, verify backup inclusion rules and Keystore behavior on the tested OS/device; hardware-backed availability and restoration behavior vary. Platform documentation defines intended semantics, but device tests verify the product flow.

Make missing-key recovery explicit

Pseudo-flow:

open database
  -> key found: decrypt and migrate
  -> key missing, ciphertext restored:
       classify data
       recover with user-held/account-wrapped key OR
       delete cache and rebuild OR
       preserve file and show support/recovery path
  -> authentication failure:
       stop; never overwrite ciphertext automatically

Distinguish key not found from authentication failed. The second can indicate the wrong key, corruption, or a migration bug. Automatically creating a new key and overwriting the database destroys evidence and may destroy recoverable data.

Evidence to collect

For each transition, record:

• whether each canary file returned;
• whether each key reference returned;
• decrypt success/failure category;
• first-launch UI state;
• recovery steps and final state;
• logs stripped of keys, tokens, and plaintext;
• elapsed recovery time, if actually measured.

Expected results should be labeled as expectations until executed on the declared devices. Simulator results do not establish hardware-backed key behavior.

Release gate

Block release when a supported restore path produces an unrecoverable blank screen, silently resets valuable data, restores a credential meant to remain device-bound, or lacks a user-understandable recovery action. A passed happy-path launch is insufficient.

Limits

This plan does not prescribe one universal migration policy. Products with end-to-end encryption, regulated retention, managed devices, or user-held recovery keys have different constraints. It also does not prove resistance to a compromised OS.

The useful comparison requires exact evidence: source device, target device, OS versions, backup path, key policy, and whether the app recovered, re-enrolled, or silently lost access.