Cloudinary, S3, R2, and S2 migrations

​

Goal

​

Prerequisites

• A verified Cloudinary, S3-compatible, or native S2 source connection
• A destination Assets bucket and its UUID
• An assets:process token to start, cancel, or retry and assets:read to inspect or download reports

​

Workflow

​

What discovery preserves

• Cloudinary: paginated image, video, and raw upload resources; the public ID becomes the folder/key, the format becomes the extension when needed, and tags, context, structured metadata, resource type, source URL, and source identity remain in migration evidence.
• Cloudinary imports originals only. StackShift regenerates transforms, thumbnails, video outputs, and other derivatives under the destination bucket policy.
• S3-compatible and S2: recursive objects beneath source_prefix; relative keys preserve folder structure, and content type, cache control, object metadata, object tags, size, provider checksum when available, and source identity are recorded.
• Discovery is paginated and idempotent by (migration, source_identity), so worker restarts do not duplicate inventory rows.

​

Create Cloudinary and S2 source connections

const cloudinary = await stackshift.assets.connections.create({
  provider: 'cloudinary',
  name: 'Legacy Cloudinary library',
  configuration: {
    cloud_name: 'acme',
    api_key: process.env.CLOUDINARY_API_KEY!,
  },
  credentials: {
    api_secret: process.env.CLOUDINARY_API_SECRET!,
  },
})
await stackshift.assets.connections.verify(cloudinary.id)

const s2Source = await stackshift.assets.connections.create({
  provider: 'stackshift_s2',
  name: 'Existing S2 library',
  s2_bucket_id: ownedS2Bucket.id,
})
await stackshift.assets.connections.verify(s2Source.id)

​

Dry-run semantics

• duplicate_checksum: a known SHA-256 already maps to an asset in the same Assets space.
• key_conflict: the destination bucket already contains the proposed logical key.
• unsupported_object: the known size or MIME type violates destination bucket policy.
• pending_execution: checksum evidence requires streaming the object during the real run.
• inventory_lookup_failed: StackShift could not safely classify an item and records an error instead of guessing.

​

Start and follow a migration

const dryRun = await stackshift.assets.migrations.start({
  kind: 'import',
  source_connection_id: source.id,
  destination_bucket_id: destination.id,
  source_prefix: 'legacy/library',
  conflict_policy: 'skip',
  dry_run: true,
})

let current = await stackshift.assets.migrations.get(dryRun.id)
while (current.status === 'queued' || current.status === 'running') {
  await new Promise((resolve) => setTimeout(resolve, 2000))
  current = await stackshift.assets.migrations.get(dryRun.id)
}
const report = await stackshift.assets.migrations.downloadReport(current.id, 'json')

stackshift asset migration-start --wait --data '{
  "kind":"import",
  "source_connection_id":"SOURCE_CONNECTION_UUID",
  "destination_bucket_id":"ASSETS_BUCKET_UUID",
  "source_prefix":"legacy/library",
  "conflict_policy":"skip",
  "dry_run":true
}' --output json

​

Checksum deduplication and key conflicts

• Execution streams the source through bounded scratch space and calculates SHA-256 before asset commit.
• Deduplication is Assets-space-wide. Duplicate content is skipped, mapped_asset_id points to the existing asset, and no alias asset is created.
• skip is the default conflict policy and preserves the existing logical asset.
• new_version replaces the compatible existing logical asset through revision-aware versioning. Exact normalized MIME compatibility is required.
• rename appends the first eight hex characters of the source-identity SHA-256 before the extension, then a stable numeric suffix if that name is also occupied.

​

Progress, retries, cancellation, and reports

• Migration states are queued, running, completed, failed, and canceled; phases begin at inventory, continue through transfer, and finish at complete.
• Persisted counters include total/processed/imported/skipped/failed items and total/processed bytes. They remain available independently of the worker process.
• Each item stores attempt count, maximum attempts, checksum state, mapped asset, warning/error code and message, timestamps, and source/destination identity. Automatic transfer retries use bounded backoff.
• Cancellation is cooperative: active work stops, unclaimed items become canceled, incomplete multipart work is aborted, and scratch data is removed. Start a new migration to continue a canceled migration.
• JSON contains the complete migration and item objects. CSV columns are source_identity, source_key, destination_key, status, checksum_sha256, mapped_asset_id, warning_code, error_code, and error_message.

stackshift asset migration-report MIGRATION_UUID --query format=csv --download ./migration.csv
stackshift asset migration-item-retry MIGRATION_UUID ITEM_UUID --output json
stackshift asset migration-cancel MIGRATION_UUID --yes --output json

​

Expected result

​

Common failures

​

Related guides