System Design · Design Question
Design Google Drive
Cloud file storage: upload/download, metadata, sync, sharing, and dedupe — without mixing file bytes into the metadata DB.
1. Requirements
- Upload/download files; folder hierarchy; rename/move.
- Sync across devices; conflict handling when two devices edit offline.
- Share with users/groups; permissions (viewer/editor).
- Version history; large files via chunked upload.
Ask: max file size, collaboration editing (Docs) in scope?
Skip: full Docs OT/CRDT unless interviewer asks
2. API
POST /v1/files/upload-session { name, parentId, size, checksum }
PUT /upload/{session}/chunks/{n}
POST /v1/files/upload-session/{id}/complete
GET /v1/files/{id}/metadata
GET /v1/files/{id}/download → redirect to signed URL
POST /v1/files/{id}/share { principal, role }
GET /v1/sync?cursor= → changed metadata since cursor
3. Data model
files(file_id, owner_id, parent_id, name, is_dir, current_version)
file_versions(file_id, version, size, content_hash, blob_id, created_at)
blobs(blob_id, storage_path, size, content_hash) -- content addressed
permissions(file_id, principal_id, role)
sync_log(user_id, change_id, file_id, op, ts)
Content-addressed blobs enable dedupe: same hash → reuse storage. Metadata updates are frequent and small; blobs are immutable.
4. High-level design
Client
|-- metadata --> Drive API → Metadata DB (+ cache)
|-- bytes --> Upload Service → Object Store (S3)
|-- download --> signed URL via CDN / object store
Sync Service reads sync_log / change feed per user
Notification: push "file changed" to devices
5. Upload deep dive
- Client requests session; server checks quota.
- Chunked upload; each chunk checksummed; resumable.
- Complete: assemble or compose object; compute full hash.
- If hash exists → dedupe pointer; else store new blob.
- Commit new file_version + metadata TX; append sync_log.
6. Sync and conflicts
Device A offline edits v3 → uploads as v4 based on v3
Device B already created v4' from v3
→ conflict: keep both (file " (conflict)") or last-writer-wins per product policy
For binary files, creating a conflict copy is safer than silent merge. Collaborative docs need OT/CRDT — call that a different subsystem.
7. Sharing and permissions
- ACL on file/folder; inherit from parent with overrides.
- Check permission on metadata read and on signed URL minting.
- Link sharing: secret token capability URLs.
8. Scaling
- Shard metadata by owner_id / file_id.
- Object store + CDN for hot downloads.
- Block-level sync (rsync-like) for huge files optional optimization.
- Quota service to prevent abuse.
9. Reliability
- Never mark upload complete until blob durable.
- GC unreferenced blobs asynchronously after version delete.
- Multi-AZ object store replication.
10. Client sync protocol sketch
1) Client sends cursor C
2) Server returns changes (C, C'] + new cursor
3) Client applies metadata ops; downloads new blobs as needed
4) On local edit: push chunks → complete → advance local version
Conflict if server version != base_version client edited
Delta sync reduces bandwidth: only changed blocks for large files when the client speaks a block protocol (optional advanced topic).
11. Preview and search (optional)
- Async workers generate previews (PDF page images, video posters).
- Filename/full-text search via separate index updated from metadata events.
Quick revision
- Split metadata DB from content-addressed blob store.
- Chunked resumable uploads; dedupe by hash.
- Versions immutable; current pointer in metadata.
- Sync via change feed/cursor per user.
- Conflicts: keep both copies for binaries.
- Permissions checked before minting download URLs.
- CDN/signed URLs for download path.