Updating a Single Row in a Billion-Row Parquet Table
Question
If a table stored as Parquet files in S3 has billions of rows and the business needs to update one row with ACID transactions and rollback, how would you implement it?
Short interview answer
S3 itself does not provide table transactions. I would use a table format such as Apache Iceberg or Delta Lake. Data stays in Parquet files, while versioned metadata defines table snapshots. An update writes new files and atomically commits a new snapshot, giving isolation, rollback and time travel.
Detailed answer
Assume an orders table is stored as files in S3:
s3://data/orders/
part-001.parquet ← 1,000,000 orders
part-002.parquet ← 1,000,000 orders
part-003.parquet
...
Order 42 is in part-002.parquet:
order_id=42, delivery_address="Old Street", status="shipped"
The business wants to correct the address. Parquet cannot edit that one value in place, and S3 cannot tell every reader, “these are the exact files that make up version 17 of the table.” Replacing a file directly risks readers observing a half-finished update.
Iceberg or Delta Lake adds versioned table metadata above the Parquet files. Instead of treating a table as “every .parquet file in this folder,” readers use one named table version:
table
↓
metadata or transaction log
↓
snapshot 17
↓
part-001.parquet, part-002.parquet, part-003.parquet
The engine reads the table metadata, finds part-002.parquet, and writes a replacement. It does not overwrite the original file.
Before commit — snapshot 17
part-001.parquet
part-002.parquet ← contains order 42 with "Old Street"
part-003.parquet
After commit — snapshot 18
part-001.parquet
part-002-v2.parquet ← contains order 42 with "New Street"
part-003.parquet
The atomic operation is the small metadata change from snapshot 17 to snapshot 18. A report that started on snapshot 17 keeps reading the old file. A report that starts after the commit reads the new file. Neither report sees a mixed table.
If another writer also changed part-002.parquet, the table format notices that snapshot 17 is no longer current. The second writer retries using the latest version or fails with a conflict; it does not silently erase the first update.
If the address correction was wrong, snapshot 17 is still available for time travel or can be restored as the current version. This is a good design for analytical data with occasional corrections. If the system performs constant low-latency row updates, an operational database should usually own the live record and the lakehouse should receive the analytical copy.