Skip to content
This repository was archived by the owner on May 4, 2019. It is now read-only.

Commit ad846f2

Browse files
author
mattjstar
committed
quick typo check
1 parent 7e241d9 commit ad846f2

File tree

1 file changed

+12
-17
lines changed

1 file changed

+12
-17
lines changed

_posts/2015-11-05-add-columns-with-default-values-to-large-tables-in-rails-postgres.markdown

Lines changed: 12 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,7 @@ image: http://res.cloudinary.com/wework/image/upload/s--GnhXQxhq--/c_scal
77
categories: data
88
---
99

10-
We had a fairly simple task of adding a couple of columns to a table for our
11-
Rails app. This is normally a straight forward operation and a boring task at
12-
best but for us, the fun only just started. The table in question was a fairly
13-
large table with lots of reads on it and in the spirit of no down time, this
14-
is the adventure we had.
10+
We had a fairly simple task of adding a couple of columns to a table for our Rails app. This is normally a straight forward operation and a boring task at best, but for us the fun only just started. The table in question was a fairly large table with lots of reads on it and in the spirit of no down time, this is the adventure we had.
1511

1612
## TL:DR;
1713

@@ -40,7 +36,7 @@ end
4036
### Reason
4137
- Column creation with default values causes all rows to be touched at the same time
4238
- Updates are a slow operation in Postgres since it has to guarantee consistency
43-
- That guarantee results in whole table locking
39+
- That guarantee results in locking the entire table
4440

4541
### Solution
4642
- Postgres can create null columns extremely fast! Even on a huge table!
@@ -83,15 +79,15 @@ end
8379

8480
### Reason
8581
- Rails migration tasks are always wrapped in a transaction to allow for rollbacks
86-
- The column adds AND the row updates are in one gigantic transaction!
82+
- Adding the new column AND updating the rows are in one gigantic transaction!
8783
- Transactions guarantee consistency
88-
- That guarantee results in whole table locking again!
84+
- That guarantee results in the whole table locking again!
8985

9086
### Solution
91-
- You can disable the transaction handle in Rails migration by calling “disable_ddl_transaction!” in your migration task
87+
- You can disable the transaction handle in Rails migrations by calling “disable_ddl_transaction!” in your migration task
9288
- But you have to handle transactions on your own
9389
- We can then run each step in its own transaction
94-
- Add our own error handling to rollback operation
90+
- And add our own error handling to the rollback operation
9591

9692
## Attempt #3
9793

@@ -148,10 +144,9 @@ end
148144
- Leverage Postgres features
149145

150146
### Possible alternate solution
151-
- Handle NULL case in code to treat as the desired default value
152-
- Clean solution and quick turn around but required us to muck up the model to abstract out that case. Give that we may or may not have complete control over how that those values are extracted from the model, this may turn into lots of defensive code.
153-
- Add view in database to do mapping for us
154-
- Very clean solution though this would require us to maintain both the schema and the view whenever we do schema changes on to that table. Though we don't do changes on the schema often on this table, the extra maintance overhead was deemed not worth the value.
155-
- Add trigger to only update rows that are actively queried
156-
- Also very clean solution though it came down to data integrity and since our data eventually gets slurped up by our data team, having a sane state on our data was highest priority. This meant that having a NULL state on a Boolean was not desired. Ultimately, we could of added the trigger to handle any current requests and just made the migration run slowly to backfill lesser accessed rows. Since we were able to run the entire migration within a night, we decided it wasn't worth the additional hassle.
157-
147+
- Handle NULL case in code to treat as the desired default value:
148+
- Clean solution and quick turn around, but it required us to muck up the model to abstract out that case. Given that we may or may not have complete control over how those values are extracted from the model, this may turn into lots of defensive code.
149+
- Add view in database to do mapping for us:
150+
- Very clean solution though this would require us to maintain both the schema and the view whenever we do schema changes to that table. Though we don't do changes to the schema often in this table, the extra maintance overhead was deemed not worth the value.
151+
- Add trigger to only update rows that are actively queried:
152+
- Also very clean solution though it came down to data integrity and since our data eventually gets consumed by our data team, having a sane state on our data was highest priority. This meant that having a NULL state on a Boolean was not desired. Ultimately, we could have added the trigger to handle any current requests and just made the migration run slowly to backfill less frequently accessed rows. Since we were able to run the entire migration within a night, we decided it wasn't worth the additional hassle.

0 commit comments

Comments
 (0)