You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on May 4, 2019. It is now read-only.
We had a fairly simple task of adding a couple of columns to a table for our
11
-
Rails app. This is normally a straight forward operation and a boring task at
12
-
best but for us, the fun only just started. The table in question was a fairly
13
-
large table with lots of reads on it and in the spirit of no down time, this
14
-
is the adventure we had.
10
+
We had a fairly simple task of adding a couple of columns to a table for our Rails app. This is normally a straight forward operation and a boring task at best, but for us the fun only just started. The table in question was a fairly large table with lots of reads on it and in the spirit of no down time, this is the adventure we had.
15
11
16
12
## TL:DR;
17
13
@@ -40,7 +36,7 @@ end
40
36
### Reason
41
37
- Column creation with default values causes all rows to be touched at the same time
42
38
- Updates are a slow operation in Postgres since it has to guarantee consistency
43
-
- That guarantee results in whole table locking
39
+
- That guarantee results in locking the entire table
44
40
45
41
### Solution
46
42
- Postgres can create null columns extremely fast! Even on a huge table!
@@ -83,15 +79,15 @@ end
83
79
84
80
### Reason
85
81
- Rails migration tasks are always wrapped in a transaction to allow for rollbacks
86
-
-The column adds AND the row updates are in one gigantic transaction!
82
+
-Adding the new column AND updating the rows are in one gigantic transaction!
87
83
- Transactions guarantee consistency
88
-
- That guarantee results in whole table locking again!
84
+
- That guarantee results in the whole table locking again!
89
85
90
86
### Solution
91
-
- You can disable the transaction handle in Rails migration by calling “disable_ddl_transaction!” in your migration task
87
+
- You can disable the transaction handle in Rails migrations by calling “disable_ddl_transaction!” in your migration task
92
88
- But you have to handle transactions on your own
93
89
- We can then run each step in its own transaction
94
-
-Add our own error handling to rollback operation
90
+
-And add our own error handling to the rollback operation
95
91
96
92
## Attempt #3
97
93
@@ -148,10 +144,9 @@ end
148
144
- Leverage Postgres features
149
145
150
146
### Possible alternate solution
151
-
- Handle NULL case in code to treat as the desired default value
152
-
- Clean solution and quick turn around but required us to muck up the model to abstract out that case. Give that we may or may not have complete control over how that those values are extracted from the model, this may turn into lots of defensive code.
153
-
- Add view in database to do mapping for us
154
-
- Very clean solution though this would require us to maintain both the schema and the view whenever we do schema changes on to that table. Though we don't do changes on the schema often on this table, the extra maintance overhead was deemed not worth the value.
155
-
- Add trigger to only update rows that are actively queried
156
-
- Also very clean solution though it came down to data integrity and since our data eventually gets slurped up by our data team, having a sane state on our data was highest priority. This meant that having a NULL state on a Boolean was not desired. Ultimately, we could of added the trigger to handle any current requests and just made the migration run slowly to backfill lesser accessed rows. Since we were able to run the entire migration within a night, we decided it wasn't worth the additional hassle.
157
-
147
+
- Handle NULL case in code to treat as the desired default value:
148
+
- Clean solution and quick turn around, but it required us to muck up the model to abstract out that case. Given that we may or may not have complete control over how those values are extracted from the model, this may turn into lots of defensive code.
149
+
- Add view in database to do mapping for us:
150
+
- Very clean solution though this would require us to maintain both the schema and the view whenever we do schema changes to that table. Though we don't do changes to the schema often in this table, the extra maintance overhead was deemed not worth the value.
151
+
- Add trigger to only update rows that are actively queried:
152
+
- Also very clean solution though it came down to data integrity and since our data eventually gets consumed by our data team, having a sane state on our data was highest priority. This meant that having a NULL state on a Boolean was not desired. Ultimately, we could have added the trigger to handle any current requests and just made the migration run slowly to backfill less frequently accessed rows. Since we were able to run the entire migration within a night, we decided it wasn't worth the additional hassle.
0 commit comments