Skip to content
This repository was archived by the owner on May 4, 2019. It is now read-only.

Commit 7e241d9

Browse files
author
Sai Wong
committed
Made post more SEO friendly
1 parent 0fa8b9f commit 7e241d9

File tree

1 file changed

+15
-15
lines changed

1 file changed

+15
-15
lines changed

_posts/2015-10-16-add-columns-with-default-values-to-large-tables-in-rails-postgres.markdown renamed to _posts/2015-11-05-add-columns-with-default-values-to-large-tables-in-rails-postgres.markdown

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
layout: post
3-
title: Rails Migration - How to add columns with default values to really large tables in Postgres + Rails
3+
title: Adding columns with default values to really large tables in Postgres + Rails
44
author: Sai Wong
55
summary:
66
image: http://res.cloudinary.com/wework/image/upload/s--GnhXQxhq--/c_scale,q_jpegmini:1,w_1000/v1445269362/engineering/shutterstock_262325693.jpg
@@ -13,16 +13,16 @@ best but for us, the fun only just started. The table in question was a fairly
1313
large table with lots of reads on it and in the spirit of no down time, this
1414
is the adventure we had.
1515

16-
# TL:DR;
16+
## TL:DR;
1717

1818
Jump straight to the [solution](#attempt-3)!
1919

20-
# The Task
20+
## The Task
2121
- Add two columns to the notifications table
2222
- Both columns have default values
2323
- Table has 2.2 MM rows!
2424

25-
# Attempt #1
25+
## Attempt #1
2626
```ruby
2727
class AddPhoneFlagsToNotifications < ActiveRecord::Migration
2828
def change
@@ -32,21 +32,21 @@ class AddPhoneFlagsToNotifications < ActiveRecord::Migration
3232
end
3333
```
3434

35-
## Problem
35+
### Problem
3636
- Migration takes hours!
3737
- The notifications table is locked
3838
- Entire application grinds to a halt
3939

40-
## Reason
40+
### Reason
4141
- Column creation with default values causes all rows to be touched at the same time
4242
- Updates are a slow operation in Postgres since it has to guarantee consistency
4343
- That guarantee results in whole table locking
4444

45-
## Solution
45+
### Solution
4646
- Postgres can create null columns extremely fast! Even on a huge table!
4747
- We can split the work to two tasks, creating the columns and populating the default value
4848

49-
# Attempt #2
49+
## Attempt #2
5050

5151
```ruby
5252
class AddPhoneFlagsToNotifications < ActiveRecord::Migration
@@ -76,24 +76,24 @@ class AddPhoneFlagsToNotifications < ActiveRecord::Migration
7676
end
7777
```
7878

79-
## Problem
79+
### Problem
8080
- Migration takes hours!
8181
- The notifications table is still locked!
8282
- Entire application grinds to a halt
8383

84-
## Reason
84+
### Reason
8585
- Rails migration tasks are always wrapped in a transaction to allow for rollbacks
8686
- The column adds AND the row updates are in one gigantic transaction!
8787
- Transactions guarantee consistency
8888
- That guarantee results in whole table locking again!
8989

90-
## Solution
90+
### Solution
9191
- You can disable the transaction handle in Rails migration by calling “disable_ddl_transaction!” in your migration task
9292
- But you have to handle transactions on your own
9393
- We can then run each step in its own transaction
9494
- Add our own error handling to rollback operation
9595

96-
# Attempt #3
96+
## Attempt #3
9797

9898
```ruby
9999
class AddPhoneFlagsToNotifications < ActiveRecord::Migration
@@ -136,18 +136,18 @@ class AddPhoneFlagsToNotifications < ActiveRecord::Migration
136136
end
137137
```
138138

139-
## Result
139+
### Result
140140
- Migration takes hours!
141141
- There is no table locking
142142
- Application is slower due to all the writes to notifications table
143143
- Nothing grinds to a halt
144144

145-
# Takeaways
145+
## Takeaways
146146
- Always be mindful of the number of rows affected in the migration
147147
- Be mindful of the transaction size
148148
- Leverage Postgres features
149149

150-
## Possible alternate solution
150+
### Possible alternate solution
151151
- Handle NULL case in code to treat as the desired default value
152152
- Clean solution and quick turn around but required us to muck up the model to abstract out that case. Give that we may or may not have complete control over how that those values are extracted from the model, this may turn into lots of defensive code.
153153
- Add view in database to do mapping for us

0 commit comments

Comments
 (0)