Skip to content

Commit 5f627a9

Browse files
bennzhangbennzhang
committed
first draft
0 parents  commit 5f627a9

14 files changed

+312
-0
lines changed

‎README.md

+206
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
# Percona XtraBackup innodb backup and restores file copy
2+
This document shows how to use `Percona XtraBackup` to backup and recover the mysql innodb tables using file copy ways for MySQL 5.6 above. You can download and install `Percona XtraBackup` from [here](https://www.percona.com/downloads/XtraBackup/LATEST/).
3+
4+
```
5+
yum install https://www.percona.com/downloads/XtraBackup/Percona-XtraBackup-2.4.9/binary/redhat/7/x86_64/percona-xtrabackup-24-2.4.9-1.el7.x86_64.rpm
6+
```
7+
8+
```
9+
$ xtrabackup --version
10+
xtrabackup version 2.4.9 based on MySQL server 5.7.13 Linux (x86_64) (revision id: a467167cdd4)
11+
```
12+
13+
## 2 Backup database
14+
15+
### 2.1 get schema table list
16+
17+
Run `get_tables.sh` below to get all tables from one schema and save into a file `${TABLE_SCHEMA}_tables.lst`. You need to pass in the schema name use `$1`. for example `./get_tables.sh test` will get all tables from schema `test`.
18+
19+
```
20+
#!/bin/sh
21+
22+
SCRIPT_DIR=/usr/local/scripts/mysql_scripts
23+
source $SCRIPT_DIR/vars.sh
24+
25+
SCHEMA_NAME=$1
26+
27+
mysql --host=${HOST_NAME} --user=${USER_NAME} --password=${PASSWD} -N -e "SELECT CONCAT(TABLE_SCHEMA,'.',TABLE_NAME) FROM information_schema.TABLES WHERE TABLE_SCHEMA = '${SCHEMA_NAME}'" > $SCHEMA_NAME\_tables.lst
28+
```
29+
30+
`-N` : ignore column header.
31+
32+
33+
### 2.2 --backup
34+
35+
All `Xtrabackup` backup options can be found [here](https://www.percona.com/doc/percona-xtrabackup/LATEST/xtrabackup_bin/xbk_option_reference.html)
36+
```
37+
sudo /usr/bin/xtrabackup --user=${USER_NAME} --password=${PASSWD} --tables-file=${TABLES_LIST_FILE} --backup --parallel=4 --target-dir=${BACKUP_DIR} --DATA_DIR=${DATA_DIR}
38+
```
39+
40+
### 2.3 --prepare --export
41+
42+
+ --prepare
43+
You need to prepare backup in order to restore it. Data files are not point-in-time consistent until they’ve been prepared, because they were copied at different times as the program ran, and they might have been changed while this was happening. If you try to start InnoDB with these data files, it will detect corruption and crash itself to prevent you from running on damaged data. The xtrabackup --prepare step makes the files perfectly consistent at a single instant in time, so you can run InnoDB on them [Reference](https://www.percona.com/doc/percona-xtrabackup/LATEST/backup_scenarios/full_backup.html#preparing-a-backup). You can run the prepare operation on any machine; it does not need to be on the originating server or the server to which you intend to restore.
44+
45+
+ --export
46+
This command makes it possible to copy table files for backups. It creates files necessary for exporting tables. See [Restoring Individual Tables](https://www.percona.com/doc/percona-xtrabackup/LATEST/xtrabackup_bin/restoring_individual_tables.html). This command is combined with `--prepare`
47+
48+
```
49+
sudo /usr/bin/xtrabackup --prepare --export --target-dir=${BACKUP_DIR}
50+
```
51+
52+
After run it, you will see some files for each table like below. These files are all you need to import the table into a server running Percona Server with XtraDB or MySQL 5.7
53+
54+
```
55+
$ find ${BACKUP_DIR}/${SCHEMA_NAME}
56+
57+
${BACKUP_DIR}/${SCHEMA_NAME}/table1.cfg
58+
${BACKUP_DIR}/${SCHEMA_NAME}/table1.exp
59+
${BACKUP_DIR}/${SCHEMA_NAME}/table1.ibd
60+
${BACKUP_DIR}/${SCHEMA_NAME}/table1.frm
61+
```
62+
63+
One script `backup.sh` is provided. This script needds pass in one parameter - SCHEMA_NAME. For example, `backup.sh test` will backup whole innodb tables of `test` database.
64+
65+
## 3 Restore databse
66+
67+
### 3.1 drop/recreate tables
68+
69+
+ Get whole DDLs
70+
71+
You can get whole DDLs from this command.
72+
```
73+
mysqldump -d -u ${USER_NAME} -p${PASSWD} ${SCHEMA_NAME} > ${SCHEMA_NAME}\_DDL.sql
74+
```
75+
76+
+ Drop and recreate tables
77+
```
78+
mysql --host=${HOST_NAME} --user=${USER_NAME} --password=${PASSWD} < ${SCHEMA_NAME}\_DDL.sql
79+
```
80+
81+
### 3.2 Discard Tablespaces
82+
83+
Use this script to generate discard tablespace sql file `./create_discard_tablespace_sql.sh SCHEMA_NAME` and save into a file `$SCHEMA_NAME\_discard_tablespace.sql`
84+
85+
```
86+
#!/bin/sh
87+
SCHEMA_NAME=$1
88+
89+
echo "USE $SCHEMA_NAME;"
90+
while read line
91+
do
92+
echo "ALTER TABLE $line DISCARD TABLESPACE;"
93+
done < $SCHEMA_NAME\_tables.lst
94+
```
95+
96+
run discard tablespace sql.
97+
98+
```
99+
mysql --host=${HOST_NAME} --user=${USER_NAME} --password=${PASSWD} < ${SCHEMA_NAME}\_discard_tablespace.sql
100+
```
101+
102+
### 3.3 Copy backed up files from BACKUP_DIR to DATA_DIR
103+
You just needs to copy `*.cfg` and `*ibd` files. `mv_back_backups.sh`
104+
105+
```
106+
#!/bin/sh
107+
SCRIPT_DIR=/usr/local/scripts/mysql_scripts
108+
source $SCRIPT_DIR/vars.sh
109+
110+
SCHEMA_NAME=$1
111+
112+
while read line
113+
do
114+
sudo cp -rpv ${BACKUP_DIR}/${SCHEMA_NAME}/*.cfg ${DATA_DIR}/${SCHEMA_NAME}
115+
sudo cp -rpv ${BACKUP_DIR}/${SCHEMA_NAME}/*.ibd ${DATA_DIR}/${SCHEMA_NAME}
116+
done < $SCHEMA_NAME\_tables.lst
117+
118+
sudo chown -R mysql:mysql ${DATA_DIR}
119+
```
120+
121+
### 3.4 Import tablesapce
122+
123+
Use this script to generate import tablespace sql file `./create_import_tablespace_sql.sh SCHEMA_NAME` and save into a file `$SCHEMA_NAME\_import_tablespace.sql`
124+
125+
```
126+
#!/bin/sh
127+
SCHEMA_NAME=$1
128+
129+
echo "USE $SCHEMA_NAME;"
130+
while read line
131+
do
132+
echo "ALTER TABLE $line IMPORT TABLESPACE;"
133+
done < $SCHEMA_NAME\_tables.lst
134+
```
135+
136+
run import discard tablespace sql.
137+
138+
```
139+
mysql --host=${HOST_NAME} --user=${USER_NAME} --password=${PASSWD} < ${SCHEMA_NAME}\_import_tablespace.sql
140+
```
141+
142+
### 3.5 Cleanup after import
143+
144+
```
145+
rm ${DATA_DIR}/${SCHEMA_NAME}/*.cfg
146+
```
147+
148+
### 3.6 Put them together
149+
```
150+
$ cat restore_backup.sh
151+
152+
#!/bin/sh
153+
154+
SCRIPT_DIR=/usr/local/scripts/mysql_scripts
155+
source $SCRIPT_DIR/vars.sh
156+
157+
SCHEMA_NAME=$1
158+
TABLES_LIST_FILE=$SCRIPT_DIR/$SCHEMA_NAME\_tables.lst
159+
160+
echo "dump original DDL ......"
161+
mysqldump -d -u ${USER_NAME} -p${PASSWD} ${SCHEMA_NAME} > ${SCHEMA_NAME}\_DDL.sql
162+
163+
echo "drop/recreate tables ......"
164+
mysql --host=${HOST_NAME} --user=${USER_NAME} --password=${PASSWD} < ${SCHEMA_NAME}\_DDL.sql
165+
166+
echo "discard tablespaces ......"
167+
echo " + create discard tablespaces sql"
168+
$SCRIPT_DIR/create_discard_tablespace_sql.sh ${SCHEMA_NAME} > $SCHEMA_NAME\_discard_tablespace.sql
169+
echo " + run discard tablespaces sql"
170+
mysql --host=${HOST_NAME} --user=${USER_NAME} --password=${PASSWD} < ${SCHEMA_NAME}\_discard_tablespace.sql
171+
172+
echo "copy backed up files from ${BACKUP_DIR} to ${DATA_DIR}
173+
$SCRIPT_DIR/mv_back_backups.sh ${SCHEMA_NAME}
174+
175+
echo "import tablespaces ......"
176+
echo " + create import tablespaces sql"
177+
$SCRIPT_DIR/create_import_tablespace_sql.sh ${SCHEMA_NAME} > $SCHEMA_NAME\_import_tablespace.sql
178+
echo " + run import tablespaces sql"
179+
mysql --host=${HOST_NAME} --user=${USER_NAME} --password=${PASSWD} < ${SCHEMA_NAME}\_import_tablespace.sql
180+
181+
echo "clean up cfg files"
182+
sudo rm ${DATA_DIR}/${SCHEMA_NAME}/*.cfg
183+
184+
echo "Restore Complete!!!!"
185+
```
186+
187+
## 4 whole picture
188+
```
189+
$ tree .
190+
.
191+
├── create_discard_tablespace_sql.sh
192+
├── create_import_tablespace_sql.sh
193+
├── dump_tables_ddl.sh
194+
├── export_backup.sh
195+
├── get_tables.sh
196+
├── mv_back_backups.sh
197+
├── restore_backup.sh
198+
├── test_DDL.sql
199+
├── test_discard_tablespace.sql
200+
├── test_import_tablespace.sql
201+
├── test_tables.lst
202+
└── vars.sh
203+
```
204+
205+
`vars.sh` is your configuration files. `export_backup.sh` is main script to backup and `restore_backup.sh` is main script to restore.
206+

‎create_discard_tablespace_sql.sh

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
#!/bin/sh
2+
SCHEMA_NAME=$1
3+
4+
echo "USE $SCHEMA_NAME;"
5+
while read line
6+
do
7+
echo "ALTER TABLE $line DISCARD TABLESPACE;"
8+
done < $SCHEMA_NAME\_tables.lst

‎create_import_tablespace_sql.sh

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
#!/bin/sh
2+
SCHEMA_NAME=$1
3+
4+
echo "USE $SCHEMA_NAME;"
5+
while read line
6+
do
7+
echo "ALTER TABLE $line IMPORT TABLESPACE;"
8+
done < $SCHEMA_NAME\_tables.lst

‎dump_tables_ddl.sh

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
#!/bin/sh
2+
SCRIPT_DIR=/usr/local/scripts/mysql_scripts
3+
source $SCRIPT_DIR/vars.sh
4+
5+
SCHEMA_NAME=$1
6+
7+
while read line
8+
do
9+
mysql --host=${HOST_NAME} --user=${USER_NAME} --password=${PASSWD} -N -e "show create table $line"
10+
done < $SCHEMA_NAME\_tables.lst

‎export_backup.sh

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
#!/bin/sh
2+
3+
SCRIPT_DIR=/usr/local/scripts/mysql_scripts
4+
source $SCRIPT_DIR/vars.sh
5+
6+
SCHEMA_NAME=$1
7+
TABLES_LIST_FILE=$SCRIPT_DIR/$SCHEMA_NAME\_tables.lst
8+
9+
echo "--backup"
10+
sudo /usr/bin/xtrabackup --user=${USER_NAME} --password=${PASSWD} --tables-file=${TABLES_LIST_FILE} --backup --parallel=4 --target-dir=${BACKUP_DIR} --datadir=${DATA_DIR}
11+
12+
echo "--prepare --export"
13+
sudo /usr/bin/xtrabackup --prepare --export --target-dir=${BACKUP_DIR}
14+

‎get_tables.sh

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
#!/bin/sh
2+
SCRIPT_DIR=/usr/local/scripts/mysql_scripts
3+
source $SCRIPT_DIR/vars.sh
4+
5+
SCHEMA_NAME=$1
6+
7+
mysql -h $HOST_NAME -u $USER_NAME -p$PASSWD -N -e "SELECT CONCAT(TABLE_SCHEMA,'.',TABLE_NAME) FROM information_schema.TABLES WHERE TABLE_SCHEMA = '${SCHEMA_NAME}'" > $SCHEMA_NAME\_tables.lst

‎gitcommand

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
echo "# mysql-innodb-backup-with-Percona-XtraBackup" >> README.md
2+
git init
3+
git add README.md
4+
git commit -m "first commit"
5+
git remote add origin https://github.com/bennzhang/mysql-innodb-backup-with-Percona-XtraBackup.git
6+
git push -u origin master

‎mv_back_backups.sh

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
#!/bin/sh
2+
3+
SCRIPT_DIR=/usr/local/scripts/mysql_scripts
4+
source $SCRIPT_DIR/vars.sh
5+
6+
SCHEMA_NAME=$1
7+
8+
while read line
9+
do
10+
cp -rpv ${BACKUP_DIR}/${SCHEMA_NAME}/*.cfg ${DATA_DIR}/${SCHEMA_NAME}/
11+
cp -rpv ${BACKUP_DIR}/${SCHEMA_NAME}/*.ibd ${DATA_DIR}/${SCHEMA_NAME}/
12+
done < $SCHEMA_NAME\_tables.lst

‎restore_backup.sh

+33
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
#!/bin/sh
2+
3+
SCRIPT_DIR=/usr/local/scripts/mysql_scripts
4+
source $SCRIPT_DIR/vars.sh
5+
6+
SCHEMA_NAME=$1
7+
TABLES_LIST_FILE=$SCRIPT_DIR/$SCHEMA_NAME\_tables.lst
8+
9+
echo "dump original DDL ......"
10+
mysqldump -d -u ${USER_NAME} -p${PASSWD} ${SCHEMA_NAME} > ${SCHEMA_NAME}\_DDL.sql
11+
12+
echo "drop/recreate tables ......"
13+
mysql --host=${HOST_NAME} --user=${USER_NAME} --password=${PASSWD} < ${SCHEMA_NAME}\_DDL.sql
14+
15+
echo "discard tablespaces ......"
16+
echo " + create discard tablespaces sql"
17+
$SCRIPT_DIR/create_discard_tablespace_sql.sh ${SCHEMA_NAME} > $SCHEMA_NAME\_discard_tablespace.sql
18+
echo " + run discard tablespaces sql"
19+
mysql --host=${HOST_NAME} --user=${USER_NAME} --password=${PASSWD} < ${SCHEMA_NAME}\_discard_tablespace.sql
20+
21+
echo "copy backed up files from ${BACKUP_DIR} to ${DATA_DIR}
22+
$SCRIPT_DIR/mv_back_backups.sh ${SCHEMA_NAME}
23+
24+
echo "import tablespaces ......"
25+
echo " + create import tablespaces sql"
26+
$SCRIPT_DIR/create_import_tablespace_sql.sh ${SCHEMA_NAME} > $SCHEMA_NAME\_import_tablespace.sql
27+
echo " + run import tablespaces sql"
28+
mysql --host=${HOST_NAME} --user=${USER_NAME} --password=${PASSWD} < ${SCHEMA_NAME}\_import_tablespace.sql
29+
30+
echo "clean up cfg files"
31+
sudo rm ${DATA_DIR}/${SCHEMA_NAME}/*.cfg
32+
33+
echo "Restore Complete!!!!"

‎test_DDL.sql

Whitespace-only changes.

‎test_discard_tablespace.sql

Whitespace-only changes.

‎test_import_tablespace.sql

Whitespace-only changes.

‎test_tables.lst

Whitespace-only changes.

‎vars.sh

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
#!/bin/sh
2+
3+
TODAY=`date +%Y%m%d`
4+
HOST_NAME=hostname
5+
USER_NAME=yourusername
6+
PASSWD=yourpasswd
7+
BACKUP_DIR=/u01/mysql_backup/
8+
DATA_DIR=/u01/mysql_data

0 commit comments

Comments
 (0)