懒癌晚期的中年大叔
0%

扩容软 RAID1

发表于 分类于

扩容软 RAID1

测试数据库无法访问，检查发现 MySQL 数据分区磁盘使用率达到 100%，MySQL 一直在进行 crash recovery 操作。服务器使用了 2 块 1TB 硬盘，配置成软 RAID1。

/dev/md0 为数据分区 410GB 100% 占用率
/dev/md1 为备份分区 400GB 空

扩容软 RAID1

因为备份分区小于现有的数据分区，所以不能把备份分区直接加入到数据分区中。

方案 1：把数据分区的 RAID1 改为 RAID10。备份分区重新做成 RAID0 加入到数据分区中，同步完成后，再把数据分区的磁盘取出配置成 RAID0 加入到数据分区中。

方案 2：保留 RAID1 结构，扩展磁盘分区容量。

因为只有 2 块硬盘，所有改成 RAID10 磁盘性能反而会下降。所以选择第 2 个方案。

磁盘分区扩容

  • 查看备份分区信息
1
2
3
4
5
6
7
8
9
10
11
12
13
mdadm -D /dev/md0
/dev/md0:
......
           Number   Major   Minor   RaidDevice State
       0       8        1        0      active sync   /dev/sda1
       1       8       17        1      active sync   /dev/sdb1

mdadm -D /dev/md1
/dev/md1:
......
    Number   Major   Minor   RaidDevice State
       0       8        2        0      active sync   /dev/sda2
       1       8       18        1      active sync   /dev/sdb2
  • 删除备份分区
1
2
3
4
umount /dev/md1
mdadm -S /dev/md1
mdadm --misc --zero-superblock /dev/sda2
mdadm --misc --zero-superblock /dev/sdb2

修改 /etc/mdadm.conf 配置文件，注释 /dev/md1 相关配置。

修改 /etc/fstab 配置文件，注释备份分区 mount 信息。（操作中忘记这一步，造成重启服务器无法进入系统。）

  • 扩展 sdb1 磁盘分区

1.从数据分区中去除 sdb1 分区：

1
2
mdadm /dev/md0 --fail /dev/sdb1 --remove /dev/sdb1
mdadm --misc --zero-superblock /dev/sdb1

2.重新划分 sdb 盘分区，把 /dev/sdb1 和 /dev/sdb2 分区合并为一个分区：（如果省略第 2 步，容易出现逻辑扇区和物理扇区无法对齐情况，有可能影响 IO 性能。）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
fdisk /dev/sdb

1. p,Enter
# 显示分区信息
2. u,Enter
# 将显示输入单元更改为扇区
3. d,1,Enter
# 删除第 1 个分区
4. d,2,Enter
# 删除第 2 个分区
5. n,p,1,Enter,Enter
# 创建第 1 个主分区
6. p,Enter
# 显示分区信息
7. w,Enter
# 写入分区表信息并退出

3.重启系统更新 /dev/sdb 分区表：

因为没有修改 /etc/fstab 配置文件，在服务器重启中出现如下提示：

1
2
3
GIVE root password for maintenance

(or type control-D to continue):

输入 root 用户密码后，系统是只读状态，需要重新 mount 根分区，再修改 /etc/fstab 配置文件，最后重启即可。

1
2
mount –o remount,rw /
vi /etc/fstab

4.数据分区添加 /dev/sdb1 磁盘，等待数据同步完成。

1
mdadm --manage /dev/md0 --add /dev/sdb1
  • 扩展 sda1 磁盘分区

1.从数据分区中去除 sda1 分区：

1
2
mdadm /dev/md0 --fail /dev/sda1 --remove /dev/sda1
mdadm --misc --zero-superblock /dev/sda1

2.重新划分 sda 盘分区，把 /dev/sda1 和 /dev/sda2 分区合并为一个分区。

3.重启系统更新 /dev/sda 分区表。

4.数据分区添加 /dev/sda1 磁盘，等待数据同步完成：

1
mdadm --manage /dev/md0 --add /dev/sda1

至此磁盘分区扩容完成。

文件系统扩容

虽然磁盘分区扩容完成，但是文件系统大小没有变化，需要再对文件系统进行扩容：

1
2
3
mdadm --grow --size max /dev/md0
e2fsck -f /dev/md0
resize2fs /dev/md0

PS：查到 mdadm --grow /dev/md0 -z max --assume-clean 命令也可以扩容，但是未经测试。

恢复 MySQL 服务

启动 MySQL 服务:

1
service mysqld start

日志信息：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
181219 12:04:57 mysqld_safe Starting mysqld daemon with databases from /data/mysql
181219 12:04:57 [Note] Plugin 'FEDERATED' is disabled.
181219 12:04:57 InnoDB: The InnoDB memory heap is disabled
181219 12:04:57 InnoDB: Mutexes and rw_locks use GCC atomic builtins
181219 12:04:57 InnoDB: Compressed tables use zlib 1.2.3
181219 12:04:57 InnoDB: Initializing buffer pool, size = 4.0G
181219 12:04:57 InnoDB: Completed initialization of buffer pool
181219 12:04:57 InnoDB: highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 848305710317
181219 12:04:57  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 848310952960
InnoDB: Doing recovery: scanned up to log sequence number 848316195840
InnoDB: Doing recovery: scanned up to log sequence number 848321438720
InnoDB: Doing recovery: scanned up to log sequence number 848326681600
InnoDB: Doing recovery: scanned up to log sequence number 848331924480
InnoDB: Doing recovery: scanned up to log sequence number 848337167360
InnoDB: Doing recovery: scanned up to log sequence number 848342410240
InnoDB: Doing recovery: scanned up to log sequence number 848347653120
InnoDB: Doing recovery: scanned up to log sequence number 848352896000
InnoDB: Doing recovery: scanned up to log sequence number 848358138880
InnoDB: Doing recovery: scanned up to log sequence number 848363381760
InnoDB: Doing recovery: scanned up to log sequence number 848368624640
InnoDB: Doing recovery: scanned up to log sequence number 848373867520
InnoDB: Doing recovery: scanned up to log sequence number 848379110400
InnoDB: Doing recovery: scanned up to log sequence number 848384353280
InnoDB: Doing recovery: scanned up to log sequence number 848389596160
InnoDB: Doing recovery: scanned up to log sequence number 848394839040
InnoDB: Doing recovery: scanned up to log sequence number 848400081920
InnoDB: Doing recovery: scanned up to log sequence number 848405324800
InnoDB: Doing recovery: scanned up to log sequence number 848410567680
InnoDB: Doing recovery: scanned up to log sequence number 848415810560
InnoDB: Doing recovery: scanned up to log sequence number 848421053440
InnoDB: Doing recovery: scanned up to log sequence number 848426296320
InnoDB: Doing recovery: scanned up to log sequence number 848431539200
InnoDB: Doing recovery: scanned up to log sequence number 848436782080
InnoDB: Doing recovery: scanned up to log sequence number 848442024960
InnoDB: Doing recovery: scanned up to log sequence number 848447267840
InnoDB: Doing recovery: scanned up to log sequence number 848452510720
InnoDB: Doing recovery: scanned up to log sequence number 848457753600
InnoDB: Doing recovery: scanned up to log sequence number 848462996480
InnoDB: Doing recovery: scanned up to log sequence number 848468239360
InnoDB: Doing recovery: scanned up to log sequence number 848473482240
InnoDB: Doing recovery: scanned up to log sequence number 848478725120
InnoDB: Doing recovery: scanned up to log sequence number 848483968000
InnoDB: Doing recovery: scanned up to log sequence number 848489210880
InnoDB: Doing recovery: scanned up to log sequence number 848494453760
InnoDB: Doing recovery: scanned up to log sequence number 848499696640
InnoDB: Doing recovery: scanned up to log sequence number 848504939520
InnoDB: Doing recovery: scanned up to log sequence number 848510182400
InnoDB: Doing recovery: scanned up to log sequence number 848515425280
InnoDB: Doing recovery: scanned up to log sequence number 848520668160
InnoDB: Doing recovery: scanned up to log sequence number 848525911040
InnoDB: Doing recovery: scanned up to log sequence number 848531153920
InnoDB: Doing recovery: scanned up to log sequence number 848536396800
InnoDB: Doing recovery: scanned up to log sequence number 848540988197
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 1 row operations to undo
InnoDB: Trx id counter is EC300
181219 12:05:01  InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
InnoDB: Starting in background the rollback of uncommitted transactions
181219 12:05:05  InnoDB: Rolling back trx with id EC1B4, 1 rows to undo
181219 12:05:05  InnoDB: Waiting for the background threads to start

InnoDB: Rolling back of trx id EC1B4 completed
181219 12:05:06  InnoDB: Rollback of non-prepared transactions completed
181219 12:05:06 InnoDB: 5.5.40 started; log sequence number 848540988197
181219 12:05:06 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
181219 12:05:06 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
181219 12:05:06 [Note] Server socket created on IP: '0.0.0.0'.
181219 12:05:06 [Note] Event Scheduler: Loaded 0 events
181219 12:05:06 [Note] /usr/local/mysql/bin/mysqld: ready for connections.
Version: '5.5.40'  socket: '/data/mysql/mysql.sock'  port: 3306  Source distribution

至此数据库服务恢复正常。