Removing multipath disks
in one of our RHEL4 blade servers, i need to remove
1x500G HP XP1000 LUN:
1x500G HP XP1000 LUN:
PV VG Fmt Attr PSize
PFree
/dev/dm-17 lx0080apps00vg lvm2 a- 200.00G
11.50G
/dev/dm-18 lx0080apps00vg lvm2 a- 200.00G
11.50G
/dev/dm-19 lx0080apps02vg lvm2 a- 500.00G 100.00G
/dev/dm-20 lx0080apps01vg lvm2 a- 250.00G
50.00G
/dev/dm-21 lx0080apps01vg lvm2 a- 100.00G
50.00G
/dev/dm-22 lx0080apps01vg lvm2 a- 200.00G
10.00G
/dev/dm-23 lx0080apps01vg lvm2 a- 200.00G
10.00G
/dev/dm-24 lx0080apps01vg lvm2 a- 200.00G
10.00G
/dev/sda2
vg00 lvm2 a- 67.88G
4.47G
[root@lx0080 ~]# pvdisplay /dev/dm-19
--- Physical volume
---
PV Name /dev/dm-19
VG Name lx0080apps02vg
PV Size 500.00 GB / not usable 1.25 MB
Allocatable yes
PE Size (KByte) 4096
Total PE 128000
Free PE 25600
Allocated PE 102400
PV UUID
O9b3hw-oSUO-pUpe-oGJw-zEBp-tKXo-3ba2g9
so here's how to do it without causing disruptions on the
machine:
btw, this machine hosts a test environment.
CAUTION: DON'T DO THIS IN PRODUCTION ENVIRONMENT ON A REGULAR
BUSINESS DAY (Even if you are sure it won't cause any problems)
make sure no i/o activity on the disk
- i've removed it from nfs exports
- unmounted the FS
remove from VG group
remove from multipath definitions
delete from system
below here are the actual steps.
1. remove from VG group
[root@lx0080 ~]# vgremove lx0080apps02vg
btw, this machine hosts a test environment.
CAUTION: DON'T DO THIS IN PRODUCTION ENVIRONMENT ON A REGULAR
BUSINESS DAY (Even if you are sure it won't cause any problems)
make sure no i/o activity on the disk
- i've removed it from nfs exports
- unmounted the FS
remove from VG group
remove from multipath definitions
delete from system
below here are the actual steps.
1. remove from VG group
[root@lx0080 ~]# vgremove lx0080apps02vg
Found duplicate PV
O9b3hwoSUOpUpeoGJwzEBptKXo3ba2g9: using /dev/sdb1 not /dev/sdq1
Do you really want to
remove volume group "lx0080apps02vg" containing 1 logical volumes?
[y/n]: y
Do you really want to
remove active logical volume "lx0080apps02LV"? [y/n]: y
Logical volume "lx0080apps02LV"
successfully removed
Volume group "lx0080apps02vg"
successfully removed
pvs
PV VG Fmt Attr PSize
PFree
/dev/dm-17 lx0080apps00vg lvm2 a- 200.00G
11.50G
/dev/dm-18 lx0080apps00vg lvm2 a- 200.00G
11.50G
/dev/dm-19 lvm2 -- 500.00G 500.00G
/dev/dm-20
lx0080apps01vg lvm2 a- 250.00G 50.00G
/dev/dm-21 lx0080apps01vg lvm2 a- 100.00G
50.00G
/dev/dm-22 lx0080apps01vg lvm2 a- 200.00G
10.00G
/dev/dm-23 lx0080apps01vg lvm2 a- 200.00G
10.00G
/dev/dm-24 lx0080apps01vg lvm2 a- 200.00G
10.00G
/dev/sda2
vg00 lvm2 a- 67.88G
4.47G
as you can see - /dev/dm-19 is now an orphan. since its the only
500G
LUN in the server, its easy to identify.
2. remove from multipath
from multipath -ll (other entries removed):
LUN in the server, its easy to identify.
2. remove from multipath
from multipath -ll (other entries removed):
mpath8
(360060e8004a51a000000a51a00000200)
[size=500
GB][features="1 queue_if_no_path"][hwhandler="0"]
\_ round-robin 0
[active]
\_ 3:0:0:0 sdb 8:16 [active][ready]
\_ 4:0:0:0 sdq 65:0 [active][ready]
it has two active paths.
multipath -f mpath8
(-f flush a multipath device map specified as parameter, if unused)
[root@lx0080 ~]# multipath -f mpath8
[root@lx0080 ~]# multipath -ll | grep 360060e8004a51a000000a51a00000200
360060e8004a51a000000a51a00000200 is the UUID of the disk from
scsi_id -gus /block/sdb (or /block/sdq)
now the multipath map is removed. but the disk is still in the system.
[root@lx0080 ~]# fdisk -l /dev/sdq
multipath -f mpath8
(-f flush a multipath device map specified as parameter, if unused)
[root@lx0080 ~]# multipath -f mpath8
[root@lx0080 ~]# multipath -ll | grep 360060e8004a51a000000a51a00000200
360060e8004a51a000000a51a00000200 is the UUID of the disk from
scsi_id -gus /block/sdb (or /block/sdq)
now the multipath map is removed. but the disk is still in the system.
[root@lx0080 ~]# fdisk -l /dev/sdq
Disk /dev/sdq: 536.8
GB, 536877465600 bytes
255 heads, 63
sectors/track, 65271 cylinders
Units = cylinders of
16065 * 512 = 8225280 bytes
Device Boot Start End Blocks
Id System
/dev/sdq1 1 65271
524289276 8e Linux LVM
[root@lx0080 ~]# fdisk -l /dev/sdb
Disk /dev/sdb: 536.8
GB, 536877465600 bytes
255 heads, 63
sectors/track, 65271 cylinders
Units = cylinders of
16065 * 512 = 8225280 bytes
Device Boot Start End Blocks
Id System
/dev/sdb1 1 65271
524289276 8e Linux LVM
[root@lx0080 ~]#
[root@lx0080 ~]# pvs
[root@lx0080 ~]# pvs
Found duplicate PV
O9b3hwoSUOpUpeoGJwzEBptKXo3ba2g9: using /dev/sdb1 not /dev/sdq1
PV
VG Fmt Attr PSize
PFree
/dev/dm-17 lx0080apps00vg lvm2 a- 200.00G
11.50G
/dev/dm-18 lx0080apps00vg lvm2 a- 200.00G
11.50G
/dev/dm-20 lx0080apps01vg lvm2 a- 250.00G
50.00G
/dev/dm-21 lx0080apps01vg lvm2 a- 100.00G
50.00G
/dev/dm-22 lx0080apps01vg lvm2 a- 200.00G
10.00G
/dev/dm-23 lx0080apps01vg lvm2 a- 200.00G
10.00G
/dev/dm-24 lx0080apps01vg lvm2 a- 200.00G
10.00G
/dev/sda2
vg00 lvm2 a- 67.88G
4.47G
/dev/sdb1 lvm2 -- 500.00G 500.00G
3. delete from the system
[root@lx0080 ~]# echo 1 > /sys/block/sdb/device/delete
[root@lx0080 ~]# echo 1 > /sys/block/sdq/device/delete
[root@lx0080 ~]# dmesg
Synchronizing SCSI
cache for disk sdb:
Synchronizing SCSI
cache for disk sdq:
[root@lx0080 ~]# fdisk -l /dev/sdb
[root@lx0080 ~]# fdisk -l /dev/sdq
here are some entries generated thru syslog:
- when multipath -f was run:
[root@lx0080 ~]# fdisk -l /dev/sdq
here are some entries generated thru syslog:
- when multipath -f was run:
Apr 22 09:17:20 lx0080
multipathd: dm map mpath8 removed
Apr 22 09:17:20 lx0080
udevd[2444]: udev done!
- when deleting sdb
Apr 22 09:22:52 lx0080
kernel: Synchronizing SCSI cache for disk sdb:
Apr 22 09:22:52 lx0080
multipathd: remove sdb path checker
Apr 22 09:22:52 lx0080
udevd[2444]: udev done!
- when deleting sdq
Apr 22 09:22:57 lx0080
kernel: Synchronizing SCSI cache for disk sdq:
Apr 22 09:22:57 lx0080
multipathd: remove sdq path checker
Apr 22 09:22:57 lx0080
udevd[2444]: udev done!
Apr 22 09:23:02 lx0080
udevd[2444]: udev done!
our storage admin can now remove the disk definition from storage
server.
