~dsmythies/serverguide/serverguide-ftp : revision 57

1

<?xml version="1.0" encoding="UTF-8"?>

2

<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"

3

"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" [

4

<!ENTITY % globalent SYSTEM "../../../../libs/global.ent">

5

%globalent;

6

<!ENTITY % gnome-menus-C SYSTEM "../../../../libs/gnome-menus-C.ent">

7

%gnome-menus-C;

8

<!ENTITY % xinclude SYSTEM "../../../../libs/xinclude.mod">

9

%xinclude;

10

<!ENTITY language "&EnglishAmerican;">

11

<!ENTITY DMM "DM-Multipath">

12

<!ENTITY multipath_c "<command>multipath</command>">

13

<!ENTITY multipathd_c "<command>multipathd</command>">

14

<!ENTITY udev_c "<command>udev</command>">

15

<!ENTITY multipath-conf "<filename>multipath.conf</filename>">

16

<!ENTITY multipath-conf-full "<filename>/etc/multipath.conf</filename>">

17

]>

18

19

<!-- This work was signifigantly enabled by a tool called XMLmind:

20

http://www.xmlmind.com/xmleditor/

21

and is approved for open source use. It will dramatically increase

22

your productivity. -->

23

24

25

26

27

<title>Device Mapper Multipathing</title>

28

29

<para>Device mapper multipathing (DM-Multipath) allows you to configure

30

multiple I/O paths between server nodes and storage arrays into a single

31

device. These I/O paths are physical SAN connections that can include

32

separate cables, switches, and controllers. Multipathing aggregates the

33

I/O paths, creating a new device that consists of the aggregated paths.

34

This chapter provides a summary of the features of DM-Multipath that are

35

new for the initial release of Ubuntu Server 12.04. Following that, this

36

chapter provides a high-level overview of DM Multipath and its components,

37

as well as an overview of DM-Multipath setup.</para>

38

39

40

<title>New and Changed Features for Ubuntu Server 12.04</title>

41

42

<para>Migrated from multipath-0.4.8 to multipath-0.4.9</para>

43

44

45

<title>Migration from 0.4.8</title>

46

47

<para>The priority checkers are no longer run as standalone binaries,

48

but as shared libraries. The key value name for this feature has also

49

slightly changed. Copy the attribute named <emphasis

50

role="bold">prio_callout</emphasis> to <emphasis

51

role="bold">prio</emphasis>, also modify the argument the name of the

52

priority checker, a system path is no longer necessary. Example

53

conversion:<screen>device {

54

vendor "NEC"

55

product "DISK ARRAY"

56

prio_callout mpath_prio_alua /dev/%n

57

prio alua

58

}</screen></para>

59

60

<para>See Table <link

61

linkend="priority-checker-conversion-table">"Priority Checker

62

Conversion"</link> for a complete listing</para>

63

64

<table>

65

<title id="priority-checker-conversion-table">Priority Checker

66

Conversion</title>

67

68

69

<thead>

70

<row>

71

72

73

74

</row>

75

</thead>

76

77

<tbody>

78

<row>

79

<entry><emphasis role="bold">prio_callout mpath_prio_emc

80

/dev/%n</emphasis></entry>

81

82

83

</row>

84

85

<row>

86

<entry><emphasis role="bold">prio_callout mpath_prio_alua

87

/dev/%n</emphasis></entry>

88

89

90

</row>

91

92

<row>

93

<entry><emphasis role="bold">prio_callout mpath_prio_netapp

94

/dev/%n</emphasis></entry>

95

96

<entry><emphasis role="bold">prio netapp</emphasis></entry>

97

</row>

98

99

<row>

100

<entry><emphasis role="bold">prio_callout mpath_prio_rdac

101

/dev/%n</emphasis></entry>

102

103

104

</row>

105

106

<row>

107

<entry><emphasis role="bold">prio_callout mpath_prio_hp_sw

108

/dev/%n</emphasis></entry>

109

110

111

</row>

112

113

<row>

114

<entry><emphasis role="bold">prio_callout

115

mpath_prio_hds_modular %b</emphasis></entry>

116

117

118

</row>

119

</tbody>

120

</tgroup>

121

</table>

122

123

<para>Since the multipath config file parser essentially parses all

124

key/value pairs it finds and then makes use of them, it is safe for

125

both <emphasis role="bold">prio_callout</emphasis> and <emphasis

126

role="bold">prio</emphasis> to coexist and is recommended that the

127

<emphasis role="bold">prio</emphasis> attribute be inserted before

128

beginning migration. After which you can safely delete the legacy

129

<emphasis role="bold">prio_calliout</emphasis> attribute without

130

interrupting service.</para>

131

</sect3>

132

</sect2>

133

134

135

<title>Overview</title>

136

137

<para>DM-Multipath can be used to provide:</para>

138

139

140

141

<para><emphasis> Redundancy </emphasis> DM-Multipath can provide

142

failover in an active/passive configuration. In an active/passive

143

configuration, only half the paths are used at any time for I/O. If

144

any element of an I/O path (the cable, switch, or controller) fails,

145

DM-Multipath switches to an alternate path.</para>

146

</listitem>

147

148

149

<para><emphasis> Improved Performance </emphasis> Performance

150

DM-Multipath can be configured in active/active mode, where I/O is

151

spread over the paths in a round-robin fashion. In some

152

configurations, DM-Multipath can detect loading on the I/O paths and

153

dynamically re-balance the load.</para>

154

</listitem>

155

</itemizedlist>

156

</sect2>

157

158

159

<title>Storage Array Overview</title>

160

161

<para>By default, DM-Multipath includes support for the most common

162

storage arrays that support DM-Multipath. The supported devices can be

163

found in the multipath.conf.defaults file. If your storage array

164

supports DM-Multipath and is not configured by default in this file, you

165

may need to add them to the DM-Multipath configuration file,

166

multipath.conf. For information on the DM-Multipath configuration file,

167

see Section, <link linkend="multipath-dm-multipath-config-file">The

168

DM-Multipath Configuration File</link>. Some storage arrays require

169

special handling of I/O errors and path switching. These require

170

separate hardware handler kernel modules.</para>

171

</sect2>

172

173

174

<title>&DMM; components</title>

175

176

<para>Table “<link linkend="multipath-components-table">DM-Multipath

177

Components”</link> describes the components of the DM-Multipath package.

178

<xi:include href="multipath-components-table.xml"

179

xmlns:xi="http://www.w3.org/2001/XInclude"/></para>

180

</sect2>

181

182

183

<title>&DMM; Setup Overview</title>

184

185

<para>DM-Multipath includes compiled-in default settings that are

186

suitable for common multipath configurations. Setting up DM-multipath is

187

often a simple procedure. The basic procedure for configuring your

188

system with DM-Multipath is as follows: <orderedlist>

189

190

<para>Install the <emphasis role="bold">multipath-tools</emphasis>

191

and <emphasis role="bold">multipath-tools-boot</emphasis>

192

packages</para>

193

</listitem>

194

195

196

<para>Create an empty config file, &multipath-conf-full;, that

197

re-defines the <link

198

linkend="multipath-skel-config">following</link></para>

199

</listitem>

200

201

202

<para>If necessary, edit the <emphasis

203

role="bold">multipath.conf</emphasis> configuration file to modify

204

default values and save the updated file.</para>

205

</listitem>

206

207

208

<para>Start the multipath daemon</para>

209

</listitem>

210

211

212

<para>Update initial ramdisk</para>

213

</listitem>

214

</orderedlist> For detailed setup instructions for multipath

215

configuration see Section, <link

216

linkend="multipath-setup-overview">Setting Up

217

DM-Multipath</link>.</para>

218

</sect2>

219

</sect1>

220

221

222

<title>Multipath Devices</title>

223

224

<para>Without DM-Multipath, each path from a server node to a storage

225

controller is treated by the system as a separate device, even when the

226

I/O path connects the same server node to the same storage controller.

227

DM-Multipath provides a way of organizing the I/O paths logically, by

228

creating a single multipath device on top of the underlying

229

devices.</para>

230

231

232

<title>Multipath Device Identifiers</title>

233

234

<para>Each multipath device has a World Wide Identifier (WWID), which is

235

guaranteed to be globally unique and unchanging. By default, the name of

236

a multipath device is set to its WWID. Alternately, you can set the

237

<emphasis role="bold"><link

238

linkend="attribute-user_friendly_names">user_friendly_names</link>

239

</emphasis>option in the multipath configuration file, which causes

240

DM-Multipath to use a node-unique alias of the form <emphasis

241

role="bold">mpathn</emphasis> as the name. For example, a node with two

242

HBAs attached to a storage controller with two ports via a single

243

unzoned FC switch sees four devices: <emphasis

244

role="bold">/dev/sda</emphasis>, <emphasis

245

role="bold">/dev/sdb</emphasis>, <emphasis

246

role="bold">/dev/sdc</emphasis>, and <emphasis

247

role="bold">/dev/sdd</emphasis>. DM-Multipath creates a single device

248

with a unique WWID that reroutes I/O to those four underlying devices

249

according to the multipath configuration. When the <emphasis

250

role="bold"><link

251

linkend="attribute-user_friendly_names">user_friendly_names</link></emphasis>

252

configuration option is set to <emphasis role="bold">yes</emphasis>, the

253

name of the multipath device is set to <emphasis

254

role="bold">mpathn</emphasis>. When new devices are brought under the

255

control of DM-Multipath, the new devices may be seen in two different

256

places under the <emphasis role="bold">/dev</emphasis> directory:

257

<emphasis role="bold">/dev/mapper/mpathn</emphasis> and <emphasis

258

role="bold">/dev/dm-n</emphasis>. <itemizedlist>

259

260

<para>The devices in <emphasis role="bold">/dev/mapper</emphasis>

261

are created early in the boot process. Use these devices to access

262

the multipathed devices, for example when creating logical

263

volumes.</para>

264

</listitem>

265

266

267

<para>Any devices of the form <emphasis

268

role="bold">/dev/dm-n</emphasis> are for internal use only and

269

should never be used.</para>

270

</listitem>

271

</itemizedlist>For information on the multipath configuration

272

defaults, including the <emphasis role="bold"><link

273

linkend="attribute-user_friendly_names">user_friendly_names</link></emphasis>

274

configuration option, see Section , <link

275

linkend="multipath-config-defaults">“Configuration File

276

Defaults”</link>. You can also set the name of a multipath device to a

277

name of your choosing by using the <emphasis role="bold"><link

278

linkend="attribute-alias">alias</link></emphasis> option in the

279

<emphasis role="bold">multipaths</emphasis> section of the multipath

280

configuration file. For information on the <emphasis

281

role="bold">multipaths</emphasis> section of the multipath configuration

282

file, see Section, <link

283

linkend="multipath-config-multipath">“Multipaths Device Configuration

284

Attributes”</link>.</para>

285

</sect2>

286

287

288

<title>Consistent Multipath Device Names in a Cluster</title>

289

290

<para>When the <emphasis role="bold">user_friendly_names</emphasis>

291

configuration option is set to yes, the name of the multipath device is

292

unique to a node, but it is not guaranteed to be the same on all nodes

293

using the multipath device. Similarly, if you set the <emphasis

294

role="bold">alias</emphasis> option for a device in the <emphasis

295

role="bold">multipaths</emphasis> section of the &multipath-conf;

296

configuration file, the name is not automatically consistent across all

297

nodes in the cluster. This should not cause any difficulties if you use

298

LVM to create logical devices from the multipath device, but if you

299

require that your multipath device names be consistent in every node it

300

is recommended that you leave the <emphasis

301

role="bold">user_friendly_names</emphasis> option set to <emphasis

302

role="bold">no</emphasis> and that you not configure aliases for the

303

devices. By default, if you do not set <emphasis

304

role="bold">user_friendly_names</emphasis> to yes or configure an alias

305

for a device, a device name will be the WWID for the device, which is

306

always the same. If you want the system-defined user-friendly names to

307

be consistent across all nodes in the cluster, however, you can follow

308

this procedure:</para>

309

310

311

312

<para>Set up all of the multipath devices on one machine.</para>

313

</listitem>

314

315

316

<para>Disable all of your multipath devices on your other machines

317

by running the following commands:</para>

318

319

<screen># service multipath-tools stop

320

# multipath -F

321

</screen>

322

</listitem>

323

324

325

<para>Copy the <filename>/etc/multipath/bindings</filename> file

326

from the first machine to all the other machines in the

327

cluster.</para>

328

</listitem>

329

330

331

<para>Re-enable the multipathd daemon on all the other machines in

332

the cluster by running the following command:</para>

333

334

<screen># service multipath-tools start</screen>

335

</listitem>

336

</orderedlist>

337

338

<para>If you add a new device, you will need to repeat this

339

process.</para>

340

341

<para>Similarly, if you configure an alias for a device that you would

342

like to be consistent across the nodes in the cluster, you should ensure

343

that the <filename>/etc/multipath.conf</filename> file is the same for

344

each node in the cluster by following the same procedure:</para>

345

346

347

348

<para>Configure the aliases for the multipath devices in the in the

349

<filename>multipath.conf</filename> file on one machine.</para>

350

</listitem>

351

352

353

<para>Disable all of your multipath devices on your other machines

354

by running the following commands:</para>

355

356

<screen># service multipath-tools stop

357

# multipath -F

358

</screen>

359

</listitem>

360

361

362

<para>Copy the &multipath-conf; file from the first machine to all

363

the other machines in the cluster.</para>

364

</listitem>

365

366

367

<para>Re-enable the multipathd daemon on all the other machines in

368

the cluster by running the following command:</para>

369

370

<screen># service multipath-tools start</screen>

371

</listitem>

372

</orderedlist>

373

374

<para>When you add a new device you will need to repeat this

375

process.</para>

376

</sect2>

377

378

379

<title>Multipath Device attributes</title>

380

381

<para>In addition to the <emphasis

382

role="bold">user_friendly_names</emphasis> and <emphasis

383

role="bold">alias</emphasis> options, a multipath device has numerous

384

attributes. You can modify these attributes for a specific multipath

385

device by creating an entry for that device in the <emphasis

386

role="bold">multipaths</emphasis> section of the <emphasis

387

role="bold">multipath</emphasis> configuration file. For information on

388

the <emphasis role="bold">multipaths</emphasis> section of the multipath

389

configuration file, see Section, "<link endterm="config-multipath-title"

390

linkend="multipath-config-multipath"/>".</para>

391

</sect2>

392

393

394

<title>Multipath Devices in Logical Volumes</title>

395

396

<para>After creating multipath devices, you can use the multipath device

397

names just as you would use a physical device name when creating an LVM

398

physical volume. For example, if /dev/mapper/mpatha is the name of a

399

multipath device, the following command will mark /dev/mapper/mpatha as

400

a physical volume. <screen># pvcreate /dev/mapper/mpatha</screen> You

401

can use the resulting LVM physical device when you create an LVM volume

402

group just as you would use any other LVM physical device.</para>

403

404

<note>

405

<para>If you attempt to create an LVM physical volume on a whole

406

device on which you have configured partitions, the pvcreate command

407

will fail.</para>

408

</note>

409

410

<para>When you create an LVM logical volume that uses active/passive

411

multipath arrays as the underlying physical devices, you should include

412

filters in the <emphasis role="bold">lvm.conf</emphasis> to exclude the

413

disks that underlie the multipath devices. This is because if the array

414

automatically changes the active path to the passive path when it

415

receives I/O, multipath will failover and failback whenever LVM scans

416

the passive path if these devices are not filtered. For active/passive

417

arrays that require a command to make the passive path active, LVM

418

prints a warning message when this occurs. To filter all SCSI devices in

419

the LVM configuration file (lvm.conf), include the following filter in

420

the devices section of the file. <screen>filter = [ "r/block/", "r/disk/", "r/sd.*/", "a/.*/" ]</screen>After

421

updating <filename>/etc/lvm.conf</filename>, it's necessary to update

422

the <emphasis role="bold">initrd</emphasis> so that this file will be

423

copied there, where the filter matters the most, during boot.

424

Perform:<screen>update-initramfs -u -k all</screen><note>

425

<para>Every time either <filename>/etc/lvm.conf</filename> or

426

<filename>/etc/multipath.conf</filename> is updated, the initrd

427

should be rebuilt to reflect these changes. This is imperative when

428

blacklists and filters are necessary to maintain a stable storage

429

configuration.</para>

430

</note></para>

431

</sect2>

432

</sect1>

433

434

435

<title>Setting up &DMM; Overview</title>

436

437

<para>This section provides step-by-step example procedures for

438

configuring &DMM;. It includes the following procedures:</para>

439

440

441

442

<para>Basic &DMM; setup</para>

443

</listitem>

444

445

446

<para>Ignoring local disks</para>

447

</listitem>

448

449

450

<para>Adding more devices to the configuration file</para>

451

</listitem>

452

</itemizedlist>

453

454

455

<title>Setting Up &DMM;</title>

456

457

<para>Before setting up &DMM; on your system, ensure that your system

458

has been updated and includes the <emphasis

459

role="bold"><application>multipath-tools</application></emphasis> package. If

460

boot from SAN is desired, then the <emphasis

461

role="bold"><application>multipath-tools-boot</application></emphasis> package

462

is also required.</para>

463

464

<para>A basic <emphasis role="bold">/etc/multipath.conf </emphasis> need

465

not even exist, when <emphasis role="bold">multpath</emphasis> is run

466

without an accompanying <filename>/etc/multipath.conf</filename>, it

467

draws from it's internal database to find a suitable configuration, it

468

also draws from it's internal blacklist. If after running <emphasis

469

role="bold">multipath -ll</emphasis> without a config file, no

470

multipaths are discovered. One must proceed to increase the verbosity to

471

discover why a multipath was not created. Consider referencing the SAN

472

vendor's documentation, the multipath example config files found in

473

<filename>/usr/share/doc/multipath-tools/examples</filename>, and the

474

live multipathd database:<screen id="multipath-skel-config"># echo 'show config' | multipathd -k > multipath.conf-live</screen><note>

475

<para>To work around a quirk in multipathd, when an

476

<filename>/etc/multipath.conf</filename> doesn't exist, the previous

477

command will return nothing, as it is the result of a

478

<emphasis>merge</emphasis> between the

479

<filename>/etc/multipath.conf</filename> and the database in memory.

480

To remedy this, either define an empty

481

<filename>/etc/multipath.conf</filename>, by using <emphasis

482

role="bold">touch</emphasis>, or create one that redefines a default

483

value like:<screen>defaults {

484

user_friendly_names no

485

}

486

</screen>and restart multipathd:<screen># service multipath-tools restart</screen>Now

487

the "show config" command will return the live database.</para>

488

</note></para>

489

</sect2>

490

491

<sect2>

492

<title>Installing with Multipath Support</title>

493

494

<para>To enable <ulink

495

url="http://wiki.debian.org/DebianInstaller/MultipathSupport">multipath

496

support during installation</ulink> use<screen>install disk-detect/multipath/enable=true</screen>at

497

the installer prompt. If multipath devices are found these will show up

498

as <emphasis role="bold">/dev/mapper/mpath<X></emphasis> during

499

installation.</para>

500

</sect2>

501

502

503

<title>Ignoring Local Disks When Generating Multipath Devices</title>

504

505

<para>Some machines have local SCSI cards for their internal disks.

506

DM-Multipath is not recommended for these devices. The following

507

procedure shows how to modify the multipath configuration file to ignore

508

the local disks when configuring multipath.</para>

509

510

511

512

<para>Determine which disks are the internal disks and mark them as

513

the ones to blacklist. In this example, <emphasis

514

role="bold"><filename>/dev/sda</filename></emphasis> is the internal

515

disk. Note that as originally configured in the default multipath

516

configuration file, executing the <emphasis role="bold">multipath

517

-v2</emphasis> shows the local disk, <emphasis

518

role="bold">/dev/sda</emphasis>, in the multipath map. For further

519

information on the <emphasis role="bold">multipath</emphasis>

520

command output, see Section <link

521

linkend="multipath-command-output">“Multipath Command

522

Output”</link>.</para>

523

524

525

# multipath -v2

526

create: SIBM-ESXSST336732LC____F3ET0EP0Q000072428BX1 undef WINSYS,SF2372

527

size=33 GB features="0" hwhandler="0" wp=undef

528

`-+- policy='round-robin 0' prio=1 status=undef

529

|- 0:0:0:0 sda 8:0 [---------

530

531

device-mapper ioctl cmd 9 failed: Invalid argument

532

device-mapper ioctl cmd 14 failed: No such device or address

533

create: 3600a0b80001327d80000006d43621677 undef WINSYS,SF2372

534

size=12G features='0' hwhandler='0' wp=undef

535

`-+- policy='round-robin 0' prio=1 status=undef

536

|- 2:0:0:0 sdb 8:16 undef ready running

537

`- 3:0:0:0 sdf 8:80 undef ready running

538

539

create: 3600a0b80001327510000009a436215ec undef WINSYS,SF2372

540

size=12G features='0' hwhandler='0' wp=undef

541

`-+- policy='round-robin 0' prio=1 status=undef

542

|- 2:0:0:1 sdc 8:32 undef ready running

543

`- 3:0:0:1 sdg 8:96 undef ready running

544

545

create: 3600a0b80001327d800000070436216b3 undef WINSYS,SF2372

546

size=12G features='0' hwhandler='0' wp=undef

547

`-+- policy='round-robin 0' prio=1 status=undef

548

|- 2:0:0:2 sdd 8:48 undef ready running

549

`- 3:0:0:2 sdg 8:112 undef ready running

550

551

create: 3600a0b80001327510000009b4362163e undef WINSYS,SF2372

552

size=12G features='0' hwhandler='0' wp=undef

553

`-+- policy='round-robin 0' prio=1 status=undef

554

|- 2:0:0:3 sdd 8:64 undef ready running

555

`- 3:0:0:3 sdg 8:128 undef ready running

556

</screen>

557

</listitem>

558

559

560

<para>In order to prevent the device mapper from mapping <emphasis

561

role="bold">/dev/sda</emphasis> in its multipath maps, edit the

562

blacklist section of the &multipath-conf-full; file to include this

563

device. Although you could blacklist the <emphasis

564

role="bold">sda</emphasis> device using a <emphasis

565

role="bold">devnode</emphasis> type, that would not be safe

566

procedure since <emphasis role="bold">/dev/sda</emphasis> is not

567

guaranteed to be the same on reboot. To blacklist individual

568

devices, you can blacklist using the WWID of that device. Note that

569

in the output to the <emphasis role="bold">multipath -v2</emphasis>

570

command, the WWID of the <filename>/dev/sda</filename> device is

571

SIBM-ESXSST336732LC____F3ET0EP0Q000072428BX1. To blacklist this

572

device, include the following in the &multipath-conf-full;

573

file.</para>

574

575

576

blacklist {

577

wwid SIBM-ESXSST336732LC____F3ET0EP0Q000072428BX1

578

}

579

</screen>

580

</listitem>

581

582

583

<para>After you have updated the &multipath-conf-full; file, you

584

must manually tell the <emphasis role="bold">multipathd</emphasis>

585

daemon to reload the file. The following command reloads the updated

586

&multipath-conf-full; file.</para>

587

588

<screen># service multipath-tools reload</screen>

589

</listitem>

590

591

592

<para>Run the following command to remove the multipath

593

device:</para>

594

595

596

# multipath -f SIBM-ESXSST336732LC____F3ET0EP0Q000072428BX1

597

</screen>

598

</listitem>

599

600

601

<para>To check whether the device removal worked, you can run the

602

<command>multipath -ll</command> command to display the current

603

multipath configuration. For information on the <command>multipath

604

-ll</command> command, see Section <link

605

linkend="multipath-queries-and-commands">“Multipath Queries with

606

multipath Command”</link>. To check that the blacklisted device was

607

not added back, you can run the multipath command, as in the

608

following example. The multipath command defaults to a verbosity

609

level of <emphasis role="bold">v2</emphasis> if you do not specify a

610

<emphasis role="bold">-v</emphasis> option.</para>

611

612

613

# multipath

614

615

create: 3600a0b80001327d80000006d43621677 undef WINSYS,SF2372

616

size=12G features='0' hwhandler='0' wp=undef

617

`-+- policy='round-robin 0' prio=1 status=undef

618

|- 2:0:0:0 sdb 8:16 undef ready running

619

`- 3:0:0:0 sdf 8:80 undef ready running

620

621

create: 3600a0b80001327510000009a436215ec undef WINSYS,SF2372

622

size=12G features='0' hwhandler='0' wp=undef

623

`-+- policy='round-robin 0' prio=1 status=undef

624

|- 2:0:0:1 sdc 8:32 undef ready running

625

`- 3:0:0:1 sdg 8:96 undef ready running

626

627

create: 3600a0b80001327d800000070436216b3 undef WINSYS,SF2372

628

size=12G features='0' hwhandler='0' wp=undef

629

`-+- policy='round-robin 0' prio=1 status=undef

630

|- 2:0:0:2 sdd 8:48 undef ready running

631

`- 3:0:0:2 sdg 8:112 undef ready running

632

633

create: 3600a0b80001327510000009b4362163e undef WINSYS,SF2372

634

size=12G features='0' hwhandler='0' wp=undef

635

`-+- policy='round-robin 0' prio=1 status=undef

636

|- 2:0:0:3 sdd 8:64 undef ready running

637

`- 3:0:0:3 sdg 8:128 undef ready running

638

</screen>

639

</listitem>

640

</orderedlist>

641

</sect2>

642

643

644

<title>Configuring Storage Devices</title>

645

646

<para>By default, DM-Multipath includes support for the most common

647

storage arrays that support DM-Multipath. The default configuration

648

values, including supported devices, can be found in the

649

<filename>multipath.conf.defaults</filename> file.</para>

650

651

<para>If you need to add a storage device that is not supported by

652

default as a known multipath device, edit the &multipath-conf-full; file

653

and insert the appropriate device information.</para>

654

655

<para>For example, to add information about the HP Open-V series the

656

entry looks like this, where <emphasis role="bold">%n</emphasis> is the

657

device name:</para>

658

659

<screen>devices {

660

device {

661

vendor "HP"

662

product "OPEN-V."

663

getuid_callout "/lib/udev/scsi_id --whitelisted --device=/dev/%n"

664

}

665

}

666

</screen>

667

668

<para>For more information on the devices section of the configuration

669

file, see Section <xref endterm="config-device-title"

670

linkend="multipath-config-device"/>.</para>

671

</sect2>

672

</sect1>

673

674

675

<title>The &DMM; Configuration File</title>

676

677

<para>By default, DM-Multipath provides configuration values for the most

678

common uses of multipathing. In addition, DM-Multipath includes support

679

for the most common storage arrays that support DM-Multipath. The default

680

configuration values and the supported devices can be found in the

681

<filename>multipath.conf.defaults</filename> file.</para>

682

683

<para>You can override the default configuration values for DM-Multipath

684

by editing the &multipath-conf-full; configuration file. If necessary, you

685

can also add a storage array that is not supported by default to the

686

configuration file. This chapter provides information on parsing and

687

modifying the &multipath-conf; file. It contains sections on the following

688

topics: <itemizedlist>

689

690

<para><xref endterm="config-overview-title"

691

linkend="multipath-config-overview"/></para>

692

</listitem>

693

694

695

<para><xref endterm="config-blacklist-title"

696

linkend="multipath-config-blacklist"/></para>

697

</listitem>

698

699

700

<para><xref endterm="config-defaults-title"

701

linkend="multipath-config-defaults"/></para>

702

</listitem>

703

704

705

<para><xref endterm="config-multipath-title"

706

linkend="multipath-config-multipath"/></para>

707

</listitem>

708

709

710

<para><xref endterm="config-device-title"

711

linkend="multipath-config-device"/></para>

712

</listitem>

713

</itemizedlist></para>

714

715

<para>In the multipath configuration file, you need to specify only the

716

sections that you need for your configuration, or that you wish to change

717

from the default values specified in the

718

<filename>multipath.conf.defaults</filename> file. If there are sections

719

of the file that are not relevant to your environment or for which you do

720

not need to override the default values, you can leave them commented out,

721

as they are in the initial file.</para>

722

723

<para>The configuration file allows regular expression description

724

syntax.</para>

725

726

<para>An annotated version of the configuration file can be found in

727

<filename><filename>/usr/share/doc/multipath-tools/examples/multipath.conf.annotated.gz</filename></filename>.</para>

728

729

730

<title id="config-overview-title">Configuration File Overview</title>

731

732

<para>The multipath configuration file is divided into the following

733

sections:</para>

734

735

736

737

<term><emphasis role="bold">blacklist</emphasis></term>

738

739

740

<para>Listing of specific devices that will not be considered for

741

multipath.</para>

742

</listitem>

743

</varlistentry>

744

745

746

<term><emphasis role="bold">blacklist_exceptions</emphasis></term>

747

748

749

<para>Listing of multipath candidates that would otherwise be

750

blacklisted according to the parameters of the blacklist

751

section.</para>

752

</listitem>

753

</varlistentry>

754

755

756

<term><emphasis role="bold">defaults</emphasis></term>

757

758

759

<para>General default settings for DM-Multipath.</para>

760

</listitem>

761

</varlistentry>

762

763

764

<term><emphasis role="bold">multipath</emphasis></term>

765

766

767

<para>Settings for the characteristics of individual multipath

768

devices. These values overwrite what is specified in the <emphasis

769

role="bold">defaults</emphasis> and <emphasis

770

role="bold">devices</emphasis> sections of the configuration

771

file.</para>

772

</listitem>

773

</varlistentry>

774

775

776

<term><emphasis role="bold">devices</emphasis></term>

777

778

779

<para>Settings for the individual storage controllers. These

780

values overwrite what is specified in the <emphasis

781

role="bold">defaults</emphasis> section of the configuration file.

782

If you are using a storage array that is not supported by default,

783

you may need to create a devices subsection for your array.</para>

784

</listitem>

785

</varlistentry>

786

</variablelist>

787

788

<para>When the system determines the attributes of a multipath device,

789

first it checks the multipath settings, then the per devices settings,

790

then the multipath system defaults.</para>

791

</sect2>

792

793

794

<title id="config-blacklist-title">Configuration File Blacklist</title>

795

796

<para>The blacklist section of the multipath configuration file

797

specifies the devices that will not be used when the system configures

798

multipath devices. Devices that are blacklisted will not be grouped into

799

a multipath device.</para>

800

801

802

803

<para>If you do need to blacklist devices, you can do so according

804

to the following criteria:</para>

805

806

807

808

<para>By WWID, as described <xref

809

endterm="config-blacklist-by-wwid-title"

810

linkend="multipath-config-blacklist-by-wwid"/></para>

811

</listitem>

812

813

814

<para>By device name, as described in <xref

815

endterm="config-blacklist-by-device-name-title"

816

linkend="multipath-config-blacklist-by-device-name"/></para>

817

</listitem>

818

819

820

<para>By device type, as described in <xref

821

endterm="config-blacklist-by-device-type-title"

822

linkend="multipath-config-blacklist-by-device-type"/></para>

823

</listitem>

824

</itemizedlist>

825

826

<para>By default, a variety of device types are blacklisted, even

827

after you comment out the initial blacklist section of the

828

configuration file. For information, see <xref

829

endterm="config-blacklist-by-device-name-title"

830

linkend="multipath-config-blacklist-by-device-name"/></para>

831

</listitem>

832

</itemizedlist>

833

834

835

<title id="config-blacklist-by-wwid-title">Blacklisting By

836

WWID</title>

837

838

<para>You can specify individual devices to blacklist by their

839

World-Wide IDentification with a <emphasis role="bold">wwid</emphasis>

840

entry in the <emphasis role="bold">blacklist</emphasis> section of the

841

configuration file.</para>

842

843

<para>The following example shows the lines in the configuration file

844

that would blacklist a device with a WWID of 26353900f02796769.</para>

845

846

847

blacklist {

848

wwid 26353900f02796769

849

}

850

</screen>

851

</sect3>

852

853

854

<title id="config-blacklist-by-device-name-title">Blacklisting By

855

Device Name</title>

856

857

<para>You can blacklist device types by device name so that they will

858

not be grouped into a multipath device by specifying a <emphasis

859

role="bold">devnode</emphasis> entry in the <emphasis

860

role="bold">blacklist</emphasis> section of the configuration

861

file.</para>

862

863

<para>The following example shows the lines in the configuration file

864

that would blacklist all SCSI devices, since it blacklists all sd*

865

devices. <screen>

866

blacklist {

867

devnode "^sd[a-z]"

868

}</screen></para>

869

870

<para>You can use a <emphasis role="bold">devnode</emphasis> entry in

871

the <emphasis role="bold">blacklist</emphasis> section of the

872

configuration file to specify individual devices to blacklist rather

873

than all devices of a specific type. This is not recommended, however,

874

since unless it is statically mapped by udev rules, there is no

875

guarantee that a specific device will have the same name on reboot.

876

For example, a device name could change from

877

<filename>/dev/sda</filename> to <filename>/dev/sdb</filename> on

878

reboot.</para>

879

880

<para>By default, the following <emphasis

881

role="bold">devnode</emphasis> entries are compiled in the default

882

blacklist; the devices that these entries blacklist do not generally

883

support DM-Multipath. To enable multipathing on any of these devices,

884

you would need to specify them in the <emphasis

885

role="bold">blacklist_exceptions</emphasis> section of the

886

configuration file, as described in <xref

887

endterm="config-blacklist-exceptions-title"

888

linkend="multipath-config-blacklist-exceptions"/> <screen>

889

blacklist {

890

devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"

891

devnode "^hd[a-z]"

892

}

893

</screen></para>

894

</sect3>

895

896

897

<title id="config-blacklist-by-device-type-title">Blacklisting By

898

Device Type</title>

899

900

<para>You can specify specific device types in the <emphasis

901

role="bold">blacklist</emphasis> section of the configuration file

902

with a device section. The following example blacklists all IBM DS4200

903

and HP devices. <screen>

904

blacklist {

905

device {

906

vendor "IBM"

907

product "3S42" #DS4200 Product 10

908

}

909

device {

910

vendor "HP"

911

product "*"

912

}

913

}

914

</screen></para>

915

</sect3>

916

917

918

<title id="config-blacklist-exceptions-title">Blacklist

919

Exceptions</title>

920

921

<para>You can use the <emphasis

922

role="bold">blacklist_exceptions</emphasis> section of the

923

configuration file to enable multipathing on devices that have been

924

blacklisted by default.</para>

925

926

<para>For example, if you have a large number of devices and want to

927

multipath only one of them (with the WWID of

928

3600d0230000000000e13955cc3757803), instead of individually

929

blacklisting each of the devices except the one you want, you could

930

instead blacklist all of them, and then allow only the one you want by

931

adding the following lines to the &multipath-conf-full; file. <screen>

932

blacklist {

933

wwid "*"

934

}

935

936

blacklist_exceptions {

937

wwid "3600d0230000000000e13955cc3757803"

938

}

939

</screen></para>

940

941

<para>When specifying devices in the <emphasis

942

role="bold">blacklist_exceptions</emphasis> section of the

943

configuration file, you must specify the exceptions in the same way

944

they were specified in the <emphasis role="bold">blacklist</emphasis>.

945

For example, a WWID exception will not apply to devices specified by a

946

<emphasis role="bold">devnode</emphasis> blacklist entry, even if the

947

blacklisted device is associated with that WWID. Similarly, devnode

948

exceptions apply only to devnode entries, and device exceptions apply

949

only to device entries.</para>

950

</sect3>

951

</sect2>

952

953

954

<title id="config-defaults-title">Configuration File Defaults</title>

955

956

<para>The &multipath-conf-full; configuration file includes a <emphasis

957

role="bold">defaults</emphasis> section that sets the <emphasis

958

role="bold">user_friendly_names</emphasis> parameter to <emphasis

959

role="bold">yes</emphasis>, as follows.</para>

960

961

962

defaults {

963

user_friendly_names yes

964

}

965

</screen>

966

967

<para>This overwrites the default value of the <emphasis

968

role="bold">user_friendly_names</emphasis> parameter.</para>

969

970

<para>The configuration file includes a template of configuration

971

defaults. This section is commented out, as follows.</para>

972

973

974

#defaults {

975

# udev_dir /dev

976

# polling_interval 5

977

# selector "round-robin 0"

978

# path_grouping_policy failover

979

# getuid_callout "/lib/dev/scsi_id --whitelisted --device=/dev/%n"

980

# prio const

981

# path_checker directio

982

# rr_min_io 1000

983

# rr_weight uniform

984

# failback manual

985

# no_path_retry fail

986

# user_friendly_names no

987

#}

988

</screen>

989

990

<para>To overwrite the default value for any of the configuration

991

parameters, you can copy the relevant line from this template into the

992

<emphasis role="bold">defaults</emphasis> section and uncomment it. For

993

example, to overwrite the <emphasis

994

role="bold">path_grouping_policy</emphasis> parameter so that it is

995

<emphasis role="bold">multibus</emphasis> rather than the default value

996

of <emphasis role="bold">failover</emphasis>, copy the appropriate line

997

from the template to the initial <emphasis

998

role="bold">defaults</emphasis> section of the configuration file, and

999

uncomment it, as follows.</para>

1000

1001

1002

defaults {

1003

user_friendly_names yes

1004

path_grouping_policy multibus

1005

}

1006

</screen>

1007

1008

<para>Table <xref endterm="config-defaults-table-title"

1009

linkend="multipath-config-defaults-table"/> describes the attributes

1010

that are set in the <emphasis role="bold">defaults</emphasis> section of

1011

the <filename>multipath.conf</filename> configuration file. These values

1012

are used by DM-Multipath unless they are overwritten by the attributes

1013

specified in the <emphasis role="bold">devices</emphasis> and <emphasis

1014

role="bold">multipaths</emphasis> sections of the &multipath-conf;

1015

file.</para>

1016

1017

<xi:include href="multipath-config-defaults-table.xml"

1018

xmlns:xi="http://www.w3.org/2001/XInclude"/>

1019

</sect2>

1020

1021

1022

<title id="config-multipath-title">Configuration File Multipath

1023

Attributes</title>

1024

1025

<para>Table <xref endterm="attributes-table-title"

1026

linkend="multipath-attributes-table"/> shows the attributes that you can

1027

set in the <emphasis role="bold">multipaths</emphasis> section of the

1028

<filename>multipath.conf</filename> configuration file for each specific

1029

multipath device. These attributes apply only to the one specified

1030

multipath. These defaults are used by DM-Multipath and override

1031

attributes set in the <emphasis role="bold">defaults</emphasis> and

1032

<emphasis role="bold">devices</emphasis> sections of the multipath.conf

1033

file.</para>

1034

1035

<xi:include href="multipath-attributes-table.xml"

1036

xmlns:xi="http://www.w3.org/2001/XInclude"/>

1037

1038

<para>In addition, the following parameters may be overridden in this

1039

<emphasis role="bold">multipath</emphasis> section</para>

1040

1041

1042

1043

1044

<parameter>path_grouping_policy</parameter> </link></para>

1045

</listitem>

1046

1047

1048

1049

<parameter>path_selector</parameter> </link></para>

1050

</listitem>

1051

1052

1053

1054

<parameter>failback</parameter> </link></para>

1055

</listitem>

1056

1057

1058

1059

</link></para>

1060

</listitem>

1061

1062

1063

1064

</listitem>

1065

1066

1067

1068

<parameter>no_path_retry</parameter> </link></para>

1069

</listitem>

1070

1071

1072

1073

1074

</listitem>

1075

1076

1077

1078

<parameter>rr_weight</parameter> </link></para>

1079

</listitem>

1080

1081

1082

1083

<parameter>flush_on_last_del</parameter> </link></para>

1084

</listitem>

1085

</itemizedlist>

1086

1087

<para>The following example shows multipath attributes specified in the

1088

configuration file for two specific multipath devices. The first device

1089

has a WWID of 3600508b4000156d70001200000b0000 and a symbolic name of

1090

yellow.</para>

1091

1092

<para>The second multipath device in the example has a WWID of

1093

1DEC_____321816758474 and a symbolic name of red. In this example, the

1094

1095

attributes is set to priorities.</para>

1096

1097

1098

multipaths {

1099

multipath {

1100

wwid 3600508b4000156d70001200000b0000

1101

alias yellow

1102

path_grouping_policy multibus

1103

path_selector "round-robin 0"

1104

failback manual

1105

rr_weight priorities

1106

no_path_retry 5

1107

}

1108

multipath {

1109

wwid 1DEC_____321816758474

1110

alias red

1111

rr_weight priorities

1112

}

1113

}

1114

</screen>

1115

</sect2>

1116

1117

1118

<title id="config-device-title">Configuration File Devices</title>

1119

1120

<para>Table <xref endterm="device-attributes-table-title"

1121

linkend="multipath-device-attributes-table"/> shows the attributes that

1122

you can set for each individual storage device in the devices section of

1123

the multipath.conf configuration file. These attributes are used by

1124

DM-Multipath unless they are overwritten by the attributes specified in

1125

the <emphasis role="bold">multipaths</emphasis> section of the

1126

<filename>multipath.conf</filename> file for paths that contain the

1127

device. These attributes override the attributes set in the <emphasis

1128

role="bold">defaults</emphasis> section of the

1129

<filename>multipath.conf</filename> file.</para>

1130

1131

<para>Many devices that support multipathing are included by default in

1132

a multipath configuration. The values for the devices that are supported

1133

by default are listed in the

1134

<filename>multipath.conf.defaults</filename> file. You probably will not

1135

need to modify the values for these devices, but if you do you can

1136

overwrite the default values by including an entry in the configuration

1137

file for the device that overwrites those values. You can copy the

1138

device configuration defaults from the

1139

<filename>multipath.conf.annotated.gz</filename> or if you wish to have

1140

a brief config file, <filename>multipath.conf.synthetic</filename> file

1141

for the device and override the values that you want to change.</para>

1142

1143

<para>To add a device to this section of the configuration file that is

1144

not configured automatically by default, you must set the <emphasis

1145

role="bold">vendor</emphasis> and <emphasis

1146

role="bold">product</emphasis> parameters. You can find these values by

1147

looking at <emphasis

1148

role="bold">/sys/block/device_name/device/vendor</emphasis> and

1149

<emphasis role="bold">/sys/block/device_name/device/model</emphasis>

1150

where device_name is the device to be multipathed, as in the following

1151

example:</para>

1152

1153

1154

# cat /sys/block/sda/device/vendor

1155

WINSYS

1156

# cat /sys/block/sda/device/model

1157

SF2372

1158

</screen>

1159

1160

<para>The additional parameters to specify depend on your specific

1161

device. If the device is active/active, you will usually not need to set

1162

additional parameters. You may want to set <link

1163

linkend="attribute-path_grouping_policy">path_grouping_policy</link> to

1164

<emphasis role="bold">multibus</emphasis>. Other parameters you may need

1165

to set are <link linkend="attribute-no_path_retry">no_path_retry</link>

1166

and <link linkend="attribute-rr_min_io">rr_min_io</link>, as described

1167

in Table <xref endterm="attributes-table-title"

1168

linkend="multipath-attributes-table"/>.</para>

1169

1170

<para>If the device is active/passive, but it automatically switches

1171

paths with I/O to the passive path, you need to change the checker

1172

function to one that does not send I/O to the path to test if it is

1173

working (otherwise, your device will keep failing over). This almost

1174

always means that you set the <link

1175

linkend="attribute-path_checker">path_checker</link> to <emphasis

1176

role="bold">tur</emphasis>; this works for all SCSI devices that support

1177

the Test Unit Ready command, which most do.</para>

1178

1179

<para>If the device needs a special command to switch paths, then

1180

configuring this device for multipath requires a hardware handler kernel

1181

module. The current available hardware handler is emc. If this is not

1182

sufficient for your device, you may not be able to configure the device

1183

for multipath.</para>

1184

1185

<xi:include href="multipath-device-attributes-table.xml"

1186

xmlns:xi="http://www.w3.org/2001/XInclude"/>

1187

1188

<para>In addition, the following parameters may be overridden in this

1189

<emphasis role="bold">device</emphasis> section</para>

1190

1191

1192

1193

1194

<parameter>path_grouping_policy</parameter> </link></para>

1195

</listitem>

1196

1197

1198

1199

<parameter>getuid_callout</parameter> </link></para>

1200

</listitem>

1201

1202

1203

1204

<parameter>path_selector</parameter> </link></para>

1205

</listitem>

1206

1207

1208

1209

<parameter>path_checker</parameter> </link></para>

1210

</listitem>

1211

1212

1213

1214

<parameter>features</parameter> </link></para>

1215

</listitem>

1216

1217

1218

1219

<parameter>failback</parameter> </link></para>

1220

</listitem>

1221

1222

1223

1224

</link></para>

1225

</listitem>

1226

1227

1228

1229

</listitem>

1230

1231

1232

1233

<parameter>no_path_retry</parameter> </link></para>

1234

</listitem>

1235

1236

1237

1238

1239

</listitem>

1240

1241

1242

1243

<parameter>rr_weight</parameter> </link></para>

1244

</listitem>

1245

1246

1247

1248

1249

</listitem>

1250

1251

1252

1253

1254

</listitem>

1255

1256

1257

1258

<parameter>flush_on_last_del</parameter> </link></para>

1259

</listitem>

1260

</itemizedlist>

1261

1262

<note>

1263

<para>Whenever a hardware_handler is specified, it is your

1264

responsibility to ensure that the appropriate kernel module is loaded

1265

to support the specified interface. These modules can be found in

1266

<emphasis role="bold"><filename>/lib/modules/`uname

1267

-r`/kernel/drivers/scsi/device_handler/ </filename></emphasis>. The

1268

requisite module should be integrated into the initrd to ensure the

1269

necessary discovery and failover-failback capacity is available during

1270

boot time. Example,<screen># cat scsi_dh_alua >> /etc/initramfs-tools/modules ## append module to file

1271

# update-initramfs -u -k all</screen></para>

1272

</note>

1273

1274

<para>The following example shows a device entry in the multipath

1275

configuration file.</para>

1276

1277

1278

1279

#devices {

1280

# device {

1281

# vendor "COMPAQ "

1282

# product "MSA1000 "

1283

# path_grouping_policy multibus

1284

# path_checker tur

1285

# rr_weight priorities

1286

# }

1287

#}

1288

</screen>

1289

1290

<para>The spacing reserved in the <emphasis

1291

role="bold">vendor</emphasis>, <emphasis role="bold">product</emphasis>,

1292

and <emphasis role="bold">revision</emphasis> fields are significant as

1293

multipath is performing a direct match against these attributes, whose

1294

format is defined by the SCSI specification, specifically the <ulink

1295

url="http://en.wikipedia.org/wiki/SCSI_Inquiry_Command">Standard

1296

INQUIRY</ulink> command. When quotes are used, the vendor, product, and

1297

revision fields will be interpreted strictly according to the spec.

1298

Regular expressions may be integrated into the quoted strings. Should a

1299

field be defined without the requisite spacing, multipath will copy the

1300

string into the properly sized buffer and pad with the appropriate

1301

number of spaces. The specification expects the entire field to be

1302

populated by printable characters or spaces, as seen in the example

1303

above</para>

1304

1305

1306

1307

<para>vendor: 8 characters</para>

1308

</listitem>

1309

1310

1311

<para>product: 16 characters</para>

1312

</listitem>

1313

1314

1315

<para>revision: 4 characters</para>

1316

</listitem>

1317

</itemizedlist>

1318

1319

<para>To create a more robust configuration file, regular expressions

1320

can also be used. Operators include <emphasis role="bold">^ $ [ ] . * ?

1321

+</emphasis>. Examples of functional regular expressions can be found by

1322

examining the live multipath database and <filename>multipath.conf

1323

</filename>example files found in

1324

<filename>/usr/share/doc/multipath-tools/examples:</filename></para>

1325

1326

<para><screen># echo 'show config' | multipathd -k</screen></para>

1327

</sect2>

1328

</sect1>

1329

1330

1331

<title>&DMM; Administration and Troubleshooting</title>

1332

1333

1334

<title>Resizing an Online Multipath Device</title>

1335

1336

<para>If you need to resize an online multipath device, use the

1337

following procedure</para>

1338

1339

1340

<step>

1341

<para>Resize your physical device. This is storage platform

1342

specific.</para>

1343

</step>

1344

1345

<step>

1346

<para>Use the following command to find the paths to the LUN:</para>

1347

1348

<screen># multipath -l</screen>

1349

</step>

1350

1351

<step>

1352

<para>Resize your paths. For SCSI devices, writing 1 to the

1353

<filename>rescan</filename> file for the device causes the SCSI

1354

driver to rescan, as in the following command:</para>

1355

1356

<screen># echo 1 > /sys/block/device_name/device/rescan</screen>

1357

</step>

1358

1359

<step>

1360

<para>Resize your multipath device by running the multipathd resize

1361

command:</para>

1362

1363

<screen># multipathd -k 'resize map mpatha'</screen>

1364

</step>

1365

1366

<step>

1367

<para>Resize the file system (assuming no LVM or DOS partitions are

1368

used):</para>

1369

1370

<screen># resize2fs /dev/mapper/mpatha</screen>

1371

</step>

1372

</procedure>

1373

</sect2>

1374

1375

1376

<title>Moving root File Systems from a Single Path Device to a Multipath

1377

Device</title>

1378

1379

<para>This is dramatically simplified by the use of UUIDs to identify

1380

devices as an intrinsic label. Simply install <emphasis

1381

role="bold">multipath-tools-boot</emphasis> and reboot. This will

1382

rebuild the initial ramdisk and afford multipath the opportunity to

1383

build it's paths before the root file system is mounted by UUID.</para>

1384

1385

<note>

1386

<para>Whenever <filename>multipath.conf</filename> is updated, so

1387

should the initrd by executing <command>update-initramfs -u -k

1388

all</command>. The reason being is <filename>multipath.conf</filename>

1389

is copied to the ramdisk and is integral to determining the available

1390

devices for grouping via it's blacklist and device sections.</para>

1391

</note>

1392

</sect2>

1393

1394

1395

<title>Moving swap File Systems from a Single Path Device to a Multipath

1396

Device</title>

1397

1398

<para>The procedure is exactly the same as illustrated in the previous

1399

section called <link

1400

linkend="multipath-moving-rootfs-from-single-path-to-multipath-device">Moving

1401

root File Systems from a Single Path to a Multipath

1402

Device</link>.</para>

1403

</sect2>

1404

1405

1406

<title>The Multipath Daemon</title>

1407

1408

<para>If you find you have trouble implementing a multipath

1409

configuration, you should ensure the multipath daemon is running as

1410

described in <link linkend="multipath-setting-up-dm-multipath">"Setting

1411

up DM-Multipath"</link>. The &multipathd_c; daemon must be running in

1412

order to use multipathd devices. Also see section <link

1413

linkend="multipath-interacting-with-multipathd">Troubleshooting with the

1414

multipathd interactive console</link> concerning interacting with

1415

&multipathd_c; as a debugging aid.</para>

1416

</sect2>

1417

1418

<!--Removing this whole section for now as I think we depend

1419

on the udev scripts for simple discovery in the initramfs

1420

so by following these suggestions you would not get any mp

1421

devices on boot. We'll need a new solution for our infrastructure -->

1422

1423

<!--

1424

1425

<title>Issues with Large Number of LUNs</title>

1426

1427

<para>When a large number of LUNs are added to a node, using multipathed

1428

devices can significantly increase the time it takes for the &udev_c;

1429

device manager to create device nodes for them. If you experience this

1430

problem, you can correct it by deleting the following line in

1431

<filename>/etc/udev/rules.d/40-multipath.rules</filename>: <emphasis

1432

role="bold">XXX VERIFY!</emphasis></para>

1433

1434

<screen>KERNEL!="dm-[0-9]*", ACTION=="add", PROGRAM=="/bin/bash -c '/sbin/lsmod | /bin/grep ^dm_multipath'", RUN+="/sbin/multipath -v0 %M:%m"</screen>

1435

1436

<para>This line causes the &udev_c; device manager to run &multipath_c;

1437

every time a block device is added to the node. Even with this line

1438

removed, the &multipathd_c; daemon will still automatically create

1439

multipathed devices, and &multipath_c; will still be called during the

1440

boot process for nodes with mutipathed root file systems. The only

1441

change is that multipathed devices will not be automatically created

1442

when the multipathd daemon is not running, which should not be a problem

1443

for the vast majority of multipath user.</para>

1444

</sect2>

1445

-->

1446

1447

1448

<title>Issues with queue_if_no_path</title>

1449

1450

<para>If <emphasis role="bold">features "1 queue_if_no_path"</emphasis>

1451

is specified in the <filename>/etc/multipath.conf</filename> file, then

1452

any process that uses I/O will hang until one or more paths are

1453

restored. To avoid this, set the <emphasis role="bold"><link

1454

linkend="attribute-no_path_retry">no_path_retry</link> N</emphasis>

1455

parameter in the <filename>/etc/multipath.conf</filename>.</para>

1456

1457

<para>When you set the <emphasis role="bold">no_path_retry</emphasis>

1458

parameter, remove the <emphasis role="bold">features "1

1459

queue_if_no_path"</emphasis> option from the

1460

<filename>/etc/multipath.conf</filename> file as well. If, however, you

1461

are using a multipathed device for which the <option>features "1

1462

queue_if_no_path"</option> option is set as a compiled in default, as it

1463

is for many SAN devices, you must add <option>features "0"</option> to

1464

override this default. You can do this by copying the existing <emphasis

1465

role="bold">devices</emphasis> section, and just that section (not the

1466

entire file), from

1467

<filename>/usr/share/doc/multipath-tools/examples/multipath.conf.annotated.gz</filename>

1468

into <filename>/etc/multipath.conf</filename> and editing to suit your

1469

needs.</para>

1470

1471

<para>If you need to use the <option>features "1

1472

queue_if_no_path"</option> option and you experience the issue noted

1473

here, use the <command>dmsetup</command> command to edit the policy at

1474

runtime for a particular LUN (that is, for which all the paths are

1475

unavailable). For example, if you want to change the policy on the

1476

multipath device <filename>mpathc</filename> from

1477

<option>"queue_if_no_path"</option> to <option>

1478

"fail_if_no_path"</option>, execute the following command.</para>

1479

1480

<screen># dmsetup message mpathc 0 "fail_if_no_path"</screen>

1481

1482

<note>

1483

<para>You must specify the <filename>mpathN</filename> alias rather

1484

than the path</para>

1485

</note>

1486

</sect2>

1487

1488

1489

<title>Multipath Command Output</title>

1490

1491

<para>When you create, modify, or list a multipath device, you get a

1492

printout of the current device setup. The format is as follows. For each

1493

multipath device:</para>

1494

1495

<screen> action_if_any: alias (wwid_if_different_from_alias) dm_device_name_if_known vendor,product

1496

size=size features='features' hwhandler='hardware_handler' wp=write_permission_if_known</screen>

1497

1498

<para>For each path group:</para>

1499

1500

<screen> -+- policy='scheduling_policy' prio=prio_if_known

1501

status=path_group_status_if_known</screen>

1502

1503

1504

1505

<screen> `- host:channel:id:lun devnode major:minor dm_status_if_known path_status

1506

online_status</screen>

1507

1508

<para>For example, the output of a multipath command might appear as

1509

follows:</para>

1510

1511

<screen> 3600d0230000000000e13955cc3757800 dm-1 WINSYS,SF2372

1512

size=269G features='0' hwhandler='0' wp=rw

1513

|-+- policy='round-robin 0' prio=1 status=active

1514

| `- 6:0:0:0 sdb 8:16 active ready running

1515

`-+- policy='round-robin 0' prio=1 status=enabled

1516

`- 7:0:0:0 sdf 8:80 active ready running</screen>

1517

1518

<para>If the path is up and ready for I/O, the status of the path is

1519

<emphasis role="bold">ready</emphasis> or <emphasis

1520

role="emphasis">ghost</emphasis>. If the path is down, the status is

1521

<emphasis role="bold">faulty</emphasis> or <emphasis

1522

role="bold">shaky</emphasis>. The path status is updated periodically by

1523

the &multipathd_c; daemon based on the polling interval defined in the

1524

<filename>/etc/multipath.conf</filename> file.</para>

1525

1526

<para>The dm status is similar to the path status, but from the kernel's

1527

point of view. The dm status has two states: <emphasis

1528

role="bold">failed</emphasis>, which is analogous to <emphasis

1529

role="bold">faulty</emphasis>, and <emphasis

1530

role="bold">active</emphasis> which covers all other path states.

1531

Occasionally, the path state and the dm state of a device will

1532

temporarily not agree.</para>

1533

1534

<para>The possible values for <emphasis

1535

role="bold">online_status</emphasis> are <emphasis

1536

role="bold">running</emphasis> and <emphasis

1537

role="bold">offline</emphasis>. A status of <emphasis

1538

role="emphasis">offline</emphasis> means that the SCSI device has been

1539

disabled.</para>

1540

1541

<note>

1542

<para>When a multipath device is being created or modified , the path

1543

group status, the dm device name, the write permissions, and the dm

1544

status are not known. Also, the features are not always correct</para>

1545

</note>

1546

</sect2>

1547

1548

1549

<title>Multipath Queries with multipath Command</title>

1550

1551

<para>You can use the <emphasis role="bold">-l </emphasis>and <emphasis

1552

role="bold">-ll</emphasis> options of the <emphasis

1553

role="bold">multipath</emphasis> command to display the current

1554

multipath configuration. The <emphasis role="bold">-l</emphasis> option

1555

displays multipath topology gathered from information in sysfs and the

1556

device mapper. The <emphasis role="bold">-ll</emphasis> option displays

1557

the information the <emphasis role="bold">-l</emphasis> displays in

1558

addition to all other available components of the system.</para>

1559

1560

<para>When displaying the multipath configuration, there are three

1561

verbosity levels you can specify with the <emphasis

1562

role="bold">-v</emphasis> option of the multipath command. Specifying

1563

<emphasis role="bold">-v0</emphasis> yields no output.

1564

Specifying<emphasis role="bold"> -v1</emphasis> outputs the created or

1565

updated multipath names only, which you can then feed to other tools

1566

such as kpartx. Specifying <emphasis role="bold">-v2</emphasis> prints

1567

all detected paths, multipaths, and device maps.</para>

1568

1569

<note>

1570

<para>The default <emphasis role="bold">verbosity</emphasis> level of

1571

multipath is <emphasis role="bold">2</emphasis> and can be globally

1572

modified by defining the <link linkend="attribute-verbosity">verbosity

1573

attribute</link> in the <emphasis role="bold">defaults</emphasis>

1574

section of <filename>multipath.conf</filename>.</para>

1575

</note>

1576

1577

<para>The following example shows the output of a <emphasis

1578

role="bold">multipath -l</emphasis> command.</para>

1579

1580

<screen># multipath -l

1581

3600d0230000000000e13955cc3757800 dm-1 WINSYS,SF2372

1582

size=269G features='0' hwhandler='0' wp=rw

1583

|-+- policy='round-robin 0' prio=1 status=active

1584

| `- 6:0:0:0 sdb 8:16 active ready running

1585

`-+- policy='round-robin 0' prio=1 status=enabled

1586

`- 7:0:0:0 sdf 8:80 active ready running</screen>

1587

1588

<para>The following example shows the output of a <emphasis

1589

role="bold">multipath -ll</emphasis> command.</para>

1590

1591

<screen># multipath -ll

1592

3600d0230000000000e13955cc3757801 dm-10 WINSYS,SF2372

1593

size=269G features='0' hwhandler='0' wp=rw

1594

|-+- policy='round-robin 0' prio=1 status=enabled

1595

| `- 19:0:0:1 sdc 8:32 active ready running

1596

`-+- policy='round-robin 0' prio=1 status=enabled

1597

`- 18:0:0:1 sdh 8:112 active ready running

1598

3600d0230000000000e13955cc3757803 dm-2 WINSYS,SF2372

1599

size=125G features='0' hwhandler='0' wp=rw

1600

`-+- policy='round-robin 0' prio=1 status=active

1601

|- 19:0:0:3 sde 8:64 active ready running

1602

`- 18:0:0:3 sdj 8:144 active ready running</screen>

1603

</sect2>

1604

1605

1606

<title>Multipath Command Options</title>

1607

1608

<para>Table <link linkend="useful-multipath-command-options">"Useful

1609

multipath Command Options"</link> describes some options of the

1610

<emphasis role="bold">multipath</emphasis> command that you find

1611

useful</para>

1612

1613

<table>

1614

<title id="useful-multipath-command-options">Useful multipath Command

1615

Options</title>

1616

1617

1618

1619

1620

1621

<thead>

1622

<row>

1623

<entry align="left">Option</entry>

1624

<entry align="left">Description</entry>

1625

</row>

1626

</thead>

1627

1628

<tbody>

1629

<row>

1630

1631

<entry>Display the current multipath configuration gathered from

1632

<emphasis role="bold">sysfs</emphasis> and the device

1633

mapper.</entry>

1634

</row>

1635

1636

<row>

1637

1638

<entry>Display the current multipath configuration gathered from

1639

<emphasis role="bold">sysfs</emphasis>, the device mapper, and

1640

all other available components on the system.</entry>

1641

</row>

1642

1643

<row>

1644

<entry align="left"><emphasis role="bold">-f device</emphasis></entry>

1645

<entry>Remove the named multipath device.</entry>

1646

</row>

1647

1648

<row>

1649

1650

<entry>Remove all unused multipath devices.</entry>

1651

</row>

1652

</tbody>

1653

</tgroup>

1654

</table>

1655

</sect2>

1656

1657

1658

<title>Determining Device Mapper Entries with dmsetup Command</title>

1659

1660

<para>You can use the <emphasis role="bold">dmsetup</emphasis> command

1661

to find out which device mapper entries match the <emphasis

1662

role="bold">multipathed</emphasis> devices.</para>

1663

1664

<para>The following command displays all the device mapper devices and

1665

their major and minor numbers. The minor numbers determine the name of

1666

the dm device. For example, a minor number of <emphasis

1667

role="bold">3</emphasis> corresponds to the multipathed device <emphasis

1668

role="bold"><filename>/dev/dm-3</filename></emphasis>.</para>

1669

1670

1671

# dmsetup ls

1672

mpathd (253, 4)

1673

mpathep1 (253, 12)

1674

mpathfp1 (253, 11)

1675

mpathb (253, 3)

1676

mpathgp1 (253, 14)

1677

mpathhp1 (253, 13)

1678

mpatha (253, 2)

1679

mpathh (253, 9)

1680

mpathg (253, 8)

1681

VolGroup00-LogVol01 (253, 1)

1682

mpathf (253, 7)

1683

VolGroup00-LogVol00 (253, 0)

1684

mpathe (253, 6)

1685

mpathbp1 (253, 10)

1686

mpathd (253, 5)

1687

</screen>

1688

</sect2>

1689

1690

1691

<title>Troubleshooting with the multipathd interactive console</title>

1692

1693

<para>The <emphasis role="bold">multipathd -k</emphasis> command is an

1694

interactive interface to the <emphasis role="bold">multipathd</emphasis>

1695

daemon. Entering this command brings up an interactive multipath

1696

console. After entering this command, you can enter help to get a list

1697

of available commands, you can enter a interactive command, or you can

1698

enter <emphasis role="bold">CTRL-D</emphasis> to quit.</para>

1699

1700

<para>The multipathd interactive console can be used to troubleshoot

1701

problems you may be having with your system. For example, the following

1702

command sequence displays the multipath configuration, including the

1703

defaults, before exiting the console. See the IBM article <ulink

1704

url="http://www-01.ibm.com/support/docview.wss?uid=isg3T1011985">"Tricks

1705

with Multipathd"</ulink> for more examples.</para>

1706

1707

<screen># multipathd -k

1708

> > show config

1709

> > CTRL-D</screen>

1710

1711

<para>The following command sequence ensures that multipath has picked

1712

up any changes to the multipath.conf,</para>

1713

1714

1715

# multipathd -k

1716

> > reconfigure

1717

> > CTRL-D

1718

</screen>

1719

1720

<para>Use the following command sequence to ensure that the path checker

1721

is working properly.</para>

1722

1723

1724

# multipathd -k

1725

> > show paths

1726

> > CTRL-D

1727

</screen>

1728

1729

<para>Commands can also be streamed into multipathd using stdin like

1730

so:<screen># echo 'show config' | multipathd -k</screen></para>

1731

</sect2>

1732

</sect1>

1733

1734

1735

</chapter>

1736

<!--

1737

# vim: set ts=2 sw=2 expandtab:

1738

-->