~ubuntu-branches/ubuntu/oneiric/nmap/oneiric : revision 17

1

2

<title>Introduction</title>

3

4

<indexterm><primary>Nmap Scripting Engine</primary></indexterm>

5

<para>The Nmap Scripting Engine (NSE) is one of Nmap's most

6

powerful and flexible features. It allows users to write (and

7

share) simple scripts to automate a wide variety of networking

8

tasks. Those scripts are then executed in parallel with the speed

9

and efficiency you expect from Nmap. Users can rely on the

10

growing and diverse set of scripts distributed with Nmap, or write

11

their own to meet custom needs.</para>

12

13

<para>The Nmap project would like to thank Diman Todorov for

14

his excellent work building the initial NSE implementation and

15

writing much of this documentation. Stoiko Ivanov also

16

contributed greatly. The tasks we had in mind when

17

creating the system are:</para>

18

19

20

21

<term>Network discovery</term>

22

23

<para>This is Nmap's bread and butter. Examples include

24

looking up whois data based on the target domain,

25

querying ARIN, RIPE, or APNIC for the target IP to determine ownership,

26

performing identd lookups on open ports, SNMP queries, and

27

listing available NFS/SMB/RPC shares and services.</para>

28

</listitem>

29

30

</varlistentry>

31

32

<term>More sophisticated version detection</term>

33

34

<para>The Nmap version detection system (<xref linkend="vscan"/>)

35

is able to recognize thousands of different services through

36

its probe and regular expression based matching system, but it

37

cannot recognize everything. For example, identifying the Skype v2 service requires two

38

independent probes. Nmap could also recognize more SNMP services

39

if it tried a few hundred different community names by brute

40

force. Neither of these tasks are well suited to traditional

41

Nmap version detection, but both are easily accomplished with

42

NSE. For these reasons, version detection now calls NSE by

43

default to handle some tricky services. This is described in

44

45

46

</listitem>

47

</varlistentry>

48

49

<term>Vulnerability detection</term>

50

51

52

<para>When a new vulnerability is discovered, you often want

53

to scan your networks quickly to identify vulnerable systems

54

before the bad guys do. While Nmap isn't a

55

comprehensive

56

<web>

57

<ulink url="http://sectools.org/vuln-scanners.html">vulnerability scanner</ulink>,

58

</web><print>

59

vulnerability scanner,

60

</print>

61

62

we plan to distribute scripts for some very severe or common vulnerabilities and misconfigurations.</para>

63

64

</listitem>

65

</varlistentry>

66

67

<term>Backdoor detection</term>

68

69

<para>

70

Many attackers and some automated worms leave

71

backdoors to enable later reentry. Some of these can be

72

detected by Nmap's regular expression based version detection.

73

For example, within hours of the MyDoom worm hitting the

74

Internet, Jay Moran posted an Nmap version detection probe and

75

signature so that others could quickly scan their networks.

76

For more complex worms and backdoors, NSE is needed

77

instead.

78

</para>

79

</listitem>

80

</varlistentry>

81

82

<term>Vulnerability exploitation</term>

83

84

<para>

85

As a general scripting language, NSE could even

86

be used to exploit vulnerabilities rather than just find them.

87

The capability to add custom exploit scripts may be valuable

88

for some people (particularly penetration testers), though we aren't

89

planning to turn Nmap into an exploitation framework like

90

<ulink url="http://www.metasploit.com">Metasploit</ulink>.

91

</para>

92

</listitem>

93

</varlistentry>

94

</variablelist>

95

<para>

96

The listed items are just the initial script classes. It is

97

likely that Nmap users will come up with even more inventive

98

uses for NSE.

99

</para>

100

101

102

<para>

103

Scripts are written in the

104

embedded <ulink url="http://www.lua.org/">Lua programming language</ulink>.

105

The language itself is well documented in the books

106

<citetitle><ulink url="http://www.amazon.com/exec/obidos/ASIN/8590379825/secbks-20">Programming

107

in Lua, Second Edition</ulink></citetitle> and

108

<citetitle><ulink url="http://www.amazon.com/exec/obidos/ASIN/8590379825/secbks-20">Lua

109

5.1 Reference Manual</ulink></citetitle>. The reference manual is also

110

<ulink url="http://www.lua.org/manual/5.1/">freely available

111

online</ulink>, as is the

112

<ulink url="http://www.lua.org/pil/">first edition of Programming in

113

Lua</ulink>. Given the availability of these excellent general

114

Lua programming references, this document only covers aspects and

115

extensions specific to the Nmap implementation.

116

</para>

117

<para>

118

119

NSE is activated with the <option>-sC</option> option (or

120

<option>--script</option> if you wish to specify a custom set of

121

scripts) and results are integrated into Nmap normal and XML

122

output. Two types of scripts are supported: service and host

123

scripts. Service scripts relate to a certain open port

124

(service) on the target host, and any results they produce are included

125

next to that port in the Nmap output port table. Host scripts,

126

on the other hand, run no more than once against each target IP

127

and produce results below the port table. <xref

128

linkend="nse-ex1"/> shows a typical script scan. Examples of

129

service scripts producing output are <literal>Stealth SSH

130

Version</literal>, which tricks some SSH servers into divulging

131

version information without logging the attempt as they normally

132

would, <literal>Service Owner</literal>, which connects to open

133

ports, then performs a reverse-identd query to determine what

134

username it is running under, and <literal>HTML Title</literal>,

135

which simply grabs the title of the root path of any web servers

136

found. A sample host script is <literal>RIPE Query</literal>,

137

which looks up and reports target IP ownership information.

138

139

</para>

140

141

<title>Typical NSE Output</title>

142

143

$ ./nmap -sC localhost -p 22,23,80,113

144

145

Starting Nmap 4.20ALPHA9-NSE ( http://insecure.org )

146

Interesting ports on localhost (127.0.0.1):

147

PORT STATE SERVICE

148

22/tcp open ssh

149

|_ Stealth SSH version: SSH-1.99-OpenSSH_4.2

150

|_ SSH protocol version 1: Server supports SSHv1

151

23/tcp closed telnet

152

80/tcp open http

153

|_ HTML title:Test Page for Apache Installation

154

113/tcp closed auth

155

156

Host script results:

157

|_ RIPE Query: IP belongs to: Internet Assigned Numbers Authority

158

159

Nmap finished: 1 IP address (1 host up) scanned in 0.907 seconds

160

</screen>

161

</example>

162

163

</sect1>

164

165

<title>Usage and Examples</title>

166

<para>

167

While NSE has a complex implementation for efficiency, it is

168

strikingly easy to use. Simply specify <option>-sC</option> to

169

enable the most common scripts. Or specify the

170

<option>--script</option> option to choose your own scripts to

171

execute by providing categories, script file names, or the name of

172

directories full of scripts you wish to execute. You can customize

173

some scripts by providing arguments to them via the

174

<option>--script-args</option> option. The two

175

remaining options, <option>--script-trace</option> and

176

<option>--script-updatedb</option>, are generally only used for

177

script debugging and development.

178

</para>

179

180

<sect2 id="nse-categories"><title>Script Categories</title>

181

182

<para>NSE scripts define a list of categories they belong to.

183

Currently defined categories are <literal>safe</literal>,

184

<literal>intrusive</literal>, <literal>malware</literal>,

185

<literal>version</literal>, <literal>discovery</literal> and

186

<literal>vulnerability</literal>. By default, Nmap runs all

187

scripts in either the <literal>safe</literal> or

188

<literal>intrusive</literal> categories. Categories are not

189

case sensitive. The following list describes each category.</para>

190

191

192

193

<term>

194

195

</term>

196

197

<para>Scripts

198

which weren't designed to crash services, use large

199

amounts of network bandwidth or other resources, or

200

exploit security holes. These are less likely to offend

201

remote sysadmins. Of course (as with all other Nmap

202

features) we cannot guarantee that they won't ever cause

203

adverse reactions. Most of these perform general

204

network discovery. Examples are echoTest (sends a string

205

to the UDP echo service) and showHTMLTitle (grabs the

206

title from a web page).</para>

207

</listitem>

208

</varlistentry>

209

210

211

<term>

212

<option>intrusive</option>

213

</term>

214

215

<para>These are not intended to

216

crash or damage anything, but are more likely to leave

217

suspicious logs or otherwise arouse sysadmin ire. Scripts

218

which attempt to login to services with default passwords

219

fall into this class.</para>

220

</listitem>

221

</varlistentry>

222

223

224

<term>

225

<option>malware</option>

226

</term>

227

228

<para>These scripts test if the target platform is

229

infected by malware or backdoors.</para>

230

</listitem>

231

</varlistentry>

232

233

234

<term>

235

<option>version</option>

236

</term>

237

238

<para>This category cannot be selected explicitly. It is only

239

run if <option>-sV</option>

240

was supplied. The scripts in this category are an

241

extension to the version detection service. Their output

242

cannot be distinguished from version detection output

243

and they do not produce script scanning

244

output. </para>

245

</listitem>

246

</varlistentry>

247

248

249

<term>

250

<option>discovery</option>

251

</term>

252

253

<para>These scripts try to actively learn more about the

254

network by querying public registries, SNMP-enabled

255

devices, directory services, and the like.</para>

256

</listitem>

257

</varlistentry>

258

259

260

<term>

261

<option>vulnerability</option>

262

</term>

263

264

<para>These scripts check for a specific vulnerability and report results only if it is found.</para>

265

</listitem>

266

</varlistentry>

267

</variablelist>

268

269

</sect2>

270

271

<title>Arguments to Scripts</title>

272

<para>

273

You can pass arguments to NSE scripts via the

274

<option>--script-args</option> option. The script-arguments generally are

275

name-value pairs, which are provided to the script as a Lua table called

276

<literal>args</literal> inside the <literal><link

277

linkend="nse-api-registry">nmap.registry</link></literal> with

278

the names as keys for the corresponding values. The values can either be

279

strings or tables. Subtables can be used to pass arguments to

280

scripts with a finer granularity (e.g. pass different usernames for

281

different scripts). A typical nmap invocation with script arguments may

282

look like:

283

</para>

284

<para>

285

286

$ nmap -sC --script-args user=foo,pass=bar,anonFTP={pass=ftp@foobar.com}

287

</userinput>

288

</para>

289

<para>

290

which would result in the Lua table:

291

</para>

292

293

{user="foo",pass="bar",anonFTP={pass=nobody@foobar.com}}

294

</programlisting>

295

296

<para>You could therefore access the username (<literal>"foo"</literal>)

297

inside your script as <literal>local username= nmap.registry.args.user

298

</literal>. As a general rule the subtables used to override

299

options for scripts should be named as the script's

300

<literal>id</literal>, since otherwise scripts can't know where to

301

search for their arguments.

302

</para>

303

</sect2>

304

305

306

<title>Command-line Arguments</title>

307

<para>

308

These are the five command line arguments specific to script-scanning:

309

</para>

310

311

312

<term>

313

314

315

316

</indexterm>

317

</term>

318

319

<para>Performs a script scan using the default set of scripts. It is

320

equivalent to

321

<option>--script=safe,intrusive</option>.</para>

322

</listitem>

323

</varlistentry>

324

325

326

<term><option>--script <script-categories|directory|filename|all></option><indexterm><primary>--script</primary></indexterm></term>

327

328

329

<para>Runs a script scan (like <option>-sC</option>) with the scripts you have chosen rather than the defaults. Arguments can be script categories, single scripts or directories with scripts which are to be run against the target hosts instead of the default set. Nmap will try to interpret the arguments at first as categories and afterwards as files or directories. Absolute paths are used as is, relative paths are searched in the following places until found:

330

<filename>--datadir/</filename>;

331

<filename>$(NMAPDIR)/</filename>;

332

<filename>~user/nmap/</filename> (not searched on Windows);

333

<filename>NMAPDATADIR/</filename> or

334

<filename>./</filename>. A <filename>scripts/</filename> subdirectory is also tried in each of these. Give the argument <literal>all</literal> to execute all scripts in the Nmap script database.

335

</para>

336

337

<para>If a directory is specified and found, Nmap loads all NSE

338

scripts (any filenames ending with <literal>.nse</literal>) from that

339

directory. They must have the filename extension

340

<literal>nse</literal>. Nmap does not recurse into subdirectories to

341

find scripts. When individual file names are specified, the file

342

extension does not have to be <literal>nse</literal>.

343

</para>

344

345

<para>Nmap scripts are stored in a <filename>scripts</filename>

346

subdirectory of the Nmap data directory

347

348

<notbook>(see the <ulink url="http://nmap.org/book/man-misc-options.html"><option>--datadir</option>

349

option</ulink>)</notbook>

350

by default. Scripts are indexed in a database stored in

351

<filename>scripts/script.db</filename>. The database lists all of the

352

scripts in each category. A single script may be in several

353

categories.</para>

354

355

</listitem>

356

</varlistentry>

357

358

359

<term>

360

<option>--script-args</option>

361

362

<primary>--script-args</primary>

363

</indexterm>

364

</term>

365

366

<para>provides arguments to the scripts. See <xref

367

linkend="nse-args"/> for a detailed explanation.</para>

368

</listitem>

369

</varlistentry>

370

371

<term>

372

<option>--script-trace</option>

373

374

<primary>--script-trace</primary>

375

</indexterm>

376

</term>

377

378

<para>

379

380

This option is similar to

381

<option>--packet-trace</option>, but works at the

382

application level rather than packet by packet. If this

383

option is specified, all incoming and outgoing

384

communication performed by scripts is printed. The

385

displayed information includes the communication

386

protocol, source and target addresses, and the

387

transmitted data. If more than 5% of transmitted data is

388

unprintable, hex dumps are given instead.

389

390

</para>

391

</listitem>

392

</varlistentry>

393

394

<term>

395

<option>--script-updatedb</option>

396

397

<primary>--script-updatedb</primary>

398

</indexterm>

399

</term>

400

401

<para>This option is only useful if you have added or

402

removed NSE scripts from the default

403

<literal>scripts</literal> directory, or if you have

404

changed any of the scripts' <literal>categories</literal>

405

fields. This field contains categories such as

406

<literal>safe</literal> and <literal>discovery</literal>

407

which the script belongs to. Categories may be

408

specified with the <option>--script</option> option. For

409

efficiency reasons, NSE generates a

410

<filename>script.db</filename> file which maps

411

categories to the scripts they contain. If you changed

412

tag directives or added/removed scripts, run

413

<command>nmap --script-updatedb</command>.

414

</para>

415

</listitem>

416

</varlistentry>

417

</variablelist>

418

419

<para>

420

Some of the Nmap options have effects on script scans. The most

421

prominent of these is <option>-sV</option>. A version scan executes

422

the scripts in the <literal>version</literal> category. The scripts

423

in this category are slightly different than other scripts. Their

424

output blends in with the version scan and they do not produce any

425

script scan output.

426

</para>

427

<para>

428

Another option which has effect on the scripting engine is

429

<option>-A</option>. The aggressive mode of Nmap implies

430

the option <option>-sC</option>.

431

</para>

432

<para>

433

</para>

434

</sect2>

435

436

<title>Usage Examples</title>

437

<para>

438

Simple script scan.

439

</para>

440

<para>

441

442

$ nmap -sC hostname

443

</userinput>

444

</para>

445

<para>

446

Tracing a specific script.

447

</para>

448

<para>

449

450

$ nmap --script=./showSSHVersion.nse --script-trace hostname

451

</userinput>

452

</para>

453

</sect2>

454

</sect1>

455

456

<title>Script Format</title>

457

<para>NSE scripts consist of four descriptive fields, a port or host rule defining when the script should be executed, and an action block containing the actual script instructions. All six of these are Lua variables that are assigned to. Their names must be lowercase as shown here.

458

459

</para>

460

461

<title><literal>id</literal> Field</title>

462

<para>

463

The script's <literal>id</literal> field is displayed in the Nmap output

464

table if the script produces any output. It should be unique so users

465

can identify exactly which script file produced a message. IDs

466

should be kept short to conserve space in Nmap output, while

467

still being meaningful enough for users to recognize. Some

468

good examples are <literal>RIPE query</literal>, <literal>HTML

469

title</literal>, and <literal>Kibuv worm</literal>.

470

</para>

471

</sect2>

472

473

<title><literal>description</literal> Field</title>

474

<para>

475

The description describes what the script is testing for and

476

any critical notes the user must be aware of. A good example

477

example is this user contributed recursive DNS script

478

description <quote>Checks whether a nameserver on UDP port 53

479

allows queries for third party names. It is expected that

480

recursion will be enabled on your own internal

481

nameserver.</quote>

482

</para>

483

</sect2>

484

485

486

<title><literal>author</literal> Field </title>

487

<para>

488

The <literal>author</literal> field contains the script authors name and contact information. If you are worried about spam, you might want to omit or obscure your email address, or give your home page URL instead. This optional field is not used by NSE, but is important for giving script authors due credit or blame.

489

</para>

490

</sect2>

491

492

493

<title><literal>license</literal> Field </title>

494

<para>This field describes the license applied to the script. All scripts currently shipped with Nmap contain:</para>

495

496

license = "Same as Nmap--See http://nmap.org/book/man-legal.html"

497

</programlisting>

498

<para>See <xref linkend="nse-license"/> for further details on contributing NSE scripts to Nmap.

499

</para>

500

</sect2>

501

502

503

<title><literal>runlevel</literal> Field</title>

504

<para>

505

This optional field determines script execution order. When

506

this section is absent the run level defaults to 1.0. A script

507

with the run level 1.0 is run before any scripts with <literal>runlevel</literal> set to

508

<literal>2.5</literal>, which in turn runs before any scripts

509

with <literal>runlevel 2.55</literal>. No particular order

510

is guaranteed for scripts with the same run level. One

511

application of run levels is allowing scripts to depend on

512

each other. If <literal>script A</literal> relies on some

513

information gathered by <literal>script B</literal>, give

514

<literal>B</literal> a lower run level than

515

<literal>A</literal>. <literal>Script B</literal> can store

516

information in the NSE registry for <literal>A</literal> to

517

retrieve later. For information on the NSE registry see to

518

<xref linkend="nse-api-registry"/>.

519

</para>

520

</sect2>

521

522

523

<title>Port and Host Rules</title>

524

<para>

525

526

There are two types of rules: <emphasis>host rules</emphasis>

527

which run only once against a target IP and <emphasis>port

528

rules</emphasis> which run against individual ports on a

529

target. A rule is a Lua function which takes a host and a

530

port table as arguments and returns a boolean. If the rule

531

evaluates to <literal>true</literal>, the script action

532

is performed. Otherwise the action is skipped. Port rules are

533

only matched against TCP or UDP ports in the

534

<literal>open</literal>, <literal>open|filtered</literal> or

535

<literal>unfiltered</literal>

536

states. Host rules are matched exactly once against every

537

scanned host. The action, like the rule, is a Lua function,

538

which takes a host and port table as arguments. If the script is

539

matched using a host rule, then <literal>nil</literal> is passed instead of a port table. Example rules are shown in

540

541

542

<sect2 id="nse-format-action"><title>Action</title>

543

544

<para>

545

The action is the heart of an NSE script. It contains all of

546

the instructions to be executed when the script's port or host

547

rule triggers. It is a Lua function which returns either

548

<literal>nil</literal> or a string. If a string is returned,

549

it is printed along with the script ID in (if it is a service

550

script) or below (if it is a host script) the Nmap port table.

551

If the script returns <literal>nil</literal>, no output is

552

produced. All variables in the

553

action and rule segments must be declared <literal>local</literal>. For an

554

example of an NSE action refer to <xref

555

linkend="nse-tutorial-action"/>.

556

</para>

557

</sect2>

558

559

</sect1>

560

561

<title>Script Language</title>

562

<para>

563

Nmap's scripting engine consists of three more or less distinct

564

parts. The largest part is the embeddable Lua interpreter. This

565

is a lightweight language designed for extensibility. It offers

566

a powerful and well documented API for interfacing with other

567

software (such as Nmap).

568

</para>

569

570

<para>

571

The second part of the Nmap scripting engine is the NSE library, which

572

connects Lua and Nmap. This layer

573

handles issues such as initialization of the Lua interpreter,

574

scheduling of parallel script execution, script retrieval and

575

more. It is also the heart of the NSE network I/O framework and the

576

exception handling mechanism.

577

</para>

578

579

<para>

580

Lua was designed with a small feature set to ease embedding. So

581

we have added extensions to support more specialized

582

functionality. These basically are

583

<ulink url="http://www.lua.org/manual/5.1/manual.html#5.3">Lua modules</ulink> written either in Lua itself, or where needed in C. This

584

NSE library is the third part of the NSE.

585

</para>

586

587

588

<title>Lua Base Language</title>

589

<para>

590

591

The Nmap scripting language is an embedded <ulink

592

url="http://www.lua.org/">Lua</ulink> interpreter which was

593

extended with libraries for interfacing with Nmap. The Nmap

594

API is in the Lua namespace <literal>nmap</literal>. This

595

means that all calls to resources provided by Nmap have an

596

<literal>nmap</literal> prefix.

597

<literal>nmap.new_socket()</literal>, for example, returns a

598

new socket wrapper object. The Nmap library layer also takes

599

care of initializing the Lua context, scheduling parallel

600

scripts and collecting the output produced by completed

601

scripts.

602

</para>

603

<para>

604

605

During the planning stages, we considered several programming

606

languages as the bases for Nmap scripting. One option was to

607

implement a completely new programming language. The criteria

608

imposed on the options were strict, NSE needed to be easy to

609

use, small in size, compatible with the Nmap license,

610

scalable, fast and parallelizable. There have been several

611

efforts to design a security auditing language from scratch

612

which have resulted in well known awkward solutions. It was

613

clear from the beginning that we would not go down this

614

road. For a while the Guile scheme interpreter was considered

615

but the preference drifted towards Elk in favor of its more

616

liberal license. But parallelizing Elk scripts would have been

617

difficult. In addition, the subset of Nmap users familiar with

618

functional programming is regarded too small to consider

619

Scheme as an option. Larger interpreters like Perl, Python or

620

Ruby are well known and loved, but are difficult to embed

621

efficiently. In the end, Lua exceeded in all criteria for

622

NSE. It is small, distributed under the MIT license, has

623

coroutines which provide a sane method for parallel script

624

execution, was designed with embeddability in mind, has

625

excellent documentation, and is actively developed by a large

626

and committed community.

627

628

</para>

629

</sect2>

630

</sect1>

631

632

<title>Lua Extensions</title>

633

<para>In addition to the significant built-in capabilities of

634

Lua, we have written or integrated several extensions to make

635

NSE scripts more powerful and convenient to write. These

636

<emphasis>modules</emphasis> are compiled and installed along with

637

Nmap. They have their own directory, <filename>nselib</filename>, which

638

is installed in the configured datadir. Scripts need only <literal>require</literal> the default modules in order to use them. The default modules are described in the following sections.

639

</para>

640

641

642

<title>Bitwise Logical Operations</title>

643

<para>

644

Lua does not provide bitwise logical operations. Since they

645

are often useful for low-level network communication, Reuben

646

Thomas' bitwise operation library for Lua has been

647

integrated into NSE. The arguments to the bitwise operation

648

functions should be integers. The number of bits available

649

for logical operations depends on the data type used to

650

represent Lua numbers—this is typically 8-byte IEEE

651

floats, which give 53 bits (the size of the mantissa).

652

653

This implies that the bitwise operations won't work (as expected)

654

for numbers larger than 10<superscript>14</superscript>. You

655

can use them with 32-bit wide numbers without any problems. Operations

656

involving 64-bit wide numbers, however, may not return the expected

657

result.

658

The logical operations start with <quote>b</quote> (for <literal>bit</literal>) to avoid

659

clashing with reserved words; although <literal>xor</literal> isn't a

660

reserved word, it seemed better to use <literal>bxor</literal> for

661

consistency. In NSE the bitwise functions are in the <literal>bit</literal>

662

namespace.

663

664

665

666

667

668

669

<para>

670

Returns the one's complement of a.

671

</para>

672

</listitem>

673

</varlistentry>

674

675

676

677

678

679

<para>

680

Returns the bitwise <literal>and</literal> of the

681

w's.

682

</para>

683

</listitem>

684

</varlistentry>

685

686

687

688

689

690

<para>

691

Returns the bitwise <literal>or</literal> of the w's.

692

</para>

693

</listitem>

694

</varlistentry>

695

696

697

698

699

700

701

<para>

702

Returns the bitwise <literal>xor</literal> of the

703

w's.

704

</para>

705

</listitem>

706

</varlistentry>

707

708

709

<term><option>bit.lshift(a,b)</option>

710

<indexterm><primary>bit.lshift(a,b)</primary></indexterm></term>

711

712

<para>

713

Returns a shifted left b places—padded with zeros.

714

</para>

715

</listitem>

716

</varlistentry>

717

718

719

<term><option>bit.rshift(a,b)</option>

720

<indexterm><primary>bit.rshift(a,b)</primary></indexterm></term>

721

722

<para>

723

Returns a shifted logically right b places.

724

</para>

725

</listitem>

726

</varlistentry>

727

728

729

<term><option>bit.arshift(a,b)</option>

730

<indexterm><primary>bit.arshift(a,b)</primary></indexterm></term>

731

732

<para>

733

Returns a shifted arithmetically right b places.

734

</para>

735

</listitem>

736

</varlistentry>

737

738

739

740

741

742

<para>

743

Returns the integer remainder of a divided by b.

744

</para>

745

</listitem>

746

</varlistentry>

747

</variablelist>

748

</para>

749

</sect2>

750

751

752

<title>Perl Compatible Regular Expressions</title>

753

754

<para>

755

One of Lua's quirks is its string patterns. While they have

756

great performance and are tightly integrated into the Lua

757

interpreter, they are very different in syntax and not as

758

powerful as standard regular expressions. So we have

759

integrated Perl compatible regular expressions into Lua

760

using libPCRE and a modified version of the Lua PCRE library

761

written by Reuben Thomas and Shmuel Zeigerman. These are

762

the same sort of regular expressions used by Nmap version

763

detection. The main modification to their library is that

764

the NSE version only supports PCRExpressions instead of both

765

PCRE and POSIX patterns. In order to maintain a high script

766

execution speed, the library interfacing with libPCRE is

767

kept very thin. It is not integrated as seamlessly as the

768

Lua string pattern API. This allows script authors to decide

769

when to use PCRE expressions versus Lua patterns. PCRE

770

involves a separate pattern compilation step, which saves

771

execution time when patterns are reused. Compiled patterns

772

can be cached in the NSE registry and reused by other

773

scripts. The PCRE functions reside inside the <literal>pcre</literal>

774

namespace.

775

</para>

776

777

<warning><para>LibPCRE has a history of security vulnerabilities

778

allowing attackers who are able to compile arbitrary regular

779

expressions to execute arbitrary code. More such

780

vulnerabilities may be discovered in the future. These have

781

never affected Nmap because it doesn't give attackers any

782

control over the regular expressions it uses. Similarly, NSE

783

scripts should never build regular expressions with untrusted

784

network input. Matching hardcoded regular expressions

785

<emphasis>against</emphasis> the untrusted input is

786

fine.</para></warning>

787

788

<para>The following documentation is derived from that supplied by

789

the PCRE Lua lib.</para>

790

791

792

793

794

<term><option>pcre.new(pattern, flags, locale)</option>

795

796

797

<para>

798

Returns a compiled regular expression. The first

799

argument is a string describing the pattern, such as

800

<literal>^foo$</literal>. The second

801

argument is a number describing which compilation

802

flags are set. The compilation flags are set

803

bitwise. If you want to set the 3rd (corresponding to

804

the number 4) and the 1st (corresponding to 1) bit

805

for example you would pass the number 5 as a second

806

argument. The compilation flags accepted are those

807

of the PCRE C library. These include flags for case

808

insensitive matching (1), matching line beginnings (^)

809

and endings ($) even in multiline strings (i.e. strings

810

containing <quote>\n</quote>) (2) and a flag for matching across

811

line boundaries (4). No compilation flags yield a default

812

value of 0. The third (optional) argument is a string

813

describing the locale which should be used to compile the

814

regular expression. The variable is a string which is

815

passed to the C standard library function

816

<function>setlocale</function>. For more

817

information on this argument refer to the

818

documentation of <function>setlocale</function>. The

819

resulting compiled regular expression is ready to be

820

matched against strings. Compiled regular

821

expressions are subject to Lua's garbage collection.

822

Generally speaking, <literal>my_regex = pcre.new("<replaceable>pcre-pattern</replaceable>",0,"C")</literal>

823

should do the job most of the time.

824

</para>

825

</listitem>

826

</varlistentry>

827

828

829

830

<term><option>pcre.flags()</option>

831

<indexterm><primary>pcre.flags</primary></indexterm></term>

832

833

<para>

834

Returns a table of the available PCRE option flags

835

(numbers) keyed by their names (strings). Possible

836

names of the available strings can be retrieved from

837

the documentation of the PCRE library used to link

838

against Nmap. The key is the option name in the

839

manual minus the <literal>PCRE</literal>

840

prefix. <literal>PCRE_CASELESS</literal> becomes

841

<literal>CASELESS</literal> for example.

842

</para>

843

</listitem>

844

</varlistentry>

845

846

847

<term><option>pcre.version()</option>

848

<indexterm><primary>pcre.version</primary></indexterm></term>

849

850

<para>

851

Returns the version of the PCRE library in use as a

852

string. For example <literal>6.4 05-Sep-2005</literal>.

853

</para>

854

</listitem>

855

</varlistentry>

856

857

858

<term><option>pcre_obj:match(string, start, flags)</option>

859

<indexterm><primary>pcre.match</primary></indexterm></term>

860

861

<para>

862

Returns the start point and the end point point of

863

the first match of the compiled regular expression

864

pcre_obj in the string. A third

865

returned value is a table which contains

866

<literal>false</literal> in the positions where the

867

pattern did not match. If named sub-patterns were

868

used the table also contains substring matches keyed

869

by their sub-pattern name. Should no match be found the

870

function returns <literal>nil</literal>.

871

The second and third arguments are optional. The second

872

argument is a number specifying where the engine should

873

start trying to apply the pattern. The third argument

874

specifies execution flags for the pattern.

875

If you want to see if a given string matches a certain expression

876

you could use:</para>

877

878

879

s = pcre_obj:match("string to be searched", 0,0);

880

if(s) code_to_be_done_on_match end

881

</programlisting>

882

883

</listitem>

884

</varlistentry>

885

886

887

<term><option>pcre_obj:exec(string, start, flags)</option>

888

889

890

<para>

891

This function is like <literal>match()</literal> except that a table returned as

892

a third result contains offsets of substring matches rather

893

than substring matches themselves. That table will not

894

contain string keys, even if named sub-patterns are used. For

895

example, if the whole match is at offsets <literal>10, 20</literal> and substring

896

matches are at offsets <literal>12, 14</literal> and <literal>16, 19</literal> then the function

897

returns the following: <literal>10, 20, {12,14,16,19}</literal>

898

</para>

899

</listitem>

900

</varlistentry>

901

902

903

<term><option>pcre_obj:gmatch(string, func, n, ef)</option>

904

<indexterm><primary>pcre.gmatch</primary></indexterm></term>

905

906

<para>

907

Tries to match the regular expression <replaceable>pcre_obj</replaceable> against <replaceable>string</replaceable>

908

up to <replaceable>n</replaceable> times (or as many as possible if <replaceable>n</replaceable> is either

909

not given or is not a positive number), subject to

910

execution flags ef. Each time there is a match, <replaceable>func</replaceable>

911

is called as <replaceable>func(m, t)</replaceable>, where <replaceable>m</replaceable> is the matched

912

string and <replaceable>t</replaceable> is a table of substring matches. This

913

table contains <literal>false</literal> in the

914

positions where the corresponding sub-pattern did

915

not match. If named sub-patterns are used then the

916

table also contains substring matches keyed by their

917

correspondent sub-pattern names (strings). If <replaceable>func</replaceable>

918

returns a <literal>true</literal> value, then gmatch

919

immediately returns; gmatch returns the number of

920

matches made.

921

</para>

922

</listitem>

923

</varlistentry>

924

925

</variablelist>

926

</sect2>

927

928

929

<title>IP Operations</title>

930

<para>

931

The <literal>ipOps</literal> module provides some functions for

932

manipulating IPv4 addresses. The functions reside inside the

933

<literal>ipOps</literal> namespace.

934

</para>

935

936

937

<term><option>bool = ipOps.isPrivate("ip-string")</option>

938

<indexterm><primary>isPrivate</primary></indexterm></term>

939

940

<para>

941

checks whether an IP address, provided as a string in

942

dotted-quad notation, is part of the non-routed private IP address

943

space, as described in <ulink role="hidepdf" url="http://www.rfc-editor.org/rfc/rfc1918.txt">RFC 1918</ulink>. These addresses are the well known

944

<literal>10.0.0.0/8</literal>,<literal>192.168.0.0/16</literal> and

945

<literal>172.16.0.0/12</literal> networks.

946

</para>

947

</listitem>

948

</varlistentry>

949

950

<term><option>DWORD = ipOps.todword("ip-string")</option>

951

<indexterm><primary>todword</primary></indexterm></term>

952

953

<para>

954

returns the IP address as DWORD value (i.e. the IP <replaceable>a.b.c.d</replaceable> becomes

955

<literal>(((a*256+b)*256+c)*256+d)</literal> )

956

</para>

957

</listitem>

958

</varlistentry>

959

960

<term><option>a,b,c,d = ipOps.get_parts_as_number("ip-string")</option>

961

<indexterm><primary>get_parts_as_number</primary></indexterm></term>

962

963

<para>

964

returns 4 numbers corresponding to the fields in dotted-quad notation.

965

For example, <literal>ipOps.get_parts_as_number("192.168.1.1")

966

</literal> returns <literal>192,168,1,1</literal>.

967

</para>

968

</listitem>

969

</varlistentry>

970

</variablelist>

971

</sect2>

972

973

<title>Short Portrules</title>

974

<para>

975

Since portrules are mostly the same for many scripts, the

976

<literal>shortport</literal> module provides functions for the most common tests.

977

The arguments in brackets (<literal>[]</literal>) are optional. If no

978

<literal>proto</literal> is provided, <literal>tcp</literal> is used. The default

979

<literal>state</literal> is <literal>open</literal>

980

</para>

981

982

983

<term><option>shortport.portnumber(port,[proto],[state])</option>

984

<indexterm><primary>portnumber</primary></indexterm></term>

985

986

<para>

987

The port argument is either a number or a table of numbers which are

988

interpreted as port numbers, against which the script should run.

989

</para>

990

</listitem>

991

</varlistentry>

992

993

<term><option>shortport.service(service,[proto],[state])</option>

994

<indexterm><primary>service</primary></indexterm></term>

995

996

<para>

997

The service argument is either a string or a table

998

of strings which are interpreted as service names

999

(e.g. <literal>"http"</literal>, <literal>"https"</literal>, <literal>"smtp"</literal> or <literal>"ftp"</literal>) against which the

1000

script should run. These service names are

1001

determined by Nmap's version scan or (if no version

1002

scan information is available) the service assigned

1003

to the port in <filename>nmap-services</filename>

1004

(i.e. "http" for TCP port 80).

1005

</para>

1006

</listitem>

1007

</varlistentry>

1008

1009

<term><option>shortport.port_or_service(port,service,[proto],[state])</option>

1010

<indexterm><primary>port_or_service</primary></indexterm></term>

1011

1012

<para>

1013

This is a combination of the above functions, since many scripts

1014

explicitly try to run against the well known ports, but want

1015

also to run against any other port which was discovered to run the

1016

named service. A typical example for this function is:

1017

<literal>portrule = shortport.port_or_service(22,"ssh")</literal>.

1018

</para>

1019

</listitem>

1020

</varlistentry>

1021

</variablelist>

1022

</sect2>

1023

1024

<title>Functional Programming Style List Operations</title>

1025

<para>

1026

People used to programming in functional languages, such as Lisp or

1027

Haskell appreciate their handling of lists very much. The <literal>listop</literal> module tries to bring much of the functionality from

1028

functional languages to Lua using Lua's central data structure, the table,

1029

as a base for its list operations. Highlights include a <literal>map</literal>

1030

function applying a given function to each element of a list.

1031

</para>

1032

1033

1034

<term><option>bool = listop.is_empty(list)</option>

1035

<indexterm><primary>is_empty</primary></indexterm></term>

1036

1037

<para>

1038

Returns <literal>true</literal> if the given list is empty.

1039

</para>

1040

</listitem>

1041

</varlistentry>

1042

1043

<term><option>bool = listop.is_list(value)</option>

1044

1045

1046

<para>

1047

Returns <literal>true</literal> if the given value is a list (or rather a table).

1048

</para>

1049

</listitem>

1050

</varlistentry>

1051

1052

<term><option>list = listop.map(function, list)</option>

1053

1054

1055

<para>

1056

The provided function is applied to each element of the list

1057

separately. The returned list contains the results of each

1058

function call. For example <literal>listop.map(tostring,{1,2,true})

1059

</literal> returns <literal>{"1","2","true"}</literal>.

1060

</para>

1061

</listitem>

1062

</varlistentry>

1063

1064

<term><option>value = listop.apply(function, list)</option>

1065

<indexterm><primary>apply</primary></indexterm></term>

1066

1067

<para>

1068

All of the elements in the list are passed to a call of <literal>

1069

function</literal>. The result is then returned. For example

1070

<literal>listop.apply(math.max,{1,5,6,7,50000})</literal>

1071

yields <literal>50000</literal>.

1072

</para>

1073

</listitem>

1074

</varlistentry>

1075

1076

<term><option>list = listop.filter(predicate, list)</option>

1077

<indexterm><primary>filter</primary></indexterm></term>

1078

1079

<para>

1080

Returns a list containing only those elements for which the predicate

1081

returns true. The predicate has to be a function, which takes an

1082

element of the list as argument and the result of which

1083

is interpreted as boolean value. If it returns true (or rather

1084

anything besides <literal>false</literal> and <literal>nil</literal>)

1085

the argument is appended to the return value of <literal>filter</literal>.

1086

For example: <literal>listop.filter(isnumber,{1,2,3,"foo",4,"bar"})</literal> returns

1087

<literal>{1,2,3,4}</literal>.

1088

</para>

1089

</listitem>

1090

</varlistentry>

1091

1092

<term><option>list = listop.flatten(list)</option>

1093

<indexterm><primary>flatten</primary></indexterm></term>

1094

1095

<para>

1096

Since a list can itself contain lists as elements,

1097

<literal>flatten</literal> returns a list which

1098

only contains values that are not themselves

1099

lists. For example:

1100

<literal>listop.flatten({1,2,3,"foo",{4,5,{"bar"}}})</literal> returns

1101

<literal>{1,2,3,"foo",4,5,"bar"}</literal>.

1102

</para>

1103

</listitem>

1104

</varlistentry>

1105

1106

<term><option>list = listop.append(list1, list2)</option>

1107

<indexterm><primary>append</primary></indexterm></term>

1108

1109

<para>

1110

Returns a list containing all elements of list1 appended by all

1111

elements of <replaceable>list2</replaceable>.

1112

</para>

1113

</listitem>

1114

</varlistentry>

1115

1116

<term><option>list = listop.cons(value1, value2)</option>

1117

1118

1119

<para>

1120

Returns a list containing <replaceable>value1</replaceable> appended by <replaceable>value2</replaceable>, which may be

1121

of any type.

1122

</para>

1123

</listitem>

1124

</varlistentry>

1125

1126

<term><option>list = listop.reverse(list)</option>

1127

<indexterm><primary>reverse</primary></indexterm></term>

1128

1129

<para>

1130

Returns a list containing all elements of the given list in inverted

1131

order.

1132

</para>

1133

</listitem>

1134

</varlistentry>

1135

1136

<term><option>value = listop.car(list)</option>

1137

1138

1139

<para>

1140

Returns the first element of the given list.

1141

</para>

1142

</listitem>

1143

</varlistentry>

1144

1145

<term><option>value = listop.ncar(list,n)</option>

1146

1147

1148

<para>

1149

Returns the nth (or first if n is omitted) element of the given list.

1150

</para>

1151

</listitem>

1152

</varlistentry>

1153

1154

<term><option>value = listop.cdr(list)</option>

1155

1156

1157

<para>

1158

Returns a list containing all elements but the first of the

1159

given list.

1160

</para>

1161

</listitem>

1162

</varlistentry>

1163

1164

<term><option>value = listop.ncdr(list, n)</option>

1165

1166

1167

<para>

1168

Returns a list containing all elements but the first n of the

1169

given list, where n is 2 if it is omitted.

1170

</para>

1171

</listitem>

1172

</varlistentry>

1173

</variablelist>

1174

</sect2>

1175

1176

<title>String Buffer Operations</title>

1177

<para>

1178

Lua's string operations are very flexible and offer an easy-to-use way

1179

to manipulate strings. Concatenation using the <literal>..</literal>

1180

operator is such an operation. The drawback of the built-in API however is the way it handles

1181

concatenation of many string values. Since strings in Lua are

1182

immutable values, each time you concatenate two strings both get copied

1183

into the result string. The <literal>strbuf</literal> module offers a

1184

workaround for this problem, while maintaining the nice syntax. This

1185

is accomplished by overloading the concatenation operator (<literal>..</literal>) the equality operator (<literal>==</literal>) and the

1186

tostring operator. By overloading

1187

these operators, we reduce the overhead of using a string buffer instead

1188

of a plain string to wrap the first literal string assigned to a

1189

variable inside a <literal>strbuf.new()</literal> call. Afterwards you can append to the string buffer, or compare

1190

two string buffers for equality just as you would do with normal strings.

1191

When looking at the details there are some more restrictions/oddities:

1192

The concatenation operator requires its left-hand value to be a

1193

string buffer. Therefore, if you want to prepend a string to a given

1194

string buffer you have to create a new string buffer out of the string

1195

you want to prepend.

1196

The string buffer's <literal>tostring</literal> operator concatenates the

1197

strings inside the buffer using newlines by default, since this appears to

1198

be the separator used most often.

1199

</para>

1200

1201

1202

<term><option>buffer = strbuf.new("first-string")</option>

1203

1204

1205

<para>

1206

Creates a new string buffer. The argument is optional and is the

1207

first string to be added to the buffer.

1208

</para>

1209

</listitem>

1210

</varlistentry>

1211

1212

<term><option>buffer = strbuf.concat(strbuf1, value)</option>

1213

<indexterm><primary>concat</primary></indexterm></term>

1214

1215

<para>

1216

Concatenates the <literal>value</literal> (which has to be either

1217

a string or a string buffer) to <literal>strbuf1</literal>. This

1218

is also the function serving as the string buffer's concatenation operator.

1219

The above function call can thus also be expressed as:

1220

<literal>buffer = strbuf1 .. value</literal>

1221

</para>

1222

</listitem>

1223

</varlistentry>

1224

1225

<term><option>bool = strbuf.eqbuf(strbuf1, strbuf2)</option>

1226

<indexterm><primary>eqbuf</primary></indexterm></term>

1227

1228

<para>

1229

Compares <literal>strbuf1</literal> and <literal>strbuf2</literal>

1230

for equality. For the function to return <literal>true</literal>, both values must be

1231

string buffers containing exactly the same strings. The <literal>eqbuf</literal> function is called to compare two strings for equality.

1232

</para>

1233

</listitem>

1234

</varlistentry>

1235

1236

<term><option>strbuf.clear(strbuf)</option>

1237

<indexterm><primary>clear</primary></indexterm></term>

1238

1239

<para>

1240

Deletes all strings in <literal>strbuf</literal>.

1241

</para>

1242

</listitem>

1243

</varlistentry>

1244

1245

<term><option>string = strbuf.dump(strbuf, "delimiter")</option>

1246

1247

1248

<para>

1249

Dumps <literal>strbuf</literal>'s contents as string. The second

1250

parameter is used as a delimiter between the strings stored inside

1251

<literal>strbuf</literal>. <literal>dump(strbuf, "\n")</literal> is

1252

used as the <literal>tostring</literal> function of string buffers.

1253

</para>

1254

</listitem>

1255

</varlistentry>

1256

</variablelist>

1257

</sect2>

1258

1259

<title>URL Manipulation Functions</title>

1260

1261

<para>URL manipulation functions have obvious uses. Fortunately

1262

there is already an implementation of URL generation functions

1263

inside the Lua-socket package, which is fairly complete and

1264

<ulink

1265

url="http://www.cs.princeton.edu/~diego/professional/luasocket/old/luasocket-2.0-alpha/url.html">well

1266

documented</ulink>. For NSE, Lua-socket's URL module was

1267

extended with two functions:</para>

1268

1269

1270

1271

<term><option>table = url.parse_query("query-string")</option>

1272

<indexterm><primary>parse_query</primary></indexterm></term>

1273

1274

<para>

1275

This function takes a <replaceable>query-string</replaceable> of the form <literal>name1=value1&name2=value2...</literal> and returns a table

1276

containing the name-value pairs, with the <literal>name</literal>

1277

as the key and the <literal>value</literal> as its associated value.

1278

The table corresponding to the above <replaceable>query-string</replaceable> would have two

1279

entries: <literal>table["name1"]="value1"</literal> and

1280

<literal>table["name2"]="value2"</literal>.</para>

1281

</listitem>

1282

</varlistentry>

1283

1284

<term><option>query_string = url.build_query(table)</option>

1285

<indexterm><primary>build_query</primary></indexterm></term>

1286

1287

<para>

1288

This is the inverse function to <literal>parse_query()</literal>.

1289

</para>

1290

</listitem>

1291

</varlistentry>

1292

</variablelist>

1293

</sect2>

1294

1295

<title>Buffered Network I/O Helper Functions</title>

1296

<para>

1297

The <literal>match</literal> module was written to provide

1298

functions which can be used for delimiting data received by the

1299

<literal>receive_buf()</literal> function from the Network I/O API:

1300

</para>

1301

1302

1303

<term><option>start,end = match.regex("regexpattern")</option>

1304

<indexterm><primary>regex</primary></indexterm></term>

1305

1306

<para>

1307

This is actually a wrapper around NSE' PCRE library <literal>exec</literal> function (see <xref linkend="nse-pcre"/>, thus

1308

giving script developers the possibility to use regular expressions

1309

for delimiting instead of Lua's string patterns. If you want to get

1310

the data in chunks separated by <function>regex</function> (which has to be a valid

1311

regular expression), you would write <literal>status, val =

1312

sockobj:receive_buf(match.lua("regex"))</literal>.

1313

</para>

1314

</listitem>

1315

</varlistentry>

1316

1317

<term><option>start,end = match.numbytes(number)</option>

1318

<indexterm><primary>numbytes</primary></indexterm></term>

1319

1320

<para>

1321

Takes a number as argument and returns that

1322

many bytes. It can be used to get a buffered

1323

version of

1324

<literal>sockobj:receive_bytes(n)</literal> in

1325

case a script requires more than one

1326

fixed-size chunk, as the unbuffered version

1327

may return more bytes than requested and thus

1328

would require you to do the parsing on your

1329

own.

1330

</para>

1331

</listitem>

1332

</varlistentry>

1333

</variablelist>

1334

</sect2>

1335

1336

<title>HTTP Functions</title>

1337

<para>

1338

The <literal>http</literal> module provides functions for dealing with the client side of the http protocol.

1339

The functions reside inside the <literal>http</literal> namespace.

1340

The return value of each function in this module is a table with the following keys:

1341

<literal>status</literal>, <literal>header</literal> and <literal>body</literal>.

1342

<literal>status</literal> is the status code of the http request

1343

In case of an error status is <literal>nil</literal>. <literal>header</literal>

1344

is a table with the headers received from the server. The header names are

1345

lower-cased and multiple headers of the same name are concatenated with comma.

1346

<literal>body</literal> holds a string with the request body.

1347

</para>

1348

1349

1350

<term><option>table = http.get(host,port,path,[options])</option>

1351

1352

1353

<para>

1354

Fetches a resource with a <literal>GET</literal> request.

1355

The first argument is either a string with the hostname or a

1356

table like the host table passed by nmap. The second argument

1357

is either the port number or a table like the port table passed

1358

by nmap. The third argument is the path of the resource. The fourth

1359

argument is a table for further options. The table may have 2 keys:

1360

<literal>timeout</literal> and <literal>header</literal>.

1361

<literal>timeout</literal> is the timeout used for the socket

1362

operations. <literal>header</literal> is a table with additional

1363

headers to be used for the request.

1364

The function builds the request and calls <literal>http.request</literal>

1365

</para>

1366

</listitem>

1367

</varlistentry>

1368

1369

<term><option>table = http.request(host,port,request,[options])</option>

1370

<indexterm><primary>request</primary></indexterm></term>

1371

1372

<para>

1373

Sends <literal>request</literal> to <literal>host</literal>:<literal>port</literal>

1374

and parses the answer.

1375

The first argument is either a string with the hostname or a

1376

table like the host table passed by nmap. The second argument

1377

is either the port number or a table like the port table passed

1378

by nmap. SSL is used for the request if either <literal>port.service</literal>

1379

equals <literal>"https"</literal> or <literal>port.version.service_tunnel</literal>

1380

equals <literal>"ssl"</literal>. The third argument is the request. The fourth

1381

argument is a table for further options. You can specify a timeout

1382

for the socket operations with the timeout key.

1383

</para>

1384

</listitem>

1385

</varlistentry>

1386

1387

<term><option>table = http.get_url(url,[options])</option>

1388

1389

1390

<para>

1391

Parses <literal>url</literal> and calls <literal>http.get</literal>

1392

with the result.

1393

The second argument is a table for further options. The table may have 2 keys:

1394

<literal>timeout</literal> and <literal>header</literal>.

1395

<literal>timeout</literal> is the timeout used for the socket

1396

operations. <literal>header</literal> is a table with additional

1397

headers to be used for the request.

1398

</para>

1399

</listitem>

1400

</varlistentry>

1401

</variablelist>

1402

</sect2>

1403

1404

<title>Data File Parsing Functions</title>

1405

<para>

1406

The <literal>datafiles</literal> module provides functions for reading and parsing

1407

Nmap's data files (e.g. <filename>nmap-protocol</filename>, <filename>nmap-rpc</filename>,

1408

etc.). These functions' return values are setup for use with exception handling via

1409

<literal>nmap.new_try()</literal>.

1410

</para>

1411

1412

1413

1414

<term><option>bool, table = datafiles.parse_protocols()</option>

1415

<indexterm><primary>parse_protocols</primary></indexterm></term>

1416

1417

<para>

1418

This function reads and parses Nmap's <filename>nmap-protocols</filename>

1419

file. <literal>bool</literal> is a boolen value indicating success.

1420

If <literal>bool</literal> is true, then the second returned

1421

value is a table with protocol numbers indexing the protocol

1422

names. If <literal>bool</literal> is false, an error message

1423

is returned as the second value instead of the table.

1424

</para>

1425

</listitem>

1426

</varlistentry>

1427

1428

1429

<term><option>bool, table = datafiles.parse_rpc()</option>

1430

<indexterm><primary>parse_rpc</primary></indexterm></term>

1431

1432

<para>

1433

This function reads and parses Nmap's <filename>nmap-rpc</filename>

1434

file. <literal>bool</literal> is a boolen value indicating success.

1435

If <literal>bool</literal> is true, then the second returned

1436

value is a table with RPC numbers indexing the RPC names. If

1437

<literal>bool</literal> is false, an error message is returned

1438

as the second value instead of the table.

1439

</para>

1440

</listitem>

1441

</varlistentry>

1442

1443

1444

<term><option>bool, table = datafiles.parse_services([protocol])</option>

1445

<indexterm><primary>parse_services</primary></indexterm></term>

1446

1447

<para>

1448

This function reads and parses Nmap's <filename>nmap-services</filename>

1449

file. <literal>bool</literal> is a boolen value indicating success.

1450

If <literal>bool</literal> is true, then the second returned

1451

value is a table containing two other tables:

1452

<literal>tcp{}</literal> and <literal>udp{}</literal>.

1453

<literal>tcp{}</literal> contains services indexed by TCP port

1454

numbers. <literal>udp{}</literal> is the same, but for UDP.

1455

You can pass "tcp" or "udp" as an argument to

1456

<literal>parse_services()</literal> to only get the corresponding

1457

table. If <literal>bool</literal> is false, an error message is

1458

returned as the second value instead of the table.

1459

</para>

1460

</listitem>

1461

</varlistentry>

1462

</variablelist>

1463

1464

</sect2>

1465

1466

<title>Various Utility Functions</title>

1467

<para>

1468

The <literal>stdnse</literal> library contains various handy

1469

functions which are too small to justify modules of their own:

1470

</para>

1471

1472

1473

<term><option>stdnse.print_debug(...)</option>

1474

<indexterm><primary>print_debug</primary></indexterm></term>

1475

1476

<para>

1477

Wrapper function around <literal>print_debug_unformatted()</literal>

1478

in the <literal>nmap</literal> namespace. The first argument, if numeric, is

1479

used as the necessary debug level to print the message (it defaults

1480

to 1 if omitted). All remaining arguments are processed with

1481

Lua's <literal>string.format()</literal> function, which provides a

1482

C-style printf interface.

1483

</para>

1484

</listitem>

1485

</varlistentry>

1486

1487

<term><option>table = stdnse.strsplit("delimiter","text")</option>

1488

<indexterm><primary>strsplit</primary></indexterm></term>

1489

1490

<para>

1491

This function will certainly be appreciated by Perl programmers.

1492

It takes two strings as arguments and splits the second one around

1493

all occurrences of the first one, returning a table, which contains

1494

the substrings without the delimiting string.

1495

</para>

1496

</listitem>

1497

</varlistentry>

1498

1499

<term><option>string = stdnse.strjoin("delimiter", table)</option>

1500

<indexterm><primary>strjoin</primary></indexterm></term>

1501

1502

<para>

1503

Inverse function to <literal>strsplit()</literal>. Basically this is

1504

Lua's <literal>table.concat()</literal> function with the parameters

1505

swapped for coherence.

1506

</para>

1507

</listitem>

1508

</varlistentry>

1509

</variablelist>

1510

</sect2>

1511

</sect1>

1512

1513

1514

1515

<para>

1516

NSE scripts have access to several Nmap facilities for writing

1517

flexible and elegant scripts. The API provides target host

1518

details such as port states and version detection results. It

1519

also offers an interface to the Nsock library for efficient

1520

network I/O.

1521

</para>

1522

1523

1524

<title>Information Passed to a Script</title>

1525

<para>

1526

An effective Nmap scripting engine requires more than just a

1527

Lua interpreter. Users need easy access to the information

1528

Nmap has learned about the target hosts. This data is passed

1529

as arguments to the NSE <literal>action</literal> method. The

1530

arguments, <literal>host</literal> and

1531

<literal>port</literal>, are Lua tables which contain

1532

information on the target against which the script is

1533

executed. The following list describes each variable in the

1534

<literal>host</literal> and <literal>port</literal> tables.

1535

</para>

1536

1537

<para>

1538

1539

1540

1541

1542

1543

<para>

1544

This table is passed as a parameter to the rule and action

1545

functions. It contains information on the operating system run by

1546

the host (if the <option>-O</option> switch was supplied), the

1547

IP address and the host name of the scanned target.

1548

</para>

1549

</listitem>

1550

</varlistentry>

1551

1552

1553

1554

1555

1556

<para>

1557

The <literal>os</literal> entry in the host table is

1558

an array of strings. The strings (maximally 8) are the

1559

names of the operating systems the target is possibly

1560

running. Strings are only entered in this array if the

1561

target machine is a perfect match for one or more OS

1562

database entries. If Nmap was run without the

1563

<option>-O</option> option, then

1564

<literal>host.os</literal> is <literal>nil</literal>.

1565

</para>

1566

</listitem>

1567

</varlistentry>

1568

1569

1570

1571

1572

1573

<para>Contains a string representation of the IP address of the

1574

target host. If the scan was run against a host name and the

1575

reverse DNS query returned more than one IP addresses then the

1576

same IP address is used as the one chosen for the scan.

1577

</para>

1578

</listitem>

1579

</varlistentry>

1580

1581

1582

1583

1584

1585

<para>Contains the reverse DNS entry of the scanned target host

1586

represented as a string. If the host has no reverse DNS entry,

1587

the value of the field is an empty string.

1588

</para>

1589

</listitem>

1590

</varlistentry>

1591

1592

<term><option>host.targetname</option>

1593

<indexterm><primary>host.targetname</primary></indexterm></term>

1594

1595

<para>Contains the name of the host as specified on the commandline.

1596

If the target given on the commandline contains a netmask or is an IP

1597

address the value of the field is <literal>nil</literal>.

1598

</para>

1599

</listitem>

1600

</varlistentry>

1601

1602

<term><option>host.directly_connected</option>

1603

<indexterm><primary>host.directly_connected</primary></indexterm></term>

1604

1605

<para> A boolean value indicating whether or not the target host is

1606

directly connected (i.e. on the same network segment).

1607

</para>

1608

</listitem>

1609

</varlistentry>

1610

1611

1612

1613

1614

<para>MAC address of the destination host (6-byte long binary

1615

string) or <literal>nil</literal>, if the host is not directly connected.

1616

</para>

1617

</listitem>

1618

</varlistentry>

1619

1620

1621

1622

1623

<para>Our own MAC address, which was used to connect to the

1624

host (either our network card's, or (with <option>--spoof-mac</option>) the spoofed address).

1625

</para>

1626

</listitem>

1627

</varlistentry>

1628

1629

<term><option>host.interface</option>

1630

<indexterm><primary>host.interface</primary></indexterm></term>

1631

1632

<para>A string containing the interface name (dnet-style) through

1633

which packets to the host are sent.

1634

</para>

1635

</listitem>

1636

</varlistentry>

1637

1638

1639

1640

1641

<para>The hosts IP as 4 byte long binary value.

1642

</para>

1643

</listitem>

1644

</varlistentry>

1645

1646

1647

1648

1649

<para>Our hosts IP as 4 byte long binary value.

1650

</para>

1651

</listitem>

1652

</varlistentry>

1653

1654

1655

1656

1657

<para>

1658

The port table is passed to the Lua script in the same

1659

fashion as the host table. It contains information about the port

1660

against which the script is running. If the script is run

1661

according to a host rule, then no port table is passed to the

1662

script. Port states on the target can still be requested from Nmap

1663

using the <literal>nmap.get_port_state()</literal> call.

1664

</para>

1665

</listitem>

1666

</varlistentry>

1667

1668

1669

<term><option>port.number</option>

1670

<indexterm><primary>port.number</primary></indexterm></term>

1671

1672

<para>

1673

Contains the number of the currently scanned port.

1674

</para>

1675

</listitem>

1676

</varlistentry>

1677

1678

1679

1680

<term><option>port.protocol</option>

1681

<indexterm><primary>port.protocol</primary></indexterm></term>

1682

1683

<para>

1684

Defines the protocol of the port. Valid values are

1685

<literal>tcp</literal> and <literal>udp</literal>.

1686

</para>

1687

</listitem>

1688

</varlistentry>

1689

1690

1691

1692

<term><option>port.service</option>

1693

<indexterm><primary>port.service</primary></indexterm></term>

1694

1695

<para>

1696

Contains a string representation of the service running on

1697

<literal>port.number</literal> as detected by the Nmap service

1698

detection. If the <literal>port.version</literal> field is

1699

<literal>nil</literal> then Nmap has guessed the service based

1700

only on the port number. Otherwise this field is equal to

1701

<literal>port.version.name</literal>.

1702

</para>

1703

</listitem>

1704

</varlistentry>

1705

1706

1707

<term><option>port.version</option>

1708

<indexterm><primary>port.version</primary></indexterm></term>

1709

1710

<para>

1711

This entry is a table which contains information

1712

retrieved by the Nmap version scanning engine. Some

1713

of the values (like service name, service type

1714

confidence, RPC related values) may be retrieved by

1715

Nmap even if a version scan was not required. Values

1716

which were not retrieved default to

1717

<literal>nil</literal>. The meaning of each value is given in the following table:</para>

1718

1719

1720

<title><literal>port.version</literal> values</title>

1721

1722

1723

1724

1725

1726

<entry>Description</entry>

1727

</row></thead>

1728

<tbody>

1729

<row>

1730

1731

<entry>Contains the service name Nmap will use for the port.</entry>

1732

</row>

1733

1734

<row>

1735

<entry><literal>name_confidence</literal></entry>

1736

<entry>Evaluates how confident the version detection is is about the accuracy of <literal>name</literal>, from one (least confident) to 10.</entry>

1737

</row>

1738

1739

<row>

1740

<entry><literal>product</literal>, <literal>version</literal>, <literal>extrainfo</literal>, <literal>hostname</literal>, <literal>ostype</literal>, <literal>devicetype</literal></entry>

1741

<entry>These five variables are described in

1742

1743

<notbook>the <ulink url="http://nmap.org/vscan/vscan-fileformat.html#vscan-versioninfo">versioninfo section</ulink> of our version scanning documentation.</notbook>

1744

</entry>

1745

</row>

1746

1747

<row>

1748

<entry><literal>service_tunnel</literal></entry>

1749

<entry>Contains the string <literal>none</literal> or <literal>ssl</literal> based on whether or not Nmap used SSL tunnelling to detect the service.</entry>

1750

</row>

1751

1752

<row>

1753

<entry><literal>service_fp</literal></entry>

1754

<entry>The service fingerprint, if any, is provided in this value. This is described in

1755

1756

<notbook>our <ulink url="http://nmap.org/vscan/vscan-community.html">version detection documentation</ulink>.</notbook>

1757

</entry>

1758

</row>

1759

1760

<row>

1761

<entry><literal>rpc_status</literal></entry>

1762

<entry>Contains a string value of <literal>good_prog</literal> if

1763

we were able to determine the program number of an RPC

1764

service listening on the port, <literal>unknown</literal>

1765

if the port appears to be RPC but we couldn't determine the

1766

program number, <literal>not_rpc</literal> if the port

1767

doesn't appear be RPC, or <literal>untested</literal> if we

1768

haven't checked for RPC status. The

1769

<literal>rpc_program</literal>,

1770

<literal>rpc_lowver</literal>, and

1771

<literal>rpc_highver</literal> variables are <literal>nil</literal> unless

1772

<literal>rpc_status</literal> is

1773

1774

</row>

1775

1776

<row>

1777

<entry><literal>rpc_program</literal>, <literal>rpc_lowver</literal>, <literal>rpc_highver</literal></entry>

1778

<entry>The detected RPC program number and the range of version

1779

numbers supported by that program. These will be

1780

<literal>nil</literal> if <literal>rpc_status</literal> is

1781

anything other than <literal>good_prog</literal>.</entry>

1782

</row>

1783

</tbody></tgroup></table>

1784

1785

</listitem>

1786

</varlistentry>

1787

1788

1789

<term><option>port.state</option>

1790

<indexterm><primary>port.state</primary></indexterm></term>

1791

1792

<para>

1793

Contains information on the state of the port.

1794

Service scripts are only run against ports in the

1795

<literal>open</literal> or

1796

<literal>open|filtered</literal> states, so

1797

<literal>port.state</literal> generally contains one

1798

of those values. Other values might appear if the port

1799

table is a result of the

1800

<literal>get_port_state</literal> function. You can

1801

adjust the port state using the

1802

<literal>nmap.set_port_state()</literal> call. This is

1803

normally done when an <literal>open|filtered</literal>

1804

port is determined to be <literal>open</literal>.

1805

</para>

1806

</listitem>

1807

</varlistentry>

1808

1809

1810

</variablelist>

1811

</para>

1812

1813

<para>

1814

Scripts also have access to some of Nmap’s functions and state

1815

variables that are exposed through functions in the <literal>nmap</literal>

1816

table.

1817

1818

1819

<term><option>nmap.debugging()</option>

1820

<indexterm><primary>debugging</primary><secondary><literal>nmap.debugging</literal></secondary></indexterm></term>

1821

1822

<para>

1823

Returns the debugging level as a non-negative integer. The

1824

debugging level can be set with the <option>-d</option>

1825

option<bookex> (see <xref linkend="port-scanning-options-output"/>)</bookex>.

1826

</para>

1827

</listitem>

1828

</varlistentry>

1829

1830

1831

1832

1833

<para>

1834

Returns true if Nmap was compiled with SSL support, false

1835

otherwise. This can be used to avoid sending SSL probes

1836

when SSL is not available.

1837

</para>

1838

</listitem>

1839

</varlistentry>

1840

1841

<term><option>nmap.verbosity()</option><indexterm><primary>verbosity</primary><secondary><literal>nmap.verbosity</literal></secondary></indexterm></term>

1842

1843

<para>

1844

Returns the verbosity level as a non-negative integer. The

1845

verbosity level can be set with the <option>-v</option>

1846

option<bookex> (see <xref linkend="port-scanning-options-output"/>)</bookex>.

1847

</para>

1848

</listitem>

1849

</varlistentry>

1850

1851

<term>

1852

<option>nmap.fetchfile(filename)</option>

1853

<indexterm><primary>fetchfile</primary></indexterm>

1854

</term>

1855

1856

<para>

1857

Allows access to Nmap's data files. <literal>fetchfile()</literal>

1858

searches for the specified file and returns a string containing

1859

it's path if it is found and readable (to the process). If the

1860

file is not found, not readable, or is a directory,

1861

<literal>nil</literal> is returned. The call

1862

1863

nmap.fetchfile("nmap-rpc")

1864

</programlisting>

1865

will search for the data file <filename>nmap-rpc</filename> and,

1866

assuming it's found (which it should be), return a location like

1867

<filename>/usr/local/share/nmap/nmap-rpc</filename>.

1868

</para>

1869

</listitem>

1870

</varlistentry>

1871

</variablelist>

1872

</para>

1873

</sect2>

1874

1875

1876

<title>Target Information Retrieving by a Script</title>

1877

<para>

1878

Often the information passed to the script is not enough. Sometimes

1879

a script might want to correct target information or set it in the

1880

first place. The following API methods handle this.

1881

</para>

1882

1883

<para>

1884

1885

1886

<term><option>nmap.get_port_state(host, port, protocol)</option>

1887

<indexterm><primary>get_port_state</primary></indexterm></term>

1888

1889

<para>

1890

The <literal>get_port_state()</literal> call takes a

1891

host table, a port table and a protocol

1892

(<literal>tcp</literal> or <literal>udp</literal>) and

1893

returns a port table for the queried port. The host

1894

and port table are similar in structure to the ones

1895

passed to the rule and action functions. The host

1896

table should have an IP address field. The port table

1897

needs a port number and a protocol field. A call could

1898

look like this:

1899

1900

nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"})

1901

</programlisting>

1902

You can of course reuse the host and port tables

1903

passed to the port rule function. The purpose of this

1904

call is to be able to match scripts against more than

1905

one open port. For example if the target host has an

1906

open port 22 and a running identd server, then you can

1907

write a script which will only fire if both ports are

1908

open and there is an identification server on port

1909

113. While it is possible to specify IP addresses

1910

different to the currently scanned target, the result

1911

will only be correct if the target is in the currently

1912

scanned group of hosts.

1913

1914

</para>

1915

</listitem>

1916

</varlistentry>

1917

1918

1919

<term><option>nmap.set_port_state(host, port, state)</option>

1920

<indexterm><primary>set_port_state</primary></indexterm></term>

1921

1922

<para>

1923

The <literal>set_port_state()</literal> call takes a host table,

1924

a port table, and a port state (<literal>open</literal> or

1925

<literal>closed</literal>). With this method the final port

1926

state can be changed. This is useful when Nmap detects a port as

1927

<literal>open|filtered</literal> but the script successfully connects to it. In this

1928

case the port state can be set to <literal>open</literal>. Note

1929

that the <literal>port.state</literal> value, which was passed

1930

to the script's <literal>action</literal> function will not be changed

1931

by this call.

1932

</para>

1933

</listitem>

1934

</varlistentry>

1935

1936

1937

<term><option>nmap.set_port_version(host, port, probestate)</option>

1938

<indexterm><primary>set_port_version</primary></indexterm></term>

1939

1940

<para>

1941

To provide a flexible extension to Nmap's version

1942

detection NSE scripts can set the version and service

1943

variables of a port.

1944

1945

The method takes a host and a port

1946

table as arguments. The third argument describes the

1947

state in which the script completed. It is a string

1948

which is one of:

1949

<literal>hardmatched</literal>,

1950

<literal>softmatched</literal>,

1951

<literal>nomatch</literal>,

1952

<literal>tcpwrapped</literal>, or

1953

<literal>incomplete</literal>.

1954

1955

A hard match will almost always be used, as it means

1956

that the script was able to determine the protocol.

1957

You can pass in <literal>nomatch</literal> if the

1958

script fails to match the target port, but it is

1959

probably already set that way anyway. One of the

1960

other states should only be used if you know exactly

1961

what you are doing.</para>

1962

1963

<para>The host and port arguments to this function

1964

should either be the tables passed to the

1965

<literal>action</literal> method or they should have

1966

the same structure. The version detection fields this

1967

function looks at are <literal>name</literal>,

1968

<literal>product</literal>,

1969

<literal>version</literal>,

1970

<literal>extrainfo</literal>,

1971

<literal>hostname</literal>,

1972

<literal>ostype</literal>,

1973

<literal>devicetype</literal>, and

1974

<literal>service_tunnel</literal>. All values in this

1975

table are optional. It is possible to pass a table in

1976

which all these values are set to

1977

<literal>nil</literal> or not to set the values at

1978

all.

1979

</para>

1980

</listitem>

1981

</varlistentry>

1982

</variablelist>

1983

</para>

1984

</sect2>

1985

1986

<title>Various Utility Functions for Raw Packet Support</title>

1987

<para>

1988

NSE has support for sending raw ethernet frames and capturing

1989

packets. The following two functions may be handy in this context:

1990

</para>

1991

1992

1993

<term><option>nmap.clock_ms()</option>

1994

<indexterm><primary>nmap.clock_ms()</primary></indexterm></term>

1995

1996

<para>

1997

Returns a number representing the current time as milliseconds

1998

since the start of the epoch (on most systems this is 01/01/1970).

1999

</para>

2000

</listitem>

2001

</varlistentry>

2002

2003

<term><option>nmap.get_interface_link("interface_name")</option>

2004

<indexterm><primary>nmap.get_interface_link(interface_name)</primary></indexterm></term>

2005

2006

<para>

2007

For the provided dnet-style

2008

<literal>interface_name</literal>,

2009

<literal>nmap.get_interface_link()</literal> returns

2010

what kind of link level hardware the interface

2011

belongs. Return values are:

2012

<literal>ethernet</literal>,

2013

<literal>loopback</literal> or

2014

<literal>p2p</literal>. If the provided

2015

<literal>interface_name</literal> is not one of

2016

those types, <literal>nil</literal> is returned.

2017

</para>

2018

</listitem>

2019

</varlistentry>

2020

</variablelist>

2021

</sect2>

2022

2023

2024

<title>Network I/O API</title>

2025

<para>

2026

To allow for efficient and parallelizable network I/O, NSE

2027

provides an interface to Nsock, the Nmap socket library. The

2028

smart callback mechanism Nsock uses is fully transparent to

2029

NSE scripts. The main benefit of Nmap-NSE sockets is that they

2030

never block on I/O operations, allowing many scripts to be run in parallel.

2031

The I/O parallelism is fully transparent to authors of NSE scripts.

2032

In NSE you can either program as if you were using a single non

2033

blocking socket or you can program as if your connection is

2034

blocking. Seemingly blocking I/O calls still return once a

2035

specified timeout has been exceeded. Two flavors of Network I/O are

2036

supported:

2037

</para>

2038

2039

<title>Connect-style network I/O</title>

2040

<para>This part of the network API should be suitable for most

2041

classical network uses: Users create a socket, connect it to a

2042

remote address, send and receive data and close the socket again.

2043

Everything up to the Transport layer (which is either TCP, UDP or

2044

SSL) is handled by the library. The following socket API methods

2045

are supported:

2046

</para>

2047

2048

<para>

2049

2050

2051

<term><option>nmap.new_socket()</option>

2052

<indexterm><primary>nmap.new_socket()</primary></indexterm></term>

2053

2054

<para>

2055

The <literal>new_socket()</literal> Nmap call returns an

2056

Nmap-NSE socket object which is the recommended method for network

2057

I/O. It provides facilities to perform communication using the

2058

UDP, TCP and SSL protocol in a uniform manner.

2059

</para>

2060

</listitem>

2061

</varlistentry>

2062

2063

2064

<term><option>status, error = socket_object:connect(hostid, port, [protocol])</option>

2065

<indexterm><primary>connect</primary></indexterm></term>

2066

2067

2068

2069

<para>

2070

The connect method of Nmap-NSE socket objects will put

2071

the socket in a state ready for communication. It

2072

takes as arguments a host descriptor (either an IP

2073

address or a host name), a port number and optionally

2074

a protocol. The protocol must be one of

2075

<literal>"tcp"</literal>, <literal>"udp"</literal> or

2076

<literal>"ssl"</literal>. By default the connect call

2077

will attempt to open a TCP connection. On success the

2078

returned value of status is

2079

<literal>true</literal>. If the connection attempt has

2080

failed, the error value contains a description of the

2081

error condition stored as a string.

2082

Those strings are

2083

taken from the <function> gai_strerror()</function>

2084

C function. They are (with the errorcode in parentheses):</para>

2085

2086

2087

<para><quote>Address family for hostname not supported</quote> (<literal>EAI_ADDRFAMILY</literal>)</para>

2088

</listitem>

2089

2090

<para><quote>Temporary failure in name resolution</quote> (<literal>EAI_AGAIN</literal>)</para>

2091

</listitem>

2092

2093

<para><quote>Bad value for ai_flags</quote> (<literal>EAI_BADFLAGS</literal>)</para>

2094

</listitem>

2095

2096

<para><quote>Non-recoverable failure in name resolution</quote> (<literal>EAI_FAIL</literal>)</para>

2097

</listitem>

2098

2099

<para><quote>ai_family not supported</quote> (<literal>EAI_FAMILY</literal>)</para>

2100

</listitem>

2101

2102

<para><quote>Memory allocation failure</quote> (<literal>EAI_MEMORY</literal>)</para>

2103

</listitem>

2104

2105

<para><quote>No address associated with hostname</quote> (<literal>EAI_NODATA</literal>)</para>

2106

</listitem>

2107

2108

<para><quote>Name or service not known</quote> (<literal>EAI_NONAME</literal>)</para>

2109

</listitem>

2110

2111

<para><quote>Servname not supported for ai_socktype</quote> (<literal>EAI_SERVICE</literal>)</para>

2112

</listitem>

2113

2114

<para><quote>ai_socktype not supported</quote> (<literal>EAI_SOCKTYPE</literal>)</para>

2115

</listitem>

2116

2117

<para><quote>System error</quote> (<literal>EAI_SYSTEM</literal>)</para>

2118

</listitem>

2119

</itemizedlist>

2120

<para>In addition to these standard system error based messages are the following two NSE-specific errors:</para>

2121

2122

2123

<para><quote>Sorry, you don't have OpenSSL.</quote> occurs

2124

if <literal>ssl</literal> is passed as third argument, but Nmap was compiled

2125

without OpenSSL support.</para>

2126

</listitem>

2127

2128

<para><quote>invalid connection method</quote> occurs if

2129

the second parameter is not one of <literal>tcp</literal>, <literal>udp</literal>, <literal>ssl</literal>.</para>

2130

</listitem>

2131

</itemizedlist>

2132

</listitem>

2133

</varlistentry>

2134

2135

2136

<term><option>status, error = socket_object:send(data)</option>

2137

2138

2139

<para>

2140

The send method sends the data contained in the

2141

<literal>data</literal> string through an open

2142

connection. On success the returned value of status is

2143

<literal>true</literal>. If the send operation

2144

has failed, the error value contains a description of

2145

the error condition stored as a string. The error strings are:

2146

2147

2148

<para><quote>Trying to send through a closed socket</quote>—if there was no

2149

call to socket_object:connect before the send operation.</para>

2150

</listitem>

2151

2152

<para><quote>TIMEOUT</quote>—if the operation took longer than the

2153

specified timeout for the socket.</para>

2154

</listitem>

2155

2156

<para><quote>ERROR</quote>—if an error occurred inside the underlying

2157

Nsock library.</para>

2158

</listitem>

2159

2160

<para><quote>CANCELLED</quote>—if the operation was cancelled.</para>

2161

</listitem>

2162

2163

<para><quote>KILL</quote>—if for example the script scan is aborted due

2164

to a faulty script.</para>

2165

</listitem>

2166

2167

<para><quote>EOF</quote>—if an EOF was read—will probably not occur

2168

for a send operation.</para>

2169

</listitem>

2170

2171

</itemizedlist>

2172

</para>

2173

</listitem>

2174

</varlistentry>

2175

2176

2177

<term><option>status, value = socket_object:receive()</option>

2178

<indexterm><primary>receive</primary></indexterm></term>

2179

2180

<para>

2181

The receive method does a non-blocking receive operation on

2182

an open socket. On success the returned value of

2183

<literal>status</literal> is

2184

<literal>true</literal> and the received data is stored in

2185

<literal>value</literal>. If receiving data has failed,

2186

<literal>value</literal> contains a description of the error

2187

condition stored as a string. A failure occurs for example

2188

if receive is called on a closed socket. The receive call

2189

returns to the NSE script all the data currently stored

2190

in the receive buffer of the socket. Error conditions

2191

are the same as for the send operation.

2192

</para>

2193

</listitem>

2194

</varlistentry>

2195

2196

2197

<term><option>status, value = socket_object:receive_lines(n)</option>

2198

<indexterm><primary>receive_lines</primary></indexterm></term>

2199

2200

<para>

2201

Tries to receive at least <replaceable>n</replaceable>

2202

lines from an open connection. A line is a string

2203

delimited with <literal><quote>\n</quote></literal> characters. If

2204

it was not possible to receive at least

2205

<replaceable>n</replaceable> lines before the operation times

2206

out a TIMEOUT error occurs. On the other hand, if more

2207

than <replaceable>n</replaceable> lines were received, all are

2208

returned, not just <replaceable>n</replaceable>. On success

2209

the returned value of <replaceable>status</replaceable> is

2210

<literal>true</literal> and the received data is

2211

stored in <replaceable>value</replaceable>. If the connection

2212

attempt has failed, <replaceable>value</replaceable> contains

2213

a description of the error condition stored as string.

2214

Error conditions are the same as for the send operation.

2215

2216

</para>

2217

</listitem>

2218

</varlistentry>

2219

2220

2221

<term><option>status, value = socket_object:receive_bytes(n)</option>

2222

<indexterm><primary>receive_bytes</primary></indexterm></term>

2223

2224

<para>

2225

Tries to receive at least <replaceable>n</replaceable>

2226

bytes from an open connection. On success the returned

2227

value of <replaceable>status</replaceable> is <literal>true</literal> and the

2228

received data is stored in

2229

<replaceable>value</replaceable>. If operation fails,

2230

<replaceable>value</replaceable> contains a description of the

2231

error condition stored as a string. Similarly to

2232

<literal>receive_lines()</literal>

2233

<replaceable>n</replaceable> is the minimum amount of

2234

characters we would like to receive. If more arrive,

2235

we get all of them. If less than <replaceable>n</replaceable> characters arrive

2236

before the operation times out, a TIMEOUT error occurs.

2237

Other error conditions are the same as for the send operation.

2238

2239

</para>

2240

</listitem>

2241

</varlistentry>

2242

2243

2244

<term><option>status, value = socket_object:receive_buf(func/"string", keeppattern)</option>

2245

<indexterm><primary>receive_buf</primary></indexterm></term>

2246

2247

<para>

2248

<literal>receive_buf</literal> tries to circumvent several

2249

limitations in the other <literal>receive*</literal> functions.

2250

<literal>receive_line(n)</literal>, for example, tries to ensure that

2251

there are at least <replaceable>n</replaceable> lines received and returns everything it has

2252

already read from the connection (even though there may be much more

2253

data than requested). It also leaves line-parsing to the user.</para>

2254

2255

2256

<para><literal>receive_buf</literal> on the other hand returns only the

2257

part of the received data until the first match of a delimiter,

2258

with the rest being saved inside a buffer for later calls to

2259

<literal>receive_buf</literal>. This buffer gets cleared on calls to

2260

other functions inside the Network I/O API. Should the data not

2261

contain the delimiter another read request is sent and the buffer is

2262

checked again when more data is present.</para>

2263

2264

<para><literal>receive_buf</literal> takes two arguments.

2265

The first one is either a string or a function. If it is

2266

a string it gets passed to Lua's <literal><ulink url="http://www.lua.org/manual/5.1/manual.html#5.4">string.find</ulink></literal> function as the (second) pattern

2267

parameter, with the buffer data being searched. If it is a function

2268

it is expected to take exactly one parameter (the buffer) and its

2269

return values have to be like those of <literal>string.find</literal>

2270

(i.e. offsets of the start and the end of the delimiter inside the

2271

buffer, or <literal>nil</literal>, if the delimiter is not found).</para>

2272

2273

<para>The second argument is a boolean value which indicates whether the

2274

delimiting pattern should be returned along with the received data or

2275

discarded.</para>

2276

2277

<para>A module inside the

2278

nselib <literal>match.lua</literal> (<xref linkend="nse-lib-match"/>) provides

2279

functions for matching received data against regular expressions or

2280

for receiving a defined number of bytes. <literal>receive_buf</literal>'s return values behave exactly as the return values of

2281

the other <literal>receive*</literal> functions. Two values are returned (status,val)—

2282

the first indicating whether the request was successful, the other

2283

containing the returned data (or the case of a failure, an error message).</para>

2284

2285

<para>Possible error messages are those of the other

2286

<literal>receive*</literal> functions and, in addition, the following:

2287

2288

2289

<para><quote>Error inside splitting-function</quote>—if the first argument was

2290

a function which caused an error while being called.

2291

</para>

2292

</listitem>

2293

2294

<para><quote>Error in <literal>string.find</literal> (<literal>nsockobj:receive_buf</literal>)!</quote>—if a string

2295

was provided as the first argument, and string.find() yielded an

2296

error while being called.</para>

2297

</listitem>

2298

2299

<para><quote>Expected either a function or a string!</quote>—if the

2300

first argument was neither a function nor a string.</para>

2301

</listitem>

2302

2303

<para><quote>Delimiter has negative size!</quote>—if the returned start offset

2304

is greater than the end offset.</para>

2305

</listitem>

2306

</itemizedlist>

2307

</para>

2308

2309

</listitem>

2310

</varlistentry>

2311

2312

2313

<term><option>status, err = socket_object:close()</option>

2314

<indexterm><primary>close</primary></indexterm></term>

2315

2316

<para>

2317

Closes an open connection. On success the returned value of

2318

<literal>status</literal> is <literal>true</literal>. If the connection

2319

attempt has failed, <literal>value</literal> contains a description

2320

of the error condition stored as a string. Currently the only error

2321

message is: <quote>Trying to close a closed socket</quote>, which is issued if the socket

2322

has already been closed. Sockets are subject to garbage collection.

2323

Should you forget to close a socket, it will get closed before it gets

2324

deleted (on the next occasion Lua's garbage collector is run).

2325

However since garbage collection cycles are difficult to predict, it

2326

is considered good practice to close opened sockets.

2327

</para>

2328

</listitem>

2329

</varlistentry>

2330

2331

2332

<term><option>status,localip,localport,remoteip,remoteport=socket_object:get_info()</option>

2333

2334

2335

<para>

2336

This function returns information about the socket

2337

object. It returns 5 values. If an error occurred, the

2338

first value is <literal>nil</literal> and the second

2339

value describes the error condition. Otherwise the

2340

first value describes the success of the operation and

2341

the remaining 4 values describe both endpoints of the

2342

TCP connection. If you put the call in a <literal>try()</literal> statement

2343

the status value is consumed. The call can be used for example if

2344

you want to query an authentication server.

2345

</para>

2346

</listitem>

2347

</varlistentry>

2348

2349

2350

2351

<term><option>socket_object:set_timeout(t)</option>

2352

<indexterm><primary>set_timeout</primary></indexterm></term>

2353

2354

<para>

2355

Sets the time, in milliseconds, after which input and

2356

output operations on a socket should time out and

2357

return. The default value is 30,000 (30 seconds). The lowest

2358

allowed value is 10ms, since this is

2359

the granularity of NSE network I/O.

2360

</para>

2361

</listitem>

2362

</varlistentry>

2363

2364

</variablelist>

2365

</para>

2366

</sect3>

2367

2368

<title>Raw packet network I/O</title>

2369

<para>For those cases where the connection oriented approach is too inflexible,

2370

NSE provides script developers with a more powerful option:

2371

raw packet network I/O. The greater flexibility comes, however, at

2372

the cost of a slightly more complex API. Receiving raw packets is

2373

accomplished via a wrapper around Libpcap inside

2374

the Nsock library. In order to keep the

2375

capturing efficient it works in a three tiered approach: Opening a

2376

device for capturing, registering listeners to it and receiving

2377

packets. With each call to <literal>pcap_open()</literal> you have

2378

to provide a callback function, which receives the packet (along with

2379

it's layer 2 and 3 headers) and is used to compute a so-called

2380

packet hash. Each call to <literal>pcap_register()</literal> takes a

2381

binary string as argument. For every packet captured the computed

2382

hash is matched against all registered strings.

2383

Those scripts for which the compare yields true are then provided

2384

with the packet as a return value to <literal>pcap_receive()</literal>.

2385

The more general the packet hash computing function is kept,

2386

the more scripts may receive the packet and proceed with their

2387

execution. To use the packet capturing inside your script you have to

2388

create (and afterwards close) a socket with

2389

<literal>nmap.newsocket()</literal>

2390

(or <literal>socket_object:close()</literal> respectively)—just

2391

like with the connection-based network I/O. A more detailed description

2392

of the functions for packet capturing follows:

2393

</para>

2394

2395

<para>

2396

2397

2398

<term><option>socket_object:pcap_open(device, snaplen, promisc,

2399

test_function, bpf)</option>

2400

2401

2402

<para>

2403

The <literal>pcap_open()</literal> call opens the socket for

2404

packet capturing. The parameters are:</para>

2405

2406

<listitem><para><literal>device</literal>—the dnet-style interface name of the device you want to capture from.</para></listitem>

2407

<listitem><para><literal>snaplen</literal>—defines the length of each packet you want to capture (similar to the <option>-s</option> option to <command>tcpdump</command>)</para></listitem>

2408

<listitem><para><literal>promisc</literal>—should be set to <literal>1</literal> if the interface should activate promiscuous mode, and zero otherwise.</para></listitem>

2409

<listitem><para><literal>test_function</literal>—callback function used to compute the <literal>packet-hash</literal></para></listitem>

2410

<listitem><para><literal>bpf</literal>—a string describing a Berkeley packet filter expression (like those provided to <command>tcpdump</command>)</para></listitem>

2411

</itemizedlist>

2412

</listitem>

2413

</varlistentry>

2414

2415

<term><option>socket_object:pcap_register(packet-hash)</option>

2416

<indexterm><primary>pcap_register</primary></indexterm></term>

2417

2418

<para>

2419

Starts the listening for incoming packages. The provided

2420

<literal>packet-hash</literal> is a binary string which has to

2421

match the hash returned by the

2422

<literal>test_function</literal> parameter provided to

2423

<literal>pcap_open()</literal>. If you want to receive all

2424

packets, just provide the empty string (<literal>""</literal>).

2425

There has to be a call to <literal>pcap_register()</literal>

2426

before a call to <literal>pcap_receive()</literal>.

2427

</para>

2428

</listitem>

2429

</varlistentry>

2430

2431

<term><option>status, packet_len, l2_data, l3_data = socket_object:pcap_receive()</option>

2432

<indexterm><primary>pcap_receive</primary></indexterm></term>

2433

2434

<para>

2435

Receives a captured packet. If successful, the return values are:</para>

2436

2437

<listitem><para><literal>status</literal>—a boolean with the value <literal>true</literal>.</para></listitem>

2438

<listitem><para><literal>packet_len</literal>—the length of the captured packet (note, that you could have received less data if the snaplen parameter was smaller than the packet length)</para></listitem>

2439

<listitem><para><literal>l2_data</literal>—data from the second OSI layer (e.g. ethernet headers)</para></listitem>

2440

<listitem><para><literal>l3_data</literal>—data from the third OSI layer (e.g. IPv4 headers).</para></listitem>

2441

</itemizedlist>

2442

2443

<para>Should an error or timeout occur, while waiting for a packet the

2444

return values are: <literal>nil,error_message,nil,nil</literal>, where

2445

error_message describes the occurred error.</para>

2446

2447

</listitem>

2448

</varlistentry>

2449

2450

<term><option>socket_object:pcap_close()</option>

2451

<indexterm><primary>pcap_close()</primary></indexterm></term>

2452

2453

<para>Closes the pcap device.

2454

</para>

2455

</listitem>

2456

</varlistentry>

2457

</variablelist>

2458

</para>

2459

<para>

2460

Receiving raw packets is a great feature, but it is also only the

2461

half job. Now for sending raw packets: To accomplish this NSE has

2462

access to a wrapper around the <literal>dnet</literal> library.

2463

Currently NSE has the ability to send raw ethernet frames via the

2464

following API:

2465

</para>

2466

<para>

2467

2468

2469

<term><option>dnet_object=nmap.new_dnet()</option>

2470

2471

2472

<para>

2473

Creates and returns a new dnet_object, which can be used to

2474

send raw packets.

2475

</para>

2476

</listitem>

2477

</varlistentry>

2478

2479

<term><option>dnet_object:ethernet_open(interface_name)</option>

2480

<indexterm><primary>ethernet_open</primary></indexterm></term>

2481

2482

<para>Opens the interface defined by the provided

2483

<replaceable>interface_name</replaceable> for sending ethernet frames

2484

through it. An error (<quote>device is not valid ethernet

2485

interface</quote>) is thrown in case the provided argument

2486

is not valid.

2487

</para>

2488

</listitem>

2489

</varlistentry>

2490

2491

<term><option>dnet_object:ethernet_send(packet)</option>

2492

<indexterm><primary>ethernet_send</primary></indexterm></term>

2493

2494

<para>

2495

Sends the provided data as ethernet frame across the previously

2496

opened interface. Note that you have to provide the packet

2497

including IP header and ethernet header. If there was no

2498

previous valid call to <literal>ethernet_open()</literal> an

2499

error is thrown (<quote>dnet is not valid opened ethernet

2500

interface</quote>).

2501

</para>

2502

</listitem>

2503

</varlistentry>

2504

2505

<term><option>dnet_object:ethernet_close()</option>

2506

<indexterm><primary>ethernet_close</primary></indexterm></term>

2507

2508

<para>Closes the interface. The only error which may be thrown

2509

is the same as for the <literal>ethernet_send()</literal>

2510

operation.

2511

</para>

2512

</listitem>

2513

</varlistentry>

2514

</variablelist>

2515

</para>

2516

</sect3>

2517

</sect2>

2518

2519

<title>Exception Handling</title>

2520

<para>

2521

NSE provides an exception handling mechanism not present in

2522

the plain Lua language. The exception handling is tailored

2523

specifically for network I/O operations. The mechanism

2524

follows a functional programming paradigm rather than an

2525

object oriented programming paradigm. To create an exception

2526

handler the <literal>nmap.new_try()</literal> API method is

2527

used. This method returns a function, which takes a function

2528

as an argument. If the function passed as an argument raises

2529

an exception, then the script execution is aborted and no

2530

output is produced. Optionally you can pass a function to

2531

the <literal>new_try()</literal> method which will be called

2532

if an exception is caught. In this function you can perform

2533

required clean up operations.</para>

2534

2535

<para>

2536

<xref linkend="nse-exception-handling"/> shows cleanup

2537

exception handling at work. A new function named

2538

<literal>catch</literal> is defined to simply close the

2539

newly created socket in case of an error. It is then used

2540

to protect connection and communication attempts on that

2541

socket. If no catch function is specified, execution of the

2542

script aborts without further ado—open sockets

2543

will remain open. If the verbosity level is at least one

2544

or if the scan is performed in debugging mode a description

2545

of the uncaught error condition is printed on standard output.

2546

Note that it is currently not easily possible to group several

2547

statements in one try block. It is also important to remember

2548

that if the socket is not closed it will occupy memory

2549

until the next run of Lua's garbage collector.

2550

2551

</para>

2552

2553

2554

<title>Exception handling example</title>

2555

2556

local result, socket, try, catch

2557

2558

result = ""

2559

socket = nmap.new_socket()

2560

catch = function()

2561

socket:close()

2562

end

2563

try = nmap.newtry(catch)

2564

2565

try(socket:connect(host.ip, port.number))

2566

result = try(socket:receive_lines(1));

2567

try(socket:send(result))

2568

</programlisting>

2569

</example>

2570

2571

<para>

2572

Writing a function which is treated properly by the

2573

try/catch mechanism is straightforward. The function should

2574

return multiple values. The first value should be a boolean

2575

which is <literal>true</literal> upon successful completion of the function and

2576

<literal>false</literal> otherwise. If the function completed successfully the try

2577

construct consumes the indicator value and returns the

2578

remaining values. If the function failed then the second

2579

returned value must be a string describing the error

2580

condition. Note that that if the value is not <literal>nil</literal> it is

2581

treated as <literal>true</literal> so you can return your

2582

value in the normal case and return <literal>nil, <replaceable>error description</replaceable></literal>

2583

if an error occurs.

2584

</para>

2585

2586

</sect2>

2587

2588

2589

<title>The Registry<indexterm><primary>registry</primary></indexterm></title>

2590

<para>

2591

The registry is a normal Lua table. What is special about it

2592

is that it is visible by all scripts and it retains its state

2593

between script executions. Nmap does not scan every host

2594

specified on the command line at the same time, it puts them

2595

in smaller groups and these groups are scanned in parallel. The

2596

registry is rebuilt for every group, so information stored

2597

there is only deleted after NSE finishes processing the

2598

current target group. This implies of course that the registry

2599

is transient—it is not stored between Nmap executions. Every

2600

script can read the registry and write to it. If a script is

2601

running after another script, it can read some information in

2602

the registry which was left by the first script. This feature

2603

is particularly powerful in combination with the run level

2604

concept. A script with a higher run level can rely on entries

2605

left behind for it by scripts with lower run levels. Remember

2606

however that the registry can be written by all scripts

2607

equally, so choose the keys for your entries wisely. The

2608

registry is stored in <literal>nmap.registry</literal>. The

2609

behavior of the registry allows caching of already calculated

2610

data. The cache can be seen by all scripts until the registry

2611

is rebuilt with the next target group. <!-- If for example you have

2612

compiled a regular expression, you can store the compiled

2613

expression in the registry so that scripts which need the same

2614

pattern do not have to recompile it. -->

2615

2616

</para>

2617

</sect2>

2618

</sect1>

2619

2620

2621

<title>Script Writing Tutorial</title>

2622

2623

<para>

2624

Suppose that you are convinced of the power of NSE. How do you

2625

go about writing your own script? Let's say

2626

that you want to extract information from an identification

2627

server. Nmap used to have this functionality but it was removed

2628

because of inconsistencies in the code base. Fortunately, the

2629

protocol identd uses is pretty simple. Unfortunately, it is too

2630

complicated to be expressible in Nmap's version detection

2631

language. Let's look at how the identification protocol

2632

works. First you connect to the identification server. Next you

2633

send a query of the form <literal><replaceable>port-on-server</replaceable>,

2634

<replaceable>port-on-client</replaceable></literal> terminated with a new line

2635

character. The server should then respond with a string of the

2636

form <literal><replaceable>port-on-server</replaceable>, <replaceable>port-on-client</replaceable>:<replaceable>response-type</replaceable>:<replaceable>address-information</replaceable></literal>. In case of an error the address

2637

information is omitted. This description is sufficient for our

2638

purposes, for more details refer to <ulink role="hidepdf" url="http://www.rfc-editor.org/rfc/rfc1413.txt">RFC 1413</ulink>. The protocol cannot be modeled in Nmap's version

2639

detection language for two reasons. The first is that you need

2640

to know both the local and the remote port of a

2641

connection. Version detection does not provide this data. The

2642

second, more severe obstacle, is that you need two open

2643

connections to the target—one to the identification server and

2644

one to the port you want to query. Both obstacles are easily

2645

overcome with NSE. </para>

2646

2647

<para>

2648

The anatomy of a script is described in <xref linkend="nse-scripts"/>.

2649

In this section we will show how the described structure is utilized.

2650

</para>

2651

2652

2653

2654

<para>

2655

The head of the script is essentially its meta

2656

information. This includes the fields

2657

<literal>id</literal>, <literal>description</literal>,

2658

<literal>author</literal>, <literal>license</literal> and

2659

<literal>categories</literal>. We are not going to change the

2660

run level for now. The <literal>id</literal> of a script

2661

should uniquely identify it. If it is absent, the path to the

2662

script will be used as an id. We recommend to choose an id

2663

which concisely identifies the purpose of the script, since

2664

the ID is printed before the script's results in Nmap output.

2665

</para>

2666

<para>

2667

2668

id = "Service Owner"

2669

</programlisting>

2670

</para>

2671

<para>

2672

The description field should contain a sentence or two describing what the script does. If anything about the script results might confuse or mislead users, and you can't eliminate the issue by improving the script or results text, it should be documented in the <literal>description</literal> string.

2673

</para>

2674

<para>

2675

2676

description = "Opens a connection to the scanned port, opens a connection to \

2677

port 113, queries the owner of the service on the scanned port and prints it."

2678

</programlisting>

2679

</para>

2680

<para>

2681

2682

Users must tell the Lua interpreter that the string

2683

continues on the following line by ending the line with a

2684

backslash (‘<literal>\</literal>’). They must also decide what

2685

categories the script belongs to. This script is a good

2686

example of a script which cannot be categorized clearly. It is

2687

<literal>safe</literal> because we are not using the service

2688

for anything it was not intended for. On the other hand, it

2689

is <literal>intrusive</literal> because we connect to a

2690

service on the target and therefore potentially give out

2691

information about us. To solve this dilemma we will place our

2692

script in two categories:

2693

2694

</para>

2695

2696

categories = {"safe", "intrusive"}

2697

</programlisting>

2698

</sect2>

2699

2700

2701

2702

<para>

2703

The rule section is a Lua method which decides when the

2704

script's action should be performed and when it should be

2705

skipped. Usually this decision is based on the host and port

2706

information passed to the rule function. In the case of the

2707

identification script it is slightly more complicated than

2708

that. To decide whether to run the identification script on a

2709

given port we need to know if there is an identification

2710

server running on the target machine. Or more formally: the

2711

script should be run if (and only if) the currently scanned TCP port is open and

2712

TCP port 113 is also open. For now we will rely on the fact that

2713

identification servers listen on TCP port 113. Unfortunately NSE

2714

only gives us information about the currently scanned port.

2715

2716

To find out if port 113 is open we are going to use the

2717

<literal>nmap.get_port_state()</literal> method. If the identd

2718

port was not scanned, the <literal>get_port_state</literal>

2719

function returns <literal>nil</literal>. So we need to make

2720

sure that the table is not <literal>nil</literal>. We also

2721

check if both ports are in the <literal>open</literal> state.

2722

If this is the case, the action is executed, otherwise we skip

2723

the action.

2724

</para>

2725

<para>

2726

2727

portrule = function(host, port)

2728

local identd, decision

2729

2730

local ident_port = { number=113, protocol="tcp" }

2731

identd = nmap.get_port_state(host, ident_port)

2732

2733

if

2734

identd ~= nil and identd.state == "open" and port.state == "open"

2735

then

2736

decision = true

2737

else

2738

decision = false

2739

end

2740

2741

return decision

2742

end

2743

</programlisting>

2744

</para>

2745

2746

<para>

2747

This rule is <emphasis>almost</emphasis> correct, but still

2748

slightly buggy. Can you find the bug? It is a pretty subtle

2749

one. The problem is that this script fires on any kind of open

2750

port, TCP or UDP. The <literal>connect()</literal> method on

2751

the other hand assumes a TCP protocol unless it is explicitly

2752

told to use another protocol. Since the identification service

2753

is only defined for TCP connections, we need to narrow down

2754

the range of ports which fire our script. Our new rule only

2755

runs the script if the port is open, we are looking at a TCP

2756

port, and TCP port 113 is open. Writing the new and

2757

improved port rule is left as an exercise to the reader (or

2758

peek at the script in the latest Nmap distribution).

2759

</para>

2760

2761

2762

</sect2>

2763

2764

2765

<title>The Mechanism</title>

2766

<para>

2767

At last we implement the actual functionality. The script will

2768

first connect to the port on which we expect to find the

2769

identification server, then it will connect to the port we

2770

want information about. Afterward we construct a query string

2771

and parse the response. If we received a satisfactory

2772

response, we return the retrieved information.

2773

</para>

2774

<para>

2775

First we need to create two socket objects. These objects

2776

represent the sockets we are going to use. By using object methods

2777

like

2778

<literal>open()</literal>,

2779

<literal>close()</literal>,

2780

<literal>send()</literal> or

2781

<literal>receive()</literal> we can operate on the network

2782

socket. To avoid excessive error checking code we use NSE's

2783

exception handling mechanism. We create a function which will

2784

be executed if an error occurs and call this function

2785

<literal>catch</literal>. Using this function we generate

2786

a <literal>try</literal> function. The <literal>try</literal>

2787

function will call the <literal>catch</literal> function

2788

whenever there is an error condition in the tried block. Note

2789

that we could have ignored the last two return values

2790

of <literal>client_service:get_info()</literal> like this:

2791

2792

2793

local localip, localport = client_service:get_info()

2794

</programlisting>

2795

This would have sufficed because we know that the remote port is

2796

stored in <literal>port.number</literal>.</para>

2797

2798

<para>In this example we

2799

prefer not to tell the user if the query resulted in an

2800

error. To inform users of failed

2801

identification queries, simply uncomment the corresponding

2802

line. It is necessary that we assign the variable <literal>owner</literal>

2803

a <literal>nil</literal> value because returning <literal>nil</literal>

2804

is the only way to tell the script engine to suppress script output.

2805

</para>

2806

2807

<para>

2808

2809

action = function(host, port)

2810

local owner = ""

2811

2812

local client_ident = nmap.new_socket()

2813

local client_service = nmap.new_socket()

2814

2815

local catch = function()

2816

client_ident:close()

2817

client_service:close()

2818

end

2819

2820

local try = nmap.newtry(catch)

2821

2822

try(client_ident:connect(host.ip, 113))

2823

try(client_service:connect(host.ip, port.number))

2824

2825

local localip, localport, remoteip,

2826

remoteport = client_service:get_info()

2827

2828

local request = port.number .. ", " .. localport .. "\n"

2829

2830

try(client_ident:send(request))

2831

2832

owner = try(client_ident:receive_lines(1))

2833

2834

if string.match(owner, "ERROR") then

2835

owner = nil

2836

-- owner = "Service owner could not be determined: " .. owner

2837

else

2838

owner = string.match(owner, "USERID : .+ : (.+)\n", 1)

2839

end

2840

2841

try(client_ident:close())

2842

try(client_service:close())

2843

2844

return owner

2845

end

2846

</programlisting>

2847

</para>

2848

</sect2>

2849

</sect1>

2850

2851

2852

<title>Version Detection using NSE</title>

2853

<para>

2854

2855

The version detection system built into Nmap was designed to

2856

efficiently recognize the vast majority of protocols with a

2857

simple pattern matching syntax. Some protocols require a more

2858

complex approach, and a generalized scripting language is

2859

perfect for this. Skype2 is one such protocol. It pretends to

2860

be an http server, requiring multiple queries to determine its

2861

true nature. NSE has been integrated into Nmap's version

2862

detection framework to handle these cases. The scripts which

2863

extend the version scanner belong to the reserved category

2864

<literal>version</literal>. This category cannot be run from

2865

the command line. It is only executed if the user has required a

2866

version scan. The following listing shows a simple script which

2867

demonstrates the use of the NSE version detection API. If either

2868

the TCP port 80 is open or the service has been determined to be

2869

http, the script is triggered. Although it could be extended to

2870

recognize different http servers, its only purpose is to show off

2871

the version detection API. It is not advisable to use NSE for

2872

version detection in the simple case of http servers. The

2873

version detection variables have been filled with dummy entries

2874

to illustrate their effect on the Nmap output.</para>

2875

2876

2877

<para>

2878

2879

description = "Demonstration of a version detection NSE script. It checks \

2880

and reports the version of a remote web server. For real life purposes it is \

2881

better to use Nmap version detection (-sV)."

2882

author = "Diman Todorov <diman.todorov@gmail.at>

2883

license = "See Nmap's COPYING for license"

2884

2885

id = "HTTP version"

2886

2887

categories = {"version"}

2888

2889

runlevel = 1.0

2890

2891

portrule = function(host, port)

2892

if (port.number == 80

2893

or port.service == "http" )

2894

and port.protocol == "tcp"

2895

then

2896

return true

2897

else

2898

return false

2899

end

2900

end

2901

2902

action = function(host, port)

2903

local query = "GET / HTTP/2.1\r\n"

2904

query = query .. "Accept: */*\r\n"

2905

query = query .. "Accept-Language: en\r\n"

2906

query = query .. "User-Agent: Nmap NSE\r\n"

2907

query = query .. "Host: " .. host.ip .. ":" .. port.number .. "\r\n\r\n"

2908

2909

local socket = nmap.new_socket()

2910

local catch = function()

2911

socket:close()

2912

end

2913

2914

local try = nmap.new_try(catch)

2915

2916

try(socket:connect(host.ip, port.number))

2917

try(socket:send(query))

2918

2919

local response = ""

2920

local lines

2921

local status

2922

local value

2923

2924

while true do

2925

status, lines = socket:receive_lines(1)

2926

2927

if not status or value then

2928

break

2929

end

2930

2931

response = response .. lines

2932

value = string.match(response, "Server: (.-)\n")

2933

end

2934

2935

try(socket:close())

2936

2937

if value then

2938

port.version.name = "[Name]"

2939

port.version.name_confidence = 10

2940

port.version.product = "[Product]"

2941

port.version.version = "[Version]"

2942

port.version.extrainfo = "[ExtraInfo]"

2943

port.version.hostname = "[HostName]"

2944

port.version.ostype = "[OSType]"

2945

port.version.devicetype = "[DeviceType]"

2946

2947

port.version.service_tunnel = "none"

2948

port.version.fingerprint = nil

2949

nmap.setPortVersion(host, port, "hardmatched")

2950

end

2951

end

2952

</programlisting>

2953

</para>

2954

<para>

2955

This is what the output of this script looks like:

2956

2957

$ ./nmap -sV localhost -p 80

2958

2959

Starting Nmap ( http://insecure.org )

2960

Interesting ports on localhost (127.0.0.1):

2961

PORT STATE SERVICE VERSION

2962

80/tcp open [Name] [Product] [Version] ([ExtraInfo])

2963

Service Info: Host: [HostName]; OS: [OSType]; Device: [DeviceType]

2964

2965

Nmap finished: 1 IP address (1 host up) scanned in 9.317 seconds

2966

</screen>

2967

</para>

2968

<para>

2969

The name variable denotes the detected protocol name.

2970

The product, version and extrainfo variables are used

2971

to produce a human readable description of the server

2972

version. The remaining variables provide information deduced

2973

from the output of the server concerning the target host.

2974

</para>

2975

</sect1>

2976

2977

2978

<title>Example Scripts</title>

2979

<para>

2980

2981

This section should probably provide 2–3 scripts

2982

which show a diverse and interesting set of NSE features. Each

2983

script should probably have its own sect2 containing a brief

2984

description of the script and anything noteworthy about it,

2985

followed by the script itself with annotations (lineannotation

2986

tag) as you can see, for example, at

2987

<ulink url="http://nmap.org/vscan/vscan-technique-demo.html"/>.

2988

</remark>

2989

</para>

2990

2991

<para>

2992

2993

2994

DT: perhaps include an optional version field

2995

</remark>

2996

2997

</para>

2998

2999

<title>Finger-Test Script</title>

3000

<para>The finger script (<filename>finger.nse</filename>) is a perfect

3001

example of how short typical NSE scripts are.

3002

</para>

3003

3004

<para>first the information fields are filled out, note that the

3005

<literal>id</literal> field is kept short, this is important since it is

3006

printed in Nmap's output. A detailed description of what the script

3007

actually does should go in the <literal>description</literal> field.</para>

3008

3009

id="Finger Results"

3010

3011

description="attempts to get a list of usernames via the finger service"

3012

3013

author = "Eddie Bell <ejlbell@gmail.com>"

3014

3015

license = "See nmaps COPYING for licence"

3016

</programlisting>

3017

3018

<para>The <literal>categories</literal> field is a table

3019

containing all the categories the script belongs to—These are used for

3020

script selection through the <option>--script</option> option.</para>

3021

3022

3023

categories = {"discovery"}

3024

</programlisting>

3025

3026

<para>You can use the facilities provided by the nselib (<xref

3027

linkend="nse-library"/>) by <literal>requiring</literal> them. Here

3028

we want to use shorter port rules.</para>

3029

3030

3031

require "shortport"

3032

</programlisting>

3033

3034

<para>We want to check whether the service behind the port is finger,

3035

or whether it runs on finger's well known port 79. Through this we can

3036

use the information gathered during the version scan (if finger runs

3037

on a non-standard port) or still run against at least the port we

3038

expect it, should the version detection information not be available.</para>

3039

3040

3041

portrule = shortport.port_or_service(79, "finger")

3042

3043

action = function(host, port)

3044

local socket = nmap.new_socket()

3045

local results = ""

3046

local status = true

3047

</programlisting>

3048

3049

<para>The function <literal>err_catch()</literal> will be called for

3050

clean up, through NSE's exception handling mechanism. Here it only

3051

closes the previously opened socket (which should be enough in most

3052

cases).</para>

3053

3054

3055

local err_catch = function()

3056

socket:close()

3057

end

3058

</programlisting>

3059

3060

<para>The clean up function gets registered for exception handling via

3061

a call to <literal>nmap.new_try()</literal></para>

3062

3063

3064

local try = nmap.new_try(err_catch())

3065

</programlisting>

3066

3067

<para>The script sets a timeout of 5000, which is equivalent to 50

3068

seconds. Should any operation require more time we'll receive a

3069

<literal>TIMEOUT</literal> error message.</para>

3070

3071

3072

socket:set_timeout(5000)

3073

</programlisting>

3074

3075

<para>For actually using exception handling we need to wrap calls to

3076

functions, which may return an error inside

3077

3078

3079

3080

try(socket:connect(host.ip, port.number, port.protocol))

3081

try(socket:send("\n\r"))

3082

</programlisting>

3083

3084

<para>The call to <literal>receive_lines()</literal> is not wrapped in

3085

<literal>try()</literal>, because we don't want to abort the script

3086

just because we didn't receive the data we expected. Note that there

3087

is less data than requested (100 lines), we still receive it and the

3088

status is <literal>true</literal> —consequent calls would yield

3089

a <literal>false</literal> status.</para>

3090

3091

3092

status, results = socket:receive_lines(100)

3093

socket:close()

3094

</programlisting>

3095

3096

<para>The script returns a string only if we got the data we

3097

wanted, otherwise <literal>nil</literal> is returned (automatically, since

3098

scripts return one result).</para>

3099

3100

3101

if not(status) then

3102

return results

3103

end

3104

end

3105

</programlisting>

3106

</sect2>

3107

3108

3109

<title>Service Owner Lookup via Identd</title>

3110

<para><filename>showOwner.nse</filename> demonstrates the flexibility

3111

of the NSE, which is unmatched by other parts of Nmap. If the target

3112

is running an <literal>identd</literal> daemon it connects to it for

3113

each running service and tries to identify its owner.

3114

</para>

3115

3116

id = "Service owner"

3117

3118

description = "Opens a connection to the scanned port, opens a connection to \

3119

port 113, queries the owner of the service on the scanned port and prints it."

3120

3121

author = "Diman Todorov <diman.todorov@gmail.com>"

3122

3123

license = "See nmaps COPYING for licence"

3124

3125

categories = {"safe"}

3126

3127

</programlisting>

3128

3129

<para>Portrules are not restricted to those provided by the

3130

short-port module (<xref linkend="nse-lib-shortport"/>).

3131

They can be any function taking a host- and a porttable as argument and

3132

returning a boolean.

3133

</para>

3134

3135

3136

portrule = function(host, port)

3137

local identd, decision

3138

</programlisting>

3139

3140

<para>In order to determine the state of a port, which is not provided

3141

as argument we just have to construct a table describing the port

3142

(i.e. its number and the protocol it's using) and pass it to

3143

<literal>nmap.get_port_state()</literal> which returns a table filled

3144

with the information Nmap has about the port.</para>

3145

3146

3147

local auth_port = { number=113, protocol="tcp" }

3148

identd = nmap.get_port_state(host, auth_port)

3149

3150

if

3151

identd ~= nil

3152

and identd.state == "open"

3153

then

3154

decision = true

3155

else

3156

decision = false

3157

end

3158

3159

return decision

3160

end

3161

3162

action = function(host, port)

3163

local owner = ""

3164

</programlisting>

3165

3166

<para>Scripts can open any number of connection they want.</para>

3167

3168

3169

local client_ident = nmap.new_socket()

3170

local client_service = nmap.new_socket()

3171

3172

local catch = function()

3173

client_ident:close()

3174

client_service:close()

3175

end

3176

3177

local try = nmap.new_try(catch)

3178

3179

try(client_ident:connect(host.ip, 113))

3180

try(client_service:connect(host.ip, port.number))

3181

3182

local localip,localport,remoteip,remoteport = try(client_service:get_info())

3183

3184

local request = port.number .. ", " .. localport .. "\n"

3185

3186

try(client_ident:send(request))

3187

3188

owner = try(client_ident:receive_lines(1))

3189

3190

if string.match(owner, "ERROR") then

3191

owner = nil

3192

else

3193

owner = string.match(owner, "USERID : .+ : (.+)\n", 1)

3194

end

3195

3196

try(client_ident:close())

3197

try(client_service:close())

3198

3199

return owner

3200

end

3201

</programlisting>

3202

</sect2>

3203

</sect1>

3204

3205

<title>Implementation</title>

3206

<para>

3207

3208

We don't need a dozen pages of low-level trivial

3209

details, but it would be nice to have a few sections

3210

describing notable aspects of the NSE implementation (maybe

3211

things like how the parallelization algorithms work, how Lua

3212

is embedded, performance related notes. Information which

3213

might help script writers is particularly desirable. I tend

3214

to think reasons for choosing Lua may be better suited to

3215

<xref linkend="nse-lua"/>, but it could be placed here

3216

instead.

3217

</remark>

3218

</para>

3219

<para>

3220

Now how does all this work? The following section describes

3221

some interesting aspects of the NSE. While the focus primarily lies on

3222

giving script writers a better feeling of what happens with scripts, it

3223

should also provide a starting point for understanding (and extending) the

3224

NSE sources.

3225

</para>

3226

3227

<title>Initialization Phase</title>

3228

<para>

3229

During its initialization stage, Nmap loads the Lua interpreter and its provided

3230

libraries get loaded. These libraries are:</para>

3231

3232

3233

<para>The <emphasis>package</emphasis> library (namespace:

3234

<literal>package</literal>)—Lua's

3235

<ulink url="http://www.lua.org/manual/5.1/manual.html#5.3">package-lib</ulink> provides (among others) the <literal>require</literal> function, used to load modules from the

3236

nselib.

3237

</para>

3238

</listitem>

3239

3240

<para>The <emphasis>table</emphasis> library (namespace:

3241

<literal>table</literal>)—The

3242

<ulink url="http://www.lua.org/manual/5.1/manual.html#5.5">table manipulation library</ulink> contains many functions used

3243

to operate on <literal>tables</literal>—Lua's central data

3244

structure.

3245

</para>

3246

</listitem>

3247

3248

<para>The <emphasis>I/O</emphasis> library (namespace:

3249

<literal>io</literal>)—The

3250

<ulink url="http://www.lua.org/manual/5.1/manual.html#5.7">Input/Output library</ulink> offers functions such as reading files and reading the output from programs you execute.

3251

</para>

3252

</listitem>

3253

3254

<para>The <emphasis>OS</emphasis> library (namespace:

3255

<literal>os</literal>)—The

3256

<ulink url="http://www.lua.org/manual/5.1/manual.html#5.8">Operating System lib</ulink> provides facilities of the operating

3257

system, including filesystem operations (renaming/removing files,

3258

creating of temporary filenames) and access to the environment.

3259

</para>

3260

</listitem>

3261

3262

<para>The <emphasis>string</emphasis> library (namespace:

3263

<literal>string</literal>)—The

3264

3265

string library </ulink> helps you with functions used to manipulate

3266

strings inside Lua. Functions include: printf-style

3267

string formating, pattern matching using Lua-style patterns,

3268

substring extraction, etc.

3269

</para>

3270

</listitem>

3271

3272

<para>The <emphasis>math</emphasis> library (namespace:

3273

<literal>math</literal>)—Since usually numbers in Lua correspond

3274

to the <literal>double</literal> C-type, the

3275

<ulink url="http://www.lua.org/manual/5.1/manual.html#5.6">math library</ulink> gives you access to rounding functions,

3276

trigonometric functions, random number generation, and many more.

3277

</para>

3278

</listitem>

3279

3280

<para>The <emphasis>debug</emphasis> library (namespace:

3281

<literal>debug</literal>)—The

3282

<ulink url="http://www.lua.org/manual/5.1/manual.html#5.9">debug library</ulink> provides you with a somewhat lower level API

3283

to the Lua-interpreter. Through it you can access functions along

3284

the execution stack, get function closures and object metatables,

3285

etc.

3286

</para>

3287

</listitem>

3288

</itemizedlist>

3289

3290

<para>In addition to loading the libraries provided with Lua, the functions

3291

in the <literal>nmap</literal> namespace also get loaded. and search

3292

path for modules is set to the default one prepended by the nselib

3293

directory (which is searched in the locations Nmap searches for its

3294

data files and scripts). In this step the provided script arguments

3295

also get stored inside the <literal>registry</literal>.</para>

3296

3297

<para>

3298

The next phase of NSE initialization is loading the chosen

3299

scripts, which are the arguments provided to the

3300

<option>--script</option> option or <literal>safe,intrusive</literal>, in

3301

case of a default script scan. The string <literal>version</literal>

3302

is appended, if version detection was enabled.

3303

The arguments afterwards are tried to be

3304

interpreted as script categories. This is done via a short Lua function

3305

hard-coded into <filename>nse_init.cc</filename> called <literal>Entry</literal>. If you take a look into the <filename>script.db</filename> you'll see that the <literal>Entry</literal> lines inside

3306

it are Lua function calls with a table as argument.

3307

The arguments that didn't produce any filenames are then interpreted

3308

as file or directory names themselves. If this also fails, the script scan is aborted.</para>

3309

3310

<para>

3311

In the next stage the found files are loaded as chunks, each with

3312

its own environment, having read but not write access to the global

3313

name space and saved inside two globally accessible Lua tables:

3314

<literal>hosttests</literal> and <literal>porttests</literal>

3315

depending on the type of script. Because scripts only get loaded once, values stored inside variables during a script's execution against one host or port can be accessed when the same script runs against another target. This can be used to save computation time when a script is run

3316

against multiple targets. See <xref linkend="nse-example-persistent-locals"/>.

3317

3318

3319

During this stage scripts are

3320

also are also provided with a default <literal>runlevel</literal> (1.0), if they

3321

don't specify one themselves and a check is performed whether they

3322

contain an <literal>action</literal> and a <literal>description</literal> field.

3323

</para>

3324

3325

3326

<title>Using local variables to save data.</title>

3327

3328

id="persistent locals example"

3329

description="This sample script shows how data can be stored across \

3330

several invocations of a script against multiple targets"

3331

author="Stoiko Ivanov"

3332

categories = {"safe"}

3333

3334

require "shortport"

3335

portrule = shortport.portnumber(80)

3336

-- we have to declare the variable in the script's global scope

3337

-- because if we declare it inside the action it would get redefined

3338

-- with each call to the action

3339

local filecontent = nil

3340

require "strbuf"

3341

action= function(host, port)

3342

if(filecontent == nil) then

3343

filecontent = strbuf.new()

3344

for line in io.lines("a_filename_we_want_to_read_from")

3345

filecontent = filecontent .. line

3346

end

3347

end

3348

--rest of the script doing something with the filecontent, we just

3349

--read

3350

end

3351

</programlisting>

3352

</example>

3353

</sect2>

3354

3355

3356

<title>Matching of Scripts to Targets</title>

3357

<para>

3358

After the initialization is finished the <literal>hostrules</literal> and

3359

<literal>portrules</literal> are evaluated for each host in the current

3360

target group. At this check a list is built which contains the combinations of scripts and the hosts they will run against.

3361

3362

It should be noted that the rules of all chosen scripts are

3363

checked against all hosts and their <literal>open</literal> and <literal>open|filtered</literal> ports.

3364

Therefore it is advisable to leave the rules as simple as possible and

3365

to do all the computation inside the <literal>action</literal>, as a script will only be

3366

executed if it is run against a specific target. After the check those script-target combinations get their own <ulink url="http://www.lua.org/manual/5.1/manual.html#2.11">Lua-thread</ulink> which is anchored in Lua's C-API <ulink url="http://www.lua.org/manual/5.1/manual.html#3.5">registry</ulink> to prevent their garbage collection. These <literal>thread_records</literal> are afterwards sorted by run level and all script-target combinations of one run level are stored in a list, in order to ensure that scripts with a higher run level are run after those with a lower one.</para>

3367

3368

</sect2>

3369

3370

3371

<title>Running Scripts</title>

3372

<para>

3373

Now to the actual script scanning, and the way NSE accomplishes

3374

parallelization. Lua, through its concept of

3375

<ulink url="http://www.lua.org/manual/5.1/manual.html#2.11">coroutines

3376

</ulink> offers collaborative multi-threading, which means that scripts

3377

can suspend themselves, at defined points, and let other coroutines

3378

execute. Since network I/O, especially waiting for responses from

3379

remote host, is the part of scripts which would consume most time with

3380

waiting, this is the point where scripts suspend themselves and let

3381

others execute. Each call to some of the functions of the Nsock wrapper

3382

causes the calling script to yield (pause). Once the request is processed by the Nsock library, the

3383

callback causes the script to be pushed from the waiting queue to the

3384

running queue, which will eventually let it resume its operation.

3385

</para>

3386

</sect2>

3387

3388

<title>Adding C Modules to Nselib</title>

3389

<para>

3390

This section tries to give a short walk-through to adding

3391

nselib modules written in C (or C++) to Nmap's build system, since

3392

this has shown to be sometimes tedious. Writing C modules is

3393

described at length in <ulink

3394

url="http://www.amazon.com/exec/obidos/ASIN/8590379825/secbks-20">Programming

3395

in Lua, Second Edition</ulink>. Basically C modules consist of the

3396

functions they provide to Lua, which have to be of type <ulink url="http://www.lua.org/manual/5.1/manual.html#lua_CFunction">lua_CFunction</ulink>. Additionally they have to contain a function

3397

which is used to actually open the module. By convention these function names are <literal>luaopen_<replaceable>modulename</replaceable></literal>.

3398

A good starting point for writing such modules is provided with

3399

<filename>bit.c</filename> and <filename>pcre.c</filename> inside

3400

the <filename>nselib/</filename> subdirectory of Nmap's source tree,

3401

which are two C modules already provided by the nselib. C modules

3402

basically are shared libraries which get loaded at runtime by Lua.

3403

</para>

3404

<para>

3405

The Unix build system uses <literal>libtool</literal> for

3406

compilation in a platform independent way.

3407

3408

As long as the new module

3409

does not depend on foreign libraries, you should only need to add

3410

<literal><replaceable>modulename</replaceable>.so</literal> to the

3411

<literal>all</literal> and <literal>clean</literal> targets in

3412

<filename>Makefile.in</filename>

3413

and copy and adapt the lines from <filename>bit.so</filename>.

3414

If your module does have dependencies you will most probably have to

3415

add checks for those libraries to <filename>configure.ac</filename>

3416

and put the dependencies inside the <literal>libtool</literal>

3417

commands in <filename>Makefile.in</filename>—here you should

3418

take a look at how <literal>pcre.so</literal> handles this.

3419

So much for the way it should work. Now for some pitfalls we've

3420

come across so far: These are results from building shared libraries in

3421

general and not really specific to nselib. Linking with

3422

static libraries (e.g. <literal>libnbase</literal>) sometimes leads

3423

to problems with exporting symbols on some platforms (in our case

3424

this was a x86_64-linux platform). To our knowledge no such

3425

problems occur when linking against already existing shared

3426

libraries.

3427

</para>

3428

<para>

3429

The Windows build system requires C module developers to create a

3430

MS Visual Studio Project file for their module

3431

(<filename><modulename>.vcproj</filename>) inside the

3432

<filename>nselib</filename> subdirectory. On Windows you have to

3433

include the <filename>liblua/</filename> subdirectory as

3434

an additional include path as well as a library search path. In addition

3435

you have to tell the project to link against the

3436

<filename>liblua.lib</filename> static library provided with Nmap.

3437

Other properties of the project should be the same as for other

3438

nselib C modules (e.g. see <filename>nse_bitlib.vcproj</filename>).

3439

Afterwards you have to include the newly created project file in

3440

Nmap's Visual Studio solution file

3441

(<filename>mswin32\nmap.sln</filename>) and make sure that

3442

<filename>nse_bitlib.vcproj</filename> depends on your project,

3443

because it is there nselib-modules get copied to their final destinations and are included in Nmap.

3444

</para>

3445

</sect2>

3446

</sect1>

3447

3448

<title>NSE Script License and Community Contributions</title>

3449

<para>

3450

3451

<remark>Fyodor is working on this. The general idea is for scripts to be contributed

3452

and distributed under the same license as Nmap, as described

3453

in <ulink url="http://seclists.org/nmap-dev/2006/q3/0156.html">this

3454

nmap-dev post</ulink>. We certainly welcome script contributions!</remark>

3455

3456

</para>

3457

</sect1>