~ubuntu-branches/ubuntu/saucy/sec/saucy : revision 11

1

.\"

2

.\" SEC (Simple Event Correlator) 2.7.3 - sec.man

2

.\" SEC (Simple Event Correlator) 2.7.4 - sec.man

3

4

.\"

5

.\" This program is free software; you can redistribute it and/or

16

.\" along with this program; if not, write to the Free Software

17

.\" Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.

18

.\"

19

.TH sec 1 "May 2013" "SEC 2.7.3"

19

.TH sec 1 "June 2013" "SEC 2.7.4"

20

.SH NAME

21

sec \- simple event correlator

22

.SH SYNOPSIS

68

.br

69

[--keepopen | --nokeepopen]

70

.br

71

[--rwfifo | --norwfifo]

72

.br

73

[--childterm | --nochildterm]

74

.br

71

75

[--intevents | --nointevents]

72

76

.br

73

77

[--intcontexts | --nointcontexts]

78

82

.br

79

83

[--version]

80

84

.SH DESCRIPTION

81

SEC is a tool for accomplishing event correlation tasks in the domains of

82

log analysis, system monitoring, network and security management, etc.

85

SEC is an event correlation tool for advanced event processing which can

86

be harnessed for event log monitoring, for network and security management,

87

for fraud detection, and for any other task which involves event correlation.

83

88

Event correlation is a procedure where a stream of events is processed,

84

89

in order to detect (and act on) certain event groups that occur within

85

90

predefined time windows. Unlike many other event correlation products which

86

91

are heavyweight solutions, SEC is a lightweight and platform-independent

87

event correlator which runs as a single process.

92

event correlator which runs as a single process. The user can start it as

93

a daemon, employ it in shell pipelines, execute it interactively in

94

a terminal, run many SEC processes simultaneously for different tasks,

95

and use it in a wide variety of other ways.

88

96

.PP

89

97

SEC reads lines from files, named pipes, or standard input,

90

98

matches the lines with patterns (regular expressions, Perl subroutines, etc.)

98

106

.BR snmptrap (1)

99

107

or

100

108

.BR mail (1)),

101

by writing to files, by calling precompiled Perl subroutines, etc.

102

.PP

103

Some rules start event correlation operations, while other rules react

109

by writing to files, by sending data to TCP and UDP based servers,

110

by calling precompiled Perl subroutines, etc.

111

.PP

112

SEC can be run in various ways. For example, the following command line

113

starts it as a daemon, in order to monitor events appended to the

114

/var/log/messages syslog file with rules from /etc/sec/syslog.rules:

115

.PP

116

/usr/bin/sec --detach --conf=/etc/sec/syslog.rules \\

117

--input=/var/log/messages

118

.PP

119

Each time /var/log/messages is rotated, a new instance of /var/log/messages

120

is opened and processed from the beginning. The following command line

121

runs SEC in a shell pipeline, configuring it to process lines from standard

122

input, and to exit when the /usr/bin/nc tool closes its standard output

123

and exits:

124

.PP

125

/usr/bin/nc -l 8080 | /usr/bin/sec --notail --input=- \\

126

--conf=/etc/sec/my.conf

127

.PP

128

Some SEC rules start event correlation operations, while other rules react

104

129

immediately to input events or system clock. For example, suppose that SEC

105

130

has been started with the following command line

106

131

.PP

331

356

if an input file is in the closed state (e.g., SEC fails to open the file at

332

357

startup, because it has not been created yet), SEC will attempt

333

358

to reopen the file after every <reopen_timeout> seconds until open succeeds.

359

This option has no meaning when the

360

.B \-\-notail

361

option is also specified.

334

362

Note that

335

363

.B \-\-reopen_timeout

336

364

is a synonym for

447

475

if the

448

476

.B \-\-notail

449

477

option is specified, SEC will process all data that are currently available

450

in input files and exit after reaching all EOFs. Default is

451

.B \-\-tail

478

in input files and exit after reaching all EOFs.

479

If all input is received from a pipe and the

480

.B \-\-notail

481

option is given, SEC terminates when the last writer closes the pipe

482

(EOF condition). Please note that with named pipes

483

.B \-\-notail

484

should be used with

485

.BR \-\-norwfifo .

452

486

With the

453

487

.B \-\-tail

454

option, SEC follows an input file both by its name and i-node, and thus

455

handles input file rotations seamlessly.

488

option, SEC will jump to the end of input files and wait for new lines to

489

arrive.

490

Each input file is tracked both by its name and i-node, and

491

input file rotations are handled seamlessly.

456

492

If the input file is recreated or truncated, SEC will reopen it and process

457

493

its content from the beginning. If the input file is removed (i.e., there is

458

494

just an i-node left without a name), SEC will keep the i-node open and wait

459

495

for the input file recreation.

496

Default is

497

.BR \-\-tail .

460

498

.TP

461

499

.BR \-\-fromstart ", " \-\-nofromstart

462

500

these flags have no meaning when the

517

554

Default is

518

555

.BR \-\-nokeepopen .

519

556

.TP

557

.BR \-\-rwfifo ", " \-\-norwfifo

558

if the

559

.BR \-\-norwfifo

560

option is specified, named pipe input files are opened in read-only mode.

561

In this mode, the named pipe has to be reopened when the last writer

562

closes the pipe, in order to clear the EOF condition on the pipe. With the

563

.BR \-\-rwfifo

564

option, named pipe input files are opened in read-write mode, although

565

SEC never writes to the pipes. In this mode, the pipe does not need to be

566

reopened when an external writer closes it, since there is always at least

567

one writer on the pipe and EOF will never appear. Therefore, if the

568

.BR \-\-notail

569

option has been given,

570

.BR \-\-norwfifo

571

should also be specified.

572

Default is

573

.BR \-\-rwfifo .

574

.TP

575

.BR \-\-childterm ", " \-\-nochildterm

576

if the

577

.B \-\-childterm

578

option is specified, SEC will send the SIGTERM signal to all its child processes

579

when it terminates or goes through a full restart. Default is

580

.BR \-\-childterm .

581

.TP

520

582

.BR \-\-intevents ", " \-\-nointevents

521

583

SEC will generate internal events when it starts up, when it receives

522

584

certain signals, and when it terminates gracefully. Specific rules can be

1171

1233

However, several signals cause the file to be closed and reopened

1172

1234

(see SIGNALS section for more information).

1173

1235

Default value for <string> is %s.

1236

.TP

1237

.I owritecl <filename> [<string>]

1238

Similar to the

1239

.I write

1240

action, except that the file <filename> is opened and closed at each write.

1241

Also, the string <string> is written without a terminating newline.

1242

If the file has already been opened by a previous

1243

.I write

1244

action,

1245

.I owritecl

1246

does not use existing filehandle, but opens and closes the file separately.

1174

1247

.TP

1175

1248

.I udgram <filename> [<string>]

1176

1249

String <string> is written to the UNIX datagram socket <filename>

2781

2854

desc=Three SSH login failures within 1m for user $1 from $2

2782

2855

.br

2783

2856

action=pipe '%t: %s' /bin/mail root@localhost; \\

2784

.br

2785

2857

create SUPPRESS_IP_$2 3600

2786

2858

.br

2787

2859

window=60

3332

3404

signal (this event will be the first event that SEC sees after reloading

3333

3405

its configuration)

3334

3406

.PP

3407

SEC_LOGROTATE - generated after SEC has received the

3408

.B SIGUSR2

3409

signal (this event will be the first event that SEC sees after reopening

3410

its log file and closing its outputs)

3411

.PP

3335

3412

SEC_SHUTDOWN - generated when SEC receives the

3336

3413

.B SIGTERM

3337

3414

signal, or when SEC reaches all EOFs of input files after being started with

3338

3415

the

3339

3416

.B \-\-notail

3340

option. After generating SEC_SHUTDOWN event, SEC sleeps for 3 seconds

3341

before sending SIGTERM to its child processes (if a child process was created

3342

immediately before SEC_SHUTDOWN, this delay leaves the process enough

3343

time for setting the signal handler for SIGTERM).

3417

option. With the

3418

.B \-\-childterm

3419

option, SEC sleeps for 3 seconds after generating SEC_SHUTDOWN event, and then

3420

sends SIGTERM to its child processes (if a child process was triggered by

3421

SEC_SHUTDOWN, this delay leaves the process enough time for setting a signal

3422

handler for SIGTERM).

3344

3423

.PP

3345

3424

Before generating an internal event, SEC sets up a context named

3346

3425

SEC_INTERNAL_EVENT, in order to disambiguate internal events from

3395

3474

.PP

3396

3475

After forking an external program, SEC continues immediately, and checks

3397

3476

the program status periodically until the program exits. The running time of

3398

a child process is not limited in any way, and before finishing gracefully,

3399

SEC sends the

3477

a child process is not limited in any way. With the

3478

.B \-\-childterm

3479

option, SEC sends the

3400

3480

.B SIGTERM

3401

signal to all child processes that are still running.

3481

signal to all child processes when it terminates.

3402

3482

If some special exit procedures need to be accomplished in the child process

3403

3483

(or the child wishes to ignore

3404

3484

.BR SIGTERM ),

3419

3499

.PP

3420

3500

action=spawn exec /usr/local/bin/myscript.pl 2>/var/log/myscript.log

3421

3501

.PP

3422

Finally, note that if an action list includes two actions which fork

3502

Note that if an action list includes two actions which fork

3423

3503

external programs, the execution order these programs is not determined

3424

3504

by the order of actions in the list, since both programs are running

3425

3505

asynchronously.

3428

3508

.IR "action=shellcmd cmd1; shellcmd cmd2" ,

3429

3509

use the shell && operator and write

3430

3510

.IR "action=shellcmd cmd1 && cmd2" ).

3511

.PP

3512

Sometimes it is desireable to start an external program and provide it with

3513

data from several rules. In order to create such setup, names pipes can be used.

3514

For example, if /var/log/pipe is a named pipe, then

3515

.PP

3516

action=shellcmd /usr/bin/logger -f /var/log/pipe -p user.notice

3517

.PP

3518

starts the /usr/bin/logger utility which sends all lines read from /var/log/pipe

3519

to the local syslog daemon with the "user" facility and "notice" level.

3520

In order to feed events to /usr/bin/logger, the

3521

.I write

3522

action can be used (e.g.,

3523

.IR "write /var/log/pipe This is my event" ).

3524

Although SEC keeps the named pipe open across different

3525

.I write

3526

actions, the pipe will be closed on the reception of SIGHUP, SIGABRT and SIGUSR2

3527

signals.

3528

Since a number of command line tools terminate on receiving EOF from input, they

3529

need restarting after such signals have arrived. For this purpose, the

3530

.B --intevents

3531

option and SEC internal events can be used. For example, the following rule starts

3532

the /usr/bin/logger utility at SEC startup, and also restarts it after the reception

3533

of relevant signals:

3534

.PP

3535

type=Single

3536

.br

3537

ptype=RegExp

3538

.br

3539

pattern=^(?:SEC_STARTUP|SEC_RESTART|SEC_SOFTRESTART|SEC_LOGROTATE)$

3540

.br

3541

context=SEC_INTERNAL_EVENT

3542

.br

3543

desc=start the logger tool

3544

.br

3545

action=free %emptystring; owritecl /var/log/pipe %emptystring; \\

3546

shellcmd /usr/bin/logger -f /var/log/pipe -p user.notice

3547

.PP

3548

Note that if /var/log/pipe is never opened for writing by a

3549

.I write

3550

action, /usr/bin/logger will never see EOF and will thus not terminate. The

3551

.I owritecl

3552

action opens and closes /var/log/pipe, in order to ensure the presence of EOF

3553

in such cases.

3431

3554

.SH PERL INTEGRATION

3432

3555

SEC supports patterns, context expressions, and actions

3433

3556

which involve calls to the Perl

3456

3579

ptype=PerlFunc

3457

3580

.br

3458

3581

pattern=sub { my(%var); my($line) = $_[0]; \\

3459

.br

3460

3582

if ($line !~ /kernel: iptables:/g) { return 0; } \\

3461

.br

3462

3583

while ($line =~ /\\G\\s*([A-Z]+)(?:=(\\S*))?/g) { \\

3463

.br

3464

3584

$var{$1} = defined($2)?$2:1; \\

3465

.br

3466

3585

} return \\%var; }

3467

3586

.br

3468

3587

varmap=IPTABLES

3511

3630

pattern=IPTABLES

3512

3631

.br

3513

3632

context=IPTABLES :> ( sub { return exists($_[0]->{"SYN"}) && \\

3514

.br

3515

3633

exists($_[0]->{"FIN"}) ; } )

3516

3634

.br

3517

3635

desc=SYN+FIN flood from host $+{SRC}

3570

3688

desc=Load the Storable module and terminate if it is not found

3571

3689

.br

3572

3690

action=eval %ret (require Storable); \\

3573

.br

3574

3691

if %ret ( logonly Storabe loaded ) else ( eval %o exit(1) )

3575

3692

.PP

3576

3693

type=Single

3584

3701

desc=Restore all SEC contexts from /tmp/SEC_CONTEXTS on startup

3585

3702

.br

3586

3703

action=lcall %ret -> ( sub { %main::context_list = \\

3587

.br

3588

3704

%{Storable::retrieve("/tmp/SEC_CONTEXTS")}; } )

3589

3705

.PP

3590

3706

type=Single

3598

3714

desc=Save all SEC contexts into /tmp/SEC_CONTEXTS on shutdown

3599

3715

.br

3600

3716

action=lcall %ret -> ( sub { \\

3601

.br

3602

3717

Storable::store(\\%main::context_list, "/tmp/SEC_CONTEXTS"); } )

3603

3718

.PP

3604

3719

However, note that modifying data structures within SEC code is recommended

3605

3720

only for advanced users who have carefully studied relevant parts of the code.

3721

.PP

3722

Finally, sometimes larger chunks of Perl code have to be used for event

3723

processing and correlation. However, writing many lines of code directly

3724

into a rule is cumbersome and may decrease its readability. In such cases

3725

it is recommended to separate the code into a custom Perl module which

3726

is loaded at SEC startup, and use the code through the module interface

3727

(see

3728

.BR perlmod (1)

3729

for further details):

3730

.PP

3731

type=Single

3732

.br

3733

ptype=SubStr

3734

.br

3735

pattern=SEC_STARTUP

3736

.br

3737

context=SEC_INTERNAL_EVENT

3738

.br

3739

desc=Load the SecStuff module

3740

.br

3741

action=eval %ret (require '/usr/local/sec/SecStuff.pm'); \\

3742

if %ret ( none ) else ( eval %o exit(1) )

3743

.PP

3744

type=Single

3745

.br

3746

ptype=PerlFunc

3747

.br

3748

pattern=sub { return SecStuff::my_match($_[0]); }

3749

.br

3750

desc=event '$0' was matched by my_match()

3751

.br

3752

action=write - %s

3606

3753

.SH EXAMPLES

3607

.SS Example 1

3754

.SS Example 1 - a ruleset for Cisco events

3608

3755

This section presents an example rulebase for managing Cisco devices.

3609

3756

It is assumed that the managed devices have syslog

3610

3757

logging enabled, and that all syslog messages are sent to a central host

3785

3932

window=21600

3786

3933

.br

3787

3934

thresh=10

3788

.SS Example 2

3935

.SS Example 2 - hierarchically organized rulesets for iptables and sshd events

3789

3936

This section presents an example of hierarchically organized rules for

3790

3937

processing Linux iptables events from /var/log/messages and SSH login events

3791

3938

from /var/log/secure. It is assumed that all rule files reside in the

3797

3944

Also, the example assumes that SEC is started with the following command line:

3798

3945

.PP

3799

3946

/usr/bin/sec --conf=/etc/sec/*.rules --intcontexts \\

3800

.br

3801

3947

--input=/var/log/messages --input=/var/log/secure

3802

3948

.PP

3803

3949

#

3813

3959

ptype=PerlFunc

3814

3960

.br

3815

3961

pattern=sub { my(%var); my($line) = $_[0]; \\

3816

.br

3817

3962

if ($line !~ /kernel: iptables:/g) { return 0; } \\

3818

.br

3819

3963

while ($line =~ /\\G\\s*([A-Z]+)(?:=(\\S*))?/g) { \\

3820

.br

3821

3964

$var{$1} = defined($2)?$2:1; \\

3822

.br

3823

3965

} return \\%var; }

3824

3966

.br

3825

3967

varmap=IPTABLES

3877

4019

pattern=IPTABLES

3878

4020

.br

3879

4021

context=IPTABLES :> ( sub { return exists($_[0]->{"SYN"}) && \\

3880

.br

3881

4022

exists($_[0]->{"FIN"}) ; } ) \\

3882

.br

3883

4023

&& !SUPPRESS_IP_$+{SRC}

3884

4024

.br

3885

4025

desc=SYN+FIN flood from host $+{SRC}

3886

4026

.br

3887

4027

action=pipe '%t: %s' /bin/mail -s 'iptables alert' root@localhost; \\

3888

.br

3889

4028

create SUPPRESS_IP_$+{SRC} 3600

3890

4029

.br

3891

4030

window=10

3899

4038

pattern=IPTABLES

3900

4039

.br

3901

4040

context=IPTABLES :> ( sub { return exists($_[0]->{"SYN"}) && \\

3902

.br

3903

4041

!exists($_[0]->{"ACK"}) ; } ) \\

3904

.br

3905

4042

&& !SUPPRESS_IP_$+{SRC}

3906

4043

.br

3907

4044

desc=SYN flood from host $+{SRC}

3908

4045

.br

3909

4046

action=pipe '%t: %s' /bin/mail -s 'iptables alert' root@localhost; \\

3910

.br

3911

4047

create SUPPRESS_IP_$+{SRC} 3600

3912

4048

.br

3913

4049

window=10

3933

4069

pattern=SSH_LOGIN

3934

4070

.br

3935

4071

context=SSH_LOGIN :> ( sub { return $_[0]->{"status"} eq "Failed" && \\

3936

.br

3937

4072

$_[0]->{"srcport"} < 1024 && \\

3938

.br

3939

4073

defined($_[0]->{"invuser"}); } )

3940

4074

.br

3941

4075

continue=TakeNext

3951

4085

pattern=SSH_LOGIN

3952

4086

.br

3953

4087

context=SSH_LOGIN :> ( sub { return $_[0]->{"status"} eq "Failed" && \\

3954

.br

3955

4088

defined($_[0]->{"invuser"}); } )

3956

4089

.br

3957

4090

desc=Ten login probes for invalid users from $+{srcip} within 60s

3979

4112

pattern2=SSH_LOGIN

3980

4113

.br

3981

4114

context2=SSH_LOGIN :> \\

3982

.br

3983

4115

( sub { return $_[0]->{"status"} eq "Accepted"; } ) && \\

3984

.br

3985

4116

$+{user} %+{user} $+{srcip} %+{srcip} -> \\

3986

.br

3987

4117

( sub { return $_[0] eq $_[1] && $_[2] eq $_[3]; } )

3988

4118

.br

3989

4119

desc2=User $+{user} logged in successfully from $+{srcip} within 60s

4065

4195

(these will be reopened on demand), reload its configuration, and

4066

4196

drop *all* event correlation state (all event correlation operations

4067

4197

will be terminated, all contexts will be deleted, all action list variables

4068

will be erased, etc.).

4069

SEC will also send the

4198

will be erased, etc.). With the

4199

.B \-\-childterm

4200

option, SEC will also send the

4070

4201

.B SIGTERM

4071

4202

signal to its child processes.

4072

4203

.TP

4107

4238

if SEC is running non-interactively (e.g., in daemon mode).

4108

4239

.TP

4109

4240

.B SIGTERM

4110

SEC will terminate gracefully (all SEC child processes will receive

4111

.BR SIGTERM ).

4241

SEC will terminate gracefully. With the

4242

.B \-\-childterm

4243

option, all SEC child processes will receive

4244

.BR SIGTERM .

4112

4245

.SH BUGS

4113

4246

With some locale settings, apostrophes (') in this man page might

4114

4247

be displayed incorrectly. As a workaround, set the LANG environment

4131

4264

.BR fork (2),

4132

4265

.BR mail (1),

4133

4266

.BR perl (1),

4267

.BR perlmod (1),

4134

4268

.BR perlre (1),

4135

4269

.BR pipe (2),

4136

4270

.BR sh (1),