« back to all changes in this revision
Viewing changes to doc/user/usermanual.sgml
- browse files at revision 28
- compare with another revision
- download diff
- download tarball
- view history from revision 28
- Committer: Package Import Robot
- Author(s): Kartik Mistry
- Date: 2012-03-27 12:15:51 UTC
- mfrom: (1.3.8)
- Revision ID: package-import@ubuntu.com-20120327121551-nmntidzpehudushy
Tags: 1.17.1-1
* New upstream release.
* Enable Python module resulting into new binary: python-recoll.
* debian/control:
+ Updated Build-Deps: libqtwebkit-dev, python-all-dev.
+ Added python-recoll binary.
+ Updated Standards-Version to 3.9.3
* debian/rules:
+ Build Python module by default.
* debian/recoll.menu, debian/python-recoll.install, debian/recoll.install:
+ Changes for new binary package.
* debian/copyright:
+ Updated to copyright-format 1.0
+ Updated upstream and Debian copyright.
+ Fixed unicode.org/copyright.html URL.
* Enable Python module resulting into new binary: python-recoll.
* debian/control:
+ Updated Build-Deps: libqtwebkit-dev, python-all-dev.
+ Added python-recoll binary.
+ Updated Standards-Version to 3.9.3
* debian/rules:
+ Build Python module by default.
* debian/recoll.menu, debian/python-recoll.install, debian/recoll.install:
+ Changes for new binary package.
* debian/copyright:
+ Updated to copyright-format 1.0
+ Updated upstream and Debian copyright.
+ Fixed unicode.org/copyright.html URL.
- files added:
- .pc
- .pc/fix-python-install.patch
- debian/patches
- desktop/unity-lens-recoll
- desktop/unity-lens-recoll/bin
- desktop/unity-lens-recoll/data
- desktop/unity-lens-recoll/recollscope
- files removed:
- debian/menu
- files modified:
- INSTALL
added
removed
doc/user/usermanual.sgml
1
<!-- Use this header for the FreeBSD sgml toolchain -->
1
2
<!DOCTYPE BOOK PUBLIC "-//FreeBSD//DTD DocBook V4.1-Based Extension//EN" [
2
3
4
<!-- Use this header for going XML -->
5
<!-- <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
6
"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ -->
7
3
8
<!ENTITY RCL "<application>Recoll</application>">
4
9
<!ENTITY RCLAPPS "<ulink url='http://www.recoll.org/features.html'>Recoll helper applications page</ulink>">
5
<!ENTITY RCLVERSION "1.16">
10
<!ENTITY RCLVERSION "1.17">
6
11
<!ENTITY XAP "<application>Xapian</application>">
7
12
]>
8
13
20
25
</author>
21
26
22
27
<copyright>
23
<year>2005-2011</year>
28
<year>2005-2012</year>
24
29
<holder role="mailto:jfd@recoll.org">Jean-Francois
25
30
Dockes</holder>
26
31
</copyright>
27
32
28
<releaseinfo>$Id: usermanual.sgml,v 1.71 2008-12-15 09:33:49 dockes Exp $</releaseinfo>
29
30
33
<abstract>
31
34
<para>This document introduces full text search notions
32
35
and describes the installation and use of the &RCL;
60
63
applications</link> for document types that need them (for
61
64
example <application>antiword</application> for ms-word
62
65
files).</para>
63
66
</sect1>
67
64
68
<sect1 id="rcl.introduction.search">
65
69
<title>Full text search</title>
66
70
140
144
currently makes no attempt at automatic language recognition.</para>
141
145
142
146
<para>&RCL; has many parameters which define exactly what to
143
index, and how to classify and decode the source
144
documents. These are kept in <link
145
linkend="rcl.indexing.config">configuration files</link>. A
146
default configuration is copied into a standard location
147
(usually something like
148
<filename>/usr/[local/]share/recoll/examples</filename>)
149
during installation. The default parameters from this file may
150
be overridden by values that you set inside your personal
151
configuration, found by default in the
152
<filename>.recoll</filename> sub-directory of your home
153
directory. The default configuration will index your home
154
directory with default parameters and should be sufficient for
155
giving &RCL; a try, but you may want to adjust it
156
later.</para>
147
index, and how to classify and decode the source documents. These
148
are kept in <link linkend="rcl.indexing.config">configuration
149
files</link>. A default configuration is copied into a standard
150
location (usually something like
151
<filename>/usr/[local/]share/recoll/examples</filename>) during
152
installation. The default parameters from this file may be
153
overridden by values that you set inside your personal
154
configuration, found by default in the <filename>.recoll</filename>
155
sub-directory of your home directory. The default configuration
156
will index your home directory with default parameters and should
157
be sufficient for giving &RCL; a try, but you may want to adjust it
158
later, which can be done either by editing the text files or by
159
using configuration menus in the <command>recoll</command>
160
GUI</para>
157
161
158
162
<para><link linkend="rcl.indexing.periodic.exec">Indexing</link>
159
163
is started automatically the first time you execute the
184
188
<para>Indexing is the process by which the set of documents is
185
189
analyzed and the data entered into the database. &RCL; indexing
186
190
is normally incremental: documents will only be processed if
187
they have been modified. On the first execution, of course, all
191
they have been modified. On the first execution, all
188
192
documents will need processing. A full index build can be forced
189
193
later by specifying an option to the indexing command
190
194
(<command>recollindex -z</command>).</para>
195
199
<itemizedlist>
196
200
197
201
<listitem>
198
<formalpara><title>Periodic indexing:</title>
202
<formalpara><title>Periodic (or Batch) indexing:</title>
199
203
<para>indexing takes place at discrete
200
204
times, by executing the <command>recollindex</command>
201
205
command. The typical usage is to have a nightly indexing run
222
226
indexes (ie: use periodic indexing on a big documentation
223
227
directory, and real time indexing on a small home
224
228
directory). Monitoring a big file system tree can consume
225
significant system resources.<para>
229
significant system resources.</para>
226
230
227
231
<para>&RCL; knows about quite a few different document
228
232
types. The parameters for document types recognition and
238
242
a folder file archived inside a zip file...</para>
239
243
240
244
<para>&RCL; indexing processes plain text, HTML, openoffice
241
and e-mail files internally (a few more actually).</para>
245
and e-mail files, and a few others internally.</para>
242
246
243
247
<para>Other file types (ie: postscript, pdf, ms-word, rtf ...)
244
248
need external applications for preprocessing. The list is in the
299
303
stored in <filename>~/.indexes-email/</filename> and,
300
304
(unless specified otherwise in
301
305
<filename>recoll.conf</filename>) would look for
302
the index in <filename>~/.indexes-email/xapiandb/</filename>.
306
the index in
307
<filename>~/.indexes-email/xapiandb/</filename>.</para>
303
308
304
309
<para>Using multiple configuration directories and
305
310
<link linkend="rcl.install.config.recollconf">configuration
336
341
total amount of data on the computer.</para>
337
342
338
343
<para>The index data directory (<filename>xapiandb</filename>)
339
only contains data that can be completely rebuilt by an index
340
run, and it can always be destroyed safely.</para>
344
only contains data that can be completely rebuilt by an index run
345
(as long as the original documents exist), and it can always be
346
destroyed safely.</para>
341
347
342
348
<sect2 id="rcl.indexing.storage.format">
343
349
<title>Xapian index formats</title>
344
350
345
<para>If your first installation of &RCL; was 1.9.0 or more
346
recent, you can skip this section.</para>
347
348
<para>&XAP; has had two possible index formats for quite some
349
time. The "old" one named <literal>Quartz</literal>, and the
350
new one named <literal>Flint</literal>. &XAP; 0.9 used
351
<literal>Quartz</literal> by default, but could use
352
<literal>Flint</literal> if a specific environment variable
353
(<literal>XAPIAN_PREFER_FLINT</literal>) was set. &XAP; 1.0
354
still supports <literal>Quartz</literal> but will use
355
<literal>Flint</literal> by default for new index
356
creations.</para>
357
358
<para>The number of disk accesses performed during indexing
359
has been much optimized in the new <literal>Flint</literal>
360
engine and you may see indexing times improved by 50% in some
361
cases (compared to <literal>Quartz</literal>), typically for
362
big indexes where disk accesses dominate the indexing
363
time. There is also a more modest improvement of index
364
size.</para>
351
<para>&XAP; versions usually support several formats for index
352
storage. A given major &XAP; version will have a current format,
353
used to create new indexes, and will also support the format from
354
the previous major version.</para>
365
355
366
356
<para>&XAP; will not convert automatically an existing index
367
from the <literal>Quartz</literal> to the
368
<literal>Flint</literal> format. If you have an older index
369
and want to take advantage of the new format (which can be
370
done without setting the environment variable as of &RCL;
371
1.8.2 and &XAP; 1.0.0), you will have to explicitly delete
372
the old index, then run a normal indexing process.</para>
357
from the older format to the newer one. If you want to upgrade to
358
the new format, or if a very old index needs to be converted
359
because its format is not supported any more, you will have to
360
explicitly delete the old index, then run a normal indexing
361
process.</para>
373
362
374
363
<para>Unfortunately, using the <literal>-z</literal> option to
375
364
<command>recollindex</command> is not sufficient to change the
376
format, you have to delete all files inside the index
365
format, you will have to delete all files inside the index
377
366
directory (typically <filename>~/.recoll/xapiandb</filename>)
378
before starting indexing.</para>
367
before starting the indexing.</para>
379
368
380
369
</sect2>
381
370
387
376
complete reconstruction. If confidential data is indexed,
388
377
access to the database directory should be restricted. </para>
389
378
390
<para>As of version 1.4, &RCL; will create the configuration
379
<para>&RCL; (since version 1.4) will create the configuration
391
380
directory with a mode of 0700 (access by owner only). As the
392
381
index data directory is by default a sub-directory of the
393
382
configuration directory, this should result in appropriate
422
411
will be asked whether or not you would like it to build the
423
412
index. If you want to adjust the configuration before indexing,
424
413
just click <guilabel>Cancel</guilabel> at this point, which will get
425
you into the configuration interface. If you exit,
414
you into the configuration interface. If you exit at this point,
426
415
<filename>recoll</filename> will have created a ~/.recoll directory
427
416
containing empty configuration files, which you can edit by hand.</para>
428
417
502
491
<ulink url="https://bitbucket.org/medoc/recoll/wiki/IndexBeagleWeb">
503
492
Recoll wiki</ulink>.</para>
504
493
494
<para>Unfortunately, it seems that the plugin does not work anymore
495
with recent Firefox versions (tried with 10.0). This is not the
496
trival installation version check issue, explicit manual indexing
497
requests still work, but automatic indexing on page load does
498
not.</para>
499
505
500
</sect1>
506
501
507
502
<sect1 id="rcl.indexing.periodic">
510
505
<sect2 id="rcl.indexing.periodic.exec">
511
506
<title>Running indexing</title>
512
507
513
<para>Indexing is performed either by the
514
<command>recollindex</command> program, or by the
515
indexing thread inside the <command>recoll</command>
516
program (use the <guimenu>File</guimenu> menu). Both programs
517
will use the <literal>RECOLL_CONFDIR</literal>
518
variable or accept a <literal>-c</literal>
519
<replaceable>confdir</replaceable> option to specify a non-default
520
configuration directory.</para>
521
522
<para>Reasons to use either the indexing thread or the
523
<command>recollindex</command> command:
524
<itemizedlist>
525
<listitem><para>Starting the indexing thread is more convenient,
526
being just one click away.</para>
527
</listitem>
528
<listitem><para>The <command>recollindex</command> command has
529
more options, especially the one to reset the index
530
(<literal>-z</literal>).</para>
531
</listitem>
532
<listitem><para>The <command>recollindex</command> command will
533
not take down your GUI if it crashes (a rare occurrence,
534
but who knows...)</para>
535
</listitem>
536
<listitem><para>The <command>recollindex</command> command uses
537
<command>setpriority/nice</command> to lower its priority while
538
indexing
539
(it will also use <command>ionice</command> when this becomes
540
more widely available), the thread can't do it, else it would
541
also slow down the user/search interface.</para>
542
</listitem>
543
</itemizedlist>
544
I'll let the reader decide where my heart belongs...</para>
508
<para>Indexing is always performed by the
509
<command>recollindex</command> program, which can be started
510
either from the command line or from the <guimenu>File</guimenu>
511
menu in the <command>recoll</command> GUI program. When started
512
from the GUI, the indexing will run on the same configuration
513
<command>recoll</command> was started on. When started from the
514
command line, <command>recollindex</command> will use the
515
<literal>RECOLL_CONFDIR</literal> variable or accept a
516
<literal>-c</literal> <replaceable>confdir</replaceable> option
517
to specify a non-default configuration directory.</para>
545
518
546
519
<para>If the <command>recoll</command> program finds no index
547
520
when it starts, it will automatically start indexing (except
548
521
if canceled).</para>
549
522
550
523
<para>The <command>recollindex</command> indexing process can be
551
interrupted by sending an
552
interrupt (^C, SIGINT) or terminate (SIGTERM) signal. Some time may
553
elapse before the process exits, because it needs to properly flush
554
and close the index. The indexing thread can be equivalently
555
stopped from the menu.</para>
524
interrupted by sending an interrupt (^C, SIGINT) or terminate
525
(SIGTERM) signal. Some time may elapse before the process exits,
526
because it needs to properly flush and close the index. The
527
indexing thread can be equivalently stopped from the menu.</para>
556
528
557
529
<para>After such an interruption, the index will be somewhat
558
530
inconsistent because some operations which are normally performed
595
567
3:30AM (supposing <command>recollindex</command> is in your
596
568
PATH):
597
569
598
<programlisting>30 3 * * * recollindex > /some/tmp/dir/recolltrace 2>&1</programlisting>
570
<programlisting>30 3 * * * recollindex > /some/tmp/dir/recolltrace 2>&1</programlisting>
599
571
600
572
Or, using <command>anacron</command>:
601
<programlisting>1 15 su mylogin -c "recollindex recollindex > /tmp/rcltraceme 2>&1"</programlisting>
573
<programlisting>1 15 su mylogin -c "recollindex recollindex > /tmp/rcltraceme 2>&1"</programlisting>
602
574
</para>
603
575
576
<para>As of version 1.17 the &RCL; GUI has dialogs to manage
577
<filename>crontab</filename> entries for
578
<command>recollindex</command>. You can reach them from the
579
<guimenu>Preferences->Indexing Schedule</guimenu> menu. They only
580
work with the good old <command>cron</command>, and do not give
581
access to all features of <command>cron</command> scheduling.</para>
582
604
583
<para>The usual command to edit your
605
584
<filename>crontab</filename> is
606
585
<userinput>crontab -e</userinput> (which will usually start
609
588
system.</para>
610
589
611
590
<para>Please be aware that there may be differences between your
612
usual interactive command line environment and the one seen by
613
crontab commands. Especially the PATH variable may be of
614
concern. Please check the crontab manual pages about possible
615
issues.</para>
591
usual interactive command line environment and the one seen by
592
crontab commands. Especially the PATH variable may be of
593
concern. Please check the crontab manual pages about possible
594
issues.</para>
595
616
596
617
597
</sect2>
618
598
</sect1>
621
601
<title>Real time indexing</title>
622
602
623
603
<para>Real time monitoring/indexing is performed by starting the
624
<command>recollindex -m</command> command. With this option,
625
<command>recollindex</command> will detach from the terminal and
626
become a daemon, permanently monitoring file changes and updating
627
the index.</para>
604
<command>recollindex -m</command> command. With this option,
605
<command>recollindex</command> will detach from the terminal and
606
become a daemon, permanently monitoring file changes and updating
607
the index.</para>
628
608
629
<para>The real time indexing support can be customised during package
630
<link linkend="rcl.install.building.build">configuration</link>
631
with the <literal>--with[out]-fam</literal> or
632
<literal>--with[out]-inotify</literal> options. The default is
633
currently to include inotify monitoring on systems that support
634
it.</para>
609
<para>Under KDE, Gnome and some other desktop environments, the daemon
610
can automatically started when you log in, by creating a desktop
611
file inside the <filename>~/.config/autostart</filename> directory.
612
This can be done for you by the &RCL; GUI. Use the
613
<guimenu>Preferences->Indexing Schedule</guimenu> menu.</para>
614
615
<para>With older X11 setups, starting the daemon is normally
616
performed as part of the user session script.</para>
635
617
636
618
<para>The <filename>rclmon.sh</filename> script can be used to
637
619
easily start and stop the daemon. It can be found in the
638
620
<filename>examples</filename> directory (typically
639
621
<filename>/usr/local/[share/]recoll/examples</filename>).</para>
640
622
641
<para>Starting the daemon is normally performed as part
642
of the user session script. For example, my out of fashion
643
xdm-based session has a <filename>.xsession</filename> script
644
with the following lines at the end:</para>
623
<para>For example, my out of fashion xdm-based session has a
624
<filename>.xsession</filename> script with the following lines at
625
the end:</para>
645
626
646
627
<programlisting>recollconf=$HOME/.recoll-home
647
628
recolldata=/usr/local/share/recoll
652
633
</programlisting>
653
634
654
635
<para>The indexing daemon gets started, then the window manager,
655
for which the session waits.</para> <para>By default the
656
indexing daemon will monitor the state of the X11 session, and
657
exit when it finishes, it is not necessary to kill it
658
explicitly. (The X11 server monitoring can be disabled with option
659
<literal>-x</literal> to <command>recollindex</command>).
660
</para>
636
for which the session waits.</para> <para>By default the
637
indexing daemon will monitor the state of the X11 session, and
638
exit when it finishes, it is not necessary to kill it
639
explicitly. (The X11 server monitoring can be disabled with option
640
<literal>-x</literal> to <command>recollindex</command>).</para>
661
641
662
<para>Under KDE, you can place a small script to start
663
<command>recollindex -m</command> under
664
<filename>$HOME/.kde/Autostart</filename>. This will be executed
665
when the session begins.</para>
666
667
<para>There is a similar mechanism under Gnome (find the session
668
control tool in the menus and use the "Startup programs" tab).</para>
642
<para>If you use the daemon completely out of an X11 session, you
643
need to add option <literal>-x</literal> to disable X11 session
644
monitoring (else the daemon will not start).</para>
669
645
670
646
<para>By default, the messages from the indexing daemon will be
671
647
discarded. You may want to change this by setting the
675
651
daemon runs permanently, the log file may grow quite big, depending
676
652
on the log level.</para>
677
653
654
<para>When building &RCL;, the real time indexing support can be
655
customised during package
656
<link linkend="rcl.install.building.build">configuration</link>
657
with the <literal>--with[out]-fam</literal> or
658
<literal>--with[out]-inotify</literal> options. The default is
659
currently to include inotify monitoring on systems that support
660
it, and, as of recoll 1.17, gamin support on FreeBSD.</para>
661
678
662
<para>While it is convenient that data is indexed in real time,
679
663
repeated indexing can generate a significant load on the
680
664
system when files such as email folders change. Also,
900
884
901
885
<para>The format of the result list entries is entirely
902
886
configurable by using the preference dialog to
903
<link linkend="rcl.search.custom.reslistpara">edit an HTML
904
fragment</link>.
887
<link linkend="rcl.search.custom.reslist">edit an HTML
888
fragment</link>.</para>
905
889
906
890
<para>You can click on the <literal>Query details</literal> link
907
891
at the top of the results page to see the query actually
1000
984
<para>Hovering over a table row will update the detail area at the
1001
985
bottom of the window with the corresponding values. You can click
1002
986
the row to freeze the display. The bottom area is equivalent to a
1003
classical result list paragraph, with links for
1004
starting a preview or a native application, and an equivalent
1005
right-click menu. Typing <keycap>Esc</keycap> (the Escape key) will
1006
unfreeze the display.</para>
987
result list paragraph, with links for starting a preview or a
988
native application, and an equivalent right-click menu. Typing
989
<keycap>Esc</keycap> (the Escape key) will unfreeze the
990
display.</para>
1007
991
1008
992
</sect2>
1009
993
1071
1055
through the <guilabel>Tools</guilabel> menu or through the main
1072
1056
toolbar.</para>
1073
1057
1074
<para>The dialog has three parts:</para>
1075
1076
<itemizedlist>
1077
<listitem><para>The top part allows constructing a query by
1078
combining multiple clauses of different types.
1079
Each entry field is configurable for the following modes:</para>
1058
<para>The dialog has two tabs:</para>
1059
<orderedlist>
1060
1061
<listitem><para>The first tab lets you specify terms to search
1062
for, and permits specifying multiple clauses which are combined
1063
to build the search.</para>
1064
</listitem>
1065
1066
<listitem><para>The second tab lets filter the results according
1067
to file size, date of modification, mime type, or
1068
location.</para>
1069
</listitem>
1070
1071
</orderedlist>
1072
1073
<para>Click on the <guilabel>Start Search</guilabel> button in
1074
the advanced search dialog, or type <keycap>Enter</keycap> in
1075
any text field to start the search. The button in
1076
the main window always performs a simple search.</para>
1077
1078
<para>Click on the <literal>Show query details</literal> link at
1079
the top of the result page to see the query expansion.</para>
1080
1081
<sect3 id="rcl.search.complex.terms">
1082
<title>Avanced search: the "find" tab</title>
1083
1084
<para>This part of the dialog lets you constructc a query by
1085
combining multiple clauses of different types. Each entry
1086
field is configurable for the following modes:</para>
1080
1087
1081
1088
<itemizedlist>
1082
1089
<listitem><para>All terms.</para>
1107
1114
a mix of single words and phrases enclosed in double quotes.
1108
1115
Stemming and wildcard expansion will be performed as for simple
1109
1116
search. </para>
1110
</listitem>
1111
1112
<listitem><para>The next part allows filtering the
1113
results by their mime types.</para>
1114
<para>The state of the file type selection can be saved as
1115
the default (the file type filter will not be activated at
1116
program start-up, but the lists will be in the restored
1117
state).</para>
1118
</listitem>
1119
1120
<listitem>
1121
<para>The bottom part allows restricting the search results to a
1122
sub-tree of the indexed area. You can use the
1123
<guilabel>Invert</guilabel> checkbox to search for files not in
1124
the sub-tree instead. If you use directory filtering often and on
1125
big subsets of the file system, you may think of setting up
1126
multiple indexes instead, as the performance may be
1127
better. </para>
1128
</listitem>
1129
1130
</itemizedlist>
1131
1132
1117
1133
1118
<formalpara><title>Phrases and Proximity searches</title>
1134
1119
<para>These two clauses work in similar ways, with the
1144
1129
a slack of 1 it will match the latter, but not <literal>fox
1145
1130
quick</literal>. A proximity search for <literal>quick
1146
1131
fox</literal> with the default slack will match the
1147
latter, and also <literal>a fox is a cunning and quick animal</literal>.
1132
latter, and also <literal>a fox is a cunning and quick
1133
animal</literal>.</para>
1148
1134
</formalpara>
1149
1135
1150
<para>Click on the <guilabel>Start Search</guilabel> button in
1151
the advanced search dialog, or type <keycap>Enter</keycap> in
1152
any text field to start the search. The button in
1153
the main window always performs a simple search.</para>
1154
<para>Click on the <literal>Show query details</literal> link at
1155
the top of the result page to see the query expansion.</para>
1136
</sect3>
1137
1138
<sect3 id="rcl.search.complex.filter">
1139
<title>Avanced search: the "filter" tab</title>
1140
1141
<para>This part of the dialog has several sections which allow
1142
filtering the results of a search according to a number of
1143
criteria</para>
1144
1145
<itemizedlist>
1146
1147
<listitem>
1148
<para>The first section allows filtering by dates of last
1149
modification. You can specify both a minimum and a maximum date. The
1150
initial values are set according to the oldest and newest documents
1151
found in the index.</para>
1152
</listitem>
1153
1154
<listitem>
1155
<para>The next section allows filtering the results by
1156
file size. There are two entries for minimum and maximum
1157
size. Enter decimal numbers. You can use suffix multipliers:
1158
<literal>k/K</literal>, <literal>m/M</literal>,
1159
<literal>g/G</literal>, <literal>t/T</literal> for 1E3, 1E6,
1160
1E9, 1E12 respectively.</para>
1161
</listitem>
1162
1163
<listitem>
1164
<para>The next section allows filtering the results by their mime
1165
types, or mime categories (ie: media/text/message/etc.).</para>
1166
<para>You can transfer the types between two boxes, to define
1167
which will be included or excluded by the search.</para>
1168
<para>The state of the file type selection can be saved as
1169
the default (the file type filter will not be activated at
1170
program start-up, but the lists will be in the restored
1171
state).</para>
1172
</listitem>
1173
1174
<listitem>
1175
<para>The bottom section allows restricting the search results to a
1176
sub-tree of the indexed area. You can use the
1177
<guilabel>Invert</guilabel> checkbox to search for files not in
1178
the sub-tree instead. If you use directory filtering often and on
1179
big subsets of the file system, you may think of setting up
1180
multiple indexes instead, as the performance may be
1181
better.</para>
1182
<para>You can use relative/partial paths for filtering. Ie,
1183
entering <literal>dirA/dirB</literal> would match either
1184
<filename>/dir1/dirA/dirB/myfile1</filename> or
1185
<filename>/dir2/dirA/dirB/someother/myfile2</filename>.</para>
1186
</listitem>
1187
1188
</itemizedlist>
1189
1190
</sect3>
1156
1191
1157
1192
</sect2>
1158
1193
1303
1338
entry.</para>
1304
1339
<para>You can erase the document history by using the
1305
1340
<guilabel>Erase document history</guilabel> entry in the
1306
<guimenu>File</guimenu> menu.
1341
<guimenu>File</guimenu> menu.</para>
1307
1342
1308
1343
</sect2>
1309
1344
1311
1346
<title>Sorting search results and collapsing duplicates</title>
1312
1347
1313
1348
<para>The documents in a result list are normally sorted in
1314
order of relevance. It is possible to specify different sort
1315
parameters by using the <guimenu>Sort parameters</guimenu>
1316
dialog (located in the <guimenu>Tools</guimenu> menu).</para>
1317
1318
<para>The tool sorts a specified number of the most
1319
relevant documents in the result list, according to specified
1320
criteria. The currently available criteria are
1321
<emphasis>date</emphasis> and <emphasis>mime
1322
type</emphasis>.</para>
1323
1324
<para>The sort parameters stay in effect until they are
1325
explicitly reset, or the program exits. An activated sort is
1326
indicated in the result list header.</para>
1349
order of relevance. It is possible to specify a different sort
1350
order, either by using the vertical arrows in the GUI toolbox to
1351
sort by date, or switching to the result table display and clicking
1352
on any header. The sort order chosen inside the result table
1353
remains active if you switch back to the result list, until you
1354
click one of the vertical arrows, until both are unchecked (you are
1355
back to sort by relevance).</para>
1327
1356
1328
1357
<para>Sort parameters are remembered between program
1329
1358
invocations, but result sorting is normally always inactive
1427
1456
1428
1457
<formalpara><title>AutoPhrases</title>
1429
1458
<para>This option can be set in the preferences dialog. If it is
1430
set, a phrase will be automatically built and added to simple
1431
searches when looking for <literal>Any terms</literal>. This
1432
will not change radically the results, but will give a relevance
1433
boost to the results where the search terms appear as a
1434
phrase. Ie: searching for <literal>virtual reality</literal>
1435
will still find all documents where either
1436
<literal>virtual</literal> or <literal>reality</literal> or
1437
both appear, but those which contain <literal>virtual
1438
reality</literal> should appear sooner in the list.</para>
1459
set, a phrase will be automatically built and added to simple
1460
searches when looking for <literal>Any terms</literal>. This
1461
will not change radically the results, but will give a relevance
1462
boost to the results where the search terms appear as a
1463
phrase. Ie: searching for <literal>virtual reality</literal>
1464
will still find all documents where either
1465
<literal>virtual</literal> or <literal>reality</literal> or
1466
both appear, but those which contain <literal>virtual
1467
reality</literal> should appear sooner in the list.</para>
1468
</formalpara>
1469
1470
<para>Phrase searches can strongly slow down a query if most of the
1471
terms in the phrase are common. This is why the
1472
<literal>autophrase</literal> option is off by default for &RCL;
1473
versions before 1.17. As of version 1.17,
1474
<literal>autophrase</literal> is on by default, but very common
1475
terms will be removed from the constructed phrase. The removal
1476
threshold can be adjusted from the search preferences.</para>
1477
1478
<formalpara><title>Phrases and abbreviations</title> <para>As of
1479
&RCL; version 1.17, dotted abbreviations like
1480
<literal>I.B.M.</literal> are also automatically indexed as a word
1481
without the dots: <literal>IBM</literal>. Searching for the word
1482
inside a phrase (ie: <literal>"the IBM company"</literal>) will only
1483
match the dotted abrreviation if you increase the phrase slack (using the
1484
advanced search panel control, or the <literal>o</literal> query
1485
language modifier). Literal occurences of the word will be matched
1486
normally.</para></formalpara>
1487
1439
1488
1440
1489
</sect3>
1441
1490
1452
1501
</para>
1453
1502
</formalpara>
1454
1503
1504
<formalpara><title>Ajusting the result table columns</title>
1505
<para>When displaying results in table mode, you can use a
1506
right click on the table headers to activate a pop-up menu
1507
which will let you adjust what columns are displayed. You can
1508
drag the column headers to adjust their order. You can click
1509
them to sort by the field displayed in the column. You can
1510
also save the result list in CSV format.</para>
1511
</formalpara>
1512
1455
1513
<formalpara><title>Query explanation</title>
1456
1514
<para>You can get an exact description of what the query
1457
1515
looked for, including stem expansion, and Boolean operators
1510
1568
interface itself, the parameters used for searching and
1511
1569
returning results, and what indexes are searched.</para>
1512
1570
1571
1513
1572
<formalpara id="rcl.search.custom.ui">
1514
1573
<title>User interface parameters:</title>
1515
1574
<para>
1516
1575
<itemizedlist>
1517
1576
1518
<listitem><para><guilabel>Number of results in a result
1519
page</guilabel>: </para>
1520
</listitem>
1521
1522
<listitem><para><guilabel>Hide duplicate results</guilabel>:
1523
decides if result list entries are shown for identical
1524
documents found in different places.</para>
1525
</listitem>
1526
1527
1577
<listitem><para><guilabel>Highlight color for query
1528
1578
terms</guilabel>: Terms from the user query are highlighted in
1529
1579
the result list samples and the preview window. The color can
1532
1582
default is <literal>blue</literal>.</para>
1533
1583
</listitem>
1534
1584
1535
<listitem><para><guilabel>Result list font</guilabel>: There is
1536
quite a lot of information shown in the result list, and you
1537
may want to customize the font and/or font size. The rest of
1538
the fonts used by &RCL; are determined by your generic Qt
1539
config (try the <command>qtconfig</command> command).</para>
1540
</listitem>
1541
1542
<listitem><anchor id="rcl.search.custom.resultpara">
1543
<para><guilabel>Result paragraph format string</guilabel>:
1544
allows you to change the presentation of each result list
1545
entry. This is <link linkend="rcl.search.custom.reslistpara">
1546
described in its own section.</link></para>
1547
</listitem>
1548
1549
<listitem><anchor id="rcl.search.custom.abssep">
1550
<para><guilabel>Abstract snippet separator</guilabel>:
1551
for synthetic abstracts built from index data, which are
1552
usually made of several snippets from different parts of the
1553
document, this defines the snippet separator, an ellipsis by
1554
default. </para>
1585
<listitem><para><guilabel>Style sheet</guilabel>:
1586
The name of a <application>Qt</application> style sheet
1587
text file which is applied to the whole Recoll application
1588
on startup. The default value is empty, but there is a
1589
skeleton style sheet (<filename>recoll.qss</filename>)
1590
inside the <filename>/usr/share/recoll/examples</filename>
1591
directory. Using a style sheet, you can change most Recoll
1592
graphical parameters: colors, fonts, etc. See the sample
1593
file for a few simple examples.</para>
1555
1594
</listitem>
1556
1595
1557
1596
<listitem><para><guilabel>Maximum text size highlighted for
1561
1600
text size to speed up loading.</para>
1562
1601
</listitem>
1563
1602
1603
<listitem><para><guilabel>Prefer HTML to plain text for
1604
preview</guilabel> if set, Recoll will display HTML as such
1605
inside the preview window. If this causes problems with the Qt
1606
HTML display, you can uncheck it to display the plain text
1607
version instead. </para>
1608
</listitem>
1609
1610
<listitem><para><guilabel>Use <PRE> tags instead of
1611
<BR> to display plain text as HTML in preview</guilabel>:
1612
when displaying plain text inside the preview window, &RCL;
1613
tries to preserve some of the original text line breaks and
1614
indentation. It can either use PRE HTML tags, which will
1615
well preserve the indentation but will force horizontal
1616
scrolling for long lines, or use BR tags to break at the
1617
original line breaks, which will let the editor introduce
1618
other line breaks according to the window width, but will
1619
lose some of the original indentation.</para>
1620
</listitem>
1621
1564
1622
<listitem><para><guilabel>Use desktop preferences to choose
1565
1623
document editor</guilabel>: if this is checked, the
1566
1624
<command>xdg-open</command> utility will be used to open files
1600
1658
stat between invocations. It normally starts with sorting
1601
1659
disabled.</para>
1602
1660
</listitem>
1603
<listitem><para><guilabel>Prefer HTML to plain text for preview
1604
1605
</guilabel> if set, Recoll will display HTML as such inside the
1606
preview window. If this causes problems with the Qt HTML
1607
display, you can uncheck it to display the plain text version
1608
instead. </para>
1609
</listitem>
1610
1661
1611
1662
</itemizedlist>
1612
1663
</para>
1613
1664
</formalpara>
1614
1665
1615
1666
1667
<formalpara id="rcl.search.custom.rl">
1668
<title>Result list parameters:</title>
1669
<para>
1670
<itemizedlist>
1671
1672
<listitem><para><guilabel>Number of results in a result
1673
page</guilabel></para>
1674
</listitem>
1675
1676
<listitem><para><guilabel>Result list font</guilabel>: There is
1677
quite a lot of information shown in the result list, and you
1678
may want to customize the font and/or font size. The rest of
1679
the fonts used by &RCL; are determined by your generic Qt
1680
config (try the <command>qtconfig</command> command).</para>
1681
</listitem>
1682
1683
<listitem id="rcl.search.custom.resultpara">
1684
<para><guilabel>Edit result list paragraph format string</guilabel>:
1685
allows you to change the presentation of each result list
1686
entry. See the <link linkend="rcl.search.custom.reslist">
1687
result list customisation section</link>.</para>
1688
</listitem>
1689
1690
<listitem id="rcl.search.custom.resulthead">
1691
<para><guilabel>Edit result page html header insert</guilabel>:
1692
allows you to define text inserted at the end of the result
1693
page html header.
1694
More detail in the <link linkend="rcl.search.custom.reslist">
1695
result list customisation section.</link></para>
1696
</listitem>
1697
1698
<listitem>
1699
<para><guilabel>Date format</guilabel>: allows specifying the
1700
format used for displaying dates inside the result list. This
1701
should be specified as an strftime() string (man strftime).</para>
1702
</listitem>
1703
1704
<listitem id="rcl.search.custom.abssep">
1705
<para><guilabel>Abstract snippet separator</guilabel>:
1706
for synthetic abstracts built from index data, which are
1707
usually made of several snippets from different parts of the
1708
document, this defines the snippet separator, an ellipsis by
1709
default. </para>
1710
</listitem>
1711
1712
</itemizedlist></para>
1713
</formalpara>
1714
1616
1715
<formalpara id="rcl.search.custom.search">
1617
1716
<title>Search parameters:</title>
1618
1717
<para>
1619
1718
<itemizedlist>
1620
1719
1720
<listitem><para><guilabel>Hide duplicate results</guilabel>:
1721
decides if result list entries are shown for identical
1722
documents found in different places.</para>
1723
</listitem>
1724
1621
1725
<listitem><para><guilabel>Stemming language</guilabel>:
1622
1726
stemming obviously depends on the document's language. This
1623
1727
listbox will let you chose among the stemming databases which
1630
1734
file.</para>
1631
1735
</listitem>
1632
1736
1633
<listitem><para><guilabel>Dynamically add phrase to simple
1737
<listitem><para><guilabel>Automatically add phrase to simple
1634
1738
searches</guilabel>: a phrase will be automatically built and
1635
1739
added to simple searches when looking for <literal>Any
1636
1740
terms</literal>. This will give a relevance boost to the
1638
1742
and in order).</para>
1639
1743
</listitem>
1640
1744
1745
<listitem><para><guilabel>Autophrase term frequency threshold
1746
percentage</guilabel>: very frequent terms should not be included
1747
in automatic phrase searches for performance reasons. The
1748
parameter defines the cutoff percentage (percentage of the
1749
documents where the term appears).</para>
1750
</listitem>
1751
1641
1752
<listitem><para><guilabel>Replace abstracts from
1642
1753
documents</guilabel>: this decides if we should synthesize and
1643
1754
display an abstract in place of an explicit abstract found
1680
1791
their database directory (ie:
1681
1792
<filename>/home/someothergui/.recoll/xapiandb</filename>,
1682
1793
<filename>/usr/local/recollglobal/xapiandb</filename>).</para>
1794
</formalpara>
1683
1795
1684
1796
<para>Once entered, the indexes will appear in the
1685
1797
<guilabel>External indexes</guilabel> list, and you can
1693
1805
need to implement a way of purging the index from stale data,
1694
1806
</para>
1695
1807
1696
<sect3 id="rcl.search.custom.reslistpara">
1697
<title>The result list paragraph format</title>
1698
1699
<para>The presentation of each result inside the result list can be
1700
customized by setting the result list paragraph format inside the
1701
<guilabel>User Interface</guilabel> tab of the <guilabel>Query
1702
configuration</guilabel>.</para>
1703
1704
<para>This is a Qt HTML string where the following printf-like
1705
<literal>%</literal> substitutions will be performed:
1808
<sect3 id="rcl.search.custom.reslist">
1809
<title>The result list format</title>
1810
1811
<para>The result list presentation can be exhaustively customized
1812
by adjusting two elements:</para>
1813
<itemizedlist>
1814
<listitem><para>The paragraph format</para></listitem>
1815
<listitem><para>Html code inside the header
1816
section</para></listitem>
1817
</itemizedlist>
1818
1819
<para>These can be edited from the <guilabel>Result list</guilabel>
1820
tab of the <guilabel>Query configuration</guilabel>.</para>
1821
1822
<para>Newer versions of Recoll (from 1.17) use a WebKit HTML
1823
object by default (this may be disabled at build time), and
1824
total customisation is possible with full support for CSS and
1825
Javascript. Conversely, there are limits to what you can do with
1826
the older Qt QTextBrowser, but still, it is possible to decide
1827
what data each result will contain, and how it will be
1828
displayed.</para>
1829
1830
<para>No more detail will be given about the header part (only
1831
useful with the WebKit build), if there are restrictions to
1832
what you can do, they are beyond this author's HTML/CSS/Javascript
1833
abilities... There are a few exemples on the
1834
<ulink url="http://www.recoll.org/custom.html">page about
1835
customising the result list</ulink> on the &RCL; web site.</para>
1836
1837
<sect4 id="rcl.search.custom.reslist.para">
1838
<title>The paragraph format</title>
1839
1840
<para>This is an arbitrary HTML string where the following printf-like
1841
<literal>%</literal> substitutions will be performed:
1706
1842
1707
1843
<itemizedlist>
1708
1844
<listitem>
1710
1846
</listitem>
1711
1847
<listitem><formalpara><title>%D</title><para>Date</para></formalpara>
1712
1848
</listitem>
1713
<listitem><formalpara><title>%I</title><para>Icon image name
1714
</para></formalpara>
1849
<listitem><formalpara><title>%I</title><para>Icon image
1850
name. This is normally determined from the mime type. The
1851
associations are defined inside the
1852
<link linkend="rcl.install.config.mimeconf">
1853
<filename>mimeconf</filename> configuration file</link>.
1854
If a thumbnail for the file is found at
1855
the standard Freedesktop location, this will be displayed
1856
instead.</para></formalpara>
1715
1857
</listitem>
1716
1858
<listitem><formalpara><title>%K</title><para>Keywords (if
1717
1859
any)</para></formalpara>
1718
1860
</listitem>
1719
<listitem><formalpara><title>%L</title><para>Preview and
1720
Edit links</para></formalpara>
1861
<listitem><formalpara><title>%L</title><para>Precooked Preview and
1862
Edit links</para></formalpara>
1721
1863
</listitem>
1722
1864
<listitem><formalpara><title>%M</title><para>Mime
1723
1865
type</para></formalpara>
1724
1866
</listitem>
1725
<listitem><formalpara><title>%N</title><para>result Number
1726
</para></formalpara>
1867
<listitem><formalpara><title>%N</title><para>result Number inside
1868
the result page</para></formalpara>
1727
1869
</listitem>
1728
1870
<listitem><formalpara><title>%R</title><para>Relevance
1729
percentage</para></formalpara>
1871
percentage</para></formalpara>
1730
1872
</listitem>
1731
1873
<listitem><formalpara><title>%S</title><para>Size
1732
1874
information</para></formalpara>
1733
1875
</listitem>
1734
<listitem><formalpara><title>%T</title><para>Title</para>
1735
</formalpara>
1876
<listitem><formalpara><title>%T</title><para>Title or Filename if
1877
not set.</para></formalpara>
1878
</listitem>
1879
<listitem><formalpara><title>%t</title><para>Title or Filename if
1880
not set.</para></formalpara>
1736
1881
</listitem>
1737
1882
<listitem><formalpara><title>%U</title><para>Url</para></formalpara>
1738
1883
</listitem>
1742
1887
<literal><a href="P%N"></literal>
1743
1888
and
1744
1889
<literal><a href="E%N"></literal>
1745
where <replaceable>docnum</replaceable> (%N expands to the document
1746
number inside the result list).</para>
1890
where <replaceable>docnum</replaceable> (%N) expands to the document
1891
number inside the result page).</para>
1747
1892
1748
1893
<para>In addition to the predefined values above, all strings like
1749
1894
<literal>%(fieldname)</literal> will be replaced by the value of
1753
1898
search process (see <link linkend="rcl.program.fields">field
1754
1899
configuration</link>). There are currently very few fields stored
1755
1900
by default, apart from the values above (only
1756
<literal>author</literal>), so this feature will need some custom
1757
local configuration to be useful. For example, you could look at
1758
the fields for the document types of interest (use the right-click
1759
menu inside the preview window), and add what you want to the list
1760
of stored fields. A candidate example would be the
1761
<literal>recipient</literal> field which is generated by the
1762
message filters.</para>
1901
<literal>author</literal> and <literal>filename</literal>), so this
1902
feature will need some custom local configuration to be useful. For
1903
example, you could look at the fields for the document types of
1904
interest (use the right-click menu inside the preview window), and
1905
add what you want to the list of stored fields. A candidate example
1906
would be the <literal>recipient</literal> field which is generated
1907
by the message filters.</para>
1763
1908
1764
1909
<para>The default value for the paragraph format string is:
1765
1910
<programlisting><img src="%I" align="left">%R %S %L &nbsp;&nbsp;<b>%T</b><br>
1783
1928
preview link.
1784
1929
</para>
1785
1930
1931
<para>These samples, and some others are
1932
<ulink url="http://www.recoll.org/custom.html">on the web
1933
site, with pictures to show how they look.</ulink></para>
1934
1786
1935
<para>It is also possible to
1787
1936
<link linkend="rcl.search.custom.abssep">
1788
1937
define the value of the snippet separator inside the abstract
1789
1938
section</link>.</para>
1939
</sect4>
1790
1940
</sect3>
1791
1941
</sect2>
1792
1942
1891
2041
<para><command>recollq</command> has a man page (not installed by
1892
2042
default, look in the <filename>doc/man</filename> directory). The
1893
2043
Usage string is as follows:</para>
1894
<programlisting>recollq [-o|-a|-f] <query string>
2044
<programlisting>
2045
recollq: usage:
2046
-P: Show the date span for all the documents present in the index
2047
[-o|-a|-f] [-q] <query string>
1895
2048
Runs a recoll query and displays result lines.
1896
Default: will interpret the argument(s) as a query language string
1897
-o Emulate the gui simple search in ANY TERM mode
1898
-a Emulate the gui simple search in ALL TERMS mode
1899
-f Emulate the gui simple search in filename mode
2049
Default: will interpret the argument(s) as a xesam query string
2050
query may be like:
2051
implicit AND, Exclusion, field spec: t1 -t2 title:t3
2052
OR has priority: t1 OR t2 t3 OR t4 means (t1 OR t2) AND (t3 OR t4)
2053
Phrase: "t1 t2" (needs additional quoting on cmd line)
2054
-o Emulate the GUI simple search in ANY TERM mode
2055
-a Emulate the GUI simple search in ALL TERMS mode
2056
-f Emulate the GUI simple search in filename mode
2057
-q is just ignored (compatibility with the recoll GUI command line)
1900
2058
Common options:
1901
-c <configdir> : specify config directory, overriding $RECOLL_CONFDIR
2059
-c <configdir> : specify config directory, overriding $RECOLL_CONFDIR
1902
2060
-d also dump file contents
1903
-n <cnt> limit the maximum number of results (0->no limit, default 2000)
2061
-n [first-]<cnt> define the result slice. The default value for [first]
2062
is 0. Without the option, the default max count is 2000.
2063
Use n=0 for no limit
1904
2064
-b : basic. Just output urls, no mime types or titles
1905
-m : dump the whole document meta[] array
1906
-S fld : sort by field name
2065
-Q : no result lines, just the processed query and result count
2066
-m : dump the whole document meta[] array for each result
2067
-A : output the document abstracts
2068
-S fld : sort by field <fld>
1907
2069
-D : sort descending
2070
-i <dbdir> : additional index, several can be given
2071
-e use url encoding (%xx) for urls
2072
-F <field name list> : output exactly these fields for each result.
2073
The field values are encoded in base64, output in one line and
2074
separated by one space character. This is the recommended format
2075
for use by other programs. Use a normal query with option -m to
2076
see the field names.
1908
2077
</programlisting>
1909
2078
1910
2079
<para>Sample execution:</para>
1929
2098
capabilities as the complex search interface in the
1930
2099
GUI.</para>
1931
2100
1932
<para>The language is roughly based on the <ulink
1933
url="http://www.xesam.org/main/XesamUserSearchLanguage95">
1934
Xesam</ulink> user search language specification.</para>
2101
<para>The language is roughly based on the (seemingly defunct)
2102
<ulink url="http://www.xesam.org/main/XesamUserSearchLanguage95">
2103
Xesam</ulink> user search language specification.</para>
1935
2104
1936
2105
<para>If the results of a query language search puzzle you and you
1937
doubt what has been actually searched for, you can use the GUI
1938
<literal>show query</literal> link at the top of the result list to
1939
check the exact query which was finally executed by Xapian.</para>
2106
doubt what has been actually searched for, you can use the GUI
2107
<literal>show query</literal> link at the top of the result list to
2108
check the exact query which was finally executed by Xapian.</para>
1940
2109
1941
2110
<para>Here follows a sample request that we are going to
1942
explain:</para>
2111
explain:</para>
1943
2112
1944
2113
<programlisting>
1945
2114
author:"john doe" Beatles OR Lennon Live OR Unplugged -potatoes
1993
2162
<replaceable>title:"prejudice pride"</replaceable> is not the same as
1994
2163
<replaceable>title:prejudice title:pride</replaceable>, and is
1995
2164
unlikely to find a result.</para>
1996
<para>Most Xesam phrase modifiers are unsupported, except for
1997
<literal>l</literal> (small ell) to disable stemming, and
1998
<literal>p</literal> to turn a phrase into a NEAR (unordered proximity)
1999
search. Exemple: <replaceable>"prejudice pride"p</replaceable></para>
2165
2166
<para>Modifiers can be set on a phrase clause, for exemple to specify
2167
a proximity search (unordered). See
2168
<link linkend="rcl.search.lang.modifiers">the modifier
2169
section</link>.</para>
2000
2170
2001
2171
<para>&RCL; currently manages the following default fields:</para>
2002
2172
<itemizedlist>
2015
2185
document-specified keywords (few documents actually have any).</para>
2016
2186
</listitem>
2017
2187
<listitem><para><literal>filename</literal> for the document's
2018
file name.</listitem>
2188
file name.</para></listitem>
2019
2189
<listitem><para><literal>ext</literal> specifies the file
2020
2190
name extension (Ex: <literal>ext:html</literal>)</para>
2021
2191
</listitem>
2028
2198
results on file location (Ex:
2029
2199
<literal>dir:/home/me/somedir</literal>). <literal>-dir</literal>
2030
2200
also works to find results out of the specified directory, only
2031
after release 1.15.8.</para>
2201
after release 1.15.8. A tilde inside the value will be expanded to
2202
the home directory. <literal>dir</literal> is not a regular field
2203
and only one value makes sense in a query (you can't use
2204
<literal>dir:dir1 OR dir:dir2</literal>). Relative paths make
2205
sense, for example,
2206
<literal>dir:share/doc</literal> would match either
2207
<filename>/usr/share/doc</filename> or
2208
<filename>/usr/local/share/doc</filename> </para>
2209
</listitem>
2210
2211
<listitem><para><literal>size</literal> for filtering the
2212
results on file size. Exemple:
2213
<literal>size<10000</literal>. You can use
2214
<literal><</literal>, <literal>></literal> or
2215
<literal>=</literal> as operators. You can specify a range like the
2216
following: <literal>size>100 size<1000</literal>. The usual
2217
<literal>k/K, m/M, g/G, t/T</literal> can be used as (decimal)
2218
multipliers. Ex: <literal>size>1k</literal> to search for files
2219
bigger than 1000 bytes.</para>
2032
2220
</listitem>
2033
2221
2034
2222
<listitem><para><literal>date</literal> for searching or filtering
2112
2300
field search possibilities may be different for you if someone
2113
2301
took care of the customisation.</para>
2114
2302
2115
<sect2>
2303
<sect2 id="rcl.search.lang.modifiers">
2116
2304
<title>Modifiers</title>
2117
2305
2118
2306
<para>Some characters are recognized as search modifiers when found
2142
2330
modifiers. Example: <literal>"Important"2.5</literal>.</para>
2143
2331
</listitem>
2144
2332
</itemizedlist>
2333
</para>
2334
2145
2335
2146
2336
</sect2> <!-- search modifiers -->
2147
2337
2327
2517
handle the protocol.</para>
2328
2518
</listitem>
2329
2519
</itemizedlist>
2330
The following will just describe the simple filters, if you are
2331
programmer enough to write one of the other kind, it shouldn't be too
2332
difficult to make sense of one of the existing modules (ie:
2333
rclzip).</para>
2520
The following will just describe the simple filters. If you can
2521
program and want to write one of the other kind, it shouldn't be too
2522
difficult to make sense of one of the existing modules. For example,
2523
look at <command>rclzip</command> which uses Zip file paths as
2524
internal identifiers (<literal>ipath</literal>), and
2525
<command>rclinfo</command>, which uses an integer index.</para>
2526
2527
<sect2 id="rcl.program.filters.simple">
2528
<title>Simple filters</title>
2334
2529
2335
2530
<para>&RCL; simple filters are usually shell-scripts, but this is in
2336
no way necessary. These programs are extremely simple and most
2337
of the difficulty lies in extracting the text from the native
2338
format, not outputting what is expected by &RCL;. Happily
2339
enough, most document formats already have translators or text
2340
extractors which handle the difficult part and can be called
2341
from the filter. In some case the output of the translating
2342
program is appropriate, and no intermediate shell-script is
2343
needed.</para>
2531
no way necessary. Extracting the text from the native format is the
2532
difficult part. Outputting the format expected by &RCL; is
2533
trivial. Happily enough, most document formats have translators or
2534
text extractors which can be called from the filter. In some cases
2535
the output of the translating program is completely appropriate,
2536
and no intermediate shell-script is needed.</para>
2344
2537
2345
2538
<para>Filters are called with a single argument which is the
2346
2539
source file name. They should output the result to stdout.</para>
2347
2540
2348
<para>The <literal>RECOLL_FILTER_FORPREVIEW</literal>
2349
environment variable (values <literal>yes</literal>,
2350
<literal>no</literal>) tells the filter if the operation is
2351
for indexing or previewing. Some filters use this to output a
2352
slightly different format. This is not essential.</para>
2541
<para>When writing a filter, you should decide if it will output
2542
plain text or html. Plain text is simpler, but you will not be able
2543
to add metadata or vary the output character encoding (this will be
2544
defined in a configuration file). Additionally, some formatting may
2545
easier to preserve when previewing html. Actually the deciding factor
2546
is metadata: &RCL; has a way to <link linkend="rcl.program.filters.html">
2547
extract metadata from the html header and use it for field
2548
searches.</link>.</para>
2549
2550
<para>The <literal>RECOLL_FILTER_FORPREVIEW</literal> environment
2551
variable (values <literal>yes</literal>, <literal>no</literal>)
2552
tells the filter if the operation is for indexing or
2553
previewing. Some filters use this to output a slightly different
2554
format, for example stripping uninteresting repeated keywords (ie:
2555
<literal>Subject:</literal> for email) when indexing. This is not
2556
essential.</para>
2557
2558
<para>You should look to one of the simple filters, for exemple
2559
<literal>rclps</literal> for a starting point.</para>
2560
2561
<para>Don't forget to make your filter executable before
2562
testing !</para>
2563
2564
</sect2>
2565
2566
<sect2 id="rcl.program.filters.association">
2567
<title>Telling &RCL; about the filter</title>
2568
2569
<para>There are two elements that link a file to the filter which
2570
should process it: the association of file to mime type and the
2571
association of a mime type with a filter.</para>
2572
2573
<para>The association of files to mime types is mostly based on
2574
name suffixes. The types are defined inside the
2575
<link linkend="rcl.install.config.mimemap">
2576
<filename>mimemap</filename> file</link>. Example:
2577
<programlisting>
2578
2579
.doc = application/msword
2580
</programlisting>
2581
If no suffix association is found for the file name, &RCL; will try
2582
to execute the <command>file -i</command> command to determine a
2583
mime type.</para>
2353
2584
2354
2585
<para>The association of file types to filters is performed in
2355
the <filename>mimeconf</filename> file. A sample:</para>
2586
the <link linkend="rcl.install.config.mimeconf">
2587
<filename>mimeconf</filename> file</link>. A sample will probably be
2588
of better help than a long explanation:</para>
2356
2589
<programlisting>
2357
2590
2358
2591
[index]
2395
2628
<literal>execm</literal> keyword.</para>
2396
2629
</listitem>
2397
2630
</itemizedlist>
2398
The easiest way to write a new filter is probably to start from an
2399
existing one.</para>
2400
2401
<para>Filters which output <literal>text/plain</literal> text
2402
are generally simpler, but they cannot specify the character set
2403
and other metadata, so they are limited to cases where these
2404
elements are not needed.</para>
2405
2631
</para>
2632
2633
</sect2>
2406
2634
2407
2635
<sect2 id="rcl.program.filters.html">
2408
2636
<title>Filter HTML output</title>
2412
2640
2413
2641
<programlisting><html><head>
2414
2642
<meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
2415
</head>
2643
</head>
2416
2644
<body>some text content</body></html>
2417
2645
</programlisting>
2418
2646
2601
2829
<sect3 id="rcl.program.python.manual">
2602
2830
<title>Interface manual</title>
2603
2831
2604
<literalLayout>
2832
<literallayout>
2605
2833
NAME
2606
2834
recoll - This is an interface to the Recoll full text indexer.
2607
2835
2787
3015
writable decides if we can index new data through this connection
2788
3016
2789
3017
2790
</literalLayout>
3018
</literallayout>
2791
3019
</sect3>
2792
3020
2793
3021
<sect3 id="rcl.program.python.examples">
2799
3027
2800
3028
<programlisting>
2801
3029
#!/usr/bin/env python
2802
3030
<![CDATA[
2803
3031
import recoll
2804
3032
2805
3033
db = recoll.connect()
2820
3048
print
2821
3049
2822
3050
2823
3051
]]>
2824
3052
</programlisting>
2825
3053
2826
3054
</sect3>
2827
3055
2828
3056
</sect2>
2829
3057
</sect1>
2830
3058
</chapter>
2831
3059
2832
3060
2949
3177
<listitem><para>Postscript files need <command>pstotext</command>.
2950
3178
The original version has an issue with shell
2951
3179
character in file names, which is corrected in recent
2952
packages. See the the &RCLAPPS; for more detail.
3180
packages. See the the &RCLAPPS; for more detail.</para>
2953
3181
</listitem>
2954
3182
2955
3183
<listitem><para>MS Word needs
3088
3316
<application>Linux</application> systems, the iconv interface
3089
3317
is part of libc and you should not need to do anything
3090
3318
special.</para>
3091
3319
3320
</sect2>
3321
3092
3322
<sect2 id="rcl.install.building.build">
3093
3323
<title>Building</title>
3094
3324
3120
3350
<para>On many Linux systems, <literal>QTDIR</literal> is set
3121
3351
by the login scripts, and <literal>QMAKESPECS</literal> is not
3122
3352
needed because there is a <filename>default</filename> link in
3123
<filename>mkspecs/</filename>.
3353
<filename>mkspecs/</filename>.</para>
3124
3354
3125
3355
<para>Neither <literal>QTDIR</literal> nor
3126
3356
<literal>QMAKESPECS</literal> should be needed with
3140
3370
real time indexing. Inotify support is enabled by default on
3141
3371
recent Linux systems.</para>
3142
3372
</listitem>
3373
<listitem><para><literal>--disable-webkit</literal> is available
3374
from version 1.17 to implement the result list with a
3375
<application>Qt</application> QTextBrowser instead of a
3376
WebKit widget if you do not or can't depend on the
3377
latter.</para>
3378
</listitem>
3143
3379
<listitem><para><literal>--enable-xattr</literal> will enable
3144
3380
code to fetch data from file extended attributes. This is only
3145
3381
useful is some application stores data in there, and also needs
3161
3397
enable the gnu version on systems where the native one is
3162
3398
bad.</para>
3163
3399
</listitem>
3164
<listitem><para><literal>--without-gui</literal> Disable the Qt
3165
interface, and auxiliary uses of X11, and compile the command
3166
line version.</para>
3400
<listitem><para><literal>--disable-qtgui</literal> Disable the Qt
3401
interface. Will allow building the indexer and the command line
3402
search program in absence of a Qt environment.</para>
3403
</listitem>
3404
<listitem><para><literal>--disable-x11mon</literal> Disable
3405
X11 connection monitoring inside recollindex. Together with
3406
--disable-qtgui, this allows building recoll without Qt and
3407
X11.</para>
3167
3408
</listitem>
3168
3409
<listitem><para>Of course the usual
3169
3410
<application>autoconf</application> <command>configure</command>
3224
3465
3225
3466
<para>Most of the parameters specific to the
3226
3467
<command>recoll</command> GUI are set through the
3227
<guilabel>Preferences</guilabel> menu and stored in the
3228
standard Qt place (<filename>$HOME/.qt/recollrc</filename>).
3468
<guilabel>Preferences</guilabel> menu and stored in the standard Qt
3469
place (<filename>$HOME/.config/Recoll.org/recoll.conf</filename>).
3229
3470
You probably do not want to edit this by hand.</para>
3230
3471
3231
3472
<para>&RCL; indexing options are set inside text configuration
3409
3650
</listitem>
3410
3651
</varlistentry>
3411
3652
3653
<varlistentry id="rcl.install.config.recollconf.skippedpathsfnmpathname">
3654
<term><literal>skippedPathsFnmPathname</literal></term>
3655
<listitem><para>The values in the
3656
<literal>*skippedPaths</literal> variables are matched by
3657
default with <literal>fnmatch(3)</literal>, with the
3658
FNM_PATHNAME and FNM_LEADING_DIR flags. This means that '/'
3659
characters must be matched explicitely. You can set
3660
<literal>skippedPathsFnmPathname</literal> to 0 to disable
3661
the use of FNM_PATHNAME (meaning that /*/dir3 will match
3662
/dir1/dir2/dir3).</para>
3663
3664
</listitem>
3665
</varlistentry>
3666
3412
3667
<varlistentry id="rcl.install.config.recollconf.followlinks">
3413
3668
<term><literal>followLinks</literal></term>
3414
3669
<listitem><para>Specifies if the indexer should follow
3599
3854
</listitem>
3600
3855
</varlistentry>
3601
3856
3857
<varlistentry><term><literal>idxstatusfile</literal></term>
3858
<listitem><para>The name of the scratch file where the indexer
3859
process updates its status. Default:
3860
<filename>idxstatus.txt</filename> inside the configuration
3861
directory.</para>
3862
</listitem>
3863
</varlistentry>
3864
3602
3865
<varlistentry><term><literal>maxfsoccuppc</literal></term>
3603
3866
<listitem><para>Maximum file system occupation before we
3604
3867
stop indexing. The value is a percentage, corresponding to
3770
4033
</varlistentry>
3771
4034
3772
4035
</variablelist>
3773
4036
</sect3>
3774
4037
</sect2>
3775
4038
3776
4039
<sect2 id="rcl.install.config.fields">
3798
4061
<varlistentry>
3799
4062
<term>[prefixes]</term>
3800
4063
<listitem><para>A field becomes indexed (searchable) by having
3801
a prefix defined in this section.
4064
a prefix defined in this section.</para>
3802
4065
</listitem>
3803
4066
</varlistentry>
3804
4067
<varlistentry>
3805
4068
<term>[stored]</term>
3806
4069
<listitem><para>A field becomes stored (displayable inside
3807
4070
results) by having its name listed in this section (typically
3808
with an empty value).
4071
with an empty value).</para>
3809
4072
</listitem>
3810
4073
</varlistentry>
3811
4074
<varlistentry>
Loggerhead is a web-based interface for Breezy
Version: 2.0.1