1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
|
README for Data Base All Enhanced version 0.7, still unreleased
===============================================================
Introduction
------------
DB-All.e is a fast on-disk database where meteorological observed and
forecast data can be stored, searched, retrieved and updated.
This framework allows to manage large amounts of data using its simple
Application Program Interface, and provides tools to visualise, import
and export in the standard formats BUFR, AOF and CREX.
The main characteristics of DB-ALL.e are:
* Fortran, C, C++ and Python APIs are provided.
* To make computation easier, data is stored as physical quantities,
that is, as measures of a variable in a specific point of space and
time, rather than as a sequence of report.
* Internal representation is similar to BUFR and CREX WMO standard
(table code driven) and utility for import and export are included
(generic and ECMWF template).
* Representation is in 7 dimensions: latitude and longitude geographic
coordinates, table driven vertical coordinate, reference time,
table driven observation and forecast specification, table driven
data type.
* It allows to store extra information linked to the data, such as
confidence intervals for quality control.
* It allows to store extra information linked to the stations.
* Variables can be represented as real, integer and characters, with
appropriate precision for the type of measured value.
* It is based on physical principles, that is, the data it contains are
defined in terms of homogeneous and consistent physical data. For
example, it is impossible for two incompatible values to exist in the
same point in space and time.
* It can manage fixed stations and moving stations such as airplanes or
ships.
* It can manage both observational and forecast data.
* It can manage data along all three dimensions in space, such as data
from soundings and airplanes.
* Report information is preserved. It can work based on physical
parameters or on report types.
* It is temporary, to be used for a limited time and then be deleted.
* Does not need backup, since it only contains replicated or derived data.
* Write access is enabled for its users.
Dependencies
------------
DB-All.e requires unixodbc, gperf and popt in order to be built.
Additionally, to build the Fortran API the CNF library is required, and
gfortran is required to build its unit tests.
To build the documentation, you need doxygen and LaTeX. If latex2html is
present, HTML documentation will also be built. If rst2html is present, the
README file will also be converted to HTML.
In Fedora, the names of the packages depended on are:
* unixODBC-devel gperf gcc-gfortran doxygen tetex tetex-latex latex2html
* For CNF there is an unofficial starmet package: contact the DB-All.e team
about it.
In Debian, the names of the packages depended on are:
* unixodbc-dev gperf libpopt-dev libcnf-dev gfortran doxygen python-docutils
* tetex-bin tetex-extra (in etch), or
* texlive-base texlive-latex-base texlive-latex-extra
texlive-latex-recommended (in lenny and sid)
* latex2html (optional, from non-free)
(see http://www.arpa.emr.it/dettaglio_documento.asp?id=514&idlivello=64)
Building DB-All.e
-----------------
DB-All.e is already packaged in both .rpm and .deb formats, and that provides
easy installation for most Linux distributions.
If you want to build and install DB-All.e yourself, you can proceed as in most
other Unix software::
./configure
make
make install
Getting started
---------------
DB-All.e requires ODBC access to a MySQL database for most of its
functionality.
To set up DB-All.e on the MySQL side, you need to create a database and grant
users create table, drop table, insert, delete and select access. MySQL needs
to run with InnoDB support.
To set up ODBC access to the database (this is called a DSN) you can run a
graphical configuration file like ODBCConfig. Otherwise, this is a sample
.odbc.ini you can use::
[dballe]
Description = MySQL
Driver = MySQL
Server = localhost
Database = dballe
Once this is set up, you can initialise the DB-All.e database using the command::
dbadb wipe --dsn=dballe --user=username [--pass=password]
"--user" can be omitted if it is the same as your Unix username; "--pass" can
be omitted if the user has no password.
If you do not already have access to datasets to import, some are available
from http://www.ncar.ucar.edu/tools/datasets/ after registering (for free) on
the website.
Documentation
-------------
Documentation for all commandline tools can be found in their manpages. All
commandline tools also have extensive commandline help that can be accessed
using the "--help" option.
The Fortran API is documented in the fapi.pdf document.
The C API and all the C internals are documented through Doxygen.
Administration and maintanance of DB-All.e are covered in the guide.pdf
document.
Testing DB-All.e
----------------
Unit testing can be run using "make check", but it requires an existing DSN
connection to a MySQL database, which should be called 'test'. Please note
that unit testing functions will wipe existing DB-All.e tables on the test DSN
database.
Useful resources
----------------
ODBC links:
* http://www.unixodbc.org/doc/ProgrammerManual/Tutorial/
* http://msdn.microsoft.com/library/default.asp?url=/library/en-us/odbc/htm/odbcabout_this_manual.asp
BUFR decoding:
* http://www.knmi.nl/~meulenvd/code/bufr/dmtn1.html
AOF decoding:
* Reading of Fortran "unformatted sequential" from C/C++:
http://astronomy.swin.edu.au/~pbourke/dataformats/fortran/
ECWMF BUFR template codes:
* http://www.ecmwf.int/research/ifsdocs/OBSERVATIONS/Chap2_Obs_types3.html
Contact and copyright information
---------------------------------
The author of DB-ALLe is Enrico Zini <enrico@enricozini.com>
DB-ALLe is Copyright (C) 2005, 2006 ARPA-SIM <urpsim@smr.arpa.emr.it>
DB-ALLe is licensed under the terms of the GNU General Public License version
2. Please see the file COPYING for details.
Contact informations for ARPA-SIM:
Agenzia Regionale Prevenzione e Ambiente (ARPA)
Servizio Idro-Meteorologico (SIM)
Address: Viale Silvani 6, 40122 Bologna, Italy
Tel: + 39 051 6497511
Fax: + 39 051 6497501
Email: urpsim@smr.arpa.emr.it
Website: http://www.arpa.emr.it/sim/
The latest version of DB-ALLe can be downloaded from:
http://www.smr.arpa.emr.it/software/DBalle.html
|