~xrg/openobject-doc/trunk-xrg

There must be a mechanism to describe, modify and reload the different data. These data are represented into a set of XML files that can possibly be loaded during start of the program in order to fill in the tables.

Files: .XML

-----------

%define=lightblue color=#27adfb%

Data can be inserted or updated into the PostgreSQL tables corresponding to the Tiny ERP objects using XML files. The general structure of a Tiny ERP XML file is as follows :

<?xml version="1.0"?>

<terp>

<data>

%lightblue%"field1 content"

</field>

%lightblue%"field2 content"

</field>

(...)

</record>

(...)

</record>

(...)

</data>

</terp>

Fields content are strings that must be encoded as UTF-8 in XML files.

Let's review an example taken from the TinyERP source (base_demo.xml in the base module):

</record>

<field name="login">admin</field>

<field name="password">admin</field>

<field name="name">Administrator</field>

<field name="signature">Administrator</field>

</record>

This last record defines the admin user :

* The fields login, password, etc are straightforward.

* The **ref** attribute allows to fill relations between the records :

->The field @@company_id@@ is a many-to-one relation from the user object to the company object, and **main_company** is the id of to associate.

* The **eval** attribute allows to put some python code in the xml: here the groups_id field is a many2many. For such a field, "[(6,0,[group_admin])]" means : Remove all the groups associated with the current user and use the list [group_admin] as the new associated groups (and group_admin is the id of another record).

* The **search** attribute allows to find the record to associate when you do not know its xml id. You can thus specify a search criteria to find the wanted record. The criteria is a list of tuples of the same form than for the predefined search method. If there are several results, an arbitrary one will be chosen (the first one):

->This is a classical example of the use of @@search@@ in demo data: here we do not really care about which partner we want to use for the test, so we give an empty list. Notice the **model** attribute is currently mandatory.

The Data

++++++++

Record Tag

""""""""""

100

Description

101

~~~~~~~~~~~

102

103

The addition of new data is made with the **record** tag. This one takes a mandatory attribute : **model**. Model is the object name where the insertion has to be done. The tag record can also take an optional attribute: **id**. If this attribute is given, a variable of this name can be used later on, in the same file, to make reference to the newly created resource ID.

104

105

A **record** tag may contain field tags. They indicate the record's **fields** value. If a field is not specified the default value will be used.

106

107

Example

108

~~~~~~~

109

110

111

112

<field name="model">account.invoice</field>

113

<field name="name">Invoices List</field>

114

<field name="report_name">account.invoice.list</field>

115

<field name="report_xsl">account/report/invoice.xsl</field>

116

<field name="report_xml">account/report/invoice.xml</field>

117

</record>

118

119

field tag

120

~~~~~~~~~

121

122

The attributes for the field tag are the following:

123

124

* **name**

125

o mandatory attribute indicating the field name

126

* **eval**

127

o python expression that indicating the value to add

128

* **ref**

129

o reference to an id defined in this file

130

131

function tag

132

~~~~~~~~~~~~

133

134

* model:

135

* name:

136

* eval

137

o should evaluate to the list of parameters of the method to be called, excluding cr and uid

138

139

Example

140

~~~~~~~

141

142

143

144

145

getitem tag

146

147

Takes a subset of the evaluation of the last child node of the tag.

148

149

* type

150

- int or list

151

* index

152

* int or string (a key of a dictionary)

153

154

Example

155

~~~~~~~

156

157

Evaluates to the first element of the list of ids returned by the function node

158

159

.. code-block:: python

160

161

162

163

</getitem>

164

165

CSV Files

166

---------

167

168

Importing from a CSV

169

++++++++++++++++++++

170

171

Instead of using .XML file, you can import .CSV files. It is simpler but the migration system does not migrate the data imported from the .CSV files. It's like the noupdate attribute in .XML files. It's also more difficult to keep track of relations between ressources and it is more slower at the installation of the server.

172

173

Use this only for demo data that will never been upgraded from one version of Tiny ERP to another.

174

175

The name of the object is the name of the file before the first '-'. You must use one file per object to import. For example, to import a file with partners (including their multiple contacts and events), the file must be named like one of the following example:

176

177

* res.partner.csv

178

* res.partner-tiny_demo.csv

179

* res.partner-tiny.demo.csv

180

181

Structure of the CSV file

182

+++++++++++++++++++++++++

183

184

Have a look at the user manual for a complete description on how to construct your .CSV file.

185

186

Usefull info:

187

188

* Separator of field: ,

189

* Quote of fields: "

190

* Encoding to use: UTF-8

191

192

Export demo data and import it from a module

193

++++++++++++++++++++++++++++++++++++++++++++

194

195

You can import .CSV file that have been exported from the Tiny ERP client. This is interesting to create your own demo module. But both formats are not exactly the same, mainly due to the conversion: Structured Data -> Flat Data -> Structured Data.

196

197

* The name of the column (first line of the .CSV file) use the end user term in his own language when you export from the client. If you want to import from a module, you must convert the first column using the fields names. Example, from the partner form:

198

199

Name,Code,Contacts/Contact Name,Contacts/Street,Contacts/Zip

200

201

becomes

202

203

name,ref,address/name,address/street,address/zip

204

205

* When you export from the Tiny ERP client, you can select any many2one fields and their child's relation. When you import from a module, Tiny ERP tries to recreate the relation between the two resources. For example, do not export something like this from a sale order form - otherwise Tiny ERP will not be able to import your file:

206

207

Order Description,Partner/Name,Partner/Payable,Partner/Address/Name

208

209

* To find the link for a many2one or many2many field, the server use the name_search function when importing. So, for a many2one field, it is better to export the field 'name' or 'code' of the related resource only. Use the more unique one. Be sure that the field you export is searchable by the name_search function. (the 'name' column is always searchable).

210

211

Order Description,Partner/Code

212

213

* Change the title of the column for all many2many or many2one fields. It's because you export the related resource and you import a link on the resource. Example from a sale order: Partner/Code should become partner_id and not partner_id/code. If you kept the @@/code@@, Tiny ERP will try to create those entries in the database instead of finding reference to existing ones.

214

215

* Many2many fields. If all the exported data contains 0 or 1 relation on each many2many fields, there will be no problem. Otherwise, the export will result in one line per many2many. The import function expect to get all many2many relations in one column, separated by a comma. So, you have to make to transformation. For example, if the categories "Customer" and "Supplier" already exists :

216

217

name,category_id

218

Smith, "Customer, Supplier"

219

220

If you want to create these two categories you can try :

221

222

name,category_id/name

223

Smith, "Customer, Supplier"

224

225

This does not work as expected: a category "Customer, Supplier" is created. The solution is to create an empty line with only the second category:

226

227

228

name,category_id/name

229

Smith, Customer

230

,Supplier

231

232

(Note the comma before "Supplier").

233

234

235

* Read only fields. Do not try to import read only fields like the amount receivable or payable for a partner. Otherwise, Tiny ERP will not accept to import your file.

236

237

* Exporting trees. You can export and import tree structures using the parent field. You just have to take care of the import order. The parent have to be created before his child's.

238

239

Use record id like in xml file:

240

+++++++++++++++++++++++++++++++

241

242

It's possible to define an id for each line of the csv file. This allow to define references between records:

243

244

id, name, parent_id:id

245

record_one, Father,

246

record_two, Child, record_one

247

248

If you do this, the line with the parent data must be before the child lines in the file.

249

250

Multiple CSV Files

251

------------------

252

253

Importing from multiple CSV a full group of linked data

254

+++++++++++++++++++++++++++++++++++++++++++++++++++++++

255

256

It' possible to import a lot of data, with multiple CSV files imported as a single operation. Assume we have a database with books and authors with a relation many2many between book and author.

257

258

And that you already have a file with a lot of books (like a library) and an other file with a lot of authors and a third file with the links between them.

259

260

How to import that easily in openERP ?

261

262

Definition of an import module

263

++++++++++++++++++++++++++++++

264

265

You can do this in the module you have defined to manage yours books and authors. but Sometimes, the tables to import can also be in several modules.

266

267

For this example, let's say that 'book' object is defined in a module called 'library_management' and that 'Author' object in a module called 'contact_name'.

268

269

In this case, you can create a 'fake' module, just to import the data for all these multiples modules. Call this importation module : 'import_my_books'.

270

271

You create this module as others modules of OpenObject :

272

273

1. create a folder 'import_my_books'

274

2. inside, create a '__init__.py' file with only one line : import import_my_books

275

3. again, in the same folder, create a '__terp__.py' file and in this file, write the following code :

276

277

.. code-block:: python

278

279

280

# -*- encoding: utf-8 -*-

281

{

282

'name': 'My Book Import',

283

'category': 'Data Module 1',

284

'init_xml':[],

285

'author': 'mySelf & I',

286

'depends': ['base','library_management','contact_name'],

287

'version': '1.0',

288

'active': False,

289

'demo_xml': [],

290

'update_xml':['contact_name.author.csv','library.book.csv'],

291

'installable': True

292

}

293

294

295

Creation of CSV files

296

+++++++++++++++++++++

297

298

For the CSV files, you'll import one the after, the other one.

299

300

So you have to choose, in which way you'll treat the many2many relation. For our example, we've choose to import all the authors, then all the books with the links to the authors.

301

302

1. authors CSV file

303

304

You have to put your data in a CSV file without any link to books (because the book ids will be known only AFTERWARDS...) For example : ("contact_name.author.csv")

305

306

307

308

id,last_name,first_name,type

309

author_1,Bradley,Marion Zimmer,Book writer

310

author_2,"Szu T'su",,Chinese philosopher

311

author_3,Zelazny,Roger,Book writer

312

author_4,Arleston,Scotch,Screen Writer

313

author_5,Magnin,Florence,Comics Drawer

314

...

315

316

1. Books CSV file

317

318

Here, you can put the data about your books, but also, the links to the authors, using the same id as the column 'id' of the author CSV file. For example : ("library.book.csv" )

319

320

321

322

id,title,isbn,pages,date,author_ids:id

323

book_a,Les Cours du Chaos,1234567890123,268,1975-12-25,"author_3"

324

book_b,"L'art de la Guerre, en 219 volumes",1234567890124,1978-01-01,"author_2"

325

book_c,"new marvellous comics",1587459248579,2009-01-01,"author_5,author_4"

326

...

327

328

Five remarks :

329

330

1. the field content must be enclosed in double quotes (") if there is a double quote or a comma in the field.

331

2. the dates are in the format YYYY-MM-DD

332

3. if you have many ids in the same column, you must separate them with a comma, and, by the way, you must enclosed the content of the column between double quotes...

333

4. the name of the field is the same as the name of the field in the class definition AND must be followed by ':id' if the content is an ID that must be interpreted by the import module. In fact, "author_4" will be transformed by the import module in an integer id for the database module and this numercial id will be put also in the table between author and book, not the literal ID (author_4).

334

5. the encoding to be used by the CSV file is the 'UTF-8' encoding

335

336

Links between id if the CSV files

337

+++++++++++++++++++++++++++++++++

338

339

Links to id already in the system

340

+++++++++++++++++++++++++++++++++

341

342

343

XML data files convention

344

-------------------------

345

346

347

Developers:Developper's Book/Data Loading/XMLFilesConventions

348

349

350

Jump to: navigation, search

351

352

The ressources are placed in different files according to their uses. By convention;

353

354

.. csv-table::

355

:header: "Name","Description"

356

:widths: 25, 25

357

358

"modulename_workflow.xml","the definitions of workflows"

359

"modulename_view.xml","the views"

360

"modulename_data.xml","the important datas to download"

361

"modulename_report.xml","the reports declarations"

362

"modulename_demo.xml","the useful datas for the demo version"

363

364

365

366

The workflow files have to be loaded before the datas ! Otherwise, the ressource created won't be integrated inside the workflow because the later is not yet defined.

367

368

369

Managing updates

370

----------------

371

372

Managing updates and migrations

373

+++++++++++++++++++++++++++++++

374

375

Open ERP has a built'in migration and upgrade system which allows updates to be nearly (or often) automatic. This system also allows to easily incorporate custom modules.

376

377

Table/Object structure

378

""""""""""""""""""""""

379

380

When you run openerp-server with option --init or --update, the table structure are updated to match the new description that is in .py files. Fields that are removed are not removed in the postgresql database not to lose data.

381

382

So, simply running --update or --init, will upgrade your table structure.

383

384

It's important to run --init=module the first time you install the module. Next time, you must use the --update=module argument instead of the init one. This is because init loads ressources that are loaded only once and never upgraded (eg: ressources with no id="" attribute or within a noupdate="1" <data> tag).

385

386

387

Data

388

""""

389

Some data is automatically loaded at the installation of Tiny ERP:

390

391

* views, actions, menus,

392

* workflows,

393

* demo data

394

395

This data is also migrated to a new version if you run --update or --init.

396

397

Workflows

398

"""""""""

399

400

Workflows are also upgraded automatically. If some activities are removed, the documents states evolves automatically to the preceding activities. That ensure that all documents are always in valid states.

401

402

You can freely remove activities in your XML files. If workitems are in this activity, they will evolve to the preceding unlinked activity. And after the activity will be removed.

403

404

Things to care about during development

405

"""""""""""""""""""""""""""""""""""""""

406

407

Since version 3.0.2 of Tiny ERP, you can not use twice the same 'id="..."' during resource creation in your XML files, unless they are in two different modules.

408

409

Resources which don't contain an id are created (and updated) only once; at the installation of the module or when you use the --init argument.

410

411

If a resource has an id and this resource is not present anymore in the next version of the XML file, Open ERP will automatically remove it from the database. If this resource is still present, Open ERP will update the modifications to this resource.

412

413

If you use a new id, the resource will be automatically created at the next update of this module.

414

415

**Use explicit id declaration !**, Example:

416

417

* view_invoice_form,

418

* view_move_line_tree,

419

* action_invoice_form_open, ...

420

421

It is important to put id="...." to all record that are important for the next version migrations. For example, do not forget to put some id="..." on all workflows transitions. This will allows Open ERP to know which transition has been removed and which transition is new or updated.

422

423

Custom modules

424

""""""""""""""

425

426

For example, if you want to override the view of an object named 'invoice_form' in your xml file (id="invoice_form"). All you have to do is redefine this view in your custom module with the same id. You can prefix ids with the name of the module to reference an id defined in another module.

427

428

Example:

429

430

431

...

432

433

434

This will override the invoice form view. You do not have to delete the old view, like in 3.0 versions of Open ERP.

435

436

Note that it is often better to use view inherytancy instead of overwritting views.

437

438

In this migration system, you do not have to delete any ressource. The migration system will detect if it is an update or a delete using id="..." attributes. This is important to preserve references duing migrations.

439

440

Demo datas

441

""""""""""

442

443

Demo datas do not have to be upgraded; because they are probably modified, deleted, ... by users. So, to avoid demo data to be upgraded, you can put a noupdate="1" attribute in the <data> tag of your .xml data files.

444

445

Summary of update and init process

446

++++++++++++++++++++++++++++++++++

447

448

init:

449

450

modify/add/delete demo data and builtin data

451

452

update:

453

454

modifiy/add/delete non demo data

455

456

Examples of builtin (non demo) data:

457

458

* Menu structure,

459

* View definition,

460

* Workflow description, ...

461

-> Everything that as an id="..." in the .XML data declaration (if no attr noupdate="1" in the header)

462

463

What's going on on a update process:

464

465

1. If you manually added data within the client:

466

* the update process will not change them

467

2. If you dropped data:

468

* if it was demo data, the update process will do nothing

469

* it it was builtin data (like a view), the update process will recreate it

470

3. If you modified data (either in the .XML or the client):

471

* if it's demo data: nothing

472

* if it's builtin data, data are updated

473

4. If builtin data have been deleted in the .XML file:

474

* this data will be deleted in the database.

475

476

Older »