~ubuntu-branches/ubuntu/trusty/libtheora/trusty-proposed

This document describes a RTP payload format for transporting Theora encoded video. It details the RTP encapsulation mechanism for raw Theora data and configuration headers necessary to configure the decoder.

</t>

<t>

Also included within the document are the necessary details for the use of Theora with MIME and Session Description Protocol (SDP).

</t>

</abstract>

<t>

All references to RFC XXXX are to be replaced by references to the RFC number of this memo, when published.

</t>

</note>

</front>

<t>

Theora is a general purpose, lossy video codec. It is based on the VP3 video codec produced by On2 Technologies and has been donated to the Xiph.org Foundation.</t>

<t>

Theora I is a block-based lossy transform codec that utilizes an 8 x 8 Type-II Discrete Cosine Transform and block-based motion compensation. This places it in the same class of codecs as MPEG-1, MPEG-2, MPEG-4, and H.263. The details of how individual blocks are organized and how DCT coefficients are stored in the bitstream differ substantially from these codecs, however. Theora supports only intra frames (I frames in MPEG) and inter frames (P frames in MPEG).

</t>

<t>

Theora provides none of its own framing, synchronization, or protection against transmission errors. Instead, the codec expects to receive a discrete sequence of data packets. Theora is a free-form variable bit rate (VBR) codec, and these packets have no minimum size, maximum size, or fixed/expected size. Theora packets are thus intended to be used with a transport mechanism that provides free-form framing, synchronization, positioning, and error correction in accordance with these design assumptions, such as Ogg <xref target="rfc3533"></xref> or RTP/AVP <xref target="rfc3550"></xref>.

</t>

<t>

Theora I currently supports progressive video data of arbitrary dimensions at a constant frame rate in one of several Y'CbCr color spaces.

Three different chroma subsampling formats are supported: 4:2:0, 4:2:2, and 4:4:4. The Theora I format does not support interlaced material, variable frame rates, bit-depths larger than 8 bits per component, nor alternate color spaces such as RGB or arbitrary multi-channel spaces. Black and white content can be efficiently encoded, however, because the uniform chroma planes compress well. For performance reason, arbitrary frame sizes will be encoded rounding both dimensions to the upper multiple of 16. The original width and height will be encoded in the header and the decoder will use this information to clip the decoded frame to the right dimensions.

</t>

<t>

Theora is similar to the Vorbis audio <xref target="vorbisrtp"></xref> in that the decoder reads the probability model for the entropy coder and all quantization parameters from special "header" packets at the start of the compressed data. It is therefore impossible to decode any video data without having previously fetched the codec info and codec setup headers, although Theora can begin to decode at an arbitrary intra-frame packet so long as the codec has been initialized with the associated headers.

</t>

<t>

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY",

and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 <xref target="rfc2119"></xref>.

</t>

</section>

<t>

For RTP based transportation of Theora encoded video the standard RTP header is followed by a 4 octets payload header, then the payload data. The payload headers are used to associate the Theora data with its associated decoding codebooks as well as indicating if the following packet contains fragmented Theora data and/or the number of whole Theora data frames. The payload data contains the raw Theora bitstream information.

</t>

<t>

For RTP based transport of Theora encoded video the standard RTP header is followed by a 4 octets payload header, then the payload data.

</t>

<t>

The format of the RTP header is specified in <xref target="rfc3550"></xref> and shown in Figure 1. This payload format uses the fields of the header in a manner consistent with that specification.

</t>

<artwork><![CDATA[

0 1 2 3

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

100

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

101

|V=2|P|X| CC |M| PT | sequence number |

102

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

103

| timestamp |

104

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

105

| synchronization source (SSRC) identifier |

106

+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+

107

| contributing source (CSRC) identifiers |

108

| ... |

109

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

110

]]></artwork>

111

</figure>

112

113

<t>

114

The RTP header begins with an octet of fields (V, P, X, and CC) to support specialized RTP uses (see <xref target="rfc3550"></xref> and <xref target="rfc3551"></xref> for details). For Theora RTP, the following values are used.

115

</t>

116

117

<t>

118

Version (V): 2 bits</t><t>

119

This field identifies the version of RTP. The version used by this specification is two (2).

120

</t>

121

122

<t>

123

Padding (P): 1 bit</t><t>

124

Padding MAY be used with this payload format according to section 5.1 of <xref target="rfc3550"></xref>.

125

</t>

126

127

<t>

128

Extension (X): 1 bit</t><t>

129

The Extension bit is used in accordance with <xref target="rfc3550"></xref>.

130

</t>

131

132

<t>

133

CSRC count (CC): 4 bits</t><t>

134

The CSRC count is used in accordance with <xref target="rfc3550"></xref>.

135

</t>

136

137

<t>

138

Marker (M): 1 bit</t><t>

139

The Marker bit is used in accordance with <xref target="rfc3550"></xref>.

140

</t>

141

142

<t>

143

Payload Type (PT): 7 bits</t><t>

144

An RTP profile for a class of applications is expected to assign a payload type for this format, or a dynamically allocated payload type SHOULD be chosen which designates the payload as Theora.

145

</t>

146

147

<t>

148

Sequence number: 16 bits</t><t>

149

The sequence number increments by one for each RTP data packet sent, and may be used by the receiver to detect packet loss and to restore packet sequence. This field is detailed further in <xref target="rfc3550"></xref>.

150

</t>

151

152

<t>

153

Timestamp: 32 bits</t><t>

154

A timestamp representing the presentation time of the first sample of the first Theora packet in the RTP packet. The clock frequency MUST be set to 90kHz.

155

</t>

156

157

<t>

158

SSRC/CSRC identifiers: </t><t>

159

These two fields, 32 bits each with one SSRC field and a maximum of 16 CSRC fields, are as defined in <xref target="rfc3550"></xref>.

160

</t>

161

162

</section>

163

164

165

166

<t>

167

The 4 octets following the RTP Header section represent the Payload Header. This header is split into a number of bitfields detailing the format of the following Payload Datagrams.

168

</t>

169

170

171

<artwork><![CDATA[

172

0 1 2 3

173

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

174

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

175

| Configuration Ident | F |TDT|# pkts.|

176

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

177

178

+-+-+-+-+-+-+-+-+

179

]]></artwork>

180

</figure>

181

182

<t>

183

Configuration Ident: 24 bits</t>

184

<t>

185

This 24 bit field is used to associate the Theora data to a decoding Packed Configuration.

186

</t>

187

188

<t>

189

Fragment type (F): 2 bit</t>

190

<t>

191

This field is set according to the following list

192

</t>

193

194

195

<t> 0 = Not Fragmented</t>

196

<t> 1 = Start Fragment</t>

197

<t> 2 = Continuation Fragment</t>

198

<t> 3 = End Fragment</t>

199

</list>

200

201

<t>This field must be zero if the number of packets field is non-zero.</t>

202

203

<t>

204

Theora Data Type (TDT): 2 bits</t>

205

<t>

206

This field sets the packet payload type for the Theora data. There are currently three Theora payload types currently used and one reserved for future use.

207

</t>

208

209

210

211

<t> 0 = Raw Theora payload</t>

212

<t> 1 = Theora Packed Configuration payload</t>

213

<t> 2 = Legacy Theora Comment payload</t>

214

<t> 3 = Reserved</t>

215

</list>

216

217

<t> The packets with a TDT of value 3 MUST be ignored </t>

218

219

<t>

220

The last 4 bits represent the number of complete packets in this payload. This provides a maximum number of 15 Theora packets in the payload. If the packet contains fragmented data the number of packets MUST be set to 0.

221

</t>

222

223

</section>

224

225

226

227

<t>

228

Each Theora payload section starts with a two octets length header that is used to represent the size of the following data payload, followed by the raw Theora packet data.

229

</t>

230

231

232

<artwork><![CDATA[

233

0 1 2 3

234

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

235

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

236

| Payload Length | Theora Data ..

237

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

238

]]></artwork>

239

</figure>

240

241

<t>

242

The Theora codec uses relatively unstructured raw packets containing binary integer fields of arbitrary width that often do not fall on an octet boundary. When a Theora encoder produces packets, unused space in the last byte of a packet is always zeroed during the encoding process. Thus, should this unused space be read, it will return binary zeros.

243

</t>

244

245

<t>

246

For payloads which consist of multiple Theora packets the payload data consists of the payload length field followed by the first Theora packet's data, then the payload length followed by the second Theora packet, and so on for each of the Theora packets in the payload.

247

</t>

248

249

</section>

250

251

252

253

<t>

254

Here is an example RTP packet containing two Theora packets.

255

</t>

256

<t>

257

RTP Packet Header:

258

</t>

259

260

261

<artwork><![CDATA[

262

0 1 2 3

263

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

264

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

265

| 2 |0|0| 0 |0| PT | sequence number |

266

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

267

| timestamp |

268

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

269

| synchronisation source (SSRC) identifier |

270

+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+

271

| contributing source (CSRC) identifiers |

272

| ... |

273

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

274

]]></artwork>

275

</figure>

276

277

278

<t>

279

Payload Data:

280

</t>

281

282

283

<artwork><![CDATA[

284

0 1 2 3

285

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

286

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

287

| Configuration Ident | 0 | 0 | 2 pks |

288

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

289

| Payload Length | ..

290

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

291

.. Theora data ..

292

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

293

.. data | Payload Length |

294

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

295

.. Theora data |

296

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

297

]]></artwork>

298

</figure>

299

300

<t>

301

The payload portion of the packet begins with the 24 bit Configuration ident field followed by 8 bits describing the payload. The Fragment type field is set to 0, indicating that this packet contains whole Theora frame data. The Data type field is set to 0 (theora raw data). The number of whole Theora data packets is set to 2.

302

</t>

303

304

<t>

305

Each of the payload blocks starts with the two octets length field followed

306

by the variable length Theora packet data.

307

</t>

308

309

</section>

310

</section>

311

312

313

314

<t>

315

To decode a Theora stream three configuration header packets are needed. The first (Identification Header) indicates frame dimensions, quality, blocks used and Theora encoder version. The second (Comment Header) contains stream metadata and the third (Setup Header) contains details of the dequantization and Huffman tables.

316

</t>

317

318

<t>

319

Since this information must be transmitted reliably, and as the RTP stream may change certain configuration data mid-session, there are different methods for delivering this configuration data to a client, both in-band and out-of-band, which are detailed below. SDP delivery is used to set up an initial state for the client application. The changes may be due to different dequantization and Huffman tables as well as different bitrates of the stream.

320

</t>

321

322

<t>

323

The delivery vectors in use are specified by an SDP attribute that indicates the method and the optional URI where the Theora <xref target="Packed Configuration">Packed Configuration</xref> Packets could be fetched. Different delivery methods MAY be advertised for the same session. The in-band codebook delivery SHOULD be considered as baseline, out-of-band delivery methods that don't use RTP will not be described in this document. For non chained streams, the RECOMMENDED Configuration delivery method is inline the <xref target="Packed Configuration">Packed Configuration</xref> in the SDP as explained in the <xref target="Mapping MIME Parameters into SDP"> IANA considerations</xref>

324

</t>

325

326

<t>

327

The 24 bit Ident field is used to map which Configuration will be used to decode a packet. When the Ident field changes, it indicates that a change in the stream has taken place. The client application MUST have in advance the correct configuration and if the client detects a change in the Ident value and does not have this information it MUST NOT decode the raw data associated until it has fetched the correct Configuration.

328

</t>

329

330

331

332

333

<t>

334

The <xref target="Packed Configuration">Packed Configuration</xref> Payload is sent in-band with the packet type bits set to match the payload type. Clients MUST be capable of dealing with periodic re-transmission of the configuration headers.

335

</t>

336

337

338

339

<t>

340

A Theora Packed Configuration is identified by a payload type field of 1. Of the three headers, defined in the <xref target="theora-spec-ref">Theora I specification</xref>, the identification and the setup will be packed together, while the comment header will be completely suppressed. It is up to the client to provide a minimal size comment header to the decoder if required by the implementation.

341

</t>

342

343

344

<artwork><![CDATA[

345

0 1 2 3

346

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

347

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

348

|V=2|P|X| CC |M| PT | xxxx |

349

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

350

| xxxxx |

351

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

352

| synchronization source (SSRC) identifier |

353

+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+

354

| contributing source (CSRC) identifiers |

355

| ... |

356

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

357

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

358

| Configuration Ident | 0 | 1 | 1|

359

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

360

| length | Identification ..

361

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

362

.. Identification ..

363

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

364

.. Identification ..

365

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

366

.. Identification |

367

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

368

.. | Setup ..

369

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

370

.. Setup ..

371

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

372

.. Setup |

373

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

374

]]></artwork>

375

</figure>

376

<t>The Ident field is set with the value that will be used by the Raw Payload Packets to address this Configuration. The Fragment type is set to 0 since the packet bears full Packed configuration, the number of packet is set to 1. In practice, Packed Headers usually need to be fragmented to fit the path MTU.

377

</t>

378

379

</section>

380

381

</section>

382

383

384

385

<t>

386

This section, as stated above, does not cover all the possible out-of-band delivery methods since they rely on different protocols and are linked to specific applications. The following packet definition SHOULD be used in out-of-band delivery and MUST be used when Configuration is inlined in the SDP.

387

</t>

388

389

390

391

<t>

392

As mentioned above, the recommended delivery vector for Theora configuration data is via a retrieval method that can be performed using a reliable transport protocol. As the RTP headers are not required for this method of delivery the structure of the configuration data is slightly different. The packed header starts with a 32 bit count field which details the number of packed headers that are contained in the bundle. Next is the Packed header payload for each setup id.

393

</t>

394

395

396

<artwork><![CDATA[

397

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

398

| Number of packed headers |

399

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

400

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

401

| Packed header |

402

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

403

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

404

| Packed header |

405

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

406

]]></artwork>

407

</figure>

408

409

<t>

410

Since the Configuration Ident and the Identification Header are fixed length there is only a 16bit Length tag to define the length of the packed headers.

411

</t>

412

413

414

<artwork><![CDATA[

415

0 1 2 3

416

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

417

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

418

| Configuration Ident | ..

419

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

420

.. Length | Identification Header ..

421

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

422

.. Identification Header |

423

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

424

| Setup Header ..

425

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

426

.. Setup Header |

427

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

428

]]></artwork></figure>

429

430

<t>The key difference from the in-band format is that there is no need for the payload header octet.

431

</t>

432

433

434

435

<t>

436

The following IANA considerations MUST only be applied to the packed headers.

437

</t>

438

439

440

441

442

<t hangText="MIME media type name:"> video </t>

443

444

445

446

<t hangText="MIME subtype:"> theora-config </t>

447

448

449

450

451

452

None

453

</t>

454

455

456

457

458

459

None

460

</t>

461

462

463

464

465

466

This media type contains binary data.

467

</t>

468

469

470

471

472

473

See Section 6 of RFC XXXX.

474

</t>

475

476

477

478

479

480

None

481

</t>

482

483

484

485

486

487

RFC XXXX [RFC Editor: please replace by the RFC number of this memo,

488

when published]

489

</t>

490

491

492

493

494

495

Theora encoded video, configuration data.

496

</t>

497

498

499

500

501

502

None

503

</t>

504

505

506

507

508

509

Luca Barbato: <lu_zero@gentoo.org>

510

511

IETF Audio/Video Transport Working Group

512

</t>

513

514

515

516

517

COMMON

518

</t>

519

520

521

522

523

524

This media type does not depend on the transport.

525

</t>

526

527

528

529

530

531

Luca Barbato</t>

532

533

534

535

536

537

IETF AVT Working Group</t>

538

</list>

539

540

</section>

541

</section>

542

543

</section>

544

545

546

547

<t>

548

Unlike the loss of raw Theora payload data, the loss of a configuration header can lead to a situation where it will not be possible to successfully decode the stream.

549

</t>

550

551

<t>

552

A loss of a Configuration Packet causes the stream decoder to halt and SHOULD be reported to the client as well as a loss report sent via RTCP.

553

</t>

554

555

</section>

556

557

</section>

558

559

560

561

<t>

562

When the payload type is set to 2, the packet contains comment metadata such as artist name, track title and so on. These metadata messages are not intended to be fully descriptive but to offer basic title information. Clients MAY choose to completely ignore them. The details on the comments format can be found in the <xref target="theora-spec-ref">Theora documentation</xref>.

563

</t>

564

565

566

<artwork><![CDATA[

567

0 1 2 3

568

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

569

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

570

|V=2|P|X| CC |M| PT | xxxx |

571

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

572

| xxxxx |

573

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

574

| synchronization source (SSRC) identifier |

575

+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+

576

| contributing source (CSRC) identifiers |

577

| ... |

578

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

579

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

580

| Configuration Ident | 0 | 2 | 1|

581

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

582

| length | Comment ..

583

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

584

.. Comment ..

585

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

586

.. Comment |

587

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

588

]]></artwork>

589

</figure>

590

591

<t>The 2 byte length field is necessary since this Theora packet could be fragmented.</t>

592

593

</section>

594

595

596

597

<t>

598

Each RTP packet contains either one complete Theora packet, one Theora packet fragment, or an integer number of complete Theora packets (up to a maximum of 15 packets, since the number of packets is defined by a 4 bit value).

599

</t>

600

601

<t>

602

Any Theora data packet that is less than path MTU SHOULD be bundled in the RTP packet with as many Theora packets as will fit, up to a maximum of 15. Path MTU is detailed in <xref target="rfc1063"></xref> and <xref target="rfc1981"></xref>.

603

</t>

604

605

<t>

606

A fragmented packet has a zero in the last four bits of the payload header. The RTP packet containing the first fragment will set the Fragment type to 1. Each RTP packet after the first will set the Fragment type to 2 in the payload header. The RTP packet containing the last fragment of the Theora packet will have the Fragment type set to 3. If the fragmented Theora packet spans only two RTP packets, the first will set the Fragment type field to 1 and the second will set it to 2. To maintain the correct sequence for fragmented packet reception the timestamp field of fragmented packets MUST be the same as the first packet sent, with the sequence number incremented as normal for the subsequent RTP packets.</t>

607

608

609

610

<t>

611

Here is an example fragmented Theora packet split over three RTP packets. Each packet contains the standard RTP headers as well as the 4 octets Theora headers.

612

</t>

613

614

615

<artwork><![CDATA[

616

Packet 1:

617

618

0 1 2 3

619

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

620

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

621

|V=2|P|X| CC |M| PT | 1000 |

622

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

623

| xxxxx |

624

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

625

| synchronization source (SSRC) identifier |

626

+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+

627

| contributing source (CSRC) identifiers |

628

| ... |

629

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

630

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

631

| Configuration Ident | 1 | 0 | 0|

632

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

633

| Payload Length | Theora data ..

634

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

635

.. Theora data ..

636

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

637

]]></artwork>

638

</figure>

639

640

<t>

641

In this packet the initial sequence number is 1000 and the timestamp is xxxxx. The Fragment type field is set to one, indicating it is the start packet of a serie of fragments. The number of packets field is set to 0, and as the payload is raw Theora data the Theora payload type field is set to 0.

642

</t>

643

644

645

<artwork><![CDATA[

646

Packet 2:

647

648

0 1 2 3

649

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

650

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

651

|V=2|P|X| CC |M| PT | 1001 |

652

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

653

| xxxxx |

654

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

655

| synchronization source (SSRC) identifier |

656

+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+

657

| contributing source (CSRC) identifiers |

658

| ... |

659

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

660

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

661

| Configuration Ident | 2 | 0 | 0|

662

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

663

| Payload Length | ..

664

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

665

.. Theora data ..

666

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

667

]]></artwork>

668

</figure>

669

670

<t>

671

The Fragment type field is set to 2 and the number of packets field is set to 0. For large Theora fragments there can be several of these type of payload packets. The maximum RTP packet size SHOULD be no greater than the path MTU, including all RTP and payload headers. The sequence number has been incremented by one but the timestamp field remains the same as the initial packet.

672

</t>

673

674

675

<artwork><![CDATA[

676

Packet 3:

677

678

0 1 2 3

679

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

680

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

681

|V=2|P|X| CC |M| PT | 1002 |

682

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

683

| xxxxx |

684

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

685

| synchronization source (SSRC) identifier |

686

+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+

687

| contributing source (CSRC) identifiers |

688

| ... |

689

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

690

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

691

| Configuration Ident | 3 | 0 | 0|

692

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

693

| Payload Length | ..

694

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

695

.. Theora data ..

696

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

697

]]></artwork>

698

</figure>

699

700

<t>

701

This is the last Theora fragment packet. The Fragment type filed is set to 3 and the packet count remains set to 0. As in the previous packets the timestamp remains set to the first packet in the sequence and the sequence number has been incremented.

702

</t>

703

704

</section>

705

706

707

708

<t>

709

As there is no error correction within the Theora stream, packet loss will result in a loss of signal. Packet loss is more of an issue for fragmented Theora packets as the client will have to cope with the handling of the Fragment type field. If we use the fragmented Theora packet example above and the first packet is lost the client MUST detect that the next packet has the packet count field set to 0 and the Fragment type is set to 2 and MUST drop it. The next packet, which is the final fragmented packet, MUST be dropped in the same manner. Feedback reports on lost and dropped packets MUST be sent back via RTCP.[note: reordering]

710

</t>

711

712

<t>

713

If a particular multicast session has a large number of participants care must be taken to prevent an RTCP feedback implosion, <xref target="rtcp-feedback"></xref>, in the event of packet loss from a large number of participants.

714

</t>

715

716

<t>

717

Loss of any of the Configuration fragment will result in the loss of the full Configuration packet as detailed in the <xref target="Loss of Configuration Headers">Loss of Configuration Headers</xref> section.

718

</t>

719

720

</section>

721

</section>

722

723

724

725

726

727

728

<t hangText="MIME media type name:"> video </t>

729

730

731

732

<t hangText="MIME subtype:"> theora </t>

733

734

735

736

737

738

739

740

741

742

<t hangText="sampling:"> Determines the chroma subsampling format.

743

</t>

744

745

746

747

<t hangText="width:"> Determines the number of pixels per line. This is an integer between 1 and 1048561 and MUST be in multiples of 16.

748

</t>

749

750

751

752

<t hangText="height:">Determines the number of lines per frame encoded. This is an integer between 1 and 1048561 and MUST be in multiples of 16.

753

</t>

754

755

756

757

<t hangText="delivery-method:"> indicates the delivery methods in use, the possible values are: inline, in_band, out_band/specific_name<vspace blankLines="0" />

758

Where "specific_name" is the name of the out of band delivery method.

759

</t>

760

761

762

763

<t hangText="configuration:"> the <xref target="rfc3548">base16</xref> (hexadecimal) representation of the <xref target="Packed Headers">Packed Headers</xref>.

764

</t>

765

</list>

766

</t>

767

768

769

770

771

772

773

774

775

<t hangText="configuration-uri:"> the URI of the configuration headers in case of out of band transmission. In the form of "protocol://path/to/resource/". Depending on the specific method the single ident packets could be retrived by their number or aggregated in a single stream, aggregates MAY be compressed using <xref target="rfc1952">gzip</xref> or <xref target="BZ2">bzip2</xref> and an <xref target="FIPS180">sha1</xref> checksum MAY be provided in the form of "protocol://path/to/resource/aggregated.bz2!sha1hash"</t>

776

</list>

777

</t>

778

779

780

781

782

783

This media type is framed and contains binary data.

784

</t>

785

786

787

788

789

790

See Section 6 of RFC XXXX.</t>

791

792

793

794

795

796

None</t>

797

798

799

800

801

802

803

804

<t> RFC XXXX [RFC Editor: please replace by the RFC number of this memo, when published]</t>

805

806

<t>Ogg Theora I specification: Codec setup and packet decode. Available from the Xiph website, http://www.xiph.org</t>

807

808

</t>

809

810

811

812

813

814

Video streaming and conferencing tools </t>

815

816

817

818

819

820

None </t>

821

822

823

824

825

826

827

828

<t>Luca Barbato: <lu_zero@gentoo.org></t>

829

<t>IETF Audio/Video Transport Working Group</t>

830

831

</t>

832

833

834

835

836

837

COMMON</t>

838

839

840

841

842

843

This media type depends on RTP framing, and hence is only defined for transfer via <xref target="rfc3550">RTP</xref></t>

844

845

846

847

848

<vspace blankLines="1"/>Luca Barbato</t>

849

850

851

852

<t hangText="Change controller:"><vspace blankLines="1"/> IETF AVT Working Group</t>

853

854

855

856

</list>

857

858

859

860

<t>

861

The information carried in the MIME media type specification has a specific mapping to fields in the Session Description Protocol (SDP) <xref target="rfc2327"></xref>, which is commonly used to describe RTP sessions. When SDP is used to specify sessions the mapping are as follows:

862

</t>

863

864

865

866

867

<t>The MIME type ("video") goes in SDP "m=" as the media name.</t>

868

869

870

<t>The MIME subtype ("theora") goes in SDP "a=rtpmap" as the encoding name.</t>

871

872

873

<t>The clock rate in the "a=rtpmap" line MUST be 90000</t>

874

875

876

<t>The mandated parameters "delivery-method" and "configuration" MUST be included in the SDP "a=fmpt" attribute.</t>

877

878

879

<t>The optional parameter "configuration-uri", when present, MUST be included in the SDP "a=fmpt" attribute and MUST follow the delivery-method that applies.</t>

880

</list>

881

882

883

<t>

884

If the stream uses multiple decoder setup configurations and all of them are known in advance, the Configuration Packet for each file SHOULD be packaged together and passed to the client using the configuration attribute.

885

</t>

886

887

<t>

888

The URI specified in the configuration-uri attribute MUST point to a location where all of the Configuration Packets needed for the life of the session reside.

889

</t>

890

891

892

<t>The following example shows a basic SDP for a single stream. The first configuration packet is inlined in the sdp, other configurations could be fetched at any time from the first provided uri using or all the known configuration could be downloaded using the second uri. The inline <xref target="rfc3548">base16</xref> configuration string is omitted because of the lenght.</t>

893

894

895

896

<t>m=video RTP/AVP 98</t>

897

<t>a=rtpmap:98 theora/90000</t>

898

<t>a=fmtp:98 sampling=YCbCr-4:2:2; width=1280; height=720; delivery-method=inline; configuration=base16string1; delivery-method=out_band/rtsp; delivery-method=out_band/rtsp; configuration-uri=rtsp://path/to/resource/; delivery-method=out_band/http; configuration-uri=http://another/path/to/resource/aggregate.bz2!sha1hash;</t>

899

</list>

900

</section>

901

902

</section>

903

904

905

906

<t>

907

The offer, as described in <xref target="rfc3264">An Offer/Answer Model Session Description Protocol</xref>, may contain a large number of delivery methods per single fmtp attribute, the answerer MUST remove every delivery-method and configuration-uri not supported. All the parameters MUST not be altered on answer otherwise.

908

</t>

909

910

</section>

911

912

</section>

913

914

915

916

<t>

917

The following examples are common usage patterns that MAY be applied in such situations, the main scope of this section is to explain better usage of the transmission vectors.

918

</t>

919

<!--

920

921

922

923

<t>This scenario implies two peers linked through a best effort network, the bandwidth isn't guaranteed and may have large variance, in order to keep the latency low enough dynamic adaptation tecniques [missing reference] are required.</t>

924

925

<t>Each peer will receive 2 streams (voice and video) from the other. To determine the quality of the stream and ensure the latency is bearable [put maximum latency here] a form of handshake is required. SIP or Jingle or TINS could be used in this phase.</t>

926

927

<t>Since changes in the bitrates will reflect on the setup header, the simplest way to get dynamic adaptation is to consider each stream as a completely different coded, have a payload number for each of them and use the dynamic coding change tecniques.</t>

928

929

<t>Due the latency requirement even if sending the Configuration in-band MAY be possible, usually it SHOULD be avoided. Other out of band methods that send Configuration on demand, since they would affect latency as the in-band method, SHOULD be avoided as well. Agree on a set of Configurations related to different bitrates during the session initiation is the best method.</t>

930

931

</section>

932

-->

933

934

935

<t>This is one of the most common situation: one single server streaming content in multicast, the clients may start a session at random time. The content itself could be a mix of live stream, as the wj's voice or studio scenes, and stored streams, as the music she plays.</t>

936

937

<t>In this situation we don't know in advance how many codebooks we will use. The clients can join anytime and users expect to start the fruition of the content in a short time.</t>

938

939

<t>On join the client will receive the current Configuration necessary to decode the current streams inlined in the SDP so that the decoding will start immediately after.</t>

940

941

<t>When the streamed content changes the new Configuration is sent in-band before the actual stream, and the Configuration that has to be sent inline in the SDP updated. Since the inline method is unreliable, an out of band fallback is provided.</t>

942

943

<t>The client could choose to fetch the Configuration from the alternate source as soon it discovers a Configuration packet got lost inline or use <xref target="rfc3611">selective retransmission</xref>, if the server supports the feature.</t>

944

945

<t>A serverside optimization would be to keep an hash list of the Configurations per session to avoid packing all of them and send the same Configuration with different Ident tags</t>

946

947

<t>A clientside optimization would be to keep a tag list of the Configurations per session and don't process configuration packets already known.</t>

948

949

</section>

950

951

</section>

952

953

954

<t>

955

RTP packets using this payload format are subject to the security considerations discussed in the RTP specification <xref target="rfc3550"></xref>. This implies that the confidentiality of the media stream is achieved by using encryption. Because the data compression used with this payload format is applied end-to-end, encryption may be performed on the compressed data. Where the size of a data block is set care MUST be taken to prevent buffer overflows in the client applications.

956

</t>

957

958

</section>

959

960

961

962

<t>This document is a continuation of draft-kerr-avt-theora-rtp-00.txt</t>

963

964

<t>

965

Thanks to the AVT, Ogg Theora Communities / Xiph.org, Fluendo, Ralph Giles, Mike Smith, Phil Kerr, Timothy Terriberry, Stefan Ehmann, Alessandro Salvatori, Politecnico di Torino (LS)³/IMG Group in particular Federico Ridolfo, Francesco Varano, Giampaolo Mancini, Juan Carlos De Martin.

966

</t>

967

968

</section>

969

970

</middle>

971

972

<back>

973

974

975

976

977

<front>

978

<title>The Ogg Encapsulation Format Version 0</title>

979

980

</front>

981

982

</reference>

983

984

985

<front>

986

<title>Key words for use in RFCs to Indicate Requirement Levels </title>

987

988

</front>

989

990

</reference>

991

992

993

<front>

994

<title>RTP: A Transport Protocol for real-time applications</title>

995

996

997

998

999

</front>

1000

1001

</reference>

1002

1003

1004

<front>

1005

<title>RTP Profile for video and Video Conferences with Minimal Control.</title>

1006

1007

1008

</front>

1009

1010

1011

</reference>

1012

1013

1014

<front>

1015

<title>An Offer/Answer Model with Session Description Protocol (SDP)</title>

1016

1017

1018

</front>

1019

1020

</reference>

1021

1022

1023

<front>

1024

<title>SDP: Session Description Protocol</title>

1025

1026

1027

</front>

1028

1029

</reference>

1030

1031

1032

<front>

1033

<title>Path MTU Discovery</title>

1034

1035

</front>

1036

1037

</reference>

1038

1039

1040

<front>

1041

<title>Path MTU Discovery for IP version 6</title>

1042

1043

</front>

1044

1045

</reference>

1046

1047

1048

<front>

1049

<title>Extended RTP Profile for RTCP-based Feedback (RTP/AVPF)</title>

1050

1051

1052

1053

1054

1055

</front>

1056

1057

</reference>

1058

1059

1060

<front>

1061

<title>RTP Payload Format for Vorbis Encoded Audio - draft-ietf-avt-vorbis-rtp-00</title>

1062

1063

</front>

1064

1065

</reference>

1066

1067

1068

<front>

1069

<title>The Base16, Base32, and Base64 Data Encodings</title>

1070

1071

</front>

1072

1073

</reference>

1074

1075

1076

<front>

1077

<title>GZIP file format specification version 4.3</title>

1078

1079

</front>

1080

1081

</reference>

1082

1083

1084

<front>

1085

<title>Secure Hash Standard</title>

1086

1087

<organization>National Institute of Standards and Technology</organization>

1088

</author>

1089

1090

</front>

1091

</reference>

1092

1093

1094

<front>

1095

<title>libbz2 and bzip2</title>

1096

1097

</front>

1098

</reference>

1099

1100

</references>

1101

1102

1103

1104

<front>

1105

<title>libTheora: Available from the Xiph website, http://www.xiph.org</title>

1106

</front>

1107

</reference>

1108

1109

1110

<front>

1111

<title>Theora I specification: Codec setup and packet decode. http://www.xiph.org/theora/doc/Theora_I_spec.pdf</title>

1112

</front>

1113

</reference>

1114

1115

1116

1117

<front>

1118

<title>RTP Control Protocol Extended Reports (RTCP XR)</title>

1119

1120

1121

1122

1123

1124

1125

1126

1127

</reference>

1128

1129

1130

1131

<front>

1132

<title>

1133

ITU-T Recommendation V.42, 1994, Rev. 1. Error-correcting Procedures for DCEs Using Asynchronous-to-Synchronous Conversion. International Telecommunications Union. Available from the ITU website, http://www.itu.int

1134

</title>

1135

</front>

1136

</reference>

1137

1138

1139

<front>

1140

<title>ISO 3309, October 1984, 3rd Edition. Information Processing Systems--Data Communication High-Level Data Link Control Procedure--Frame Structure. International Organization for Standardization.

1141

</title>

1142

</front>

1143

</reference>

1144

</references>

1145

</back>

1146

</rfc>

Older »