~dkuhlman/python-training-materials/Materials : revision 45

1

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"

2

"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

3

4

5

6

<head>

7

8

9

<title>15. Floating Point Arithmetic: Issues and Limitations — Python 3.5.1 documentation</title>

10

11

12

13

14

15

var DOCUMENTATION_OPTIONS = {

16

URL_ROOT: '../',

17

VERSION: '3.5.1',

18

COLLAPSE_INDEX: false,

19

FILE_SUFFIX: '.html',

20

HAS_SOURCE: true

21

};

22

</script>

23

24

25

26

27

28

title="Search within Python 3.5.1 documentation"

29

href="../_static/opensearch.xml"/>

30

31

32

33

34

35

36

37

38

39

40

41

42

</head>

43

44

45

<h3>Navigation</h3>

46

<ul>

47

48

<a href="../genindex.html" title="General Index"

49

accesskey="I">index</a></li>

50

51

<a href="../py-modindex.html" title="Python Module Index"

52

>modules</a> |</li>

53

54

<a href="appendix.html" title="16. Appendix"

55

accesskey="N">next</a> |</li>

56

57

<a href="interactive.html" title="14. Interactive Input Editing and History Substitution"

58

accesskey="P">previous</a> |</li>

59

<li><img src="../_static/py.png" alt=""

60

style="vertical-align: middle; margin-top: -1px"/></li>

61

<li><a href="https://www.python.org/">Python</a> »</li>

62

<li>

63

64

<a href="../index.html">Documentation </a> »

65

</li>

66

67

<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">The Python Tutorial</a> »</li>

68

</ul>

69

</div>

70

71

72

73

74

75

76

77

<span id="tut-fp-issues"></span><h1>15. Floating Point Arithmetic: Issues and Limitations<a class="headerlink" href="#floating-point-arithmetic-issues-and-limitations" title="Permalink to this headline">¶</a></h1>

78

<p>Floating-point numbers are represented in computer hardware as base 2 (binary)

79

fractions. For example, the decimal fraction</p>

80

81

</pre></div>

82

</div>

83

<p>has value 1/10 + 2/100 + 5/1000, and in the same way the binary fraction</p>

84

85

</pre></div>

86

</div>

87

<p>has value 0/2 + 0/4 + 1/8. These two fractions have identical values, the only

88

real difference being that the first is written in base 10 fractional notation,

89

and the second in base 2.</p>

90

<p>Unfortunately, most decimal fractions cannot be represented exactly as binary

91

fractions. A consequence is that, in general, the decimal floating-point

92

numbers you enter are only approximated by the binary floating-point numbers

93

actually stored in the machine.</p>

94

<p>The problem is easier to understand at first in base 10. Consider the fraction

95

1/3. You can approximate that as a base 10 fraction:</p>

96

97

</pre></div>

98

</div>

99

<p>or, better,</p>

100

101

</pre></div>

102

</div>

103

<p>or, better,</p>

104

105

</pre></div>

106

</div>

107

<p>and so on. No matter how many digits you’re willing to write down, the result

108

will never be exactly 1/3, but will be an increasingly better approximation of

109

1/3.</p>

110

<p>In the same way, no matter how many base 2 digits you’re willing to use, the

111

decimal value 0.1 cannot be represented exactly as a base 2 fraction. In base

112

2, 1/10 is the infinitely repeating fraction</p>

113

114

</pre></div>

115

</div>

116

<p>Stop at any finite number of bits, and you get an approximation. On most

117

machines today, floats are approximated using a binary fraction with

118

the numerator using the first 53 bits starting with the most significant bit and

119

with the denominator as a power of two. In the case of 1/10, the binary fraction

120

is <code class="docutils literal"><span class="pre">3602879701896397</span> <span class="pre">/</span> <span class="pre">2</span> <span class="pre">**</span> <span class="pre">55</span></code> which is close to but not exactly

121

equal to the true value of 1/10.</p>

122

<p>Many users are not aware of the approximation because of the way values are

123

displayed. Python only prints a decimal approximation to the true decimal

124

value of the binary approximation stored by the machine. On most machines, if

125

Python were to print the true decimal value of the binary approximation stored

126

for 0.1, it would have to display</p>

127

128

129

</pre></div>

130

</div>

131

<p>That is more digits than most people find useful, so Python keeps the number

132

of digits manageable by displaying a rounded value instead</p>

133

134

135

</pre></div>

136

</div>

137

<p>Just remember, even though the printed result looks like the exact value

138

of 1/10, the actual stored value is the nearest representable binary fraction.</p>

139

<p>Interestingly, there are many different decimal numbers that share the same

140

nearest approximate binary fraction. For example, the numbers <code class="docutils literal"><span class="pre">0.1</span></code> and

141

<code class="docutils literal"><span class="pre">0.10000000000000001</span></code> and

142

<code class="docutils literal"><span class="pre">0.1000000000000000055511151231257827021181583404541015625</span></code> are all

143

approximated by <code class="docutils literal"><span class="pre">3602879701896397</span> <span class="pre">/</span> <span class="pre">2</span> <span class="pre">**</span> <span class="pre">55</span></code>. Since all of these decimal

144

values share the same approximation, any one of them could be displayed

145

while still preserving the invariant <code class="docutils literal"><span class="pre">eval(repr(x))</span> <span class="pre">==</span> <span class="pre">x</span></code>.</p>

146

<p>Historically, the Python prompt and built-in <a class="reference internal" href="../library/functions.html#repr" title="repr"><code class="xref py py-func docutils literal"><span class="pre">repr()</span></code></a> function would choose

147

the one with 17 significant digits, <code class="docutils literal"><span class="pre">0.10000000000000001</span></code>. Starting with

148

Python 3.1, Python (on most systems) is now able to choose the shortest of

149

these and simply display <code class="docutils literal"><span class="pre">0.1</span></code>.</p>

150

<p>Note that this is in the very nature of binary floating-point: this is not a bug

151

in Python, and it is not a bug in your code either. You’ll see the same kind of

152

thing in all languages that support your hardware’s floating-point arithmetic

153

(although some languages may not <em>display</em> the difference by default, or in all

154

output modes).</p>

155

<p>For more pleasant output, you may wish to use string formatting to produce a limited number of significant digits:</p>

156

<div class="highlight-python3"><div class="highlight"><pre><span class="gp">>>> </span><span class="nb">format</span><span class="p">(</span><span class="n">math</span><span class="o">.</span><span class="n">pi</span><span class="p">,</span> <span class="s1">'.12g'</span><span class="p">)</span> <span class="c1"># give 12 significant digits</span>

157

158

159

<span class="gp">>>> </span><span class="nb">format</span><span class="p">(</span><span class="n">math</span><span class="o">.</span><span class="n">pi</span><span class="p">,</span> <span class="s1">'.2f'</span><span class="p">)</span> <span class="c1"># give 2 digits after the point</span>

160

161

162

163

164

</pre></div>

165

</div>

166

<p>It’s important to realize that this is, in a real sense, an illusion: you’re

167

simply rounding the <em>display</em> of the true machine value.</p>

168

<p>One illusion may beget another. For example, since 0.1 is not exactly 1/10,

169

summing three values of 0.1 may not yield exactly 0.3, either:</p>

170

171

<span class="go">False</span>

172

</pre></div>

173

</div>

174

<p>Also, since the 0.1 cannot get any closer to the exact value of 1/10 and

175

0.3 cannot get any closer to the exact value of 3/10, then pre-rounding with

176

<a class="reference internal" href="../library/functions.html#round" title="round"><code class="xref py py-func docutils literal"><span class="pre">round()</span></code></a> function cannot help:</p>

177

<div class="highlight-python3"><div class="highlight"><pre><span class="gp">>>> </span><span class="nb">round</span><span class="p">(</span><span class="o">.</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span> <span class="o">+</span> <span class="nb">round</span><span class="p">(</span><span class="o">.</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span> <span class="o">+</span> <span class="nb">round</span><span class="p">(</span><span class="o">.</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span> <span class="o">==</span> <span class="nb">round</span><span class="p">(</span><span class="o">.</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>

178

<span class="go">False</span>

179

</pre></div>

180

</div>

181

<p>Though the numbers cannot be made closer to their intended exact values,

182

the <a class="reference internal" href="../library/functions.html#round" title="round"><code class="xref py py-func docutils literal"><span class="pre">round()</span></code></a> function can be useful for post-rounding so that results

183

with inexact values become comparable to one another:</p>

184

<div class="highlight-python3"><div class="highlight"><pre><span class="gp">>>> </span><span class="nb">round</span><span class="p">(</span><span class="o">.</span><span class="mi">1</span> <span class="o">+</span> <span class="o">.</span><span class="mi">1</span> <span class="o">+</span> <span class="o">.</span><span class="mi">1</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span> <span class="o">==</span> <span class="nb">round</span><span class="p">(</span><span class="o">.</span><span class="mi">3</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span>

185

186

</pre></div>

187

</div>

188

<p>Binary floating-point arithmetic holds many surprises like this. The problem

189

with “0.1” is explained in precise detail below, in the “Representation Error”

190

section. See <a class="reference external" href="http://www.lahey.com/float.htm">The Perils of Floating Point</a>

191

for a more complete account of other common surprises.</p>

192

<p>As that says near the end, “there are no easy answers.” Still, don’t be unduly

193

wary of floating-point! The errors in Python float operations are inherited

194

from the floating-point hardware, and on most machines are on the order of no

195

more than 1 part in 2**53 per operation. That’s more than adequate for most

196

tasks, but you do need to keep in mind that it’s not decimal arithmetic and

197

that every float operation can suffer a new rounding error.</p>

198

<p>While pathological cases do exist, for most casual use of floating-point

199

arithmetic you’ll see the result you expect in the end if you simply round the

200

display of your final results to the number of decimal digits you expect.

201

<a class="reference internal" href="../library/stdtypes.html#str" title="str"><code class="xref py py-func docutils literal"><span class="pre">str()</span></code></a> usually suffices, and for finer control see the <a class="reference internal" href="../library/stdtypes.html#str.format" title="str.format"><code class="xref py py-meth docutils literal"><span class="pre">str.format()</span></code></a>

202

method’s format specifiers in <a class="reference internal" href="../library/string.html#formatstrings"><span>Format String Syntax</span></a>.</p>

203

<p>For use cases which require exact decimal representation, try using the

204

<a class="reference internal" href="../library/decimal.html#module-decimal" title="decimal: Implementation of the General Decimal Arithmetic Specification."><code class="xref py py-mod docutils literal"><span class="pre">decimal</span></code></a> module which implements decimal arithmetic suitable for

205

accounting applications and high-precision applications.</p>

206

<p>Another form of exact arithmetic is supported by the <a class="reference internal" href="../library/fractions.html#module-fractions" title="fractions: Rational numbers."><code class="xref py py-mod docutils literal"><span class="pre">fractions</span></code></a> module

207

which implements arithmetic based on rational numbers (so the numbers like

208

1/3 can be represented exactly).</p>

209

<p>If you are a heavy user of floating point operations you should take a look

210

at the Numerical Python package and many other packages for mathematical and

211

statistical operations supplied by the SciPy project. See <<a class="reference external" href="http://scipy.org">http://scipy.org</a>>.</p>

212

<p>Python provides tools that may help on those rare occasions when you really

213

<em>do</em> want to know the exact value of a float. The

214

<a class="reference internal" href="../library/stdtypes.html#float.as_integer_ratio" title="float.as_integer_ratio"><code class="xref py py-meth docutils literal"><span class="pre">float.as_integer_ratio()</span></code></a> method expresses the value of a float as a

215

fraction:</p>

216

217

<span class="gp">>>> </span><span class="n">x</span><span class="o">.</span><span class="n">as_integer_ratio</span><span class="p">()</span>

218

219

</pre></div>

220

</div>

221

<p>Since the ratio is exact, it can be used to losslessly recreate the

222

original value:</p>

223

224

225

</pre></div>

226

</div>

227

<p>The <a class="reference internal" href="../library/stdtypes.html#float.hex" title="float.hex"><code class="xref py py-meth docutils literal"><span class="pre">float.hex()</span></code></a> method expresses a float in hexadecimal (base

228

16), again giving the exact value stored by your computer:</p>

229

230

231

</pre></div>

232

</div>

233

<p>This precise hexadecimal representation can be used to reconstruct

234

the float value exactly:</p>

235

<div class="highlight-python3"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">x</span> <span class="o">==</span> <span class="nb">float</span><span class="o">.</span><span class="n">fromhex</span><span class="p">(</span><span class="s1">'0x1.921f9f01b866ep+1'</span><span class="p">)</span>

236

237

</pre></div>

238

</div>

239

<p>Since the representation is exact, it is useful for reliably porting values

240

across different versions of Python (platform independence) and exchanging

241

data with other languages that support the same format (such as Java and C99).</p>

242

<p>Another helpful tool is the <a class="reference internal" href="../library/math.html#math.fsum" title="math.fsum"><code class="xref py py-func docutils literal"><span class="pre">math.fsum()</span></code></a> function which helps mitigate

243

loss-of-precision during summation. It tracks “lost digits” as values are

244

added onto a running total. That can make a difference in overall accuracy

245

so that the errors do not accumulate to the point where they affect the

246

final total:</p>

247

248

<span class="go">False</span>

249

250

251

</pre></div>

252

</div>

253

254

<span id="tut-fp-error"></span><h2>15.1. Representation Error<a class="headerlink" href="#representation-error" title="Permalink to this headline">¶</a></h2>

255

<p>This section explains the “0.1” example in detail, and shows how you can perform

256

an exact analysis of cases like this yourself. Basic familiarity with binary

257

floating-point representation is assumed.</p>

258

<p><em class="dfn">Representation error</em> refers to the fact that some (most, actually)

259

decimal fractions cannot be represented exactly as binary (base 2) fractions.

260

This is the chief reason why Python (or Perl, C, C++, Java, Fortran, and many

261

others) often won’t display the exact decimal number you expect.</p>

262

<p>Why is that? 1/10 is not exactly representable as a binary fraction. Almost all

263

machines today (November 2000) use IEEE-754 floating point arithmetic, and

264

almost all platforms map Python floats to IEEE-754 “double precision”. 754

265

doubles contain 53 bits of precision, so on input the computer strives to

266

convert 0.1 to the closest fraction it can of the form <em>J</em>/2**<em>N</em> where <em>J</em> is

267

an integer containing exactly 53 bits. Rewriting</p>

268

269

</pre></div>

270

</div>

271

272

273

</pre></div>

274

</div>

275

<p>and recalling that <em>J</em> has exactly 53 bits (is <code class="docutils literal"><span class="pre">>=</span> <span class="pre">2**52</span></code> but <code class="docutils literal"><span class="pre"><</span> <span class="pre">2**53</span></code>),

276

the best value for <em>N</em> is 56:</p>

277

278

279

</pre></div>

280

</div>

281

<p>That is, 56 is the only value for <em>N</em> that leaves <em>J</em> with exactly 53 bits. The

282

best possible value for <em>J</em> is then that quotient rounded:</p>

283

<div class="highlight-python3"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">q</span><span class="p">,</span> <span class="n">r</span> <span class="o">=</span> <span class="nb">divmod</span><span class="p">(</span><span class="mi">2</span><span class="o">**</span><span class="mi">56</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span>

284

285

286

</pre></div>

287

</div>

288

<p>Since the remainder is more than half of 10, the best approximation is obtained

289

by rounding up:</p>

290

291

292

</pre></div>

293

</div>

294

<p>Therefore the best possible approximation to 1/10 in 754 double precision is:</p>

295

296

</pre></div>

297

</div>

298

<p>Dividing both the numerator and denominator by two reduces the fraction to:</p>

299

300

</pre></div>

301

</div>

302

<p>Note that since we rounded up, this is actually a little bit larger than 1/10;

303

if we had not rounded up, the quotient would have been a little bit smaller than

304

1/10. But in no case can it be <em>exactly</em> 1/10!</p>

305

<p>So the computer never “sees” 1/10: what it sees is the exact fraction given

306

above, the best 754 double approximation it can get:</p>

307

308

309

</pre></div>

310

</div>

311

<p>If we multiply that fraction by 10**55, we can see the value out to

312

55 decimal digits:</p>

313

314

315

</pre></div>

316

</div>

317

<p>meaning that the exact number stored in the computer is equal to

318

the decimal value 0.1000000000000000055511151231257827021181583404541015625.

319

Instead of displaying the full decimal value, many languages (including

320

older versions of Python), round the result to 17 significant digits:</p>

321

<div class="highlight-python3"><div class="highlight"><pre><span class="gp">>>> </span><span class="nb">format</span><span class="p">(</span><span class="mf">0.1</span><span class="p">,</span> <span class="s1">'.17f'</span><span class="p">)</span>

322

323

</pre></div>

324

</div>

325

<p>The <a class="reference internal" href="../library/fractions.html#module-fractions" title="fractions: Rational numbers."><code class="xref py py-mod docutils literal"><span class="pre">fractions</span></code></a> and <a class="reference internal" href="../library/decimal.html#module-decimal" title="decimal: Implementation of the General Decimal Arithmetic Specification."><code class="xref py py-mod docutils literal"><span class="pre">decimal</span></code></a> modules make these calculations

326

easy:</p>

327

<div class="highlight-python3"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">decimal</span> <span class="k">import</span> <span class="n">Decimal</span>

328

<span class="gp">>>> </span><span class="kn">from</span> <span class="nn">fractions</span> <span class="k">import</span> <span class="n">Fraction</span>

329

330

<span class="gp">>>> </span><span class="n">Fraction</span><span class="o">.</span><span class="n">from_float</span><span class="p">(</span><span class="mf">0.1</span><span class="p">)</span>

331

<span class="go">Fraction(3602879701896397, 36028797018963968)</span>

332

333

<span class="gp">>>> </span><span class="p">(</span><span class="mf">0.1</span><span class="p">)</span><span class="o">.</span><span class="n">as_integer_ratio</span><span class="p">()</span>

334

335

336

<span class="gp">>>> </span><span class="n">Decimal</span><span class="o">.</span><span class="n">from_float</span><span class="p">(</span><span class="mf">0.1</span><span class="p">)</span>

337

<span class="go">Decimal('0.1000000000000000055511151231257827021181583404541015625')</span>

338

339

<span class="gp">>>> </span><span class="nb">format</span><span class="p">(</span><span class="n">Decimal</span><span class="o">.</span><span class="n">from_float</span><span class="p">(</span><span class="mf">0.1</span><span class="p">),</span> <span class="s1">'.17'</span><span class="p">)</span>

340

341

</pre></div>

342

</div>

343

</div>

344

</div>

345

346

347

</div>

348

</div>

349

</div>

350

351

352

<h3><a href="../contents.html">Table Of Contents</a></h3>

353

<ul>

354

<li><a class="reference internal" href="#">15. Floating Point Arithmetic: Issues and Limitations</a><ul>

355

<li><a class="reference internal" href="#representation-error">15.1. Representation Error</a></li>

356

</ul>

357

</li>

358

</ul>

359

360

<h4>Previous topic</h4>

361

<p class="topless"><a href="interactive.html"

362

title="previous chapter">14. Interactive Input Editing and History Substitution</a></p>

363

<h4>Next topic</h4>

364

<p class="topless"><a href="appendix.html"

365

title="next chapter">16. Appendix</a></p>

366

367

368

<li><a href="../bugs.html">Report a Bug</a></li>

369

<li><a href="../_sources/tutorial/floatingpoint.txt"

370

rel="nofollow">Show Source</a></li>

371

</ul>

372

373

374

<h3>Quick search</h3>

375

376

377

378

379

380

</form>

381

382

Enter search terms or a module, class or function name.

383

</p>

384

</div>

385

386

</div>

387

</div>

388

389

</div>

390

391

<h3>Navigation</h3>

392

<ul>

393

394

<a href="../genindex.html" title="General Index"

395

>index</a></li>

396

397

<a href="../py-modindex.html" title="Python Module Index"

398

>modules</a> |</li>

399

400

<a href="appendix.html" title="16. Appendix"

401

>next</a> |</li>

402

403

<a href="interactive.html" title="14. Interactive Input Editing and History Substitution"

404

>previous</a> |</li>

405

<li><img src="../_static/py.png" alt=""

406

style="vertical-align: middle; margin-top: -1px"/></li>

407

<li><a href="https://www.python.org/">Python</a> »</li>

408

<li>

409

410

<a href="../index.html">Documentation </a> »

411

</li>

412

413

<li class="nav-item nav-item-1"><a href="index.html" >The Python Tutorial</a> »</li>

414

</ul>

415

</div>

416

417

418

<br />

419

The Python Software Foundation is a non-profit corporation.

420

<a href="https://www.python.org/psf/donations/">Please donate.</a>

421

<br />

422

Last updated on Jan 22, 2016.

423

<a href="../bugs.html">Found a bug</a>?

424

<br />

425

Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.3.3.

426

</div>

427

428

</body>

429

</html>

b'\\ No newline at end of file'