You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
author = a_paszke_adam # and # a_gross_sam # and # a_massa_francisco # and # a_lerer_adam # and # a_bradbury_james # and # a_chanan_gregory # and # a_killeen_trevor # and # a_lin_zeming # and # a_gimelshein_natalia # and # a_antiga_luca # and # a_desmaison_alban # and # a_klopf_andreas # and # a_yang_edward_z # and # a_devito_zachary # and # a_raison_martin # and # a_tejani_alykhan # and # a_chilamkurthy_sasank # and # a_steiner_benoit # and # a_fang_lu # and # a_bai_junjie # and # a_chintala_soumith,
2076
2119
title = {\pytorch:~{A}n Imperative Style, High-Performance Deep Learning Library},
@@ -2264,6 +2307,14 @@ @inbook{PSF2024DTIPE
2264
2307
urldate = {2024-11-07},
2265
2308
}
2266
2309
2310
+
@inbook{PSF2024FPAIAL,
2311
+
title = {Floating-Point Arithmetic:~Issues and Limitations},
Copy file name to clipboardExpand all lines: text/main/basics/simpleDataTypesAndOperations/float/float.tex
+16-5
Original file line number
Diff line number
Diff line change
@@ -65,7 +65,7 @@
65
65
However, their accuracy is always limited to about 15~digits.
66
66
In other words, regardless whether your \pythonilIdx{float} stores a very large or a very small number, you can have at most 15~digits of precision.
67
67
For example, adding~1 to~$10^{16}$ would still yield~$10^{16}$, because only 15~digits are \inQuotes{stored} and the~1 will just \inQuotes{fall off.}
68
-
You cannot represent numbers arbitrarily precisely.%
68
+
You cannot represent numbers arbitrarily precisely~\cite{PSF2024FPAIAL}.%
69
69
%
70
70
\endhsection%
71
71
%
@@ -99,7 +99,7 @@
99
99
This brings us back to the previous section:
100
100
$\sqrt{3.3}$~is not actually 1.816\decSep590\decSep212\decSep458\decSep495.
101
101
It is an irrational number~\cite{S1988WPCHD,B1991IWNT} and irrational numbers cannot be expressed as fractions of integer numbers (by definition, otherwise they would be rational numbers).
102
-
Since they cannot be expressed as integer fractions, we cannot write them in any way like $\frac{1\decSep816\decSep590\decSep212\decSep458\decSep495\dots}{1\decSep000\decSep000\decSep000\decSep000\decSep000\dots}$, regardless of how large a denominator we would pick.
102
+
Since they cannot be expressed as integer fractions, we cannot write them down exactly in any way like $\frac{1\decSep816\decSep590\decSep212\decSep458\decSep495\dots}{1\decSep000\decSep000\decSep000\decSep000\decSep000\dots}$, regardless of how large a denominator we would pick.
103
103
Hence, we cannot represent them exactly using discrete binary values of our computer's memory.
104
104
Hence, the floating point representation cuts off somewhere.
105
105
And this somewhere is after 15~decimal places.%
@@ -120,12 +120,23 @@
120
120
When we then type \pythonilIdx{pi} and \pythonilIdx{e}, we can get to see their value in floating point representations: \pythonil{3.141592653589793} and \pythonil{2.718281828459045}, respectively.
121
121
Again, these are not the exact values, but they are as close as we can get in this format.
122
122
123
-
Of course, \numberPi\ and~\numberE\ alone are not that much useful.
124
-
If you reach back into your highschool days again, you will remember many interesting functions that are related to them.
123
+
Surprisingly, we do not necessarily need irrational or transcendental numbers to experience this cut-off.
124
+
There is no way to write down all the digits of fractions like~$\frac{1}{7}$ as decimals.
125
+
We always need to cut off somewhere, e.g., we could write~$0.14285714285714285$, but that is not exactly the same as~$\frac{1}{7}$.
126
+
As we discussed before, floating point numbers are stored in a binary format, i.e., represented by bits.
127
+
In the binary system, we encounter this problem already for fractions like~$\frac{1}{10}=0.1$~\cite{PSF2024FPAIAL}.
128
+
Essentially, we could write $\frac{1}{10}\approx\frac{1}{2^4}+\frac{1}{2^5}+\frac{1}{2^8}+\frac{1}{2^9}+\frac{1}{2^{12}}+\frac{1}{2^{13}}+\frac{1}{2^{16}}+\dots$, but we would never quite get there.
129
+
This means that $0.1$~cannot be exactly represented as \pythonil{float}.
130
+
Therefore, adding it up ten times also does not exactly equal~$1$.
131
+
Indeed, adding~\pythonil{0.1} ten times and then subtracting~\pythonil{1.0} from the result yields~\pythonil{-1.1102230246251565e-16}.
132
+
So even for \inQuotes{completely normal} numbers, floating point arithmetics may already cost us (a very tiny little bit of) precision.
133
+
134
+
Anyway, back to the constants \numberPi\ and~\numberE.
135
+
Alone, they are not that much useful, but if you reach back into your high school days again, you will remember many interesting functions that are related to them.
125
136
Let us import a few of them, again from the \pythonilIdx{math} module, via \pythonil{from math import sin, cos, tan, log}\pythonIdx{sin}\pythonIdx{cos}\pythonIdx{tan}\pythonIdx{log}.
126
137
I think you can guess what these functions do.
127
138
128
-
From highschool, you may remember that~$\sin{\frac{\numberPi}{4}}=\frac{\sqrt{2}}{2}$ and thus~$\sin^2{\frac{\numberPi}{4}}=0.5$.
139
+
From high school, you may remember that~$\sin{\frac{\numberPi}{4}}=\frac{\sqrt{2}}{2}$ and thus~$\sin^2{\frac{\numberPi}{4}}=0.5$.
129
140
Let us compute this in \python\ by doing \pythonil{sin(0.25 * pi) ** 2}\pythonIdx{sin}.
130
141
Surprisingly, we get \pythonil{0.4999999999999999} instead of \pythonil{0.5}.
131
142
The reason is again the limited precision of \pythonilIdx{float}, which cannot represent~$\frac{\sqrt{2}}{2}$ exactly.
A class for representing points in the two-dimensional Euclidean plane.}%
@@ -71,14 +72,14 @@
71
72
In other words, we later want to be able to create one instance of \pythonil{Point} with the x\nobreakdashes-coordinate~5 and the y\nobreakdashes-coordinate~10 and then another instance with the x\nobreakdashes-coordinate~2 and the y\nobreakdashes-coordinate~7.
72
73
73
74
Therefore, \pythonil{Point} needs a initializer, i.e., a special method that creates these attributes.
74
-
This method is called~\pythonilIdx{\_\_init\_\_}.
75
+
This method is called~\pythonilIdx{\_\_init\_\_}\pythonIdx{dunder!\_\_init\_\_}.
75
76
As said, every method of a class must have a parameter~\pythonilIdx{self}, which is the instance of the class (the object) upon which the method is called.
76
-
The initializer \pythonilIdx{\_\_init\_\_} is a special method, so it also has this parameter~\pythonilIdx{self}.
77
+
The initializer \pythonilIdx{\_\_init\_\_}\pythonIdx{dunder!\_\_init\_\_} is a special method, so it also has this parameter~\pythonilIdx{self}.
77
78
Additionally, we demand that two parameters~\pythonil{x} and~\pythonil{y} be passed in when we create an instance of~\pythonil{Point}.
78
79
We allow the values to be either \pythonils{int} or \pythonils{float}.%
79
80
%
80
81
\bestPractice{attributes}{%
81
-
Object attributes must only be created inside the initializer~\pythonilIdx{\_\_init\_\_}. %
82
+
Object attributes must only be created inside the initializer~\pythonilIdx{\_\_init\_\_}\pythonIdx{dunder!\_\_init\_\_}. %
82
83
An initial value must immediately be assigned to each attribute.%
83
84
}
84
85
%
@@ -97,7 +98,7 @@
97
98
In other words, we do not allow the coordinates of our \pythonils{Point} to be change after creation.%
98
99
%
99
100
\bestPractice{attributeTypeHint}{%
100
-
Every attribute of an object must be annotated with a \pgls{typeHint} and a documentation comment when created in the initializer~\pythonilIdx{\_\_init\_\_}. %
101
+
Every attribute of an object must be annotated with a \pgls{typeHint} and a documentation comment when created in the initializer~\pythonilIdx{\_\_init\_\_}\pythonIdx{dunder!\_\_init\_\_}. %
101
102
\pglspl{typeHint} work as with normal variables, but the documentation is slightly different: %
102
103
In the line \emph{above} the attribute initialization, a \emph{comment} starting with \pythonilIdx{\#: } must be written which explains the meaning of the attribute.%
103
104
}%
@@ -140,7 +141,7 @@
140
141
For \pythonil{p1}, this returns~\pythonil{True}.
141
142
142
143
We now create a second instance, \pythonil{p2}, of the class \pythonil{Point}.
143
-
We assign \pythonil{7} to \pythonil{p2.x} and \pythonil{8} to \pythonil{p2.y} via the \pythonilIdx{\_\_init\_\_} initializer, which is automatically invoked when we write~\pythonil{Point(7, 8)}.
144
+
We assign \pythonil{7} to \pythonil{p2.x} and \pythonil{8} to \pythonil{p2.y} via the \pythonilIdx{\_\_init\_\_}\pythonIdx{dunder!\_\_init\_\_} initializer, which is automatically invoked when we write~\pythonil{Point(7, 8)}.
144
145
We can again print the values of these attributes using an \pgls{fstring}.
145
146
While \pythonil{isinstance(p2, Point)} is again \pythonil{True}, \pythonil{isinstance(5, Point)} returns \pythonil{False}.
146
147
@@ -208,7 +209,7 @@
208
209
This is illustrated in \cref{fig:addingFloatsError}:
209
210
If we add~\pythonil{1} to~$10^{15}=\pythonil{1e15}$, the correct result is~\pythonil{1000000000000001.0}.
210
211
However, if we add~\pythonil{1} to~$10^{16}=\pythonil{1e16}$, the result is still~\pythonil{1e16}.
211
-
It is not possible to represent the number~$1+10^{16}$ correctly with the 64~bit double precision floating point numbers that \python\ offers.
212
+
It is not possible to represent the number~$1+10^{16}$ correctly with the 64~bit double precision floating point numbers that \python\ offers~\cite{PSF2024FPAIAL}.
212
213
Usually, that itself is totally fine:
213
214
There are few application in \inQuotes{everyday programming} where we really need more than fifteen digits of precision.
214
215
@@ -333,7 +334,7 @@
333
334
%
334
335
This is going to be the interface for our new class~\pythonil{KahanSum}, which, in turn, is based on~\cite{K2006AGKBSA}.
335
336
336
-
In the initializer~\pythonilIdx{\_\_init\_\_}, we create the three attributes~\pythonil{__sum}, \pythonil{__cs}, and \pythonil{__ccs} and initialize them to~\pythonil{0}.
337
+
In the initializer~\pythonilIdx{\_\_init\_\_}\pythonIdx{dunder!\_\_init\_\_}, we create the three attributes~\pythonil{__sum}, \pythonil{__cs}, and \pythonil{__ccs} and initialize them to~\pythonil{0}.
337
338
These names are directly taken from \cref{algo:kahanSum}.
338
339
Notice the double leading underscores in front of the names.%
0 commit comments