Skip to content

Commit 84166ea

Browse files
committed
Done with complexity notes.
1 parent 6b9e746 commit 84166ea

File tree

1 file changed

+46
-23
lines changed

1 file changed

+46
-23
lines changed

source/docs/programming_and_computer_usage/complexity.md

Lines changed: 46 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ In typical usage the $O$ notation is *asymptotical* (we are interested in very l
5252
- If $f(x)$ is a sum of several terms, if there is one with largest growth rate, it can be kept, and all others omitted.
5353
- If $f(x)$ is a product of several factors, any constants (factors in the product that do not depend on $x$) can be omitted.
5454

55-
There are additionaly some very useful [properties of the big O notation](https://www.geeksforgeeks.org/dsa/properties-of-asymptotic-notations/):
55+
There are additionally some very useful [properties of the big O notation](https://www.geeksforgeeks.org/dsa/properties-of-asymptotic-notations/):
5656

5757
- Reflexivity: $f(n) = O(f(n))$,
5858
- Transitivity: $f(n) = O(g(n))$ and $g(n) = O(h(n))$ implies $f(n) = O(h(n))$,
@@ -61,24 +61,6 @@ There are additionaly some very useful [properties of the big O notation](https:
6161
- Product rule: $f(n) = O(g(n))$ and $h(n) = O(k(n))$ implies $f(n) \times h(n) = O(g(n) \times k(n))$,
6262
- Composition rule: $f(n) = O(g(n))$ and $g(n) = O(h(n))$ implies $f(g(n)) = O(h(n))$.
6363

64-
$$
65-
\begin{aligned}
66-
f(n) & = O(f(n)) && \text{Reflexivity}\\
67-
f(n) & = O(f(n)) && \text{Reflexivity}
68-
\end{aligned}
69-
$$
70-
71-
$$
72-
\begin{array}{rll}
73-
E \psi &= H\psi & \text{Expanding the Hamiltonian Operator} \\
74-
&= -\frac{\hbar^2}{2m}\frac{\partial^2}{\partial x^2} \psi + \frac{1}{2}m\omega x^2 \psi & \text{Using the ansatz $\psi(x) = e^{-kx^2}f(x)$, hoping to cancel the $x^2$ term} \\
75-
&= -\frac{\hbar^2}{2m} [4k^2x^2f(x)+2(-2kx)f'(x) + f''(x)]e^{-kx^2} + \frac{1}{2}m\omega x^2 f(x)e^{-kx^2} &\text{Removing the $e^{-kx^2}$ term from both sides} \\
76-
& \Downarrow \\
77-
Ef(x) &= -\frac{\hbar^2}{2m} [4k^2x^2f(x)-4kxf'(x) + f''(x)] + \frac{1}{2}m\omega x^2 f(x) & \text{Choosing $k=\frac{im}{2}\sqrt{\frac{\omega}{\hbar}}$ to cancel the $x^2$ term, via $-\frac{\hbar^2}{2m}4k^2=\frac{1}{2}m \omega$} \\
78-
&= -\frac{\hbar^2}{2m} [-4kxf'(x) + f''(x)] \\
79-
\end{array}
80-
$$
81-
8264
<!--
8365
Example 1: f(n) = 3n2 + 2n + 1000Logn + 5000
8466
After ignoring lower order terms, we get the highest order term as 3n2
@@ -91,15 +73,56 @@ $$
9173
Big O Notation: O(n3)
9274
-->
9375

94-
95-
9676
Have a look at the [Big-O complexity chart](https://www.bigocheatsheet.com/):
9777

9878
![Big-O Complexity Chart](https://www.bigocheatsheet.com/img/big-o-complexity-chart.png)
9979

10080

101-
This can make a *very* significant difference, as exemplified in the following code:
81+
This can make a *very* significant difference, as exemplified in [the following code](./code/projects/GrowthMagnitudes.zip):
10282

103-
```
83+
```{download="./code/projects/GrowthMagnitudes.zip"}
10484
!include code/projects/GrowthMagnitudes/GrowthMagnitudes/Program.cs
10585
```
86+
87+
88+
# An Example: Integers
89+
90+
## Abstraction and Representation
91+
92+
To measure the time and space consumption of programs, we make some simplifying assumptions:
93+
94+
- Hardware is discarded: we compare programs assuming they run on the same platform, and do *not* consider if a program would be "better" on parallel hardware.
95+
- Constants are discarded: if a program is twice as slow as another one, we still consider them to be in the same order of magnitude^[This is captured by the "constant factor" property of the big O notation.].
96+
- *Representations* are in general discarded, as programs are assumed to use the same: for example, if the implementation of the `int` datatype is "reasonable" and the same for all programs, then we can discard it.
97+
98+
## How integers are represented
99+
100+
Compare the following three ways of representing integers:
101+
102+
Name | Base | Examples | Bits to represent $n$ |
103+
--- | :---: | ---------- | ----- |
104+
Unary (Tallies) | 1 | III, IIIIIII, IIIIIIII, … | $O(n)$ |
105+
Binary | 2 | 01011, 10011101, 101101010, … | $O(\log_2(n))$ |
106+
Decimal | 10 | 123, 930, 120,439, … | $O(\log_{10}(n))$ |
107+
108+
Actually, it takes roughly [$\log_2(n) / \log_2(b)$ digits](https://math.stackexchange.com/a/4490764) to represent the number $n$ in base $b$, except if $b = 1$.
109+
110+
And indeed we can confirm that for $b = 10$, we have
111+
112+
$$\log_2(n) / \log_2(b) = \log_{10}(n)$$
113+
114+
by [changing the base](https://en.wikipedia.org/wiki/List_of_logarithmic_identities#Changing_the_base) of the logarithm.
115+
116+
## Why it (almost) does not matter
117+
118+
Now, imagine that we have a program manipulating integers in base $b$.
119+
Converting numbers in base $b'$ takes $\log_2(b) / \log_2(b')$ times more (or less!) space, so if $b = 2$ and $b' = 2$, it means we need $3.322$ times more space to store and manipulate the integers.
120+
121+
If our program in base $b$ uses $O(g(n))$, it means that a program performing the same task, with the same algorithm, but using integers in base $b'$, would require $O((\log_2(b) / \log_2(b')) \times g(n))$.
122+
By adapting the constant factor principle of the big O notation, we can see that this is a negligible factor that can be omitted.
123+
124+
However, if the $b'$ base is 1, then the new program will use $O(n \times g(n))$: if $g(n)$ is greater than linear, this will make a difference!
125+
Of course, unary representation is "not" reasonable, so we will always assume that our representations are related by some constant, making the function orger of magnitude insensible to such details.
126+
127+
You can have a look at [the complexity of various arithmetic functions](https://en.wikipedia.org/wiki/Computational_complexity_of_mathematical_operations#Arithmetic_functions) and see that the representation is not even discussed, as those results are insensible to them, provided they are "reasonable".
128+

0 commit comments

Comments
 (0)