In the 04_error.ipynb notebook the two categories of numerical errors are listed as truncation error and floating point error. I tend to think of truncation error as the error made in a single step of approximating an ODE. I would like to separate truncation error into two other different kinds of errors, discretization error (error associated with using a simpler function) and convergence error (errors that accumulate over multiple steps in an algorithm).

Would this be okay?

Hi Kyle,

I found that in your notebook '~' is used for empty spaces, however, it won't get rendered correctly in github (right above the second cell block):
https://github.com/mandli/intro-numerical-methods/blob/master/14_LA_iterative.ipynb

It is rendered correctly on nbviewer however:
http://nbviewer.jupyter.org/github/mandli/intro-numerical-methods/blob/master/14_LA_iterative.ipynb

The way to get around is to use \quad or \[empty space].

I also use jupyter notebook for self-study and came across your wonderful lecture notes, when you cited an issue I raised for nbviewer:
jupyter/nbviewer#590

Cheers,
Zhangyi

At 5 Root Finding and Optimization, Asymptotic Convergence of Newton's Method

... Let $g(x) = x - \frac{f(x)}{f'(x)}$, then
...
What about $g'(x^*)$ though:
$$g'(x) = 1 - \frac{f'(x)}{f'(x)} + \frac{f(x)}{f''(x)}$$
which simplifies when evaluated at $x = x^*$ to
$$g'(x^*) = \frac{f(x^*)}{f''(x^*)} = 0$$
...

Since the Quotient Rule, $$\left ( \frac{u}{v} \right )' = \frac{u' v - v' u}{v^2}$$,

I think it should be $$g'(x) = 1 - \frac{f'(x) f'(x) - f(x) f''(x)}{f'^2(x)} = 1 - 1 + \frac{f(x) f''(x)}{f'^2(x)}$$.

Nonetheless, $$g'(x^*) = \frac{f(x^*) f''(x^*)}{f'^2(x^*)} = 0$$, $$g''(x^*) = \frac{f''(x^*)}{f'(x^*)}$$, so it won't effect the following result.

Under the shooting method, instead of

min_{v_2(0)} |pi/2 - v_2(2)|

should be

min_{v_2(0)} |pi/2 - v_1(2)|

I saw you revised this part before, but it did not look correct.

If $f(x_2) < f(x_1)$ then we know the minimum is between $x_0$ and $x_2$.
If $f(x_2) > f(x_1)$ then we know the minimum is between $x_1$ and $x_3$.

Should not the sign be reversed?

"Interpolation Alogithms: Repeated parabolic interpolation"

c = a^2 / b → a / b = a^2 / [b (b - a^2 / b)]
a / b = a^2 / (b^2 - a^2)
(there's an extra factor of 1 / b^2 in the notes)

For Bracketing Algorithm - Basic Idea:

"If $f(x_3) > f(x_2)$ then we know the minimum is between $x_1$ and $x_4$.

If $f(x_3) < f(x_2)$ then we know the minimum is between $x_3$ and $x_2$."

Shouldn't this be:

"If $f(x_3) > f(x_2)$ then we know the minimum is between $x_1$ and $x_3$.

If $f(x_3) < f(x_2)$ then we know the minimum is between $x_2$ and $x_4$."

...or go find a good one.

Typo for secant method comments:

"Not gauranteed to converge"

Should be du / dt = lambda u in the second line under the Example: Forward Euler on a Linear Problem.

This is throughout at least 10_LA_QR. Probably want 'complement' (from set theory) and not 'compliment' which is an expression of approval.

"Numerics compliment analytical methods" --> complement

In Complimentary Projectors, line that reads

I-2P-P

should be

I-2P+P

In the block containing

x = [0.2, None, None, 0.5]

x[1] = x[3] - phi * (x[3] - x[0])

x[2] = x[0] + phi * (x[3] - x[0])

I believe there is an error. The golden ratio phi is defined as 1.61803, but if that is true, then we should be dividing by the golden ratio, not multiplying by it. The code happens to work because the interval chosen in the notes is small enough such that x[1] and x[2] happen to fall within the interval. Making the interval larger will prevent this from happening and the brackets will diverge away.

Something along these lines was mentioned in class, but I figured I'd raise the potential issue just to be thorough.

In 10_linear_algebra, "complement" is misspelled as "compliment"

Area under "Plotting Stability Regions" and "Absolute Stability of the Forward Euler Method" mentions 'complex plain' and should be 'plane'.

In Backward Substitution, Solving Ax=b, 15_LA_gaussian, it writes:

Backwards substitution requires us to move from the last row of $U$ and move upwards. We can consider again the general $i$th row with
$$
U_{i,i} x_i + U_{i,i-1} x_{i-1} + \ldots + U_{i,m-1} x_{m-1} + U_{i,m} x_m = y_i
$$
noting that we are using the fact that the matrix $L$ has 1 on its diagonal. We can now solve for $y_i$ as
$$
x_i = \frac{1}{U_{i,i}} \left( y_i - ( U_{i,i-1} x_{i-1} + \ldots + U_{i,m-1} x_{m-1} + U_{i,m} x_m) \right )
$$

1. Both equations have index mistake ($U_{i,i-1} x_{i-1}$), which should be
   $$U_{i,i} x_i + U_{i,i+1} x_{i+1} + \ldots + U_{i,m-1} x_{m-1} + U_{i,m} x_m = y_i$$,
   $$x_i = \frac{1}{U_{i,i}} \left( y_i - ( U_{i,i+1} x_{i+1} + \ldots + U_{i,m-1} x_{m-1} + U_{i,m} x_m) \right )$$.

2. Matrix $L$ has nothing to do with backwards substitution. Noting that the matrix $L$ has 1 on its diagonal is unnecessary.

3. The second sentence of ones between two equations should be

   We can now solve for $x_i$ as ...

For the floating point arithmetic Ax = b example, the solver I used generated x = [1; 16], and not x = [-0.5; 16].

Want to take content and make it more modularized and re-orderable for ease of re-use.

Typo 1.)

"Smallest number that can be represented is the underflow: $1.0 \times 10^{-2} = 0.01$ Largest number that can be represented is the overflow: $9.9 \times 10^0 = 9.9$".

Should be "Smallest number that can be represented is the underflow: $0.1 \times 10^{-2} = 0.001$ Largest number that can be represented is the overflow: $9.9 \times 10^0 = 9.9".

Typo 2.)

"Smallest number that can be represented is the underflow: $1.0 \times 2^{-1} = 0.25$ Largest number that can be represented is the overflow: $1.1 \times 2^1 = 2.2$".

Should be "Smallest number that can be represented is the underflow: $0.1 \times 2^{-1} = 0.25$ Largest number that can be represented is the overflow: $1.1 \times 2^1 = 3$".

I'm pretty sure there is a typo on the last line of this lecture. Instead of:

$(U_{i,i-1} x_{i-1}$ ...

I think it should be:

$(U_{i,i+1} x_{i+1}$ ...

When describing the adam-bashworth method, there is a minor formatting error in the list describing the steps.

In "Example: 4-stage Runge-Kutta Method"

y_2 = u_4[n] + 0.5 * delta_t * f(t_n + 0.5 * delta_t, y_1)
y_3 = u_4[n] + 0.5 * delta_t * f(t_n + 0.5, y_2)

In "Example: Vandermonde Matrix," I believe the y matrix should be y1, y2 ... ym.

In the code for the "2-digit precision base 2 system" in the 04_error notebook, we see the system defined as follows:

axes.plot( (d1 + d2 * 0.1) * 2E, 0.0, 'r+', markersize=20)
axes.plot(-(d1 + d2 * 0.1) * 2E, 0.0, 'r+', markersize=20)

Shouldn't the 0.1 be 0.5, since this is binary and not decimal (or maybe I am missing something)?

I think there is a typo in the lecture, in the "Truncation Error for Multi-Step Methods" section.

In the last expression I think the general form for the q-ith term is miss-written. A summation seems missing, and I also think that the 1/q! term should not be multiplying both the \alpha and \beta terms (it should just be multiplying the \alpha term). For example:

\Delta t^{q - 1} \left (\frac{1}{q!} \left(j^q \alpha_j - \frac{1}{(q-1)!} j^{q-1} \beta_j \right) \right) u^{(q)}(t_n)

should be:

\Delta t^{q - 1} \left( \sum^r_{j=0} \left (\frac{1}{q!} j^q \alpha_j - \frac{1}{(q-1)!} j^{q-1} \beta_j \right) \right) u^{(q)}(t_n)

"Plotting the error as a function of $\Delta x$ is a common way to show that a numerical method is doing what we expect and exhbits the correct convergence behavior" --> "...exhibits the correct..."

Plots under L-stability section have typos in labels:

"Comparison of error for backward euler" should have label for Backward Euler.

"Comparison of errors for trapezoidal rule" should have label for Trapezoidal Rule (not Forward Euler).

There seems to be an error in the graphs showing the difference between euler and the leap-frog methods. The leapfrog graph just shows one data point at 0 and a vertical, dashed black line to it.

Near the end of the global error for the forward euler example, there is a type in the line:
"In other words the global error is bounded by the original global erro(r) and"

As described on computational science.

A small typo in line 28 of the code block under Adams-Moulton Methods.
axes.set_xlabel("u(t)") should be axes.set_ylabel

Also detect same typos when plotting similar graphs in the file.

The last equation in the last block has an error. The backwards substitution method starts with i+1, not i-1. See

link

It doesn't make sense as written so hopefully people will realize it, but this could confuse people!

Under 'Newton-Cotes Quadrature,' the lecture contains the phrase:
"evaluate f(x) at these points and exactly integrate the interpolating polynomial exactly."

was this intended?

The expression given by

$$f \cdot g = p \cdot q + O(x^{n\cdot m})$$

should be

$$f \cdot g = p \cdot q + O(x^{n + m})$$

covers chapters 00 - 04,

• example given not true for Python 3
  No requirement of defining as float initially in python 3
• indexing
  add reverse indexing? i.e grades[-1], grades[-2]
• help() command
  using help function, help() and with ?, jupyter notebook autocompletion using tab, and asking them to explore each function of every data structure
• any data structure or type possible inside a list
  showing examples of list inside a list for matrices?
• python 3 keeps the function as is
  range function defined as count, does not print the values
• add list of bitwise, comparison and logical operators?
• implicit arguments in functions
• Other data structures not discussed
• should be (1,0)
• print(numpy.linspace(-1, 1, 10))
• TODO: use %precision 3
• should not mention dtype=complex, is done in the following explanation
• use %matplotlib notebook made for Jupyter notebooks, plot becomes interactive
• add plt.colorbar()
• close bracket
• explaining list comprehension in previous chapters?

The code for Example 2 (last example) should be

numpy.all(error < 8.0 * numpy.finfo(float).eps):

not

numpy.all(error < 100.0 * numpy.finfo(float).eps):

Typo or unfinished block within Taylor Series Methods?

Example (no math mode):
[ u^{(p)}(t_n) = f^{(p-1)}(t_n, u(t_n)) ]

The images for 10_linear_algebra are not in the images folder

In 5_root_finding_optimization.ipynb, Analysis of Fixed Point Iteration part, it writes:

Using a Taylor expansion we know

$$g(x^* + e_k) = g(x^) + g'(x^) e_k + \frac{g''(c) e_k^2}{2}$$

$$x^* + e_{k+1} = g(x^) + g'(x^) e_k + \frac{g''(c) e_k^2}{2}$$

Why it's $$\frac{g''(c) e_k^2}{2}$$, instead of $$\frac{g''(x^*) e_k^2}{2}$$? Where does the $$c$$ come from?

In the above picture, shouldn't their be coefficients of x and x^2 in the bottom matrix matching the equations above it?

"In nomalized floating point systems" --> "normalized"

In 7_Differentiation, Examples, Example 1: 1st order Forward and Backward Differences, Computing Order of Convergence part, there are two equations:
e(Δx) = Δx^n + b
log e(Δx) = n logb logΔx

However, I don't think log e(Δx) equals that.
If I change the e(Δx), equations seem to be correct then.
e(Δx) = b Δx^n
log e(Δx) = logb + n logΔx
And the following plot supports that.