Numerical Methods
Solving engineering math on a computer: rounding error, roots of equations, linear systems, curve fitting, derivatives and integrals from data, ODE solvers, finite differences and optimization, with six labs. Free, self-paced, and tied to NOAA weather forecasting, the Mauna Loa CO₂ record and USGS groundwater models.
Courses › MATH 340
About this course
This is a free self-study course outline written by Saving Planets. It is not a university course, it carries no academic credit, and no school is involved.
Its scope follows a one-semester junior numerical methods course for engineers. It is modeled on the topics and depth of MIT 18.330 Introduction to Numerical Analysis, MIT 2.086 Numerical Computation for Mechanical Engineers and MIT 10.34 (free on MIT OpenCourseWare), using free OpenStax calculus sections, which we link where they help. None of those authors, schools or agencies endorses this page.
What we add: every unit ties to real work with a cited public source (NOAA’s Global Forecast System and supercomputers, the Mauna Loa CO₂ record, USGS groundwater models and NASA climate models), every number on the page is re-computed by a checking script before it goes live, and every lab has a free simulation or a spreadsheet.
Work through it in order. Each unit has goals, key ideas, equations, step-by-step examples, practice with full answers, common mistakes, flashcards and a quiz. Your progress is saved only in this browser.
How this course helps save the planet
Weather forecasts, climate projections and groundwater plans are all numerical methods running on computers: the equations cannot be solved by hand, so they are broken into grids, time steps and huge linear systems.
Knowing how those methods work, and how they fail, is how engineers judge a forecast or a model. Each unit ends with a short, sourced section. Values that are not from a source are labeled as sample or textbook values.
Before you start
Prerequisites
- Calculus I and II: derivatives, integrals and Taylor series.
- Differential Equations: first-order and second-order ODEs.
- Linear Algebra: matrices, determinants and eigenvalues.
- Intro Programming: loops, functions and arrays (a spreadsheet also works).
How to study
- Before computing, estimate the answer and decide what accuracy you need.
- Always compare against an exact answer on a test problem before trusting a method on a real one.
- Track the error as you halve the step: the ratio tells you the order of the method.
- Keep units in SI and write the step size with its units.
- Watch for subtracting nearly equal numbers; rewrite the formula if you can.
- Do every practice problem on paper or in a spreadsheet before opening the answer, and retake the quizzes a few days later.
Lab equipment (for the wet labs)
- A computer or tablet for the free PhET Least-Squares Regression, Curve Fitting, Calculus Grapher and Projectile Motion simulations.
- Spreadsheet software (any free one works), or Python if you know it.
- NOAA’s Mauna Loa CO₂ data file in a web browser.
- A calculator.
Suggested 15-week schedule
About 6 to 9 hours a week: reading, practice problems and, in lab weeks, the lab.
| Week | Unit | Topics | Reading |
|---|---|---|---|
| 1 | Errors, Floating-Point Numbers and Taylor Series | Errors, floating point, Taylor series; Lab A | MIT 18.330 |
| 2 | Roots of Equations: Bisection, Newton and Secant | Bisection and fixed-point iteration | MIT 18.330 |
| 3 | Roots of Equations: Bisection, Newton and Secant | Newton and secant methods; Lab B | OpenStax Calculus 1, 4.9 |
| 4 | Linear Systems: Gaussian Elimination, LU and Conditioning | Gaussian elimination, LU and conditioning | MIT 2.086 |
| 5 | Iterative Solvers: Jacobi, Gauss–Seidel and Convergence | Jacobi, Gauss–Seidel and sparse systems | MIT 18.330 |
| 6 | Interpolation, Splines and Least-Squares Fitting | Interpolation and splines | MIT 18.330 |
| 7 | Interpolation, Splines and Least-Squares Fitting | Least squares; Lab C; midterm practice | NOAA GML |
| 8 | Numerical Differentiation | Numerical differentiation | MIT 18.330 |
| 9 | Numerical Integration | Numerical integration; Lab D | OpenStax Calculus 2, 3.6 |
| 10 | Ordinary Differential Equations: Euler, Runge–Kutta and Stability | Euler, Heun and RK4 | MIT 10.34 |
| 11 | Ordinary Differential Equations: Euler, Runge–Kutta and Stability | Stability, stiffness and systems; Lab E | MIT 10.34 |
| 12 | Partial Differential Equations: Finite Differences | Finite differences for the heat equation | MIT 2.086 |
| 13 | Partial Differential Equations: Finite Differences | Laplace, waves and CFL; Lab F | NOAA EMC |
| 14 | Optimization, Eigenvalues and Computing at Scale | Optimization and the power method | MIT 18.330 |
| 15 | Optimization, Eigenvalues and Computing at Scale | Computing at scale, verification and validation; final practice | NOAA; NASA GISS |
Your progress
Tick units and labs as you finish them, and take the quizzes. Progress and quiz scores are saved only in this browser, never sent to us.
Turn on JavaScript to track progress.
Unit 1 · Week 1
Errors, Floating-Point Numbers and Taylor Series
Absolute and relative error, significant digits, IEEE 754 floating-point numbers, machine epsilon, rounding and cancellation, and truncation error from Taylor series.
Learning objectives
By the end of this unit you can:
- Compute absolute and relative error.
- Explain machine epsilon for single and double precision.
- Spot and fix catastrophic cancellation.
- Bound truncation error with the next Taylor term.
Key concepts
- Absolute error is |approximate − exact|; relative error divides by |exact|. Relative error tells you how many digits are right.
- Computers store numbers in binary floating point. Double precision has machine epsilon 2⁻⁵² ≈ 2.22 × 10⁻¹⁶ (about 16 digits); single precision has 2⁻²³ ≈ 1.19 × 10⁻⁷ (about 7 digits).
- Most decimals, like 0.1, have no exact binary form, so 0.1 + 0.2 − 0.3 comes out as about 5.55 × 10⁻¹⁷ instead of 0.
- Subtracting nearly equal numbers loses digits (catastrophic cancellation). Rewriting the formula algebraically often fixes it.
- Truncation error comes from cutting off an infinite process, such as a Taylor series. The first dropped term estimates the error.
Key equations
Worked examples, step by step
Example 1.1 Find the relative error of 22/7 as an approximation to π.
Show the steps
- |22/7 − π| = 0.001264.
- Divide by π.
Answer: Relative error = 4.02 × 10⁻⁴, so about 3 correct digits.
Example 1.2 Compute √(10⁸ + 1) − √(10⁸) safely.
Show the steps
- Direct subtraction cancels most digits.
- Use 1/(√(10⁸ + 1) + √(10⁸)).
Answer: 5.0 × 10⁻⁵ (more exactly 4.99999999875 × 10⁻⁵).
Example 1.3 Estimate the error of e^0.1 ≈ 1 + 0.1.
Show the steps
- The next term is 0.1²/2 = 0.005.
- The true error is e^0.1 − 1.1.
Answer: Error = 0.00517, close to the 0.005 estimate.
Practice problems
Work each on paper first, then check.
How many correct digits does double precision give?
Show answer
About 16 (ε = 2.22 × 10⁻¹⁶).
Add the 0.1³/6 term to e^0.1 ≈ 1.1. New error?
Show answer
1.71 × 10⁻⁴.
Why is 0.1 + 0.2 − 0.3 not exactly 0 on a computer?
Show answer
0.1 and 0.2 have no exact binary form; the result is about 5.55 × 10⁻¹⁷.
What is single-precision epsilon?
Show answer
2⁻²³ = 1.19 × 10⁻⁷.
Common misconceptions
Myth: Computers do exact arithmetic.
Actually: Every operation rounds to the nearest floating-point number.
Myth: Smaller steps always mean smaller error.
Actually: Rounding error grows as steps shrink.
Myth: Relative and absolute error are the same.
Actually: Relative error divides by the true size.
Flashcards
Absolute error
|approx − exact|.
Relative error
Absolute error / |exact|.
Machine epsilon
2⁻⁵² ≈ 2.22 × 10⁻¹⁶ in double.
Cancellation
Digit loss when subtracting close numbers.
Truncation error
From cutting off an infinite process.
Single precision
About 7 digits.
Self-check quiz
How this helps save the planet
NOAA’s Environmental Modeling Center describes a coupled global forecast run at about 3 km resolution that ran its atmospheric dynamics in 32-bit precision and everything else in 64-bit precision, a deliberate trade between speed and accuracy.
That is this unit’s epsilon choice in practice: single precision (about 7 digits) is faster and uses half the memory, so modelers use it where the error it adds stays below the error the model already has.
On Saving Planets: Earth Observation · Intro to Programming
Sources: NOAA Environmental Modeling Center: Global Forecast System (GFS)
Free learning resources
Unit 2 · Weeks 2–3
Roots of Equations: Bisection, Newton and Secant
Bracketing with bisection, Newton’s method, the secant method, fixed-point iteration, convergence order and stopping rules.
Learning objectives
By the end of this unit you can:
- Find a root by bisection and predict the number of steps.
- Apply Newton’s method and recognize quadratic convergence.
- Use the secant method when the derivative is unknown.
- Choose a stopping rule.
Key concepts
- Bisection halves a bracket [a, b] where f changes sign. It always converges, but only one binary digit per step: n ≥ log₂((b − a)/tol).
- Newton’s method follows the tangent line: x_{n+1} = x_n − f(x_n)/f′(x_n). Near a simple root it converges quadratically, doubling the correct digits each step.
- The secant method replaces f′ with the slope through the last two points. It needs no derivative and converges with order about 1.618.
- Newton can fail if f′ is near zero or the start is far away; a bracketing fallback makes it safe.
- Stop when |x_{n+1} − x_n| and |f(x_n)| are both below tolerance, and always cap the number of iterations.
Key equations
Worked examples, step by step
Example 2.1 Use bisection on f(x) = x³ − x − 2 on [1, 2]. Give the first three midpoints and the root.
Show the steps
- f(1) = −2, f(2) = 4.
- Midpoints 1.5, 1.75, 1.625.
Answer: Root x = 1.5214.
Example 2.2 How many bisection steps shrink a width-1 bracket below 10⁻⁶?
Show the steps
- log₂(10⁶) = 19.93.
Answer: 20 steps.
Example 2.3 Use Newton’s method on x² − 2 from x₀ = 1.
Show the steps
- x₁ = 1.5, x₂ = 1.41667.
- x₃ = 1.414216.
Answer: x₃ = 1.414216, already 6 correct digits.
Practice problems
Work each on paper first, then check.
Apply Newton to x³ − x − 2 from x₀ = 1.5. First step?
Show answer
1.52174.
Apply the secant method to x² − 2 from 1 and 2. First three iterates?
Show answer
1.3333, 1.4000, 1.4146.
What convergence order does the secant method have?
Show answer
About 1.618.
Why cap the iteration count?
Show answer
A method may diverge or cycle.
Common misconceptions
Myth: Newton always converges.
Actually: It can diverge from a poor start.
Myth: Bisection is fast.
Actually: It gains one bit per step.
Myth: A small f(x) means x is close to the root.
Actually: Not if f is very flat.
Flashcards
Bisection
Halve a sign-change bracket.
Newton
x − f/f′.
Secant
Newton with a two-point slope.
Quadratic convergence
Digits double each step.
Bracket
Interval with a sign change.
Stopping rule
Step and residual below tolerance.
Self-check quiz
How this helps save the planet
The USGS says MODFLOW is considered an international standard for simulating groundwater. One of its versions, MODFLOW-NWT, uses a Newton-Raphson formulation to improve the solution of unconfined groundwater-flow problems.
Unconfined aquifers make the equations nonlinear, because the saturated thickness depends on the water level being solved for. Newton’s method from this unit is what makes those runs converge reliably.
On Saving Planets: Oceans · Calculus I
Sources: USGS: MODFLOW and Related Programs
Free learning resources
- OpenStax Calculus Volume 1, 4.9 Newton’s method: Free textbook section.
- MIT OpenCourseWare 18.330 Introduction to Numerical Analysis (Spring 2012): Free MIT lectures.
Unit 3 · Week 4
Linear Systems: Gaussian Elimination, LU and Conditioning
Gaussian elimination with partial pivoting, LU factorization, operation counts, the condition number and how it limits accuracy.
Learning objectives
By the end of this unit you can:
- Solve a 3 × 3 system by elimination.
- Explain LU factorization and reuse.
- Estimate the cost of elimination.
- Use the condition number to judge accuracy.
Key concepts
- Gaussian elimination reduces Ax = b to an upper-triangular system, then back-substitutes. Partial pivoting swaps rows so the largest pivot is used, which controls rounding error.
- LU factorization stores the elimination as A = LU. Once factored, each new right-hand side costs only two triangular solves.
- Elimination costs about 2n³/3 floating-point operations: doubling n multiplies the work by 8.
- The condition number κ(A) = ‖A‖‖A⁻¹‖ measures sensitivity. You can lose about log₁₀ κ digits of accuracy.
- Never compute A⁻¹ to solve a system; factor and solve instead.
Key equations
Worked examples, step by step
Example 3.1 Solve 2x + y − z = 8, −3x − y + 2z = −11, −2x + y + 2z = −3.
Show the steps
- Eliminate x from rows 2 and 3, then y from row 3.
- Back-substitute.
Answer: x = 2, y = 3, z = −1.
Example 3.2 Estimate the elimination cost for n = 1000.
Show the steps
- 2 × 1000³/3.
Answer: About 6.67 × 10⁸ flops.
Example 3.3 Find the 2-norm condition number of [[1, 1], [1, 1.0001]].
Show the steps
- The rows are nearly parallel.
- κ = σ_max/σ_min.
Answer: κ ≈ 40,002, so about 4–5 digits can be lost.
Practice problems
Work each on paper first, then check.
How much more work is n = 2000 than n = 1000?
Show answer
8 times.
Elimination flops for n = 3?
Show answer
2 × 27/3 = 18.
Why pivot?
Show answer
To avoid dividing by small numbers and amplifying rounding.
Digits lost with κ = 10⁸ in double precision?
Show answer
About 8 of 16.
Common misconceptions
Myth: Use A⁻¹ to solve systems.
Actually: Factor and solve; it is cheaper and more accurate.
Myth: A small determinant means ill-conditioned.
Actually: Use the condition number.
Myth: Pivoting is optional.
Actually: Without it, elimination can fail or lose accuracy.
Flashcards
Gaussian elimination
Reduce to triangular form.
LU
A = LU.
Pivoting
Row swaps for stability.
Condition number
‖A‖‖A⁻¹‖.
Back substitution
Solve triangular systems.
Cost
≈ 2n³/3.
Self-check quiz
How this helps save the planet
The USGS says a key feature of MODFLOW 6 is its ability to solve multiple tightly coupled numerical models in a single system of equations.
Its map of depth to the water table across the continental United States came from a parallel simulation of 256 tightly coupled model domains on 256 cores of a USGS supercomputer. Each domain is a large sparse linear system like the ones in this unit.
On Saving Planets: Linear Algebra · Oceans
Sources: USGS: MODFLOW and Related Programs
Free learning resources
Unit 4 · Week 5
Iterative Solvers: Jacobi, Gauss–Seidel and Convergence
Jacobi and Gauss–Seidel iteration, diagonal dominance, the iteration matrix and spectral radius, sparse systems and why iterative methods scale.
Learning objectives
By the end of this unit you can:
- Carry out Jacobi and Gauss–Seidel steps.
- Test diagonal dominance.
- Use the spectral radius to predict convergence speed.
- Explain why large sparse systems use iteration.
Key concepts
- Jacobi updates every unknown from the previous iterate; Gauss–Seidel uses new values as soon as they are computed.
- If A is strictly diagonally dominant, both methods converge from any start.
- The error shrinks each step by about the spectral radius ρ of the iteration matrix. Gauss–Seidel often has ρ_GS = ρ_J².
- Large grids give sparse matrices with only a few nonzeros per row. Iterative methods store only those nonzeros, while elimination fills in.
- Stop when the residual ‖b − Ax‖ is small relative to ‖b‖.
Key equations
Worked examples, step by step
Example 4.1 Do two Jacobi steps on 4x + y = 1, x + 3y = 2 from (0, 0).
Show the steps
- Step 1: x = 1/4 = 0.25, y = 2/3 = 0.6667.
- Step 2: x = (1 − 0.6667)/4, y = (2 − 0.25)/3.
Answer: x = 0.0833, y = 0.5833 (exact 0.0909, 0.6364).
Example 4.2 Do two Gauss–Seidel steps on the same system.
Show the steps
- Step 1: x = 0.25, y = (2 − 0.25)/3 = 0.5833.
- Step 2: x = 0.1042, y = 0.6319.
Answer: x = 0.1042, y = 0.6319, closer than Jacobi.
Example 4.3 Find the Jacobi and Gauss–Seidel spectral radii for that system.
Show the steps
- ρ_J = √(1/12).
- ρ_GS = ρ_J² = 1/12.
Answer: ρ_J = 0.289; ρ_GS = 0.0833.
Practice problems
Work each on paper first, then check.
Is [[4, 1], [1, 3]] diagonally dominant?
Show answer
Yes: 4 > 1 and 3 > 1.
Steps for Gauss–Seidel (ρ = 0.0833) to cut error by 10⁶?
Show answer
About 6 (0.0833⁶ = 3.35 × 10⁻⁷).
Third Gauss–Seidel iterate?
Show answer
x = 0.0920, y = 0.6360.
Why not eliminate on a million-unknown grid?
Show answer
Fill-in uses too much memory and time.
Common misconceptions
Myth: Iteration always converges.
Actually: It needs ρ < 1.
Myth: Jacobi and Gauss–Seidel are the same.
Actually: Gauss–Seidel uses new values immediately.
Myth: A small change between steps means converged.
Actually: Check the residual too.
Flashcards
Jacobi
Update all from old values.
Gauss–Seidel
Use new values immediately.
Spectral radius
Largest |eigenvalue|.
Diagonal dominance
|a_ii| > row sum.
Sparse
Mostly zeros.
Residual
b − Ax.
Self-check quiz
How this helps save the planet
NOAA’s GFS divides the atmosphere into 127 vertical layers at about 13 km horizontal spacing. Each grid point couples only to its neighbours, so the systems such models build are huge and sparse.
With a textbook Earth surface area of 5.1 × 10⁸ km², a 13 km grid has about 3.0 million columns and 383 million cells. Systems that size are handled with iterative and other sparse methods, not plain elimination.
On Saving Planets: Earth Observation · Linear Algebra
Sources: NOAA Environmental Modeling Center: Global Forecast System (GFS)
Free learning resources
Unit 5 · Weeks 6–7
Interpolation, Splines and Least-Squares Fitting
Linear and polynomial interpolation, Lagrange form, the danger of high-degree polynomials, cubic splines, and least-squares line and polynomial fits with residuals.
Learning objectives
By the end of this unit you can:
- Interpolate linearly between data points.
- Build a Lagrange polynomial.
- Fit a least-squares line and interpret its slope.
- Choose between interpolation and fitting.
Key concepts
- Interpolation passes exactly through the data; fitting passes near noisy data and minimizes the sum of squared residuals.
- A degree-n polynomial through n + 1 points is unique. High degrees on equally spaced points can oscillate wildly near the ends (Runge’s phenomenon).
- Cubic splines join cubic pieces with matching slope and curvature, giving smooth curves without wild swings.
- The least-squares line solves the normal equations AᵀAc = Aᵀy. The slope is a rate, with units.
- Plot the residuals: a pattern in them means the model is missing something, such as curvature.
Key equations
Worked examples, step by step
Example 5.1 NOAA’s Mauna Loa annual means are 390.10 ppm (2010) and 414.21 ppm (2020). Interpolate 2015.
Show the steps
- y = 390.10 + 0.5 × 24.11.
Answer: 402.16 ppm; NOAA’s 2015 value is 401.01, so the record curves upward.
Example 5.2 Fit a least-squares line to the decade values 1960–2020 (316.91, 325.68, 338.76, 354.45, 369.71, 390.10, 414.21 ppm).
Show the steps
- Solve the normal equations for slope and intercept.
Answer: Slope 1.613 ppm per year; the line predicts 415.0 ppm for 2025.
Example 5.3 Fit a quadratic in t = year − 1960 instead.
Show the steps
- Least squares with columns 1, t, t².
Answer: c = 316.84 + 0.810t + 0.01338t²; it predicts 426.0 ppm for 2025, close to NOAA’s 427.35.
Practice problems
Work each on paper first, then check.
Average growth 2010–2020 from the decade values?
Show answer
2.41 ppm per year.
Average growth 2000–2010?
Show answer
2.04 ppm per year.
Why does the straight line underpredict 2025?
Show answer
Growth is speeding up; the residuals curve.
Interpolate or fit noisy lab data?
Show answer
Fit.
Common misconceptions
Myth: Higher-degree polynomials always fit better.
Actually: They can oscillate between points.
Myth: A good R² means a good model.
Actually: Check the residual pattern.
Myth: Extrapolation is as safe as interpolation.
Actually: It is much riskier.
Flashcards
Interpolation
Exactly through the data.
Least squares
Minimize squared residuals.
Spline
Smooth piecewise cubic.
Runge
Wiggles of high-degree fits.
Residual
Data minus model.
Normal equations
AᵀAc = Aᵀy.
Self-check quiz
How this helps save the planet
NOAA reports that the Mauna Loa record, started by C. David Keeling of Scripps in March 1958, is the longest record of direct CO₂ measurements in the atmosphere. The monthly mean was 426.07 ppm in September 2026, up from 424.37 ppm in September 2025.
NOAA’s annual means rose from 316.91 ppm in 1960 to 427.35 ppm in 2025. The upward curve shows up in this unit’s residuals: a straight line misses it, and a quadratic fits.
On Saving Planets: Solutions · Earth Observation
Sources: NOAA Global Monitoring Laboratory: Trends in atmospheric CO₂ (Mauna Loa) · NOAA GML: Mauna Loa annual mean CO₂ data
Free learning resources
- PhET: Least-Squares Regression: Fit lines interactively.
- PhET: Curve Fitting: Polynomial fits interactively.
Unit 6 · Week 8
Numerical Differentiation
Forward, backward and central differences, their truncation error, the balance with rounding error and the best step size, second derivatives, and Richardson extrapolation.
Learning objectives
By the end of this unit you can:
- Apply forward and central differences.
- State the order of each formula.
- Choose a step size that balances truncation and rounding.
- Estimate a second derivative.
Key concepts
- The forward difference (f(x + h) − f(x))/h has error proportional to h (first order).
- The central difference (f(x + h) − f(x − h))/(2h) has error proportional to h² (second order), so halving h cuts the error by 4.
- Rounding error grows like ε/h as h shrinks. For forward differences the best h is about √ε ≈ 1.49 × 10⁻⁸; for central differences about ε^(1/3) ≈ 6.06 × 10⁻⁶.
- The second derivative is (f(x + h) − 2f(x) + f(x − h))/h², also second order.
- Richardson extrapolation combines two step sizes to cancel the leading error term.
Key equations
Worked examples, step by step
Example 6.1 Estimate the derivative of sin x at x = 1 with a forward difference, h = 0.1.
Show the steps
- (sin 1.1 − sin 1)/0.1.
Answer: 0.4974; exact cos 1 = 0.5403; error −0.0429.
Example 6.2 Repeat with a central difference.
Show the steps
- (sin 1.1 − sin 0.9)/0.2.
Answer: 0.5394; error −0.00090, about 47.7 times smaller.
Example 6.3 Find the best forward-difference step in double precision.
Show the steps
- h ≈ √(2.22 × 10⁻¹⁶).
Answer: h ≈ 1.49 × 10⁻⁸.
Practice problems
Work each on paper first, then check.
Halve h in a central difference. Error?
Show answer
About ¼ as large.
Halve h in a forward difference. Error?
Show answer
About ½ as large.
Best central-difference step in double precision?
Show answer
About ε^(1/3) = 6.06 × 10⁻⁶.
Why not use h = 10⁻¹⁵?
Show answer
Rounding error ε/h dominates.
Common misconceptions
Myth: Smaller h is always better.
Actually: Below the optimum, rounding wins.
Myth: Forward and central are equally accurate.
Actually: Central is second order.
Myth: Differentiating noisy data is easy.
Actually: Differences amplify noise; smooth or fit first.
Flashcards
Forward difference
First order.
Central difference
Second order.
Second difference
f(x+h) − 2f(x) + f(x−h) over h².
Optimal h
≈ √ε for forward.
Richardson
Cancel the leading error.
Order
Exponent of h in the error.
Self-check quiz
How this helps save the planet
NOAA says the GFS produces hourly forecast output for the first 120 hours, then 3-hourly output for days 5 to 16. Forecasters and downstream models difference those outputs to get rates such as pressure tendency and rainfall rate.
This unit shows why the time step of saved output matters: a rate taken from 3-hourly values with a central difference is far more accurate than a one-sided difference over the same interval.
On Saving Planets: GOES-19 · Calculus I
Sources: NOAA Environmental Modeling Center: Global Forecast System (GFS)
Free learning resources
- MIT OpenCourseWare 18.330 Introduction to Numerical Analysis (Spring 2012): Free MIT lectures.
- PhET: Calculus Grapher: Derivatives interactively.
Unit 7 · Week 9
Numerical Integration
Trapezoidal and Simpson’s rules, composite rules and error orders, Romberg extrapolation, Gaussian quadrature, and integrating measured data.
Learning objectives
By the end of this unit you can:
- Apply the composite trapezoidal and Simpson’s rules.
- Predict how error falls with n.
- Use Romberg to improve a result.
- Apply 2-point Gaussian quadrature.
Key concepts
- The trapezoidal rule joins points with straight lines. Its error falls like h²: doubling n cuts it by 4.
- Simpson’s rule fits parabolas through pairs of intervals (n even). Its error falls like h⁴: doubling n cuts it by 16.
- Romberg extrapolation: T₂ₙ + (T₂ₙ − Tₙ)/3 cancels the h² error of the trapezoidal rule (it equals Simpson’s rule).
- Gaussian quadrature picks the best points: 2 points at ±1/√3 integrate cubics exactly.
- For measured data at fixed spacing, use the trapezoidal or Simpson rule directly on the samples.
Key equations
Worked examples, step by step
Example 7.1 Estimate ∫₀¹ eˣ dx with the trapezoidal rule, n = 4.
Show the steps
- h = 0.25; sum the weighted values.
Answer: T₄ = 1.72722; exact e − 1 = 1.71828; error 0.00894.
Example 7.2 Repeat with Simpson’s rule, n = 4.
Show the steps
- Weights 1, 4, 2, 4, 1 times h/3.
Answer: S₄ = 1.718319; error 3.70 × 10⁻⁵.
Example 7.3 Use 2-point Gaussian quadrature on the same integral.
Show the steps
- Map [0, 1] to [−1, 1]: points 0.5 ± 0.5/√3, weight ½ each.
Answer: 1.717896; error −3.85 × 10⁻⁴ with only 2 function values.
Practice problems
Work each on paper first, then check.
Trapezoidal error with n = 8?
Show answer
0.00224, about ¼ of the n = 4 error.
Romberg from T₄ and T₈?
Show answer
1.718284.
Simpson’s rule with n = 2?
Show answer
1.718861.
Why must Simpson’s n be even?
Show answer
It uses pairs of intervals.
Common misconceptions
Myth: Simpson works for any n.
Actually: n must be even.
Myth: More points always help equally.
Actually: Higher-order rules gain much faster.
Myth: Gauss points can be anywhere.
Actually: They are fixed roots of Legendre polynomials.
Flashcards
Trapezoidal
Error ∝ h².
Simpson
Error ∝ h⁴.
Romberg
Extrapolate trapezoid results.
Gauss quadrature
Optimal points and weights.
Composite rule
Apply on many subintervals.
Even n
Needed for Simpson.
Self-check quiz
How this helps save the planet
NOAA explains that every Mauna Loa monthly mean is the average of daily means, which are in turn based on hourly averages, using only hours with “background” conditions.
An average is an integral divided by the interval, so building monthly means from hourly values is numerical integration, the trapezoidal idea from this unit applied to real measurements.
On Saving Planets: Calculus II · Solutions
Sources: NOAA Global Monitoring Laboratory: Trends in atmospheric CO₂ (Mauna Loa)
Free learning resources
- OpenStax Calculus Volume 2, 3.6 Numerical integration: Free textbook section.
- PhET: Calculus Grapher: Integrals interactively.
Unit 8 · Weeks 10–11
Ordinary Differential Equations: Euler, Runge–Kutta and Stability
Euler’s method, Heun’s method, fourth-order Runge–Kutta, local and global error, stability limits, stiff equations and implicit methods, and systems of ODEs.
Learning objectives
By the end of this unit you can:
- Step an ODE with Euler and RK4.
- Relate step size to global error.
- Find the explicit stability limit for y′ = −λy.
- Recognize stiff problems.
Key concepts
- Euler’s method y_{n+1} = y_n + h f(t_n, y_n) has global error proportional to h (first order).
- Fourth-order Runge–Kutta (RK4) uses four slope samples per step; its global error falls like h⁴.
- For y′ = −λy, explicit Euler is stable only if h < 2/λ. Outside that, the numerical solution grows even though the true one decays.
- Stiff problems have very fast and very slow parts. Implicit methods such as backward Euler stay stable with large steps.
- Higher-order ODEs become first-order systems, which the same methods solve.
Key equations
Worked examples, step by step
Example 8.1 Solve y′ = −y, y(0) = 1 to t = 1 with Euler, h = 0.1.
Show the steps
- Each step multiplies by 0.9.
- y(1) ≈ 0.9¹⁰.
Answer: 0.34868 vs exact 0.36788; error −0.0192.
Example 8.2 Take one RK4 step of h = 0.1 on the same problem.
Show the steps
- k₁ = −1, k₂ = −0.95, k₃ = −0.9525, k₄ = −0.90475.
Answer: y(0.1) = 0.9048375 vs exact 0.9048374.
Example 8.3 A coffee cools by T′ = −0.1(T − 20) from 90 °C (sample values). Use Euler with h = 1 min for 10 min.
Show the steps
- Each step: T − 20 shrinks by 0.9.
Answer: 44.41 °C vs exact 45.75 °C.
Practice problems
Work each on paper first, then check.
Euler with h = 0.05 to t = 1 on y′ = −y?
Show answer
0.95²⁰ = 0.35849, error about half.
Stability limit for y′ = −50y with Euler?
Show answer
h < 0.04.
One Heun step of h = 0.1 on y′ = −y?
Show answer
0.905.
How do you solve y″ = −y with these methods?
Show answer
Write it as y′ = v, v′ = −y.
Common misconceptions
Myth: A smaller step is always stable.
Actually: Only below the limit, and rounding grows with many steps.
Myth: RK4 is exact.
Actually: Its error is small but not zero.
Myth: Implicit methods are always slower.
Actually: For stiff problems they are much faster overall.
Flashcards
Euler
First-order explicit.
RK4
Fourth order, 4 slopes.
Stability
Errors do not grow.
Stiff
Fast and slow scales together.
Backward Euler
Implicit, very stable.
System
Higher order → first order.
Self-check quiz
How this helps save the planet
NOAA says the GFS runs four times a day and forecasts up to 16 days ahead. The EMC page also describes a 3 km coupled research run that took about 40,000 seconds of computer time for each 24-hour forecast, using 233 nodes.
That is about 11.1 hours of computing per forecast day, so the model ran only about 2.16 times faster than real time. Each forecast is a time-stepping ODE and PDE solve, and the stable step size sets the cost.
On Saving Planets: Differential Equations · GOES-19
Sources: NOAA Environmental Modeling Center: Global Forecast System (GFS)
Free learning resources
- MIT OpenCourseWare 10.34 Numerical Methods Applied to Chemical Engineering (Fall 2015): Free MIT course.
- PhET: Projectile Motion: Compare with your solver.
Unit 9 · Weeks 12–13
Partial Differential Equations: Finite Differences
Grids and stencils, the explicit heat equation and its stability limit, implicit and Crank–Nicolson schemes, Laplace’s equation and iterative solution, the CFL condition for waves, and the cost of resolution.
Learning objectives
By the end of this unit you can:
- Write finite-difference stencils for the heat and Laplace equations.
- Apply the explicit stability limit r ≤ ½.
- Explain the CFL condition.
- Estimate how cost grows with resolution.
Key concepts
- Replace derivatives with differences on a grid. For u_t = αu_xx, the explicit scheme is u_i^{n+1} = u_i^n + r(u_{i+1}^n − 2u_i^n + u_{i−1}^n) with r = αΔt/Δx².
- The explicit heat scheme is stable only if r ≤ ½, so halving Δx forces Δt down by 4.
- Implicit and Crank–Nicolson schemes are stable for any Δt but need a linear solve each step.
- Laplace’s equation u_xx + u_yy = 0 gives a 5-point stencil; Jacobi or Gauss–Seidel (Unit 4) solve it.
- For waves and flow, the CFL condition c Δt/Δx ≤ 1 links time step to grid spacing. Refining a 2D-horizontal grid by a factor k costs about k² more columns and k more time steps.
Key equations
Worked examples, step by step
Example 9.1 Find the largest stable explicit time step for α = 1.0 × 10⁻⁴ m²/s and Δx = 0.01 m (sample values).
Show the steps
- Δt ≤ 0.5Δx²/α.
Answer: Δt ≤ 0.5 s.
Example 9.2 NOAA plans to refine the GFS from 13 km to 9 km. Estimate the change in horizontal columns.
Show the steps
- (13/9)².
Answer: About 2.09 times as many columns.
Example 9.3 Including a CFL-limited time step, estimate the total cost increase.
Show the steps
- (13/9)³.
Answer: About 3.01 times the work.
Practice problems
Work each on paper first, then check.
Halve Δx in the explicit heat scheme. New Δt limit?
Show answer
¼ as large.
Columns on a 13 km global grid (Earth 5.1 × 10⁸ km², textbook)?
Show answer
About 3.02 million.
Cells with 127 layers?
Show answer
About 3.83 × 10⁸.
Cells at 9 km with 127 layers?
Show answer
About 8.0 × 10⁸.
Common misconceptions
Myth: Any time step works if the grid is fine.
Actually: Finer grids need smaller explicit steps.
Myth: Implicit schemes are always more accurate.
Actually: They are more stable, not automatically more accurate.
Myth: Doubling resolution doubles cost.
Actually: In 2D plus time it is about 8×.
Flashcards
Stencil
Pattern of neighbour points.
r
αΔt/Δx².
CFL
cΔt/Δx ≤ 1.
Crank–Nicolson
Implicit, second order.
5-point Laplacian
Average of neighbours.
Resolution cost
≈ k³ for k-fold refinement.
Self-check quiz
How this helps save the planet
NOAA’s 2023 supercomputer upgrade was meant to let the GFS move to a horizontal resolution of nine kilometres, compared with 13 in the current version, so smaller-scale features can be seen and simulated.
This unit’s scaling shows why that needed new computers: 2.09 times as many columns and, with a CFL-limited time step, about 3 times the work for every forecast.
On Saving Planets: Fluid Mechanics · Thermodynamics
Sources: NOAA: NOAA completes upgrade to weather and climate supercomputer system (Aug. 10, 2023) · NOAA Environmental Modeling Center: Global Forecast System (GFS)
Free learning resources
Unit 10 · Weeks 14–15
Optimization, Eigenvalues and Computing at Scale
Golden-section search, gradient descent and Newton’s method for minimization, the power method for eigenvalues, flop counts, parallel computing, and verifying and validating models.
Learning objectives
By the end of this unit you can:
- Minimize a 1D function by golden-section search.
- Apply gradient descent and choose a step.
- Find a dominant eigenvalue by the power method.
- Estimate run time from flop counts and machine speed.
Key concepts
- Golden-section search shrinks a bracket by 0.618 each step without derivatives.
- Gradient descent steps downhill: x ← x − α∇f. Too large an α diverges; too small crawls.
- Newton’s method for minimization solves ∇f = 0 using the Hessian; it converges quadratically near the minimum.
- The power method repeatedly multiplies by A; the vector lines up with the dominant eigenvector, and the Rayleigh quotient gives its eigenvalue.
- Verification asks whether the code solves the equations right; validation asks whether they are the right equations, checked against observations.
Key equations
Worked examples, step by step
Example 10.1 How much does golden-section search shrink a bracket in 10 steps?
Show the steps
- 0.618¹⁰.
Answer: To 0.0081 (0.81%) of its width.
Example 10.2 Minimize f = (x − 3)² by gradient descent with α = 0.1 from x = 0. Where are you after 5 steps?
Show the steps
- Each step: x − 3 shrinks by 0.8.
- x₅ = 3(1 − 0.8⁵).
Answer: x₅ = 2.017.
Example 10.3 Apply the power method to [[2, 1], [1, 2]] from (1, 0), 5 steps.
Show the steps
- v = (122, 121).
- Rayleigh quotient.
Answer: λ ≈ 2.99997 (exact 3).
Practice problems
Work each on paper first, then check.
How long does a 10,000-unknown dense solve (2n³/3 flops) take at 14.5 petaflops?
Show answer
6.67 × 10¹¹/1.45 × 10¹⁶ = 4.6 × 10⁻⁵ s.
Combined speed of NOAA’s two 14.5-petaflop computers?
Show answer
29 petaflops.
Operational plus research capacity (20 petaflops)?
Show answer
49 petaflops.
Verification or validation: comparing a model with Mauna Loa data?
Show answer
Validation.
Common misconceptions
Myth: A bigger step always speeds gradient descent.
Actually: It can overshoot and diverge.
Myth: The power method finds every eigenvalue.
Actually: Only the dominant one.
Myth: A converged code is a correct model.
Actually: It still needs validation.
Flashcards
Golden section
Bracket shrinks by 0.618.
Gradient descent
x − α∇f.
Power method
Dominant eigenvalue.
Rayleigh quotient
vᵀAv/vᵀv.
Verification
Solving the equations right.
Validation
Solving the right equations.
Self-check quiz
How this helps save the planet
NOAA reports that its twin weather and climate supercomputers, Dogwood in Manassas, Virginia, and Cactus in Phoenix, Arizona, were upgraded in August 2023 to 14.5 petaflops each, so together the forecast system can process 29 quadrillion calculations per second. With research machines totalling 20 petaflops, NOAA’s capacity is 49 petaflops, and over 20 operational numerical weather prediction models run on the system.
NASA GISS makes the code of its ModelE Earth system model freely available, and its GISTEMP surface-temperature analysis is updated about the 10th of every month. Open code and data let anyone verify and validate the numerical methods behind climate science.
On Saving Planets: Solutions · Intro to Programming
Sources: NOAA: NOAA completes upgrade to weather and climate supercomputer system (Aug. 10, 2023) · NASA GISS: GISS Earth System Model, ModelE · NASA GISS: GISS Surface Temperature Analysis (GISTEMP v4)
Free learning resources
The lab (Computation lab)
The lab is the hands-on part of the course. Here it is free and for practice only, with no credit.
Every lab runs in a free simulation or a spreadsheet, so there are no physical hazards. Take screen breaks, sit with good posture, and keep cables out of walkways.
Safety reference: OSHA: Walking-working surfaces (slip, trip and fall hazards)
Lab A: Machine epsilon and Taylor series in a spreadsheet
Week 1
Goal: Find machine epsilon by experiment and watch a Taylor series converge to e.
Procedure
- In a column, start at 1 and halve each row.
- In the next column, test whether 1 + value > 1.
- The last value that passes is machine epsilon.
- Sum 1/k! for 5 and 10 terms and compare with e.
Data table
| Quantity | Value |
|---|---|
| Machine epsilon (double) | 2.22 × 10⁻¹⁶ |
| e, 5 terms | 2.70833 |
| e, 10 terms | 2.7182815 |
| e | 2.7182818 |
Computed values; most spreadsheets use double precision.
Sample calculation
- Halving 52 times gives 2⁻⁵² = 2.22 × 10⁻¹⁶.
- 1 + 1 + ½ + ⅙ + 1/24 = 2.70833.
Safety
- Computer only.
Home or virtual alternative
- Any spreadsheet works.
MIT OpenCourseWare 18.330 Introduction to Numerical Analysis (Spring 2012)
Planet link: NOAA describes a forecast run with dynamics in 32-bit and everything else in 64-bit precision. NOAA Environmental Modeling Center: Global Forecast System (GFS)
Lab B: Bisection versus Newton
Week 3
Goal: Solve cos x = x by bisection and by Newton’s method and count the correct digits each step.
Procedure
- Bracket the root in [0, 1].
- Run 20 bisection steps in a spreadsheet.
- Run Newton from x₀ = 1.
- Plot log₁₀ of the error against the step number for both.
Data table
| Newton step | x |
|---|---|
| 1 | 0.750364 |
| 2 | 0.739113 |
| 3 | 0.739085133 |
| 4 | 0.739085133 |
Computed values. Bisection needs about 20 steps for 6 digits; Newton needs 3.
Sample calculation
- x₁ = 1 − (cos 1 − 1)/(−sin 1 − 1) = 0.750364.
- Root 0.7390851.
Safety
- Computer only.
Home or virtual alternative
- Any spreadsheet works.
OpenStax Calculus Volume 1, 4.9 Newton’s method
Planet link: The USGS says MODFLOW-NWT uses a Newton-Raphson formulation for unconfined groundwater flow. USGS: MODFLOW and Related Programs
Lab C: Fitting the Mauna Loa CO₂ record
Week 7
Goal: Fit straight-line and quadratic models to NOAA’s Mauna Loa annual means and judge them by their residuals.
Procedure
- Explore PhET Least-Squares Regression and Curve Fitting to see how residuals change with the model.
- Open NOAA’s annual-mean file and copy the 1960–2020 decade values.
- Fit a line and a quadratic (spreadsheet LINEST or trendline).
- Predict 2025 and compare with NOAA’s 427.35 ppm.
Data table
| Model | Key coefficient | 2025 prediction (ppm) |
|---|---|---|
| Line | slope 1.613 ppm/yr | 415.0 |
| Quadratic | 0.01338 ppm/yr² | 426.0 |
| NOAA 2025 | — | 427.35 |
Data from NOAA GML; fits computed.
Sample calculation
- The line underpredicts 2025 by 12.3 ppm.
- The quadratic is within 1.3 ppm.
Safety
- Computer only.
Home or virtual alternative
- Any spreadsheet works.
PhET: Least-Squares Regression · PhET: Curve Fitting · NOAA GML: Mauna Loa annual mean CO₂ data
Planet link: NOAA reports the Mauna Loa monthly mean was 426.07 ppm in September 2026. NOAA Global Monitoring Laboratory: Trends in atmospheric CO₂ (Mauna Loa)
Lab D: Trapezoid versus Simpson
Week 9
Goal: Integrate sin x from 0 to π with both rules, halve the step and measure the error order.
Procedure
- Use PhET Calculus Grapher to see the area under a curve.
- Build columns for x, sin x and the rule weights.
- Compute T and S for n = 4, then n = 8.
- Find the error ratios.
Data table
| Rule, n = 4 | Estimate | Error |
|---|---|---|
| Trapezoidal | 1.89612 | −0.10388 |
| Simpson | 2.00456 | +0.00456 |
Computed values; doubling to n = 8 cuts the errors by 4.03 and 16.9, close to the theoretical 4 and 16.
Sample calculation
- T₄ = (π/4)(sin π/4 + sin π/2 + sin 3π/4) = 1.89612.
- S₄ = (π/12)(4 sin π/4 + 2 sin π/2 + 4 sin 3π/4) = 2.00456.
Safety
- Computer only.
Home or virtual alternative
- Any spreadsheet works.
PhET: Calculus Grapher · OpenStax Calculus Volume 2, 3.6 Numerical integration
Planet link: NOAA builds each Mauna Loa monthly mean from daily means based on hourly averages. NOAA Global Monitoring Laboratory: Trends in atmospheric CO₂ (Mauna Loa)
Lab E: Euler and RK4 error orders
Week 11
Goal: Solve y′ = −y to t = 1 with several step sizes and confirm first- and fourth-order convergence.
Procedure
- Build an Euler column for h = 0.2, 0.1 and 0.05.
- Build an RK4 block for h = 0.2 and 0.1.
- Record the error at t = 1.
- Compare the error ratios when h halves; then try the same solver on PhET Projectile Motion’s no-drag case.
Data table
| Method, h | Error at t = 1 |
|---|---|
| Euler, 0.2 | −0.0402 |
| Euler, 0.1 | −0.0192 |
| Euler, 0.05 | −0.00939 |
| RK4, 0.2 | 5.80 × 10⁻⁶ |
| RK4, 0.1 | 3.33 × 10⁻⁷ |
Computed values. Euler’s error halves with h; RK4’s falls by about 17 (near 2⁴ = 16).
Sample calculation
- Euler h = 0.1: 0.9¹⁰ − e⁻¹ = −0.0192.
- RK4 ratio 5.80 × 10⁻⁶/3.33 × 10⁻⁷ = 17.4.
Safety
- Computer only.
Home or virtual alternative
- Any spreadsheet works.
PhET: Projectile Motion · MIT OpenCourseWare 10.34 Numerical Methods Applied to Chemical Engineering (Fall 2015)
Planet link: NOAA’s GFS runs four times a day, forecasting up to 16 days ahead. NOAA Environmental Modeling Center: Global Forecast System (GFS)
Lab F: The heat equation and its stability limit
Week 13
Goal: Run the explicit heat scheme on a rod with a hot spot and see what happens above r = ½.
Procedure
- Set 7 nodes with ends at 0 and the middle node at 100 (sample values).
- Step with r = 0.25 for 10 steps.
- Repeat with r = 0.6.
- Describe both results.
Data table
| Case | Middle value after 1 step | After 10 steps |
|---|---|---|
| r = 0.25 | 50.0 | 16.7 |
| r = 0.6 | −20.0 | 290.5 |
Computed values. With r = 0.6 the values oscillate and grow: the scheme is unstable.
Sample calculation
- r = 0.25: 100 + 0.25(0 − 200 + 0) = 50.
- r = 0.6: 100 + 0.6(−200) = −20.
Safety
- Computer only.
Home or virtual alternative
- Any spreadsheet works.
MIT OpenCourseWare 2.086 Numerical Computation for Mechanical Engineers (Fall 2014)
Planet link: NOAA planned to move the GFS from 13 km to 9 km resolution after its 2023 upgrade. NOAA: NOAA completes upgrade to weather and climate supercomputer system (Aug. 10, 2023)
Exam prep practice sets
Time yourself: about 10 minutes per problem. Then compare with the full solutions.
Midterm practice set (Units 1–5)
Find the relative error of 3.14 as an approximation to π.
Show full solution
- |3.14 − π|/π.
Answer: 5.07 × 10⁻⁴.
Take two Newton steps on x² − 5 from x₀ = 2.
Show full solution
- x₁ = 2 − (−1)/4.
- x₂ = 2.25 − 0.0625/4.5.
Answer: x₁ = 2.25; x₂ = 2.23611.
How many bisection steps shrink a width-4 bracket below 10⁻³?
Show full solution
- log₂(4000) = 11.97.
Answer: 12 steps.
How much more elimination work is n = 500 than n = 100?
Show full solution
- (500/100)³.
Answer: 125 times.
Interpolate Mauna Loa CO₂ for 2023 from NOAA’s 2020 (414.21) and 2025 (427.35) annual means.
Show full solution
- 414.21 + 0.6 × 13.14.
Answer: 422.09 ppm (NOAA’s 2023 value is 421.08).
Final practice set (all units)
Estimate ∫₀² x² dx with the trapezoidal rule, n = 2.
Show full solution
- h = 1: 0.5 × 0 + 1 + 0.5 × 4.
Answer: 3 (exact 2.667).
Repeat with Simpson’s rule, n = 2.
Show full solution
- (1/3)(0 + 4 + 4).
Answer: 2.667, exact for a quadratic.
Take two Euler steps of h = 0.1 on y′ = −2y, y(0) = 1.
Show full solution
- 0.8².
Answer: 0.64 (exact 0.670).
Find the largest explicit heat time step for α = 2 × 10⁻⁵ m²/s, Δx = 0.005 m.
Show full solution
- 0.5 × 0.005²/2 × 10⁻⁵.
Answer: 0.625 s.
Estimate the derivative of eˣ at 0 by central difference, h = 0.01.
Show full solution
- (e^0.01 − e^−0.01)/0.02.
Answer: 1.0000167.
How much does golden-section search shrink a bracket in 20 steps?
Show full solution
- 0.618²⁰.
Answer: To 6.6 × 10⁻⁵ of its width.
Formula sheet
Constants
| Constant | Value |
|---|---|
| Double precision (IEEE 754) | machine epsilon 2⁻⁵² = 2.22 × 10⁻¹⁶, about 16 significant digits |
| Single precision (IEEE 754) | machine epsilon 2⁻²³ = 1.19 × 10⁻⁷, about 7 significant digits |
| e | 2.718281828 |
| π | 3.141592654 |
| Earth’s surface area (textbook value) | 5.1 × 10⁸ km² |
| 1 petaflop | 10¹⁵ floating-point operations per second |
Source: NIST Guide to the SI, Appendix B.9: conversion factors
Free texts and simulations
Use these free lectures, textbook sections and simulations alongside the course.
- MIT OpenCourseWare 18.330 Introduction to Numerical Analysis (Spring 2012): Free MIT numerical analysis lectures and problem sets.
- MIT OpenCourseWare 2.086 Numerical Computation for Mechanical Engineers (Fall 2014): Free MIT engineering computation course.
- MIT OpenCourseWare 10.34 Numerical Methods Applied to Chemical Engineering (Fall 2015): Free MIT applied numerical methods course.
- OpenStax Calculus Volume 2, 3.6 Numerical integration: Free textbook section.
- OpenStax Calculus Volume 1, 4.9 Newton’s method: Free textbook section.
- PhET: Least-Squares Regression: Fit lines interactively.
Glossary
- Absolute error
- Size of the difference from the true value.
- Bisection
- Root finding by halving a sign-change bracket.
- CFL condition
- Time-step limit cΔt/Δx ≤ 1.
- Catastrophic cancellation
- Digit loss when subtracting close numbers.
- Condition number
- How much a problem amplifies input errors.
- Crank–Nicolson
- Implicit second-order time scheme.
- Diagonal dominance
- Each diagonal entry outweighs the rest of its row.
- Gaussian quadrature
- Integration with optimal points and weights.
- Gauss–Seidel
- Iterative solver using new values at once.
- LU factorization
- A = LU, reusable elimination.
- Least squares
- Fit minimizing the sum of squared residuals.
- Machine epsilon
- Gap between 1 and the next float.
- Newton’s method
- Root finding along tangent lines.
- Power method
- Repeated multiplication to find the dominant eigenvalue.
- Runge–Kutta (RK4)
- Fourth-order ODE solver.
- Simpson’s rule
- Fourth-order integration with parabolas.
- Spline
- Smooth piecewise cubic curve.
- Stiff equation
- ODE with very fast and very slow parts.
- Truncation error
- Error from cutting off an infinite process.
- Validation
- Checking a model against observations.
Self-assessment rubric
No grade or credit is given. Use this to judge honestly where you are in each unit.
| Level | What it looks like |
|---|---|
| 4 · Mastery | Chooses a method for the accuracy needed, tests it on a problem with a known answer, measures its order, and explains its stability limits. |
| 3 · Proficient | Solves the practice problems with occasional slips in step size or units. |
| 2 · Developing | Follows worked examples but cannot tell truncation from rounding error. |
| 1 · Beginning | Runs formulas without checking error or stability; needs to revisit the unit. |
Suggested weighting if you want a score
| Part | Weight |
|---|---|
| Practice problems | 20% |
| Labs | 25% |
| Midterm practice set | 25% |
| Final practice set | 30% |
Aim for level 3 or better in every unit before moving to the next course.