Linear Algebra

Vectors, matrices and systems of equations: elimination, LU, inverses, input–output models, the four fundamental subspaces, projections and least squares, determinants, eigenvalues, Markov chains and the SVD, with six labs. Free, self-paced, and tied to GPS, Landsat, sea-level satellites, weather models, recycling and mineral mapping.

NASA/Kim Shiflett (KSC-20210927-PH-KLS04_0013, Sept. 27, 2021)NASA/JPL-Caltech (PIA22431)CNES (PIA25623)NASA Blue Marble

Courses › MATH 230

Download the PDF study guide

About this course

This is a free self-study course outline written by Saving Planets. It is not a university course, it carries no academic credit, and no school is involved.

Its scope follows a one-semester sophomore linear algebra course. It is modeled on the topics and depth of MIT 18.06 Linear Algebra taught by Gilbert Strang (free on MIT OpenCourseWare), and on the Georgia Tech and Purdue sophomore linear algebra courses, using the free Georgia Tech text Interactive Linear Algebra by Dan Margalit and Joseph Rabinoff and Jim Hefferon’s free Linear Algebra, which we link where it helps. None of those authors or schools endorses this page.

What we add: every unit ties to real planet work with a cited public source (GPS, Landsat 9, NASA sea-level satellites, NOAA weather supercomputers, EPA input–output and recycling data, USGS building studies, NASA’s EMIT mineral spectrometer), every number on the page is re-computed by a checking script before it goes live, and every lab runs in a free spreadsheet or on paper.

Work through it in order. Each unit has goals, key ideas, equations, step-by-step examples, practice with full answers, common mistakes, flashcards and a quiz. Your progress is saved only in this browser.

How this course helps save the planet

Linear algebra is the language of data about the planet. A satellite pixel is a vector, a GPS fix is the solution of a linear system, a weather forecast is millions of equations solved again and again, and an economy’s pollution can be traced with one matrix.

The same few ideas, elimination, projection, eigenvalues and the SVD, turn raw measurements into maps of forests, sea-level trends and mineral dust. Each unit ends with a short, sourced section. Values that are not from a source are labeled as sample values.

Back to top

Before you start

Prerequisites

  • High-school algebra: solving two equations in two unknowns, and working with functions.
  • Calculus I helps with limits and derivatives in a few examples, but no calculus is needed for most of the course.
  • Comfort with a spreadsheet; Intro to Programming helps for the optional Python versions of the labs.
  • Calculus III and Differential Equations pair well with this course but are not required.

How to study

  • Do every computation by hand first on small (2 × 2 and 3 × 3) examples, then check it with a spreadsheet or calculator.
  • Think of Ax two ways: as dot products of the rows with x, and as a combination of the columns of A.
  • Check every answer: plug x back into Ax = b, multiply A by its inverse, or test Av = λv.
  • Keep track of sizes: an m × n matrix times an n × p matrix gives m × p, and sizes that do not match mean an error.
  • Draw pictures in 2D and 3D: lines, planes, projections and stretches make the rules easy to remember.
  • Do every practice problem on paper before opening the answer, and retake the quizzes a few days later.

Lab equipment (for the wet labs)

  • A computer with free spreadsheet software (its MMULT, MINVERSE, MDETERM and LINEST functions do matrix work).
  • Graph paper, a ruler, a protractor and a pencil.
  • Optional: free Python with NumPy for the larger versions of the labs.
  • A string and push pins or tape for the trilateration lab, and a printed map or sheet of paper.
  • A grayscale photo or a small grid of numbers for the image-compression lab.

Back to top

Suggested 15-week schedule

About 6 to 9 hours a week: reading, practice problems and, in lab weeks, the lab.

WeekUnitTopicsReading
1Vectors: Linear Combinations, Lengths and Dot ProductsVectors, linear combinations, span, lengthMIT 18.06 L1; ILA 2.1–2.2
2Vectors: Linear Combinations, Lengths and Dot ProductsDot products, angles, orthogonality; Lab A; start eliminationILA 6.1
3Systems of Linear Equations and Gaussian EliminationElimination, echelon forms, solution sets; Lab BMIT 18.06 L2; ILA 1.1–1.3
4Matrix Algebra: Multiplication, Inverses and LU FactorizationMatrix multiplication, inverses, A = LU; Lab CMIT 18.06 L3–4; ILA 3.4–3.5
5Linear Models: Input–Output (Leontief) Economics, Networks and BalancingInput–output models, networks, balancingHefferon Topics; EPA USEEIO
6Vector Spaces, Subspaces, Independence, Basis, Rank and the Four Fundamental SubspacesSubspaces, column space, null space, Ax = 0MIT 18.06 L5–7; ILA 2.6
7Vector Spaces, Subspaces, Independence, Basis, Rank and the Four Fundamental SubspacesIndependence, basis, dimension, rank, four subspaces; midterm practiceMIT 18.06 L8–10; ILA 2.5, 2.7, 2.9
8Orthogonality, Projections, Least Squares and Gram–SchmidtOrthogonality and projectionsMIT 18.06 L14–15; ILA 6.3
9Orthogonality, Projections, Least Squares and Gram–SchmidtLeast squares, Gram–Schmidt, A = QR; Lab DMIT 18.06 L16–17; ILA 6.4–6.5
10Determinants: Properties, Cofactors, Cramer’s Rule and VolumeDeterminants, cofactors, Cramer’s rule, volumeMIT 18.06 L18–20; ILA 4.1–4.3
11Eigenvalues and EigenvectorsEigenvalues, eigenvectors, power iteration, modesMIT 18.06 L21; ILA 5.1–5.2, 5.5
12Diagonalization, Matrix Powers, Markov Chains and Linear Dynamical SystemsDiagonalization, Markov chains, dynamical systems; Lab EMIT 18.06 L22–25; ILA 5.4, 5.6
13The Singular Value Decomposition, Principal Component Analysis and Positive Definite MatricesPositive definite matrices and the SVDMIT 18.06 L27–29
14The Singular Value Decomposition, Principal Component Analysis and Positive Definite MatricesLow-rank approximation, PCA, condition number; Lab FMIT 18.06 L29; Strang book site
15ReviewReview and final practice setAll units

Back to top

Your progress

Tick units and labs as you finish them, and take the quizzes. Progress and quiz scores are saved only in this browser, never sent to us.

Turn on JavaScript to track progress.

Back to top

Unit 1 · Weeks 1–2

Vectors: Linear Combinations, Lengths and Dot Products

Vectors in two, three and n dimensions, adding and scaling them, linear combinations and span, length, unit vectors, the dot product, the angle between vectors, orthogonality and the Cauchy–Schwarz and triangle inequalities.

Learning objectives

By the end of this unit you can:

  • Add, scale and combine vectors by hand and in pictures.
  • Compute lengths, unit vectors and dot products.
  • Find the angle between two vectors and test for orthogonality.
  • Describe the span of one, two or three vectors.

Key concepts

  • A vector in ℝⁿ is an ordered list of n numbers. In 2D and 3D it is an arrow; in higher dimensions it is a list, such as the 11 band values of one satellite pixel.
  • A linear combination c₁v₁ + c₂v₂ + … scales vectors and adds them. The span is the set of all linear combinations: a line, a plane or all of space.
  • Length ‖v‖ = √(v·v). Dividing by the length gives a unit vector u = v/‖v‖ with length 1.
  • Dot product v·w = v₁w₁ + v₂w₂ + … = ‖v‖‖w‖cos θ. If v·w = 0 the vectors are orthogonal (perpendicular).
  • Cauchy–Schwarz: |v·w| ≤ ‖v‖‖w‖. Triangle inequality: ‖v + w‖ ≤ ‖v‖ + ‖w‖.

Key equations

∥v∥=v12+v22+⋯+vn2\|v\|=\sqrt{v_1^2+v_2^2+\cdots+v_n^2}
Length
v⋅w=v1w1+⋯+vnwnv\cdot w=v_1w_1+\cdots+v_nw_n
Dot product
cos⁡θ=v⋅w∥v∥ ∥w∥\cos\theta=\frac{v\cdot w}{\|v\|\,\|w\|}
Angle between vectors
∣v⋅w∣≤∥v∥ ∥w∥|v\cdot w|\le\|v\|\,\|w\|
Cauchy–Schwarz inequality

Worked examples, step by step

Example 1.1 Find the length of v = (3, 4) and the unit vector in its direction.

Show the steps
  1. ‖v‖ = √(3² + 4²) = √25 = 5.
  2. u = v/5.

Answer: ‖v‖ = 5, u = (0.6, 0.8).

Example 1.2 Find the angle between v = (3, 4) and w = (1, 2).

Show the steps
  1. v·w = 3 + 8 = 11.
  2. ‖v‖ = 5, ‖w‖ = √5 = 2.236.
  3. cos θ = 11/(5 × 2.236) = 0.9839.

Answer: θ = 10.30°.

Example 1.3 Write 2(1, 0, 1) + 3(0, 1, 1) as one vector. Is (2, 3, 4) in the span of these two vectors?

Show the steps
  1. 2(1, 0, 1) + 3(0, 1, 1) = (2, 3, 5).
  2. Any combination c₁(1, 0, 1) + c₂(0, 1, 1) = (c₁, c₂, c₁ + c₂), so the third entry must equal the sum of the first two.

Answer: (2, 3, 5); (2, 3, 4) is not in the span because 2 + 3 ≠ 4.

Practice problems

Work each on paper first, then check.

  1. Find ‖(1, 2, 2)‖.

    Show answer

    √(1 + 4 + 4) = 3.

  2. Find (1, 2, 3)·(4, −5, 6).

    Show answer

    4 − 10 + 18 = 12.

  3. Are (2, −1) and (1, 2) orthogonal?

    Show answer

    Yes: 2 − 2 = 0.

  4. Find the unit vector in the direction of (6, 8).

    Show answer

    ‖(6, 8)‖ = 10, so (0.6, 0.8).

Common misconceptions

  • Myth: The dot product is a vector.

    Actually: It is a single number (a scalar).

  • Myth: ‖v + w‖ = ‖v‖ + ‖w‖ always.

    Actually: Only when v and w point the same way; otherwise it is smaller.

  • Myth: Vectors only make sense in 2D or 3D.

    Actually: Data vectors can have hundreds of entries; the same rules apply.

Flashcards

  • Linear combination

    c₁v₁ + c₂v₂ + …

  • Span

    All linear combinations of a set of vectors.

  • Length

    ‖v‖ = √(v·v).

  • Dot product

    v·w = Σ vᵢwᵢ = ‖v‖‖w‖cos θ.

  • Orthogonal

    v·w = 0.

  • Unit vector

    v/‖v‖.

Self-check quiz

1. ‖(3, 4)‖ equals…

Show answer

5. √(9 + 16) = 5.

2. v·w = 0 means the vectors are…

Show answer

orthogonal. cos θ = 0, so θ = 90°.

3. The span of two non-parallel vectors in ℝ³ is…

Show answer

a plane. Two independent directions fill a plane.

4. Doubling v changes ‖v‖ by a factor of…

Show answer

2. ‖cv‖ = |c|‖v‖.

How this helps save the planet

Every Landsat 9 pixel is a vector. According to USGS, Landsat 9’s Operational Land Imager 2 records 9 spectral bands at 30 m (15 m for the panchromatic band) and its Thermal Infrared Sensor 2 records 2 thermal bands at 100 m, so each spot on the ground becomes a list of 11 numbers. The data are 14-bit, 2¹⁴ = 16,384 shades per band, and Landsat 9 collects about 750 scenes a day, all free.

Forest and crop health is read from the red band (band 4, 0.64–0.67 µm) and the near-infrared band (band 5, 0.85–0.88 µm). The vegetation index NDVI = (NIR − red)/(NIR + red) compares two entries of the pixel vector. With sample reflectances of 0.40 (NIR) and 0.08 (red), NDVI = 0.32/0.48 = 0.667, typical of healthy vegetation; bare soil gives values near zero.

On Saving Planets: Earth Observation · Deforestation

Sources: USGS: Landsat 9 · Interactive Linear Algebra, 2.1 Vectors

Free learning resources

Back to top

Unit 2 · Weeks 2–3

Systems of Linear Equations and Gaussian Elimination

Linear systems in row and column pictures, augmented matrices, elimination to echelon form, back substitution, reduced row echelon form, pivots and free variables, and when a system has one, none or infinitely many solutions.

Learning objectives

By the end of this unit you can:

  • Write a system as Ax = b and as an augmented matrix.
  • Solve systems by elimination and back substitution.
  • Find the reduced row echelon form and identify pivots and free variables.
  • Decide whether a system has one, no or infinitely many solutions.

Key concepts

  • Row picture: each equation is a line (2D) or plane (3D) and the solution is where they meet. Column picture: b is a combination of the columns of A, with weights x.
  • Three row operations do not change the solutions: swap rows, multiply a row by a nonzero number, and add a multiple of one row to another.
  • Elimination produces an upper-triangular (echelon) form; back substitution then solves from the bottom up. Pivots are the first nonzero entries of each row.
  • A column without a pivot gives a free variable and infinitely many solutions. A row 0 = c with c ≠ 0 means no solution (inconsistent).
  • Squared-distance equations like (x − a)² + (y − b)² = d² are not linear, but subtracting one from another cancels x² and y² and leaves linear equations. That is how positioning problems become linear systems.

Key equations

Ax=bA\mathbf{x}=\mathbf{b}
Matrix form
Ri←Ri−ℓijRj,  ℓij=aij/ajjR_i\leftarrow R_i-\ell_{ij}R_j,\ \ \ell_{ij}=a_{ij}/a_{jj}
Elimination step and multiplier
solutions: one, none, or infinitely many\text{solutions: one, none, or infinitely many}
The only three cases
d=c td=c\,t
Range from signal travel time (GPS)

Worked examples, step by step

Example 2.1 Solve x + y + z = 6, 2x + y − z = 1, x − y + 2z = 5.

Show the steps
  1. R₂ − 2R₁: −y − 3z = −11. R₃ − R₁: −2y + z = −1.
  2. R₃ − 2R₂: 7z = 21, so z = 3.
  3. Back substitute: −y − 9 = −11 gives y = 2; then x = 6 − 2 − 3 = 1.

Answer: (x, y, z) = (1, 2, 3).

Example 2.2 Beacons sit at (0, 0), (10, 0) and (0, 10) km. A receiver measures squared distances 25, 65 and 45 km². Find its position.

Show the steps
  1. Equation 1 minus equation 2: x² + y² − [(x − 10)² + y²] = 25 − 65, so 20x − 100 = −40.
  2. Equation 1 minus equation 3: 20y − 100 = −20.
  3. Solve the two linear equations.

Answer: (x, y) = (3, 4) km.

Example 2.3 How many solutions do x + 2y = 3, 2x + 4y = 6 have? And x + 2y = 3, 2x + 4y = 7?

Show the steps
  1. R₂ − 2R₁ gives 0 = 0 in the first system: one pivot, one free variable.
  2. In the second, R₂ − 2R₁ gives 0 = 1.

Answer: Infinitely many; none.

Practice problems

Work each on paper first, then check.

  1. Solve x + y = 5, x − y = 1.

    Show answer

    Adding gives 2x = 6: x = 3, y = 2.

  2. A GPS-style signal takes 0.00505783 s to arrive. How far did it travel at 299,792,458 m/s?

    Show answer

    d = ct = 1,516,299 m, about 1,516 km.

  3. Find the reduced row echelon form of [[1, 2, 3], [2, 5, 8]] (augmented).

    Show answer

    R₂ − 2R₁ = [0, 1, 2]; R₁ − 2R₂ = [1, 0, −1]: x = −1, y = 2.

  4. A 3 × 5 system has 3 pivots. How many free variables?

    Show answer

    5 − 3 = 2.

Common misconceptions

  • Myth: Multiplying a row by 0 is a legal row operation.

    Actually: It destroys information; the multiplier must be nonzero.

  • Myth: A system with more unknowns than equations has no solution.

    Actually: If consistent, it has infinitely many; it can also be inconsistent.

  • Myth: A zero row means no solution.

    Actually: Only a row 0 = c with c ≠ 0 does; 0 = 0 just means a free variable.

Flashcards

  • Augmented matrix

    [A | b].

  • Pivot

    First nonzero entry in a row of echelon form.

  • Free variable

    Variable whose column has no pivot.

  • RREF

    Pivots 1, zeros above and below.

  • Inconsistent

    A row 0 = c, c ≠ 0.

  • Back substitution

    Solve the triangular system from the bottom up.

Self-check quiz

1. A row 0 0 0 | 5 in echelon form means…

Show answer

no solution. 0 = 5 is impossible.

2. Which is NOT a row operation?

Show answer

square a row. Squaring is not linear.

3. A pivot-free column gives…

Show answer

a free variable. Its variable can be chosen freely.

4. In the column picture, Ax = b asks whether b is…

Show answer

a combination of the columns of A. Ax = x₁a₁ + x₂a₂ + …

How this helps save the planet

GPS is a linear algebra machine. GPS.gov reports that at least 24 operational satellites are kept available 95% of the time and that 31 operational satellites have been flown for more than a decade, at about 20,200 km altitude, each circling Earth twice a day. They sit in six equally spaced orbital planes so that users can view at least four satellites from virtually any point on the planet.

GPS.gov’s trilateration activity explains the idea: distance = rate × time, with the rate equal to the speed of light, 299,792,458 m/s, and distances from at least 3 satellites fix a 3D position. A real receiver’s clock is also unknown, so in the standard textbook model the unknowns are x, y, z and the clock offset, four unknowns that need four satellites; differencing the range equations, as in the worked example, turns them into a linear system. A clock error of just one microsecond would put a range off by 299.8 m. Precise positions guide farm tractors that apply less fertilizer, ships on efficient routes and the survey stations that measure sea level and ice.

On Saving Planets: Earth Observation · Space

Sources: GPS.gov: Space segment · GPS.gov: Trilateration (NOAA classroom activity)

Free learning resources

Back to top

Unit 3 · Week 4

Matrix Algebra: Multiplication, Inverses and LU Factorization

Matrix–vector and matrix–matrix products, the rules (and non-rules) of matrix algebra, transposes, elimination matrices, the inverse and Gauss–Jordan, the 2 × 2 inverse formula, A = LU, and the cost of elimination.

Learning objectives

By the end of this unit you can:

  • Multiply matrices and check that sizes match.
  • Find inverses by the 2 × 2 formula and by Gauss–Jordan.
  • Factor A = LU and use it to solve systems.
  • Estimate the cost of elimination as n grows.

Key concepts

  • (AB)ᵢⱼ = row i of A dotted with column j of B. An m × n times an n × p matrix is m × p. In general AB ≠ BA.
  • The inverse A⁻¹ satisfies AA⁻¹ = A⁻¹A = I. It exists exactly when A has n pivots (det A ≠ 0). (AB)⁻¹ = B⁻¹A⁻¹ and (AB)ᵀ = BᵀAᵀ.
  • Gauss–Jordan: row-reduce [A | I] to [I | A⁻¹].
  • Elimination without row swaps records its multipliers in a lower-triangular L with ones on the diagonal, giving A = LU. Solve Lc = b, then Ux = c; with L and U saved, each new right side is cheap.
  • Elimination on an n × n matrix costs about n³/3 multiply–add operations; solving with saved L and U costs about n² per right side.

Key equations

(AB)ij=∑kaikbkj(AB)_{ij}=\sum_k a_{ik}b_{kj}
Matrix product
[abcd]−1=1ad−bc[d−b−ca]\begin{bmatrix}a&b\\c&d\end{bmatrix}^{-1}=\frac{1}{ad-bc}\begin{bmatrix}d&-b\\-c&a\end{bmatrix}
2 × 2 inverse
A=LUA=LU
LU factorization
cost≈n3/3\text{cost}\approx n^3/3
Elimination operation count

Worked examples, step by step

Example 3.1 Find AB and BA for A = [[1, 2], [3, 4]] and B = [[0, 1], [1, 0]].

Show the steps
  1. AB: row 1 · columns = (2, 1); row 2 = (4, 3).
  2. BA swaps the rows of A: [[3, 4], [1, 2]].

Answer: AB = [[2, 1], [4, 3]], BA = [[3, 4], [1, 2]], so AB ≠ BA.

Example 3.2 Invert A = [[4, 7], [2, 6]].

Show the steps
  1. ad − bc = 24 − 14 = 10.
  2. A⁻¹ = (1/10)[[6, −7], [−2, 4]].

Answer: A⁻¹ = [[0.6, −0.7], [−0.2, 0.4]].

Example 3.3 Factor A = [[2, 1], [6, 8]] as LU.

Show the steps
  1. Multiplier ℓ₂₁ = 6/2 = 3.
  2. R₂ − 3R₁ = [0, 5].

Answer: L = [[1, 0], [3, 1]], U = [[2, 1], [0, 5]].

Practice problems

Work each on paper first, then check.

  1. What size is a 3 × 2 matrix times a 2 × 4 matrix?

    Show answer

    3 × 4.

  2. Invert [[2, 1], [5, 3]].

    Show answer

    det = 1, inverse [[3, −1], [−5, 2]].

  3. About how many operations does elimination take for n = 1000?

    Show answer

    n³/3 ≈ 3.33 × 10⁸.

  4. If A and B are invertible, what is (AB)⁻¹?

    Show answer

    B⁻¹A⁻¹.

Common misconceptions

  • Myth: AB = BA.

    Actually: Matrix multiplication is usually not commutative.

  • Myth: (AB)⁻¹ = A⁻¹B⁻¹.

    Actually: The order reverses: B⁻¹A⁻¹.

  • Myth: Computers solve Ax = b by computing A⁻¹.

    Actually: They factor A = LU (or similar); forming A⁻¹ is slower and less accurate.

Flashcards

  • Identity

    AI = IA = A.

  • Inverse

    AA⁻¹ = I.

  • 2 × 2 inverse

    Swap a, d; negate b, c; divide by ad − bc.

  • Transpose rule

    (AB)ᵀ = BᵀAᵀ.

  • LU

    A = LU from elimination.

  • Cost

    About n³/3 for elimination.

Self-check quiz

1. A 2 × 3 matrix times a 3 × 5 matrix is…

Show answer

2 × 5. Outer sizes.

2. A matrix is invertible exactly when it has…

Show answer

n pivots. Full set of pivots.

3. Doubling n multiplies elimination cost by about…

Show answer

8. n³ grows 2³ = 8 times.

4. L in A = LU holds…

Show answer

the multipliers. Below-diagonal multipliers, ones on the diagonal.

How this helps save the planet

Weather and climate forecasts are, at their core, enormous linear-algebra jobs repeated many times a day. NOAA reported in August 2023 that its twin operational supercomputers, Dogwood in Manassas, Virginia and Cactus in Phoenix, Arizona, were upgraded by 20% to 14.5 petaflops each, so that together the forecast system can process 29 quadrillion calculations per second, and that with its research supercomputers its total capacity is 49 petaflops.

The n³/3 rule shows why speed matters. As a sample, a dense system with n = 100,000 unknowns needs about 3.33 × 10¹⁴ operations; at a sustained 14.5 × 10¹⁵ operations per second that is about 0.023 s, but doubling n multiplies the work by 8. Real models use sparse matrices and clever solvers, yet the same cubic growth is what NOAA says the extra power will pay for: higher-resolution models that improve forecasts and warnings.

On Saving Planets: Intro to Programming · Solutions

Sources: NOAA: NOAA completes upgrade to weather and climate supercomputer system (Aug. 10, 2023) · MIT OpenCourseWare 18.06 Linear Algebra (Strang, Spring 2010)

Free learning resources

Back to top

Unit 4 · Week 5

Linear Models: Input–Output (Leontief) Economics, Networks and Balancing

Building linear models from real systems: the Leontief input–output model x = Ax + d and its solution x = (I − A)⁻¹d, environmentally extended models that turn spending into emissions, network flows and balancing chemical equations.

Learning objectives

By the end of this unit you can:

  • Set up and solve a Leontief input–output model.
  • Interpret the entries of (I − A)⁻¹.
  • Extend an input–output model with emission intensities to find a footprint.
  • Write flow-balance and chemical-balance problems as linear systems.

Key concepts

  • In an input–output model, column j of the consumption matrix A lists what sector j buys from each sector to make one dollar of output.
  • Total output x must cover what industries use (Ax) plus final demand d: x = Ax + d, so (I − A)x = d and x = (I − A)⁻¹d.
  • Entry (i, j) of the Leontief inverse (I − A)⁻¹ is the total output of sector i needed, directly and through all supply chains, for one dollar of final demand from sector j.
  • Environmentally extended models multiply output by a row of emission intensities e (for example kg CO₂ per dollar): footprint = e·x.
  • Conservation laws give linear equations too: flow in equals flow out at every node of a network, and atoms of each element balance in a chemical reaction.

Key equations

x=Ax+d\mathbf{x}=A\mathbf{x}+\mathbf{d}
Input–output balance
x=(I−A)−1d\mathbf{x}=(I-A)^{-1}\mathbf{d}
Leontief solution
footprint=e⋅x\text{footprint}=\mathbf{e}\cdot\mathbf{x}
Environmentally extended model
∑in=∑out\sum \text{in}=\sum \text{out}
Flow balance at a node

Worked examples, step by step

Example 4.1 A two-sector economy has A = [[0.2, 0.3], [0.1, 0.4]] and final demand d = (100, 50) (sample values, $ million). Find the total output.

Show the steps
  1. I − A = [[0.8, −0.3], [−0.1, 0.6]], det = 0.48 − 0.03 = 0.45.
  2. (I − A)⁻¹ = (1/0.45)[[0.6, 0.3], [0.1, 0.8]].
  3. x = (1/0.45)(60 + 15, 10 + 40).

Answer: x = (166.7, 111.1) $ million.

Example 4.2 With sample emission intensities e = (0.5, 2.0) kg CO₂ per dollar, find the footprint of that final demand.

Show the steps
  1. footprint = 0.5 × 166.7 + 2.0 × 111.1 (in kg per $ × $ million).

Answer: 305.6 million kg = 305.6 kt CO₂ (sample).

Example 4.3 Balance x CH₄ + y O₂ → z CO₂ + w H₂O.

Show the steps
  1. C: x = z. H: 4x = 2w. O: 2y = 2z + w.
  2. Set x = 1: z = 1, w = 2, y = 2.

Answer: CH₄ + 2 O₂ → CO₂ + 2 H₂O.

Practice problems

Work each on paper first, then check.

  1. In the worked model, how much output of sector 1 does $1 of final demand for sector 2 require in total?

    Show answer

    Entry (1, 2) of (I − A)⁻¹ = 0.3/0.45 = 0.667.

  2. A single-sector economy uses 0.25 of its own output. What total output meets a demand of 300?

    Show answer

    x = 300/(1 − 0.25) = 400.

  3. A node has inflows 40 and 25 and one outflow f₁ plus an outflow of 30. Find f₁.

    Show answer

    f₁ = 65 − 30 = 35.

  4. Balance H₂ + O₂ → H₂O.

    Show answer

    2 H₂ + O₂ → 2 H₂O.

Common misconceptions

  • Myth: Total output equals final demand.

    Actually: Industries also consume each other’s output, so x is larger than d.

  • Myth: The footprint comes only from the sector you buy from.

    Actually: The Leontief inverse adds every upstream supplier.

  • Myth: Any A gives a sensible economy.

    Actually: Column sums must be below 1 for (I − A)⁻¹ to have nonnegative entries.

Flashcards

  • Consumption matrix

    Inputs per dollar of output.

  • Leontief model

    x = Ax + d.

  • Leontief inverse

    (I − A)⁻¹, total requirements.

  • Extended model

    footprint = e·x.

  • Flow balance

    In = out at each node.

  • Homogeneous system

    Ax = 0.

Self-check quiz

1. The Leontief solution is…

Show answer

x = (I − A)⁻¹d. From x = Ax + d.

2. Column j of A describes…

Show answer

what sector j buys per dollar of output. Inputs per unit of output.

3. A footprint in an extended model is…

Show answer

e·x. Intensity times output, summed.

4. Balancing a chemical equation is a…

Show answer

homogeneous linear system. Atom counts give Ax = 0.

How this helps save the planet

The U.S. EPA uses exactly this model to trace pollution through the economy. According to EPA, its USEEIO models combine data on economic transactions between 389 industry sectors with environmental data for those sectors, covering land, water, energy and mineral use, air pollution, nutrients and toxics, to evaluate the impacts of goods and services. EPA describes them as useful for carbon footprinting, and they support its sustainable materials management work.

A 389-sector consumption matrix has 389² = 151,321 entries, far too many for hand work but easy for the elimination and LU methods of Unit 3. The Leontief inverse is what lets a city, company or household see the emissions hidden in its supply chains, not just at the factory gate.

On Saving Planets: Solutions · General Chemistry I

Sources: U.S. EPA: US Environmentally-Extended Input-Output (USEEIO) models · Interactive Linear Algebra, 1.1 Systems of linear equations

Free learning resources

Back to top

Unit 5 · Weeks 6–7

Vector Spaces, Subspaces, Independence, Basis, Rank and the Four Fundamental Subspaces

Vector spaces and subspaces, the column space and null space, solving Ax = 0 and Ax = b completely, linear independence, basis and dimension, rank, the rank–nullity theorem and Strang’s four fundamental subspaces.

Learning objectives

By the end of this unit you can:

  • Test whether a set is a subspace.
  • Find bases for the column space and null space of a matrix.
  • Test vectors for linear independence.
  • Use rank–nullity and the dimensions of the four fundamental subspaces.

Key concepts

  • A subspace contains the zero vector and is closed under addition and scalar multiplication. Lines and planes through the origin are subspaces; ones that miss the origin are not.
  • Column space C(A): all Ax, the set of b for which Ax = b is solvable. Null space N(A): all x with Ax = 0.
  • Vectors are linearly independent if the only combination giving zero is the all-zero one. A basis is an independent spanning set; every basis of a space has the same number of vectors, its dimension.
  • Rank r = number of pivots = dim C(A). Rank–nullity: r + dim N(A) = n, the number of columns.
  • Four fundamental subspaces of an m × n matrix of rank r: C(A) (dim r) and N(Aᵀ) (dim m − r) in ℝᵐ; C(Aᵀ) (dim r) and N(A) (dim n − r) in ℝⁿ. N(A) is orthogonal to the row space.

Key equations

C(A)={Ax},  N(A)={x:Ax=0}C(A)=\{A\mathbf{x}\},\ \ N(A)=\{\mathbf{x}:A\mathbf{x}=0\}
Column space and null space
r+dim⁡N(A)=nr+\dim N(A)=n
Rank–nullity
x=xp+xn\mathbf{x}=\mathbf{x}_p+\mathbf{x}_n
Complete solution of Ax = b
dim⁡N(AT)=m−r\dim N(A^T)=m-r
Left null space

Worked examples, step by step

Example 5.1 Find the rank and a null-space basis of A = [[1, 2, 3], [2, 4, 6], [1, 0, 1]].

Show the steps
  1. R₂ − 2R₁ = 0; R₃ − R₁ = [0, −2, −2]. Two pivots, so r = 2.
  2. Ax = 0: x + z = 0 and −2y − 2z = 0, with z free.
  3. Take z = 1: x = −1, y = −1.

Answer: r = 2, N(A) spanned by (−1, −1, 1); dim N(A) = 3 − 2 = 1.

Example 5.2 Are (1, 2, 3), (4, 5, 6), (7, 8, 9) independent?

Show the steps
  1. Row-reduce the matrix with these columns: R₂ − 2R₁ and R₃ − 3R₁ give rows [0, −3, −6] and [0, −6, −12], which are multiples.
  2. Only two pivots.

Answer: No: (7, 8, 9) = 2(4, 5, 6) − (1, 2, 3).

Example 5.3 A 5 × 8 matrix has rank 3. Give the dimensions of its four fundamental subspaces.

Show the steps
  1. C(A) and C(Aᵀ): r = 3.
  2. N(A): n − r = 8 − 3; N(Aᵀ): m − r = 5 − 3.

Answer: 3, 3, 5 and 2.

Practice problems

Work each on paper first, then check.

  1. Is the line y = 2x + 1 a subspace of ℝ²?

    Show answer

    No: it does not contain (0, 0).

  2. A 4 × 4 matrix has a 2-dimensional null space. What is its rank?

    Show answer

    4 − 2 = 2.

  3. What is the dimension of the space of 2 × 2 matrices?

    Show answer

    4.

  4. Can 4 vectors in ℝ³ be independent?

    Show answer

    No: at most 3.

Common misconceptions

  • Myth: Any plane in ℝ³ is a subspace.

    Actually: Only planes through the origin.

  • Myth: Rank counts nonzero entries.

    Actually: It counts pivots (independent columns).

  • Myth: The null space lives in the same space as b.

    Actually: N(A) is in ℝⁿ (the x side); b is in ℝᵐ.

Flashcards

  • Subspace

    Contains 0; closed under + and scaling.

  • Column space

    All Ax.

  • Null space

    All x with Ax = 0.

  • Basis

    Independent spanning set.

  • Rank

    Number of pivots.

  • Rank–nullity

    r + dim N(A) = n.

Self-check quiz

1. Rank–nullity says r + dim N(A) equals…

Show answer

n. Number of columns.

2. Ax = b is solvable exactly when b is in…

Show answer

C(A). b must be a combination of the columns.

3. The null space is orthogonal to…

Show answer

the row space. Each row dotted with x is 0.

4. A basis of ℝ³ has how many vectors?

Show answer

3. Dimension 3.

How this helps save the planet

Dimension counts unknowns, and better forecasts need more of them. NOAA said its 2023 supercomputer upgrade would let the U.S. Global Forecast System improve its horizontal resolution from 13 km to 9 km, helping smaller-scale features be seen and simulated.

A grid with spacing h has about (1/h)² columns per unit area, so going from 13 km to 9 km multiplies the number of grid columns by (13/9)² = 2.09. Every one of those columns carries temperature, wind, humidity and pressure at many levels, so the state vector of a forecast lives in a space with a vastly higher dimension, and the matrices that update it grow to match.

On Saving Planets: Calculus III · Differential Equations

Sources: NOAA: NOAA completes upgrade to weather and climate supercomputer system (Aug. 10, 2023) · Interactive Linear Algebra, 2.7 Basis and dimension

Free learning resources

Back to top

Unit 6 · Weeks 8–9

Orthogonality, Projections, Least Squares and Gram–Schmidt

Orthogonal vectors and subspaces, projection onto a line and onto a subspace, the normal equations AᵀAx̂ = Aᵀb, fitting lines and curves to data by least squares, orthonormal bases, Gram–Schmidt and A = QR.

Learning objectives

By the end of this unit you can:

  • Project a vector onto a line and onto a column space.
  • Fit a line to data with the normal equations.
  • Build an orthonormal basis with Gram–Schmidt.
  • Explain why least squares is a projection.

Key concepts

  • The projection of b onto the line through a is p = (a·b/a·a) a; the error e = b − p is orthogonal to a.
  • When Ax = b has no solution (more equations than unknowns, noisy data), choose x̂ to make ‖b − Ax̂‖ as small as possible. The error must be orthogonal to C(A), giving the normal equations AᵀAx̂ = Aᵀb.
  • For a line y = C + Dt through points (tᵢ, yᵢ), the columns of A are all ones and the tᵢ values.
  • Orthonormal vectors qᵢ satisfy qᵢ·qⱼ = 0 (i ≠ j) and ‖qᵢ‖ = 1. With Q having orthonormal columns, QᵀQ = I and projections become p = QQᵀb.
  • Gram–Schmidt subtracts from each new vector its projections on the earlier ones, then normalizes. The result is A = QR with R upper triangular.

Key equations

p=a⋅ba⋅a a\mathbf{p}=\frac{\mathbf{a}\cdot\mathbf{b}}{\mathbf{a}\cdot\mathbf{a}}\,\mathbf{a}
Projection onto a line
ATAx^=ATbA^TA\hat{\mathbf{x}}=A^T\mathbf{b}
Normal equations
P=A(ATA)−1ATP=A(A^TA)^{-1}A^T
Projection matrix
A=QRA=QR
Gram–Schmidt factorization

Worked examples, step by step

Example 6.1 Project b = (2, 3) onto a = (1, 1).

Show the steps
  1. a·b = 5, a·a = 2.
  2. p = 2.5(1, 1); e = b − p = (−0.5, 0.5), and e·a = 0.

Answer: p = (2.5, 2.5).

Example 6.2 Fit y = C + Dt to (0, 1), (1, 2), (2, 4) by least squares.

Show the steps
  1. A has rows (1, 0), (1, 1), (1, 2); AᵀA = [[3, 3], [3, 5]], Aᵀb = (7, 10).
  2. det = 6; C = (5 × 7 − 3 × 10)/6, D = (3 × 10 − 3 × 7)/6.

Answer: C = 0.833, D = 1.5: y = 0.833 + 1.5t.

Example 6.3 Apply Gram–Schmidt to a₁ = (1, 1, 0), a₂ = (1, 0, 1).

Show the steps
  1. q₁ = (1, 1, 0)/√2.
  2. a₂ − (a₂·q₁)q₁ = (1, 0, 1) − ½(1, 1, 0) = (0.5, −0.5, 1), length √1.5 = 1.225.

Answer: q₁ = (0.707, 0.707, 0), q₂ = (0.408, −0.408, 0.816).

Practice problems

Work each on paper first, then check.

  1. Project (3, 4) onto (1, 0).

    Show answer

    (3, 0).

  2. Write the normal equations for fitting y = Dt (no intercept) to (1, 2), (2, 3).

    Show answer

    Σt² D = Σty: 5D = 8, D = 1.6.

  3. Is Q = [[0, 1], [1, 0]] orthogonal?

    Show answer

    Yes: QᵀQ = I.

  4. If b is already in C(A), what is the least-squares error?

    Show answer

    Zero: p = b.

Common misconceptions

  • Myth: Least squares solves Ax = b exactly.

    Actually: It finds the closest Ax̂ when no exact solution exists.

  • Myth: Any basis makes projections easy.

    Actually: Orthonormal bases do; otherwise you need (AᵀA)⁻¹.

  • Myth: A straight-line fit means the data are linear.

    Actually: Check the residuals; a curve may fit better (add a t² column).

Flashcards

  • Projection on a line

    p = (a·b/a·a)a.

  • Normal equations

    AᵀAx̂ = Aᵀb.

  • Projection matrix

    P = A(AᵀA)⁻¹Aᵀ.

  • Orthonormal

    qᵢ·qⱼ = 0, ‖qᵢ‖ = 1.

  • Gram–Schmidt

    Subtract projections, normalize.

  • QR

    A = QR.

Self-check quiz

1. The normal equations are…

Show answer

AᵀAx̂ = Aᵀb. Error orthogonal to C(A).

2. The least-squares error vector is orthogonal to…

Show answer

the column space. That is what makes it shortest.

3. For Q with orthonormal columns, QᵀQ =…

Show answer

I. Dot products 0 or 1.

4. Gram–Schmidt produces…

Show answer

A = QR. Orthonormal Q, triangular R.

How this helps save the planet

Sea-level rise is measured by fitting curves to noisy satellite data. NASA reports that satellite altimeters, TOPEX/Poseidon, Jason-1, -2 and -3 and now Sentinel-6 Michael Freilich, have tracked global mean sea level since 1993, that it has risen 114.4 mm since then, and that the rate of rise has increased from 0.08 inches per year (0.20 cm per year) in 1993 to 0.17 inches per year (0.44 cm per year) now, about 2.2 times faster.

A straight-line least-squares fit gives an average rate; adding a t² column to A fits a curve whose changing slope measures the acceleration NASA describes. Lab D fits sample data shaped like the NASA record with exactly the normal equations from this unit.

On Saving Planets: Oceans · Earth Observation

Sources: NASA: Global mean sea level (Vital Signs) · Interactive Linear Algebra, 6.5 The method of least squares

Free learning resources

Back to top

Unit 7 · Week 10

Determinants: Properties, Cofactors, Cramer’s Rule and Volume

The determinant from its properties, computing it by elimination and by cofactor expansion, det(AB) = det A det B, the test for invertibility, Cramer’s rule, and the determinant as the area or volume scale factor of a linear map.

Learning objectives

By the end of this unit you can:

  • Compute 2 × 2 and 3 × 3 determinants by cofactors and by elimination.
  • Use the properties of determinants under row operations.
  • Solve small systems by Cramer’s rule.
  • Interpret |det A| as an area or volume scale factor.

Key concepts

  • det A ≠ 0 exactly when A is invertible. The determinant of a triangular matrix is the product of its diagonal entries, so det A = ± (product of pivots).
  • Row operations: a swap flips the sign, scaling a row by c scales det by c, and adding a multiple of one row to another leaves det unchanged.
  • det(AB) = det A · det B, det(Aᵀ) = det A and det(A⁻¹) = 1/det A.
  • Cofactor expansion along a row: det A = Σ aᵢⱼCᵢⱼ with Cᵢⱼ = (−1)ⁱ⁺ʲ times the minor. Cramer’s rule: xᵢ = det(Bᵢ)/det A, where Bᵢ has b in column i.
  • A linear map x ↦ Ax multiplies areas (2D) or volumes (3D) by |det A|; a negative determinant also flips orientation.

Key equations

det⁡[abcd]=ad−bc\det\begin{bmatrix}a&b\\c&d\end{bmatrix}=ad-bc
2 × 2 determinant
det⁡(AB)=det⁡A det⁡B\det(AB)=\det A\,\det B
Product rule
xi=det⁡Bidet⁡Ax_i=\frac{\det B_i}{\det A}
Cramer’s rule
area(AS)=∣det⁡A∣ area(S)\text{area}(AS)=|\det A|\,\text{area}(S)
Area scale factor

Worked examples, step by step

Example 7.1 Find det [[1, 2, 0], [0, 1, 3], [2, 0, 1]] by cofactors along row 1.

Show the steps
  1. 1 × (1 × 1 − 3 × 0) = 1.
  2. − 2 × (0 × 1 − 3 × 2) = −2 × (−6) = 12.
  3. + 0 × (…) = 0.

Answer: det = 13.

Example 7.2 Solve 2x + y = 5, x + 3y = 10 by Cramer’s rule.

Show the steps
  1. det A = 6 − 1 = 5.
  2. x = det[[5, 1], [10, 3]]/5 = (15 − 10)/5.
  3. y = det[[2, 5], [1, 10]]/5 = (20 − 5)/5.

Answer: x = 1, y = 3.

Example 7.3 Find the area of the parallelogram with sides (3, 0) and (1, 2).

Show the steps
  1. Area = |det [[3, 1], [0, 2]]| = |6 − 0|.

Answer: 6 square units.

Practice problems

Work each on paper first, then check.

  1. det [[3, 1], [2, 4]]?

    Show answer

    12 − 2 = 10.

  2. det [[2, 0, 1], [1, 3, 2], [1, 1, 1]]?

    Show answer

    2(3 − 2) − 0 + 1(1 − 3) = 0, so it is singular.

  3. If det A = 4 for a 3 × 3 A, what is det(2A)?

    Show answer

    2³ × 4 = 32.

  4. If det A = 5, what is det(A⁻¹)?

    Show answer

    1/5 = 0.2.

Common misconceptions

  • Myth: det(A + B) = det A + det B.

    Actually: False in general; only det(AB) splits.

  • Myth: det(2A) = 2 det A.

    Actually: For n × n, det(2A) = 2ⁿ det A.

  • Myth: Cofactor expansion is the fast way.

    Actually: It takes about n! steps; elimination takes about n³/3.

Flashcards

  • 2 × 2 det

    ad − bc.

  • Invertibility

    det A ≠ 0.

  • Product rule

    det(AB) = det A det B.

  • Row swap

    Changes the sign.

  • Cramer’s rule

    xᵢ = det Bᵢ/det A.

  • Area factor

    |det A|.

Self-check quiz

1. Swapping two rows multiplies det by…

Show answer

−1. Sign flips.

2. det A = 0 means A is…

Show answer

singular. Not invertible.

3. det of a triangular matrix is…

Show answer

the product of the diagonal. Expand down the diagonal.

4. A map with det = 3 changes areas by a factor of…

Show answer

3. |det A|.

How this helps save the planet

Turning pixels into land area is a determinant. USGS lists Landsat 9’s multispectral bands at 30 m and its panchromatic band at 15 m. The map from pixel indices (i, j) to ground coordinates is, locally, a matrix; for an aligned 30 m grid it is diag(30, 30) m, whose determinant is 900, so each pixel covers 900 m². The 15 m panchromatic grid has determinant 225 m², so four panchromatic pixels fit in one multispectral pixel.

Counting pixels classified as forest and multiplying by this scale factor is how satellite records measure forest loss in hectares (1 ha = 10,000 m², about 11.1 Landsat pixels). When an image is rotated or sheared to fit a map, the determinant of the new matrix keeps the area right.

On Saving Planets: Deforestation · Earth Observation

Sources: USGS: Landsat 9 · Interactive Linear Algebra, 4.3 Determinants and volumes

Free learning resources

Back to top

Unit 8 · Week 11

Eigenvalues and Eigenvectors

Eigenvectors as directions a matrix only stretches, the characteristic equation det(A − λI) = 0, trace and determinant checks, eigenvalues of special matrices, complex eigenvalues and rotation, power iteration, and natural modes of vibrating structures.

Learning objectives

By the end of this unit you can:

  • Find eigenvalues from det(A − λI) = 0 and eigenvectors from the null space of A − λI.
  • Check eigenvalues with the trace and determinant.
  • Recognize real, repeated and complex eigenvalues.
  • Use power iteration to find the dominant eigenvalue.

Key concepts

  • Av = λv with v ≠ 0: the eigenvector v keeps its direction and is scaled by the eigenvalue λ.
  • The eigenvalues are the roots of det(A − λI) = 0, a polynomial of degree n. For each λ, the eigenvectors fill the null space of A − λI.
  • Checks: the eigenvalues sum to the trace (sum of the diagonal) and multiply to det A.
  • Symmetric matrices have real eigenvalues and orthogonal eigenvectors. Rotation matrices have complex eigenvalues. Triangular matrices show their eigenvalues on the diagonal.
  • Power iteration: repeatedly multiply a starting vector by A and rescale; it lines up with the eigenvector of the largest |λ|. For vibration, Kv = ω²Mv gives the natural frequencies ω and mode shapes v.

Key equations

Av=λvA\mathbf{v}=\lambda\mathbf{v}
Eigenvalue equation
det⁡(A−λI)=0\det(A-\lambda I)=0
Characteristic equation
λ1+⋯+λn=tr⁡A,  λ1⋯λn=det⁡A\lambda_1+\cdots+\lambda_n=\operatorname{tr}A,\ \ \lambda_1\cdots\lambda_n=\det A
Trace and determinant checks
Kv=ω2MvK\mathbf{v}=\omega^2M\mathbf{v}
Natural modes of a structure

Worked examples, step by step

Example 8.1 Find the eigenvalues and eigenvectors of A = [[2, 1], [1, 2]].

Show the steps
  1. det(A − λI) = (2 − λ)² − 1 = λ² − 4λ + 3 = 0.
  2. λ = 3: A − 3I = [[−1, 1], [1, −1]] gives v = (1, 1). λ = 1: v = (1, −1).
  3. Check: 3 + 1 = 4 = trace; 3 × 1 = 3 = det.

Answer: λ = 3 with (1, 1); λ = 1 with (1, −1).

Example 8.2 Find the eigenvalues of [[4, 1], [2, 3]].

Show the steps
  1. λ² − 7λ + 10 = 0 (trace 7, det 10).
  2. (λ − 5)(λ − 2) = 0.

Answer: λ = 5 and λ = 2.

Example 8.3 A two-story sample building has equal floor masses m and story stiffness k, with K/m = (k/m)[[2, −1], [−1, 1]] and k/m = 100 s⁻². Find the fundamental frequency.

Show the steps
  1. Eigenvalues of [[2, −1], [−1, 1]]: λ = (3 ± √5)/2 = 0.382 and 2.618.
  2. ω₁² = 100 × 0.382 = 38.2 s⁻², ω₁ = 6.18 rad/s.
  3. f₁ = ω₁/(2π), T₁ = 1/f₁.

Answer: f₁ = 0.984 Hz, period T₁ = 1.02 s (sample values).

Practice problems

Work each on paper first, then check.

  1. Eigenvalues of [[5, 0], [0, −2]]?

    Show answer

    5 and −2 (diagonal).

  2. A 3 × 3 matrix has eigenvalues 1, 2, 3. Find its trace and determinant.

    Show answer

    Trace 6, det 6.

  3. Eigenvalues of the 90° rotation [[0, −1], [1, 0]]?

    Show answer

    λ² + 1 = 0: λ = ±i.

  4. If Av = 4v, what is A²v?

    Show answer

    16v.

Common misconceptions

  • Myth: The zero vector is an eigenvector.

    Actually: Eigenvectors must be nonzero.

  • Myth: Row reduction keeps the eigenvalues.

    Actually: It changes them; use det(A − λI).

  • Myth: Every real matrix has real eigenvalues.

    Actually: Rotations give complex ones; symmetric matrices are the safe case.

Flashcards

  • Eigenvector

    Av = λv, v ≠ 0.

  • Characteristic equation

    det(A − λI) = 0.

  • Trace check

    Σλ = trace.

  • Determinant check

    Πλ = det A.

  • Power iteration

    Repeated multiplication finds the top eigenvector.

  • Natural modes

    Kv = ω²Mv.

Self-check quiz

1. The eigenvalues of a triangular matrix are…

Show answer

its diagonal entries. det(A − λI) is a product of diagonal terms.

2. The sum of the eigenvalues equals…

Show answer

the trace. Trace check.

3. Power iteration finds…

Show answer

the dominant eigenvalue. Largest |λ| wins.

4. Symmetric matrices have…

Show answer

real eigenvalues and orthogonal eigenvectors. Spectral theorem.

How this helps save the planet

Earthquake engineers read buildings through their eigenvalues. A USGS study by Mehmet Çelebi (2022) used the fundamental periods and critical damping percentages of 41 tall buildings in several countries. It found a clear linear trend between building height and fundamental period for steel and reinforced-concrete buildings, though with a large scatter from differing structural designs, and that critical damping was clearly lower than 3%, consistent with other recent tall-building studies.

The fundamental period is set by the smallest eigenvalue of the stiffness–mass problem, as in the two-story sample example. Low damping means a tall building can ring for a long time when long-period earthquake waves match that period, which is why instrumented buildings and their measured modes guide safer, longer-lasting cities.

On Saving Planets: Differential Equations · Dynamics

Sources: USGS: Preliminary study of correlation of natural periods and damping percentages of tall buildings in several countries (Çelebi, 2022) · Interactive Linear Algebra, 5.1 Eigenvalues and eigenvectors

Free learning resources

Back to top

Unit 9 · Week 12

Diagonalization, Matrix Powers, Markov Chains and Linear Dynamical Systems

Diagonalizing A = PDP⁻¹, computing powers Aᵏ quickly, symmetric matrices and the spectral theorem A = QΛQᵀ, Markov (stochastic) matrices and steady states, and the long-run behavior of xₖ₊₁ = Axₖ.

Learning objectives

By the end of this unit you can:

  • Diagonalize a matrix with n independent eigenvectors.
  • Compute Aᵏ and predict the long-run behavior of xₖ₊₁ = Axₖ.
  • Find the steady state of a Markov chain.
  • Use the spectral theorem for symmetric matrices.

Key concepts

  • If A has n independent eigenvectors (the columns of P), then A = PDP⁻¹ with the eigenvalues on the diagonal of D, and Aᵏ = PDᵏP⁻¹.
  • Write x₀ = c₁v₁ + … + cₙvₙ. Then xₖ = Aᵏx₀ = c₁λ₁ᵏv₁ + … + cₙλₙᵏvₙ; the largest |λ| dominates as k grows.
  • A Markov matrix has nonnegative entries and columns that sum to 1. It always has λ = 1, and the other eigenvalues satisfy |λ| ≤ 1. The steady state is the eigenvector for λ = 1, scaled so its entries sum to 1.
  • The second-largest |λ| sets how fast a Markov chain forgets its start: the gap shrinks like |λ₂|ᵏ.
  • Spectral theorem: a real symmetric matrix can be written A = QΛQᵀ with orthonormal eigenvectors in Q.

Key equations

A=PDP−1,  Ak=PDkP−1A=PDP^{-1},\ \ A^k=PD^kP^{-1}
Diagonalization and powers
xk=c1λ1kv1+⋯+cnλnkvn\mathbf{x}_k=c_1\lambda_1^k\mathbf{v}_1+\cdots+c_n\lambda_n^k\mathbf{v}_n
Solution of x_{k+1} = Ax_k
Pq=q,  ∑qi=1P\mathbf{q}=\mathbf{q},\ \ \textstyle\sum q_i=1
Markov steady state
A=QΛQTA=Q\Lambda Q^T
Spectral theorem

Worked examples, step by step

Example 9.1 Use diagonalization to find A¹⁰ for A = [[2, 1], [1, 2]].

Show the steps
  1. From Unit 8: λ = 3, 1 with v = (1, 1), (1, −1). P = [[1, 1], [1, −1]], P⁻¹ = ½[[1, 1], [1, −1]].
  2. A¹⁰ = P diag(3¹⁰, 1) P⁻¹; each entry is (3¹⁰ ± 1)/2.

Answer: A¹⁰ = [[29525, 29524], [29524, 29525]].

Example 9.2 A Markov matrix is P = [[0.8, 0.3], [0.2, 0.7]] (sample values). Find the steady state.

Show the steps
  1. (P − I)q = 0: −0.2q₁ + 0.3q₂ = 0, so q₁ = 1.5q₂.
  2. q₁ + q₂ = 1 gives q₂ = 0.4.

Answer: q = (0.6, 0.4).

Example 9.3 Starting from x₀ = (1, 0), find x₂ for the same P, and the general xₖ.

Show the steps
  1. x₁ = (0.8, 0.2); x₂ = (0.64 + 0.06, 0.16 + 0.14).
  2. Eigenvalues are 1 and 0.5 (trace 1.5, det 0.5), so xₖ = (0.6, 0.4) + 0.5ᵏ(0.4, −0.4).

Answer: x₂ = (0.70, 0.30); xₖ → (0.6, 0.4).

Practice problems

Work each on paper first, then check.

  1. Is [[0.5, 0.2], [0.5, 0.8]] a Markov matrix?

    Show answer

    Yes: nonnegative, columns sum to 1.

  2. Find its steady state.

    Show answer

    0.5q₁ = 0.2q₂, so q = (2/7, 5/7) = (0.286, 0.714).

  3. If A = PDP⁻¹ with D = diag(0.9, 0.5), what happens to Aᵏx for large k?

    Show answer

    It decays to zero, dominated by 0.9ᵏ.

  4. Can [[1, 1], [0, 1]] be diagonalized?

    Show answer

    No: λ = 1 twice but only one independent eigenvector.

Common misconceptions

  • Myth: Every matrix can be diagonalized.

    Actually: It needs n independent eigenvectors; [[1, 1], [0, 1]] has only one.

  • Myth: A Markov chain’s steady state depends on the start.

    Actually: For a regular chain it does not; every start converges to the same q.

  • Myth: Aᵏ means raising each entry to the k-th power.

    Actually: Only diagonal matrices work that way; that is why we diagonalize.

Flashcards

  • Diagonalization

    A = PDP⁻¹.

  • Matrix power

    Aᵏ = PDᵏP⁻¹.

  • Markov matrix

    Nonnegative, columns sum to 1.

  • Steady state

    Pq = q, entries sum to 1.

  • Spectral theorem

    A = QΛQᵀ.

  • Defective matrix

    Too few eigenvectors to diagonalize.

Self-check quiz

1. Every Markov matrix has the eigenvalue…

Show answer

1. Columns sum to 1.

2. Aᵏ for A = PDP⁻¹ is…

Show answer

PDᵏP⁻¹. Inner P⁻¹P cancel.

3. The convergence speed of a Markov chain is set by…

Show answer

|λ₂|. The gap shrinks like |λ₂|ᵏ.

4. Symmetric matrices diagonalize as…

Show answer

QΛQᵀ. Spectral theorem.

How this helps save the planet

Where materials end up is a Markov chain. EPA reports that the United States generated 292.4 million tons of municipal solid waste in 2018, 4.9 pounds per person per day. About 69 million tons were recycled and 25 million tons composted, a combined 32.1% rate; about 35 million tons (11.8%) were burned with energy recovery; and more than 146 million tons (50%) were landfilled.

If each year’s shares stayed fixed, a material would leave use for these destinations with these probabilities, and only the recycled share would return to the economy. Lab E builds a sample transition matrix with a 32.1% recovery share and shows, with matrix powers and the λ = 1 eigenvector, how raising recycling changes how long material stays in use and how much reaches landfill.

On Saving Planets: Solutions

Sources: U.S. EPA: National overview, facts and figures on materials, wastes and recycling · Interactive Linear Algebra, 5.6 Stochastic matrices

Free learning resources

Back to top

Unit 10 · Weeks 13–14

The Singular Value Decomposition, Principal Component Analysis and Positive Definite Matrices

Positive definite matrices and AᵀA, the SVD A = UΣVᵀ, singular values and the geometry of a matrix as rotate–stretch–rotate, best low-rank approximation and image compression, the condition number, and principal component analysis of data.

Learning objectives

By the end of this unit you can:

  • Test a symmetric matrix for positive definiteness.
  • Compute singular values from the eigenvalues of AᵀA.
  • Build a rank-k approximation and measure what it keeps.
  • Run a small principal component analysis.

Key concepts

  • A symmetric matrix is positive definite when xᵀAx > 0 for every x ≠ 0; equivalently all eigenvalues (or all pivots) are positive. AᵀA is always symmetric and positive semidefinite.
  • Every m × n matrix has an SVD A = UΣVᵀ: V and U have orthonormal columns, and Σ holds the singular values σ₁ ≥ σ₂ ≥ … ≥ 0, the square roots of the eigenvalues of AᵀA.
  • A = σ₁u₁v₁ᵀ + σ₂u₂v₂ᵀ + …. Keeping the first k terms gives the best rank-k approximation (Eckart–Young); the error in the 2-norm is σₖ₊₁.
  • The condition number σ₁/σₙ measures how much errors in b can be amplified when solving Ax = b.
  • PCA: center the data (subtract column means), then the right singular vectors (eigenvectors of the covariance matrix) are the principal directions, and σᵢ² measures the variance each explains.

Key equations

A=UΣVTA=U\Sigma V^T
Singular value decomposition
σi=λi(ATA)\sigma_i=\sqrt{\lambda_i(A^TA)}
Singular values
Ak=∑i=1kσiuiviTA_k=\sum_{i=1}^{k}\sigma_i\mathbf{u}_i\mathbf{v}_i^T
Best rank-k approximation
κ=σ1/σn\kappa=\sigma_1/\sigma_n
Condition number

Worked examples, step by step

Example 10.1 Find the singular values of A = [[3, 0], [4, 5]].

Show the steps
  1. AᵀA = [[25, 20], [20, 25]], eigenvalues 45 and 5.
  2. σ₁ = √45, σ₂ = √5. Check: σ₁σ₂ = 15 = |det A|.

Answer: σ₁ = 6.708, σ₂ = 2.236.

Example 10.2 What share of AᵀA’s “energy” Σσ² does the rank-1 approximation of that A keep?

Show the steps
  1. σ₁²/(σ₁² + σ₂²) = 45/50.

Answer: 90%.

Example 10.3 A 1000 × 1000 grayscale image is stored as a rank-50 SVD. How many numbers are stored, compared with the full image?

Show the steps
  1. Each term needs one u (1000), one v (1000) and one σ: 2001 numbers.
  2. 50 × 2001 = 100,050 versus 1,000,000.

Answer: 100,050 numbers, about 10.0% of the original.

Practice problems

Work each on paper first, then check.

  1. Is [[2, 1], [1, 2]] positive definite?

    Show answer

    Yes: eigenvalues 3 and 1 are positive.

  2. Find the condition number of diag(10, 0.1).

    Show answer

    10/0.1 = 100.

  3. Centered 2D data have covariance [[2, 1], [1, 2]]. What share of the variance does the first principal component explain?

    Show answer

    Eigenvalues 3 and 1: 3/4 = 75%, along (1, 1).

  4. What are the singular values of an orthogonal matrix?

    Show answer

    All equal to 1.

Common misconceptions

  • Myth: Singular values are the eigenvalues of A.

    Actually: They are square roots of the eigenvalues of AᵀA, and are never negative.

  • Myth: Only square matrices have an SVD.

    Actually: Every matrix does.

  • Myth: PCA works on raw data.

    Actually: Center the data first, and scale variables measured in different units.

Flashcards

  • SVD

    A = UΣVᵀ.

  • Singular value

    σᵢ = √λᵢ(AᵀA).

  • Positive definite

    xᵀAx > 0, all λ > 0.

  • Eckart–Young

    Truncated SVD is the best rank-k approximation.

  • Condition number

    σ₁/σₙ.

  • PCA

    SVD of centered data.

Self-check quiz

1. Singular values are…

Show answer

nonnegative. Square roots of eigenvalues of AᵀA ≥ 0.

2. The best rank-k approximation keeps…

Show answer

the k largest singular terms. Eckart–Young.

3. A large condition number means…

Show answer

errors can be amplified. Near-singular.

4. In PCA, the first principal direction is…

Show answer

the top right singular vector of the centered data. Direction of most variance.

How this helps save the planet

NASA JPL’s EMIT imaging spectrometer on the International Space Station maps the minerals in Earth’s dust-source regions. JPL says it collects more than 100,000 spectra every second, which is 3.6 × 10⁸ spectra an hour, and that its mission has been extended through at least 2026. Dust matters for climate: JPL notes that darker minerals absorb sunlight and warm the air, while lighter minerals reflect it and cool.

Each spectrum is a long vector of reflectances at many wavelengths, and neighboring wavelengths are strongly correlated. PCA and the SVD compress such data into a few components, separate signal from noise, and pick out the signatures of individual minerals, the same mathematics as the image-compression lab.

On Saving Planets: Earth Observation · Space

Sources: NASA JPL: EMIT, Earth Surface Mineral Dust Source Investigation · MIT OpenCourseWare 18.06 Linear Algebra (Strang, Spring 2010)

Free learning resources

Back to top

The lab (Computation lab)

The lab is the hands-on part of the course. Here it is free and for practice only, with no credit.

Every lab runs in a free spreadsheet or on paper; Python versions are optional. Take regular breaks from the screen, and handle pins and scissors with care.

Safety reference: OSHA: Computer workstations eTool

Lab A: Pixel vectors and a vegetation index

Week 2

Goal: Treat satellite pixels as vectors: compute lengths, angles between pixels and the NDVI, and see that vegetation, soil and water point in different directions.

cos⁡θ=p⋅q∥p∥∥q∥,NDVI=NIR−redNIR+red\cos\theta=\frac{\mathbf{p}\cdot\mathbf{q}}{\|\mathbf{p}\|\|\mathbf{q}\|},\quad \text{NDVI}=\frac{\text{NIR}-\text{red}}{\text{NIR}+\text{red}}
Angle between pixels and NDVI

Procedure

  1. In a spreadsheet, enter three sample pixels as rows of (green, red, NIR) reflectances: vegetation (0.10, 0.08, 0.40), soil (0.20, 0.25, 0.30), water (0.06, 0.04, 0.02).
  2. Compute each pixel’s length with SQRT(SUMSQ(…)) and each pair’s dot product with SUMPRODUCT.
  3. Find the angle between each pair with ACOS and DEGREES.
  4. Compute NDVI for each pixel. Optional: download a free Landsat scene and repeat for real pixels.

Data table

PixelGreenRedNIRNDVI
Vegetation0.100.080.400.667
Soil0.200.250.300.091
Water0.060.040.02−0.333

Sample reflectances chosen to look like typical vegetation, bare soil and water.

Sample calculation

  1. Vegetation NDVI = (0.40 − 0.08)/(0.40 + 0.08) = 0.667.
  2. Soil NDVI = 0.05/0.55 = 0.091; water NDVI = −0.02/0.06 = −0.333.
  3. Angle vegetation–soil: dot = 0.160, lengths 0.420 and 0.4387, cos θ = 0.8683, θ = 29.7°.

Safety

  • Take a short screen break every half hour.

Home or virtual alternative

  • The whole lab runs in a free spreadsheet.

PhET: Vector Addition · USGS: Landsat 9

Planet link: USGS lists Landsat 9’s red (band 4) and near-infrared (band 5) bands, and its data are free, so anyone can map vegetation this way. USGS: Landsat 9

Lab B: Trilateration, from strings to a linear system

Week 3

Goal: Find a position from distances, first with strings on a map as in the GPS.gov activity, then by turning the circle equations into a linear system and solving it by elimination.

2(a2−a1)x+2(b2−b1)y=d12−d22+a22−a12+b22−b122(a_2-a_1)x+2(b_2-b_1)y=d_1^2-d_2^2+a_2^2-a_1^2+b_2^2-b_1^2
Difference of two circle equations

Procedure

  1. Print the GPS.gov trilateration map (without scaling) or mark three “satellites” on graph paper at (0, 0), (10, 0) and (0, 10) cm.
  2. Convert signal times to distances with d = ct (c = 299,792,458 m/s); for the paper version use the sample distances below.
  3. Draw circles with a string and pencil and mark where they cross.
  4. Subtract the circle equations in pairs to get two linear equations, solve them by elimination in a spreadsheet, and compare with your drawing.

Data table

BeaconPosition (cm)Distance (cm)d²
1(0, 0)5.00025
2(10, 0)8.06265
3(0, 10)6.70845

Sample distances for a receiver at (3, 4) cm. The GPS.gov activity uses four satellites on a map at 1:21,283,839.

Sample calculation

  1. Eq. 1 − Eq. 2: 20x − 100 = −40, x = 3. Eq. 1 − Eq. 3: 20y − 100 = −20, y = 4.
  2. GPS.gov time A = 0.00505783 s gives d = 1,516,299 m; at 1:21,283,839 that is 7.12 cm on the map.

Safety

  • Handle push pins and scissors with care.

Home or virtual alternative

  • The string-and-paper version is the home lab.

GPS.gov: Trilateration activity · Interactive Linear Algebra, 1.2 Row reduction

Planet link: GPS.gov designed this activity with NOAA; real receivers solve the same kind of system, with a clock unknown, many times a second. GPS.gov: Trilateration (NOAA classroom activity) · GPS.gov: Space segment

Lab C: Matrices as transformations of the plane

Week 4

Goal: See matrices act on shapes: stretch, shear, rotate and reflect a square on graph paper, and confirm that areas scale by |det A|.

area(AS)=∣det⁡A∣ area(S)\text{area}(AS)=|\det A|\,\text{area}(S)
Area scale factor

Procedure

  1. Draw the unit square with corners (0, 0), (1, 0), (1, 1), (0, 1).
  2. Multiply each corner by A = [[2, 1], [0, 1]] (a shear and stretch), by R = [[0, −1], [1, 0]] (a 90° rotation) and by F = [[1, 0], [0, −1]] (a reflection).
  3. Draw each image and find its area by counting squares or with the shoelace formula.
  4. Compare each area with |det| and note which maps flip orientation (det < 0). Then check det(AR) = det A det R.

Data table

MatrixdetImage areaOrientation
A = [[2, 1], [0, 1]]22kept
R (90° rotation)11kept
F (reflection)−11flipped
AR22kept

Exact values for the unit square.

Sample calculation

  1. det A = 2 × 1 − 1 × 0 = 2; the image is a parallelogram with base 2 and height 1.
  2. det(AR) = det A × det R = 2 × 1 = 2.

Safety

  • None beyond normal desk work.

Home or virtual alternative

  • Graph paper and a pencil are all you need.

Interactive Linear Algebra, 3.1 Matrix transformations · Interactive Linear Algebra, 4.3 Determinants and volumes

Planet link: Map projections and image registration use these same matrices; the determinant keeps area measurements of forests and floods honest. USGS: Landsat 9

Lab D: Least-squares fit of sea-level data

Week 9

Goal: Fit a line, and then a parabola, to sea-level data with the normal equations, and compare with the spreadsheet’s built-in fit.

ATAx^=ATb,A=[ 1  t ]A^TA\hat{\mathbf{x}}=A^T\mathbf{b},\quad A=[\,\mathbf{1}\ \ \mathbf{t}\,]
Normal equations for y = C + Dt

Procedure

  1. Enter the sample data below (or download NASA’s record from its Vital Signs page): t in years since 1993 and h in mm.
  2. Build A with a column of ones and a column of t. Compute AᵀA and Aᵀb with MMULT and TRANSPOSE, then x̂ = MINVERSE(AᵀA)·Aᵀb.
  3. Check against LINEST or a chart trendline. Plot the residuals b − Ax̂.
  4. Add a t² column and refit. A positive t² coefficient means the rise is speeding up.

Data table

t (years since 1993)08162432
h (mm)0224878114

Sample values shaped like the NASA record (which reports 114.4 mm of rise since 1993); use the real data for a real answer.

Sample calculation

  1. t̄ = 16, h̄ = 52.4; Σ(t − t̄)(h − h̄) = 2272, Σ(t − t̄)² = 640.
  2. D = 2272/640 = 3.55 mm/yr; C = 52.4 − 3.55 × 16 = −4.4 mm.
  3. The residuals are positive at both ends and negative in the middle: a curve fits better, as NASA’s rising rate suggests.

Safety

  • Take a short screen break every half hour.

Home or virtual alternative

  • The whole lab runs in a free spreadsheet.

NASA: Global mean sea level · Interactive Linear Algebra, 6.5 Least squares

Planet link: NASA reports the rate of rise grew from 0.20 cm per year in 1993 to 0.44 cm per year now, an acceleration you can detect with a t² column. NASA: Global mean sea level (Vital Signs)

Lab E: A Markov model of materials and recycling

Week 12

Goal: Build a transition matrix for material in use and lost to landfill or burning, use matrix powers to follow it over time, and see how a higher recycling rate keeps material in use longer.

xk+1=Pxk,P=[1−f(1−r)0f(1−r)1]\mathbf{x}_{k+1}=P\mathbf{x}_k,\quad P=\begin{bmatrix}1-f(1-r)&0\\f(1-r)&1\end{bmatrix}
Two-state chain: in use, lost

Procedure

  1. Choose a sample discard rate f = 0.2 per year and a recovery rate r = 0.321 (EPA’s 2018 recycling-plus-composting share).
  2. Build P in a spreadsheet and start with x₀ = (1, 0): all material in use.
  3. Use MMULT to step forward 10 years, and compare with the formula (1 − f(1 − r))ᵏ for the share still in use.
  4. Repeat with r = 0.5 and compare the average years in use, 1/(f(1 − r)).

Data table

Recovery rate rStay-in-use probabilityIn use after 10 yearsAverage years in use
0.3210.86420.2327.36
0.500.90000.34910.0

Sample model: the discard rate f = 0.2 is a sample value; r = 0.321 is the EPA 2018 share.

Sample calculation

  1. Stay probability = 1 − 0.2 × 0.679 = 0.8642; 0.8642¹⁰ = 0.232.
  2. Average years in use = 1/0.1358 = 7.36; at r = 0.5 it is 1/0.1 = 10.0 years.
  3. λ = 1 belongs to the “lost” state, which absorbs everything in the long run; λ = 0.8642 sets how fast.

Safety

  • Take a short screen break every half hour.

Home or virtual alternative

  • The whole lab runs in a free spreadsheet.

Interactive Linear Algebra, 5.6 Stochastic matrices · EPA: National overview of materials and waste

Planet link: EPA reports a 32.1% recycling and composting rate in 2018 and 50% landfilled; models like this show what raising that rate is worth. U.S. EPA: National overview, facts and figures on materials, wastes and recycling

Lab F: Power iteration and SVD image compression

Week 14

Goal: Find a dominant eigenvalue by repeated multiplication, then compress a small grayscale image with a truncated SVD and see how few numbers keep the picture.

xk+1=Axk∥Axk∥,Ak=∑i=1kσiuiviT\mathbf{x}_{k+1}=\frac{A\mathbf{x}_k}{\|A\mathbf{x}_k\|},\quad A_k=\sum_{i=1}^k\sigma_i\mathbf{u}_i\mathbf{v}_i^T
Power iteration and truncated SVD

Procedure

  1. In a spreadsheet, start with x₀ = (1, 0) and multiply by A = [[2, 1], [1, 2]] ten times, rescaling each time; watch the ratio of lengths approach 3.
  2. Make an 8 × 8 grid of gray levels (0–255) for a simple picture, or load a photo in free Python with NumPy.
  3. Compute the SVD (numpy.linalg.svd) and rebuild the image with k = 1, 2 and 4 terms.
  4. Record the storage k(8 + 8 + 1) and the share of Σσ² kept, and judge when the picture looks right.

Data table

kNumbers stored (8 × 8 image)Share of 64
11726.6%
23453.1%
468106.3%

Exact storage counts. For an 8 × 8 image, k = 4 already stores more than the raw 64 numbers; the savings appear for large images (Unit 10: rank 50 of a 1000 × 1000 image keeps 10.0%).

Sample calculation

  1. Power iteration: (1, 0) → (2, 1) → (5, 4) → (14, 13); ‖(14, 13)‖/‖(5, 4)‖ = 19.10/6.403 = 2.98, near λ = 3.
  2. Storage for k = 1: 8 + 8 + 1 = 17 numbers, 17/64 = 26.6%.

Safety

  • Take a short screen break every half hour.

Home or virtual alternative

  • The spreadsheet and free Python versions are both home labs.

MIT OpenCourseWare 18.06 Linear Algebra (Strang, Spring 2010) · Interactive Linear Algebra, 5.1 Eigenvectors

Planet link: NASA JPL’s EMIT returns more than 100,000 spectra every second; compressing and separating that data uses exactly these tools. NASA JPL: EMIT, Earth Surface Mineral Dust Source Investigation

Back to top

Exam prep practice sets

Time yourself: about 10 minutes per problem. Then compare with the full solutions.

Midterm practice set (Units 1–5)

  1. Find the angle between (1, 0, 1) and (0, 1, 1).

    Show full solution
    1. Dot = 1; lengths √2 each; cos θ = 1/2.

    Answer: 60°.

  2. Solve x + 2y = 4, 3x − y = 5.

    Show full solution
    1. R₂ − 3R₁: −7y = −7.

    Answer: x = 2, y = 1.

  3. Invert [[3, 2], [1, 1]].

    Show full solution
    1. det = 1.

    Answer: [[1, −2], [−1, 3]].

  4. A one-sector economy uses 0.4 of its own output. What output meets demand 120?

    Show full solution
    1. x = 120/0.6.

    Answer: 200.

  5. A 6 × 9 matrix has rank 4. Find dim N(A) and dim N(Aᵀ).

    Show full solution
    1. n − r and m − r.

    Answer: 5 and 2.

Final practice set (all units)

  1. Fit y = C + Dt to (0, 0), (1, 2), (2, 2) by least squares.

    Show full solution
    1. AᵀA = [[3, 3], [3, 5]], Aᵀb = (4, 6); det 6.

    Answer: C = 0.333, D = 1.

  2. Find det [[2, 1, 0], [1, 2, 1], [0, 1, 2]].

    Show full solution
    1. 2(4 − 1) − 1(2 − 0) + 0.

    Answer: 4.

  3. Find the eigenvalues of [[6, 2], [2, 3]].

    Show full solution
    1. Trace 9, det 14: λ² − 9λ + 14 = 0.

    Answer: 7 and 2.

  4. Find the steady state of the Markov matrix [[0.9, 0.2], [0.1, 0.8]].

    Show full solution
    1. 0.1q₁ = 0.2q₂.

    Answer: (0.667, 0.333).

  5. Find the singular values of [[2, 0], [0, −3]].

    Show full solution
    1. AᵀA = diag(4, 9).

    Answer: 3 and 2.

  6. How many numbers does a rank-20 SVD of a 500 × 800 image store, and what share of the full image is that?

    Show full solution
    1. 20(500 + 800 + 1).

    Answer: 26,020, about 6.5%.

Back to top

Formula sheet

∥v∥=v⋅v, cos⁡θ=v⋅w∥v∥∥w∥\|v\|=\sqrt{v\cdot v},\ \cos\theta=\frac{v\cdot w}{\|v\|\|w\|}
Ax=b,  A=LUA\mathbf{x}=\mathbf{b},\ \ A=LU
[abcd]−1=1ad−bc[d−b−ca]\begin{bmatrix}a&b\\c&d\end{bmatrix}^{-1}=\frac{1}{ad-bc}\begin{bmatrix}d&-b\\-c&a\end{bmatrix}
(AB)−1=B−1A−1, (AB)T=BTAT(AB)^{-1}=B^{-1}A^{-1},\ (AB)^T=B^TA^T
x=(I−A)−1d\mathbf{x}=(I-A)^{-1}\mathbf{d}
r+dim⁡N(A)=nr+\dim N(A)=n
p=a⋅ba⋅aa,  ATAx^=ATb\mathbf{p}=\frac{a\cdot b}{a\cdot a}a,\ \ A^TA\hat{x}=A^Tb
A=QR, QTQ=IA=QR,\ Q^TQ=I
det⁡(AB)=det⁡Adet⁡B, xi=det⁡Bi/det⁡A\det(AB)=\det A\det B,\ x_i=\det B_i/\det A
det⁡(A−λI)=0, ∑λ=tr⁡A, ∏λ=det⁡A\det(A-\lambda I)=0,\ \sum\lambda=\operatorname{tr}A,\ \prod\lambda=\det A
A=PDP−1, Ak=PDkP−1, A=QΛQTA=PDP^{-1},\ A^k=PD^kP^{-1},\ A=Q\Lambda Q^T
A=UΣVT, σi=λi(ATA), κ=σ1/σnA=U\Sigma V^T,\ \sigma_i=\sqrt{\lambda_i(A^TA)},\ \kappa=\sigma_1/\sigma_n

Constants

ConstantValue
Speed of light (used for GPS ranges)299,792,458 m/s
1 mile1.609344 km
1 foot0.3048 m
1 inch25.4 mm
Flops to solve an n × n system by eliminationabout n³/3 multiply–adds (Strang)
Identity matrix IOnes on the diagonal, zeros elsewhere: AI = IA = A

Source: NIST Guide to the SI, Appendix B.9: conversion factors

Back to top

Free texts and lectures

Use these free textbooks, lectures and tools alongside the course.

Back to top

Glossary

Basis
A linearly independent set that spans a space.
Column space
All combinations Ax of the columns of A.
Condition number
σ₁/σₙ, how much errors can grow when solving Ax = b.
Determinant
Signed volume scale factor of A; zero exactly when A is singular.
Diagonalization
A = PDP⁻¹ with eigenvectors in P.
Dimension
Number of vectors in any basis.
Eigenvalue
λ with Av = λv for some nonzero v.
LU factorization
A = LU from elimination.
Least squares
The x̂ minimizing ‖b − Ax‖.
Leontief inverse
(I − A)⁻¹, total output needed per unit of final demand.
Linear independence
Only the zero combination gives the zero vector.
Markov matrix
Nonnegative entries, columns summing to 1.
Null space
All x with Ax = 0.
Orthonormal
Mutually orthogonal unit vectors.
Pivot
First nonzero entry of a row in echelon form.
Positive definite
Symmetric with xᵀAx > 0 for x ≠ 0.
Projection
Closest point to b in a subspace.
Rank
Number of pivots; dimension of the column space.
Singular value
σᵢ = √λᵢ(AᵀA).
Span
All linear combinations of a set of vectors.
Subspace
A set containing 0 that is closed under addition and scaling.

Back to top

Self-assessment rubric

No grade or credit is given. Use this to judge honestly where you are in each unit.

LevelWhat it looks like
4 · MasteryChooses the right tool (elimination, projection, eigenvalues or SVD), computes small cases by hand, checks answers (Ax = b, trace and determinant), and explains the meaning in pictures and in context.
3 · ProficientSolves the practice problems with occasional arithmetic slips.
2 · DevelopingFollows worked examples but mixes up rows and columns or the four subspaces.
1 · BeginningManipulates symbols without checking sizes or answers; needs to revisit the unit.

Suggested weighting if you want a score

PartWeight
Practice problems25%
Labs20%
Midterm practice set25%
Final practice set30%

Aim for level 3 or better in every unit before moving to the next course.

Back to top

Help / Donate