Advanced Linear Models for Data Science 1: Least Squares

About this course: Welcome to the Advanced Linear Models for Data Science Class 1: Least Squares. This class is an introduction to least squares from a linear algebraic and mathematical perspective. Before beginning the class make sure that you have the following: - A basic understanding of linear algebra and multivariate calculus. - A basic understanding of statistics and regression models. - At least a little familiarity with proof based mathematics. - Basic knowledge of the R programming language. After taking this course, students will have a firm foundation in a linear algebraic treatment of regression modeling. This will greatly augment applied data scientists' general understanding of regression models.

Who is this class for: This class is for students who already have had a class in regression modeling and are familiar with the area who would like to see a more advanced treatment of the topic.

Created by:  Johns Hopkins University

• Taught by:  Brian Caffo, PhD, Professor, Biostatistics

  Bloomberg School of Public Health

Level Advanced Commitment 6 weeks of study, 1-2 hours/week Language English How To Pass Pass all graded assignments to complete the course. User Ratings 4.4 stars Average User Rating 4.4See what learners said Coursework

Each course is like an interactive textbook, featuring pre-recorded videos, quizzes and projects.

Help from your peers

Connect with thousands of other learners and debate ideas, discuss course material, and get help mastering concepts.

Certificates

Earn official recognition for your work, and share your success with friends, colleagues, and employers.

Johns Hopkins University The mission of The Johns Hopkins University is to educate its students and cultivate their capacity for life-long learning, to foster independent and original research, and to bring the benefits of discovery to the world.

Syllabus


WEEK 1


Background



We cover some basic matrix algebra results that we will need throughout the class. This includes some basic vector derivatives. In addition, we cover some some basic uses of matrices to create summary statistics from data. This includes calculating and subtracting means from observations (centering) as well as calculating the variance.


7 videos, 4 readings expand

1. Video: Introduction
2. Reading: Welcome to the class
3. Reading: Course textbook
4. Reading: Grading
5. Reading: In this module
6. Video: Matrix derivatives
7. Video: Coding example
8. Video: Centering by matrix multiplication
9. Video: Coding example
10. Video: Variance via matrix multiplication
11. Video: Coding example

Graded: Background Quiz

WEEK 2


One and two parameter regression
In this module, we cover the basics of regression through the origin and linear regression. Regression through the origin is an interesting case, as one can build up all of multivariate regression with it.


6 videos, 2 readings expand

1. Reading: Before you begin
2. Video: Regression through the origin
3. Video: Centering first
4. Video: Coding example
5. Reading: Before you begin
6. Video: Connection with linear regression
7. Video: Coding example
8. Video: Fitted values and residuals

Graded: One Parameter Regression Quiz

WEEK 3


Linear regression
In this lecture, we focus on linear regression, the most standard technique for investigating unconfounded linear relationships.


8 videos, 2 readings expand

1. Reading: Before you begin
2. Video: Least squares
3. Video: Coding example
4. Video: Prediction
5. Video: Coding example
6. Video: Residuals
7. Video: Coding example
8. Reading: Generalizations
9. Video: Generalizations
10. Video: Generalizations example

Graded: Linear Regression Quiz

WEEK 4


General least squares
We now move on to general least squares where an arbitrary full rank design matrix is fit to a vector outcome.


6 videos, 1 reading expand

1. Reading: Before you begin
2. Video: Least squares
3. Video: Coding example
4. Video: Second derivation of least squares
5. Video: Projections
6. Video: Third derivation of least squares
7. Video: Coding example

Graded: General Least Squares Quiz

WEEK 5


Least squares examples
Here we give some canonical examples of linear models to relate them to techniques that you may already be using.


4 videos expand

1. Video: Basic examples of design matrices and fits
2. Video: Group effects
3. Video: Change of parameterization
4. Video: ANCOVA

Graded: Least Squares Examples Quiz

WEEK 6


Bases and residuals
Here we give a very useful kind of linear model, that is decomposing a signal into a basis expansion.


6 videos expand

1. Video: Bases, introduction
2. Video: Bases 2, Fourier
3. Video: Bases 3, SVDs
4. Video: Bases, coding example
5. Video: Introduction to residuals
6. Video: Partitioning variability

Graded: Bases Quiz
Graded: Residuals Quiz