Regression

Linear Regression

A predictive model fitting a linear trend line to minimize Mean Squared Error.

What is Linear Regression?

Linear Regression is a supervised machine learning model that predicts a continuous target variable from input features by fitting a linear equation to the observed data. The line equation is y = mx + c, where m is the slope and c is the y-intercept. The optimal parameters are found by minimizing the Mean Squared Error (MSE) using Gradient Descent or Ordinary Least Squares.

Key Characteristics:

  • Predicts continuous numerical values
  • Fits a straight line: y = mx + c
  • Residuals represent differences between actual and predicted points
  • Loss function: Mean Squared Error (MSE)

Complexity Analysis

Avg Time Complexity: O(Epochs * N * D) fit time, O(1) prediction
Space Complexity: O(1) parameter weights storage
Best Case: O(N) (fitting directly with OLS formula)
Worst Case: O(Epochs * N) (running slow gradient epochs)

How it Works Step-by-Step

  1. Initialize: Set weight slope m and bias intercept c to random numbers.
  2. Predict: Compute predicted y values: y_pred = m * x + c.
  3. Compute Loss: Calculate MSE: sum((y_actual - y_pred)^2) / N.
  4. Update Weights: Adjust m and c by stepping opposite to loss gradient.

Code Implementation

Worked Trace Example

Points (1,2), (2,4) with MSE line fitting:
1. Predicted values with y = 2x + 0 matches perfectly.
2. Residuals = 0, MSE loss = 0.
3. Line fits points perfectly with m=2, c=0.