Linear Regression Calculator

Linear regression finds the best-fit straight line through data points to model the relationship between a predictor variable (x) and response variable (y). The line minimizes the sum of squared vertical distances from points to the line (least squares method). Equation: ŷ = a + bx, where ŷ is predicted value, b is slope, and a is y-intercept. Example 1: Study hours (x) vs. test scores (y). Data: (2, 65), (4, 75), (6, 85), (8, 90), (10, 95). Regression gives ŷ = 52 + 4.3x. This predicts someone studying 7 hours scores ŷ = 52 + 4.3(7) = 82.1. Example 2 - Business: Advertising spend vs. sales. Regression: Sales = $10,000 + $3.50 × Ad_Spend. For every $1 in ads, expect $3.50 in additional sales. When to use: (1) Prediction: Estimate y from known x. (2) Relationship strength: Quantify how x affects y. (3) Trend analysis: Understand patterns over time. Assumptions: Linear relationship, independent observations, constant variance (homoscedasticity), normally distributed residuals. Applications: Forecasting sales, predicting house prices, analyzing scientific data, financial modeling.

Slope (b): b = Σ[(x - x̄)(y - ȳ)] / Σ(x - x̄)². Intercept (a): a = ȳ - b×x̄. Example: Study hours (x) and test scores (y). Data: (1, 60), (2, 70), (3, 75), (4, 85), (5, 90). Step 1: Calculate means: x̄ = (1+2+3+4+5)/5 = 3, ȳ = (60+70+75+85+90)/5 = 76. Step 2: Calculate deviations: x values: {-2, -1, 0, 1, 2}, y values: {-16, -6, -1, 9, 14}. Step 3: Products of deviations: (-2)(-16)=32, (-1)(-6)=6, (0)(-1)=0, (1)(9)=9, (2)(14)=28. Sum = 75. Step 4: Squared x-deviations: 4+1+0+1+4 = 10. Step 5: Slope b = 75/10 = 7.5 (each additional study hour increases score by 7.5 points). Step 6: Intercept a = 76 - 7.5(3) = 76 - 22.5 = 53.5 (expected score with 0 study hours). Regression equation: ŷ = 53.5 + 7.5x. Prediction: For x = 6 hours, ŷ = 53.5 + 7.5(6) = 98.5. Interpretation: Slope tells rate of change. Positive slope = positive relationship. Negative slope = inverse relationship. Intercept is y-value when x = 0 (may not always be meaningful if x = 0 is outside data range).

R² (R-squared) measures how well the regression line fits the data—the proportion of variance in y explained by x. Range: 0 to 1 (or 0% to 100%). Formula: R² = 1 - (SSresidual / SStotal), where SSresidual = Σ(y - ŷ)² and SStotal = Σ(y - ȳ)². Equivalently, R² = r² (square of correlation coefficient). Interpretation: R² = 1.0 (100%): Perfect fit—all points on the line. R² = 0.8 (80%): Model explains 80% of variation in y; 20% due to other factors. R² = 0.5 (50%): Moderate fit—half of variation explained. R² = 0 (0%): Model explains nothing—no linear relationship. Example 1: Study hours vs. scores with R² = 0.92 = 92%. Study time explains 92% of score variation—very good predictive power. Other factors (sleep, intelligence) explain 8%. Example 2: Height vs. income with R² = 0.05 = 5%. Height explains only 5% of income variation—poor predictor. Example 3: Advertising spend vs. sales with R² = 0.65 = 65%. Ads explain 65% of sales—meaningful but other factors (product quality, competition) matter too. Guidelines: R² > 0.7: Strong fit. R² = 0.4-0.7: Moderate fit. R² < 0.4: Weak fit. Caution: High R² doesn't prove causation! Also, R² increases with more variables—use adjusted R² for multiple regression.

Residuals are the differences between actual y values and predicted ŷ values: residual = y - ŷ. They show how far each point is from the regression line. Example: Regression ŷ = 50 + 5x. For point (4, 75): predicted ŷ = 50 + 5(4) = 70. Actual y = 75. Residual = 75 - 70 = +5 (point is 5 units above the line). Residual analysis checks model assumptions: (1) Residual plot (residuals vs. x): Should show random scatter around zero with constant spread. Good: Points randomly scattered above and below zero line. Bad - Pattern: Curved pattern suggests nonlinear relationship—linear regression inappropriate. Bad - Funnel: Spread increases with x (heteroscedasticity)—violates constant variance assumption. (2) Normal probability plot: Residuals should be approximately normal. Points should roughly follow diagonal line. (3) Independence: Residuals should be independent (especially important for time-series data). Sum of squared residuals (SSR): SSR = Σ(y - ŷ)² measures total prediction error. Smaller SSR = better fit. Least squares regression minimizes SSR. Standard error of estimate (SEE): SEE = √[SSR/(n-2)]. Average distance of points from line. Example: SEE = 5 means typical prediction is off by about 5 units. Red flags: Patterns in residuals, outliers with large residuals (|residual| > 3×SEE), non-normal distribution. If violations found, consider: transforming variables, removing outliers, or using nonlinear regression.

Point prediction: Plug x-value into regression equation ŷ = a + bx. Example: House price regression: Price = $50,000 + $100×SqFt. For 1500 sq ft house: Price = 50,000 + 100(1500) = $200,000. Prediction interval: Range likely to contain actual y-value for new observation. Accounts for uncertainty in both the line and individual variation. Wider than confidence interval. Confidence interval: Range likely to contain the true mean y-value for given x. Shows uncertainty about the line position. Example: At x = 5, predicted ŷ = 78 with 95% prediction interval [68, 88] and 95% confidence interval [74, 82]. Interpretation: We're 95% confident a new observation at x = 5 will fall between 68-88 (prediction interval—wider). We're 95% confident the mean y at x = 5 is between 74-82 (confidence interval—narrower). Uncertainty increases with distance from x̄: Predictions near the mean of x are more reliable. Extrapolating far beyond data range is risky. Extrapolation dangers: Don't predict outside observed x-range. Relationship may change. Example: Regressing child height vs. age (ages 5-10) shouldn't predict height at age 40—relationship is nonlinear beyond childhood. Sample size matters: Larger n gives narrower intervals and more reliable predictions. n = 10 gives wide intervals; n = 100 much narrower. Standard error of prediction: Quantifies prediction uncertainty. Smaller R² and smaller n increase prediction error.

Correlation (r) measures strength and direction of linear relationship between two variables—symmetric (r for x vs. y equals r for y vs. x). Regression creates a prediction equation—asymmetric (predicting y from x differs from predicting x from y). Example - Correlation: Hours studied and test scores have r = 0.85. Strong positive relationship. Doesn't specify which causes which, just that they move together. Example - Regression: Score = 50 + 5×Hours. Allows prediction: studying 8 hours predicts score of 90. Makes hours the predictor (independent) and score the outcome (dependent). Key differences: (1) Purpose: Correlation quantifies relationship strength. Regression predicts and explains. (2) Symmetry: Correlation is same both directions. Regression equation changes if you swap x and y. (3) Units: Correlation is unitless (-1 to +1). Regression slope has units (e.g., points per hour). (4) Assumptions: Correlation assumes neither variable causes the other. Regression assumes x predicts y. Relationship: R² in regression equals r². If r = 0.9, then R² = 0.81. When to use each: Correlation: Exploring relationships, no clear predictor/outcome, want strength of association. Regression: Making predictions, quantifying effect size, modeling relationships, clear predictor and outcome. Example: Correlation tells you height and weight are related (r = 0.7). Regression tells you each inch of height adds 3 lbs to expected weight (slope = 3).

Avoid these frequent errors: (1) Assuming causation: Regression shows association, not causation! Ice cream sales predict drowning deaths (both increase in summer), but ice cream doesn't cause drownings. Need experiments for causation. (2) Extrapolation beyond data: Predicting outside observed x-range is dangerous. Relationship may change. Don't use child height-age regression to predict adult heights. (3) Ignoring assumptions: If relationship is curved, residuals show pattern, or variance isn't constant, linear regression is inappropriate. Check residual plots! (4) Outliers: Single extreme point can dramatically change the line. Always identify and investigate outliers. May indicate data error or need to remove. (5) Correlation doesn't equal good prediction: r = 0.5 means R² = 0.25—only 25% variance explained. Predictions may be imprecise. (6) Confusing correlation strength with slope: Steep slope doesn't mean strong relationship. Can have slope = 10 with R² = 0.2 (weak fit). (7) Treating nonlinear as linear: Forcing linear model on curved data gives poor fit and misleading predictions. Consider polynomial or exponential regression. (8) Ignoring other variables (confounding): Shoe size predicts reading ability in kids—but age is the confounder. Multiple regression controls for confounders. (9) Small sample sizes: n = 5 gives unreliable regression. Need n ≥ 30 for stable estimates. (10) Overfitting with too many predictors: In multiple regression, using too many variables inflates R² artificially. Use adjusted R². Best practice: Visualize data with scatterplot, check assumptions with residual plots, report R² along with equation, avoid extrapolation, and never claim causation without proper experimental evidence.