Regression Modelling in Concrete and Material Science: The Ibearugbulem Approach - CHAPTER TWO: The Mathematical Basis of Regression

CHAPTER TWO

The Mathematical Basis of Regression

(By Owus M. Ibearugbulem – Light for the Living)

2.1 Overview

Having learned what regression means conceptually, we now turn to its mathematical foundation — the “engine room” that makes prediction possible. The aim of this chapter is to show how regression equations are built, how they can be expressed in matrix form, and how to compute the model coefficients when there are two, three, four or even five predictors.

2.2 From Experimental Data to Regression Equation

Every regression problem begins with experimental data — a set of observations where we know both the predictor values and the corresponding responses. Suppose we have \( n \) experimental observations. Each observation records values of the predictors \( X_1, X_2, \ldots, X_k \) and the corresponding response \( Y \).

The general regression equation can be written as:

\( Y = a_0 + a_1 X_1 + a_2 X_2 + a_3 X_3 + \cdots + a_k X_k + e_i \) (2.1)

where:

\( Y \) = measured response (e.g., compressive strength)
\( X_1, X_2, \ldots, X_k \) = predictor variables (e.g., water, cement, sand, aggregate, admixture)
\( a_0, a_1, \ldots, a_k \) = regression coefficients to be determined
\( e_i \) = residual (difference between measured and predicted value)

The objective is to find the coefficients \( a_0, a_1, \ldots, a_k \) that minimise the sum of squared residuals:

\( \text{Minimise } \sum e_i^2 = \sum (Y_i - \hat{Y}_i)^2 \) (2.2)

2.3 Matrix Representation

To handle multiple equations efficiently, regression is expressed in matrix form. Equation (2.1) for all \( n \) observations can be written compactly as:

\( [Y] = [H][a] + [e] \) (2.3)

where:

\([Y]\) = column vector of measured responses (size \( n \times 1 \))
\([H]\) = design matrix containing 1s and predictor values (size \( n \times (k + 1) \))
\([a]\) = column vector of coefficients (size \( (k + 1) \times 1 \))
\([e]\) = column vector of residuals

2.4 Determining the Coefficients

To determine \([a]\), we pre-multiply both sides of (2.3) by the transpose of \([H]\):

\( [H]^T [Y] = [H]^T [H][a] + [H]^T [e] \) (2.4)

Assuming that residuals sum to zero (orthogonal to the predictors), \( [H]^T [e] = 0 \). We then obtain the Normal Equation:

\( [H]^T [H][a] = [H]^T [Y] \) (2.5)

Solving for \([a]\):

\( [a] = ([H]^T [H])^{-1} [H]^T [Y] \) (2.6)

Equation (2.6) is the core of Ibearugbulem’s Regression Model. It works for any number of predictors as long as \( [H]^T [H] \) is invertible.

EXPLANATION BOX – “Balancing the See-saw”
Imagine you’re trying to balance a see-saw with weights (your data points) placed at different positions. The regression equation finds the best plank position (the coefficients \( a_i \)) that make the see-saw as level as possible — that is, with the smallest total imbalance (sum of squared errors).

2.5 Case 1: Two Predictors (k = 2)

\( Y = a_0 + a_1 X_1 + a_2 X_2 \) (2.7)

Example — Binary Mixture:
Let \( X_1 = \) cement content, \( X_2 = \) water content, and \( Y = \) compressive strength.

Compute:

\( [CC] = [H]^T [H], \quad [R] = [H]^T [Y], \quad [a] = [CC]^{-1} [R] \) (2.8)

This yields \( a_0, a_1, a_2 \), giving the predictive model \( Y_{mod} = a_0 + a_1 X_1 + a_2 X_2 \).

Interpretation: If \( a_1 > 0 \) and \( a_2 < 0 \), increasing cement increases strength, while increasing water decreases it — matching engineering intuition.

2.6 Case 2: Three Predictors (k = 3)

\( Y = a_0 + a_1 X_1 + a_2 X_2 + a_3 X_3 \) (2.9)

Example — Ternary Mixture:
\( X_1 = \) cement, \( X_2 = \) water, \( X_3 = \) sand; \( Y \) = measured compressive strength.

Explanatory Analogy:
Think of a fruit juice made from orange, pineapple, and mango. Each fruit (predictor) contributes to the taste (response). Regression helps you figure out exactly how much each fruit’s flavour affects the final sweetness score.

2.7 Case 3: Four Predictors (k = 4)

\( Y = a_0 + a_1 X_1 + a_2 X_2 + a_3 X_3 + a_4 X_4 \) (2.10)

Example — Ordinary Concrete:
\( X_1 = \) water, \( X_2 = \) cement, \( X_3 = \) fine aggregate, \( X_4 = \) coarse aggregate.

Once the coefficients are found, the regression model becomes:

\( Y = a_0 + a_1 X_1 + a_2 X_2 + a_3 X_3 + a_4 X_4 \) (2.11)

2.8 Case 4: Five Predictors (k = 5)

\( Y = a_0 + a_1 X_1 + a_2 X_2 + a_3 X_3 + a_4 X_4 + a_5 X_5 \) (2.12)

Example — Modified Concrete with Admixture:
\( X_1 = \) water, \( X_2 = \) cement, \( X_3 = \) fine aggregate, \( X_4 = \) coarse aggregate, \( X_5 = \) admixture (e.g., fly ash or silica fume).

2.9 Summary of the General Procedure

Collect data: measure predictors and responses for \( n \) mixes.
Form the design matrix \([H]\): include 1s for the constant term and all predictor values.
Compute \([CC] = [H]^T [H]\) and \([R] = [H]^T [Y]\).
Invert \([CC]\) to obtain \([CC]^{-1}\).
Find coefficients: \([a] = [CC]^{-1} [R]\).
Form regression equation: \( Y_{mod} = a_0 + a_1 X_1 + a_2 X_2 + \cdots + a_k X_k \).
Validate model: use \( R^2 \), adjusted \( R^2 \), SER, and F-tests (covered in Chapter 3).

Explanatory Box – “From Dots to Formula”
Plot all your experimental results as dots on a graph. Regression finds the line (or plane, or hyper-plane) that passes closest to all the dots. The coefficients \( a_0, a_1, \ldots \) tell you how the line tilts — that is, how each variable influences the outcome.

2.10 Worked Illustration (Mini Example)

Suppose a simple binary mixture experiment produced:

Mix	X₁ (Cement kg/m³)	X₂ (Water kg/m³)	Y (Strength MPa)
1	400	180	35
2	380	175	31
3	360	170	28

Then:

\( [a] = ([H]^T [H])^{-1} [H]^T [Y] \)

The result might be approximately:

\( a_0 = 5.2, \; a_1 = 0.072, \; a_2 = -0.061 \)

Hence:

\( Y = 5.2 + 0.072X_1 - 0.061X_2 \)

Interpretation: Strength increases with cement and decreases with water — exactly what we expect.

2.11 Key Takeaways

Regression equations can include any number of predictors (2–5 or more).
The same matrix equation, \( [a] = ([H]^T [H])^{-1} [H]^T [Y] \), solves all cases.
The model provides both quantitative prediction and qualitative insight.
Ibearugbulem’s formulation emphasises clarity and ease of computation — ideal for classroom or practical use.

Test Your Understanding (Chapter 2)

Write the general regression equation for four predictors.
Explain the purpose of the design matrix \([H]\). What do its rows and columns represent?
Derive the normal equation from \( [Y] = [H][a] + [e] \).
For a mixture with two predictors, outline step-by-step how you would compute \( a_0, a_1, a_2 \).
Explanatory Challenge: If each predictor were a musician and the response was the song’s volume, what does the regression model do?

Search This Blog

Light for the Living