← All notes
Given the linear model:
y=X1b1+X2b2+e
We want to find the first order conditions of the problem:
b1,b2argmine′e
b1,b2argmin(y−y^)′(y−y^)
The objective function of the problem is equivalent to:
e′e=(y−y^)′(y−y^)=y′y−2y′y^+y^′y^=y′y−2y′(X1b1+X2b2)+(X1b1+X2b2)′(X1b1+X2b2)=y′y−2y′X1b1−2y′X2b2+b1′X1′X1b1+2b1′X1′X2b2+b2′X2′X2b2
Now we take the gradient of the function with respect to b1 and b2:
∂b1∂f=0⟹−2X1′y+2X1′X1b1+2X1′X2b2=0⟹X1′X1b1+X1′X2b2=X1′y
∂b2∂f=0⟹−2X2′y+2X2′X2b2+2X2′X1b1=0⟹X2′X2b2+X2′X1b1=X2′y
Which we can package into a block of conditions like so:
[X1′X1X2′X1X1′X2X2′X2][b1b2]=[X1′yX2′y]