Understanding Fourier Series

white android tablet turned on displaying a graph
Photo by Burak K on Pexels.com

Comparing Functions and vectors.

\vec{v} function f(x)

Finite dimensional Infinite dimensional

A vector can be written in the following different ways,
\vec{V} = V_x \hat{x} + V_y \hat{y} + V_z \hat{z}
\hskip .5cm = (V \cdot \hat{x}) \hat{x} + (V \cdot \hat{y}) \hat{y} + (V \cdot \hat{z}) \hat{z}
If the decomposition is along an orthogonal frame along the vectors \vec{a},\, \vec{b} and \vec{c} then the expression would be,
\vec{V} = (\vec{V} \cdot \hat{a}) \hat{a} + (\vec{V} \cdot \hat{b}) \hat{b} + (\vec{V} \cdot \hat{c}) \hat{c}
\hskip .5cm = \frac{\vec{V} \cdot \vec{a}}{\vec{a}\cdot\vec{a}} \vec{a} + \frac{\vec{V} \cdot \vec{b}}{\vec{b}\cdot\vec{b}} \vec{b} + \frac{\vec{V} \cdot \vec{c}}{\vec{c}\cdot\vec{c}} \vec{c}

In general the dot product of two $n-$dimensional vectors \vec{V} = (V_1, V_2,...,V_n) and \vec{W} = (W_1,W_2,...,W_n), can be written as,
\vec{V} \cdot \vec{W} = \sum_{i=1}^n V_i W_i.

It is useful to think of a real function f(x) over an interval
[a,b] as a vector with infinite components. Here the argument serves
as an index and the function value as the vector component. Analogous to vector dot product, the dot product between two functions $f$ and $g$ defined over the same interval can be written as,
(f,g) = \int_a^b f(x) g(x) dx.

Using this definition of the dot product, one can show that the following functions
are orthogonal to each-other (mutual dot products are zero) on the interval
[0, 2\pi].
f_1(x) = 1, \sin{x}, \sin{2x}, \sin{3x}, ...,\cos{x}, \cos{2x}, \cos{3x},...

Thus in parallel with writing a vector in terms of it’s components, one can write any (finite, smooth and continuous on [0, 2\pi] (I am not trying to be mathematically precise, the aim is to give an intuitive feel)) function in terms of
the above basis functions in the same manner,
f(x) = \frac{(f(x),1)}{(1,1)} 1 + \frac{(f(x),\sin(x)}{(\sin(x), \sin(x))} \sin(x) + \frac{(f(x),\sin(2x)}{(\sin(2x), \sin(2x))} \sin(2x) + ...
\hskip.2cm + \frac{(f(x),\cos(x)}{(\cos(x), \cos(x))} \cos(x) + \frac{(f(x),\cos(2x)}{(\cos(2x), \cos(2x))} \cos(2x) +....
Notice the similarity of the expression of a function in terms of it’s components and a vector in terms of it’s components. Hence decomposition of a function in its Fourier components is quite akin to decomposition of a vector in its Cartesian components.

Regression, How to model

This year my linear algebra class is using regression to model real world data. The data ranges from climate change to bank interests to chemical reactions.

A standard question is, given the data, how do we choose the model. Since most of the data is in two variables say (x,y). Here is the usual process.

First Plot the data, using scatter plot. This will give you an idea as to “do you expect a linear or a nonlinear relationship between x and y?”

Consider if smoothing the data will help. If your graph looks like a noisy line or a noisy quadratic, rolling average will make it smoother.

Decide on the model. Looking at the scatter plot you should be able to get an idea if the relationship between x and y is linear, quadratic, cubic and so on.

Write out your model, for example y = a + b x + c x^2. Thus each value of the data point when plugged into the model will give you a linear equation in the parameters a,b,c. For the collection of these linear equations you can write the matrix equation A \vec{x} = \vec{b} (note that here \vec{x} contains the parameters as its elements).

Use the normal equation A^TA \vec{x} = A^T \vec{b} or the equation \vec{x} = (A^T A)^{-1} A^T \vec{b} to find the best fit parameter values (\vec{x}).

The easiest way of solving the above equations is to use matlab or mathematica, which have built-in functions for matrix manipulations. (most of the programming languages like C or python also may have corresponding libraries). However writing the code is better in terms of gaining skills and making your foundations stronger.

Please note that even if you find splines easy to use for interpolation, regression is a better choice for modeling as the resultant equation is simpler.

Remember that when the parameters are found by minimizing the magnitude square of the error vector using calculus, one would get the same result for the best fit parameters. That method is known as “the least square method“.

You may post your doubts below, and if you are at the Ahmedabad University then catch me after a class.

Regression: Reading suggestions

In every aspect of practical applications, understanding interpreting and modeling using
data is get getting more and more important. As a part of regression analysis projects, I recommend the following readings for insights into working with data.

1. Process improvement using data: “https://learnche.org/pid/
This is a detailed review discussing various aspects of data analysis. Chapter 4 reviews regression. There is also discussion on chemometrics in the book.

2. SAT scores: There exists variety of documents seeking correlation between the SAT scores and family income, month of birth, college success, etc., take your pick.

3. This research paper provides multiple examples of effective regression:

4. Information on databases can be found on wikipedia or through a simple search.

Exam time

It is the time of exam again and there is the usual stream of students asking for
clarification and solution of different concepts. So here I discuss some common doubts.

Chapter 2: problem 2: Definition of a line passing through the origin in various ways.
(1) y = m x + c is useful only in two dimensions.

(2) Defining a line as collection of position vectors given by c \vec{v}, where \vec{v} can be a vector in any dimensions and c \in R. For example c(1,0,0) with c \in R would define the x-axis in three dimensions.

(3) Defining a line as intersection of two planes in 3 dimensions, for example the
intersection of the xy-plane and the xz-plane defines the x-axis.

Given any two equations of planes passing through zero, one can find the intersection and express it in terms of scaling of a vector. As an example, the intersection of 2 x + y - z = 0 and x + y + 2 z= 0 requires that z = x+y, and x = -y substituting these conditions for the coordinates of a general point (x,y,z) gives (x,y,z) = (x, -x, 0) = x (1,-1,0) The equation of a general line (not necessarily passing through the origin) can be written as \vec{a} + c \vec{v} where \vec{a} is a constant vector.

Chapter 2: problem 4, how to find a plane that passes through the given three points. Assume a general equation of a plane ax + b y + c z = 0, insist that the given points satisfy the equation and solve for a,b,c.

Chapter 3: problem 5, notice that the B matrix has only two rows that are linearly independent. There are 3 variables, hence one variable is free. Thus either there are infinite solutions or there is no solution. When the \vec{b} is chosen such that equations 1 and 3 and equations 2 and 4 are the same, the equation is solvable with infinite solutions, whereas if \vec{b} is chosen so that two of there equations are different then the matrix equation has no solution.