Review of Differential Calculus

<== Click here to toggle light-mode.

Last update: Monday, 22 August 2022.

Purpose

This document contains the prerequisite knowledge from Calculus I for this course.

Notations

As the semester progresses, we will use the notations below quite extensively. Do make friends with it!

We use \( \R \) to denote the set of real numbers. The symbol \( \in \) means "is an element of" or "belongs to". For example, \( \pi \in \R \) says that "\( \pi \) is an element of the real numbers", or more simply, "\( \pi \) is a real number". We use \( \set{x}{x \text{ does a thing}} \) to denote the set of all real numbers that "do a thing". A good example of this is below, concerning interval notation.

Let \( a, b \in \R \) with \( a \leq b \). There are four types of interval, listed below.

Open interval: \( (a, b) = \set{x \in \R}{a < x < b} \)
Left-half-open interval: \( (a, b] = \set{x \in \R}{a < x \leq b} \)
Right-half-open interval: \( [a, b) = \set{x \in \R}{a \leq x < b} \)
Closed interval: \( [a, b] = \set{x \in \R}{a \leq x \leq b} \)

For example, the interval \( (3, 5) = \set{x \in \R}{3 < x < 5} \), whereas \( (3, 5] = \set{x \in \R}{3 < x \leq 5} \).

The notation for open intervals makes them look like points in the plane, but they are very different objects!

There are some additional special intervals, written below.

\begin{align*} (-\infty, a) & = \set{x \in \R}{x < a} \\ (-\infty, a] & = \set{x \in \R}{x \leq a} \\ (a, \infty) & = \set{x \in \R}{a < x} \\ [a, \infty) & = \set{x \in \R}{a \leq x} \end{align*}

Finally, \( \R = (-\infty, \infty) \).

Here are a few things to remember regarding infinity.

\( \infty \) is NOT a real number! We use \( \infty \) to denote the concept of "unboundedness", i.e., the idea that "it goes on forever without stopping".
We NEVER write \( [-\infty, 3) \) or anything like it! That would mean we thought \( -\infty \) was a real number, and we just agreed it isn't!

To string intervals together, we take their union, denoted \( \cup \). For example, in interval notation \( \set{x \in \R}{x \neq 4 \text{ and } x \neq \pi} = (-\infty, \pi) \cup (\pi, 4) \cup (4, \infty) \).

Given real numbers \( a_1, a_2, \dots, a_n \in \R \), their sum \( S \) can be expressed in summation notation as \( \displaystyle{S = \sum_{k = 1}^n a_k} \). That symbol should be read "the sum as \( k \) goes from \( 1 \) to \( n \), \( a_k \)". Similarly, their product \( P \) can be expressed in product notation as \( \displaystyle{P = \prod_{k = 1}^n a_k} \). That symbol should be read "the product as \( k \) goes from \( 1 \) to \( n \), \( a_k \)".

If \( a_k = 2k \) for all \( k \), then we may write \( \displaystyle{ 2 \cdot 1 + 2 \cdot 2 + 2 \cdot 3 = \sum_{k = 1}^3 2 k = \sum_{k = 1}^3 a_k }. \)

Functions

A function is a rule of assignment that takes a real number to a real number, usually via some formula. For example, \( f(x) = x^2 \) is a function that takes each real number to its square. In this example, the \( x \) is the argument of the function. We evaluate a function \( f \) at an argument \( a \) by "plugging it in". If \( a = -1 \), then for our function above we evaluate \( f(a) = f(-1) = (-1)^2 = 1 \).

Each of the arithmetic operations on real numbers extends to an operation on functions. In particular, we can add, subtract, multiply, and divide functions!

\( (f + g)(x) = f(x) + g(x) \)
\( (f - g)(x) = f(x) - g(x) \)
\( (f \cdot g)(x) = f(x) \cdot g(x) \)
\( (f / g)(x) = f(x) / g(x) \)

A polynomial is a function of the form \(\displaystyle{ p(x) = \sum_{k = 0}^n c_kx^k = c_n x^n + c_{n-1} x^{n-1} + \cdots + c_1 x + c_0 }\), where the \( c_i \)'s are all constant real numbers.

The domain of a function is the set of allowed inputs, i.e., all the possible valid arguments. We denote the domain of a function \( f \) by \( \dom(f) \).

Let \( f \) and \( g \) be functions.

The domain of every polynomial is \( \R = (-\infty, \infty) \).
\( \dom(f + g) = \set{x}{x \in \dom(f) \text{ and } x \in \dom(g)} \).
\( \dom(f - g) = \set{x}{x \in \dom(f) \text{ and } x \in \dom(g)} \).
\( \dom(f \cdot g) = \set{x}{x \in \dom(f) \text{ and } x \in \dom(g)} \).
\( \dom(f / g) = \set{x}{x \in \dom(f), x \in \dom(g), \text{ and } g(x) \neq 0} \).

A rational function is a function of the form \( f = p / q \) for polynomials \( p \) and \( q \). In light of the domain restrictions described above, we have the following.

Let \( f = p / q \) be a rational function. Then \( \displaystyle{\dom(f) = \set{x \in \R}{q(x) \neq 0}}\).

The composition of two functions \( f \) and \( g \) is \( (g \circ f)(x) = g(f(x)) \).

The domain of composition of functions \( g \circ f \) is \(\displaystyle{ \dom(g \circ f) = \set{x \in \R}{x \in \dom(f) \text{ and } f(x) \in \dom(g)} }\).

Each of the sine, cosine, and natural exponential functions has domain \( \R \). The natural logarithm has domain \( (-\infty, 0) \cup (0, \infty) \).

Limits

Let \( f \) be a function and let \( a \in \R \). The left-hand limit of \( f \) as \( x \) approaches \( a \) is the real number \( L \) (provided it exists) which \( f(x) \) approaches as \( x \) gets arbitrarily close to \( a \) and \( x < a \). This is denoted \( \displaystyle{\lim_{x \to a^-} f(x) = L}\). Similarly, the right-hand limit of \( f \) as \( x \) approaches \( a \) is denoted \( \displaystyle{\lim_{x \to a^+} f(x) = L} \). The two-sided limit of \( f \) as \( x \) approaches \( a \) is \( \displaystyle{\lim_{x \to a} f(x) = \lim_{x \to a^-} f(x) = \lim_{x \to a^+} f(x)}\), provided the left- and right-hand limits exist and are equal; otherwise this limit does not exist.

"DNE" is a common abbreviation for "does not exist", so DNE cannot equal a limit! We NEVER write \( \displaystyle{\lim_{x \to 0} \frac{1}{x} = DNE} \)!

Piecewise-defined functions behave weirdly with limits, and require careful parsing of piecewise definitions!

Algebra of Limits

The following are sometimes called Limit Laws.

Suppose \( f \) and \( g \) are functions with \( a \in \R \). If \( \displaystyle{\lim_{x \to a} f(x) = L }\) and \( \displaystyle{\lim_{x \to a} g(x) = M }\), then

\( \displaystyle{ \lim_{x \to a} (f + g)(x) = \lim_{x \to a} f(x) + \lim_{x \to a} g(x) = L + M }\).
\( \displaystyle{ \lim_{x \to a} (f \cdot g)(x) = \lim_{x \to a} f(x) \cdot \lim_{x \to a} g(x) = L \cdot M }\).
\( \displaystyle{ \lim_{x \to a} (f / g)(x) = \lim_{x \to a} f(x) / \lim_{x \to a} g(x) = L / M }\), provided \( \displaystyle{M \neq 0 }\).
\( \displaystyle{ \lim_{x \to a} (f \circ g)(x) = \lim_{x \to a} f(g(x)) = \lim_{y \to M} f(y) }\), provided the latter limit exists. In particular, under these assumptions, the limits above exist and have the given values.

Note that these rules also work for left- and right-handed limits individually.

Infinite Limits

Sometimes, a limit might fail to exist in an interesting way. In particular, the limit value might grow unboundedly. In such a case, we say that the limit is infinite.

We might also be interested in the end behaviour of a function, i.e., the limiting behaviour of the function as the input grows unboundedly. These are another type of infinite limit.

Continuity

A function \( f \) is continuous at \( x = a \) when \( \displaystyle{f(a) = \lim_{x \to a} f(x)} \). Note that this means both the left- and right-hand limits exist and are equal to the function's value at \( a \). We say \( f \) is continuous on \( S \) when \( f \) is continuous at \( a \) for all \( a \in S \).

Let \( f \) and \( g \) be continuous at \( a \), and let \( c \) be a constant. Then…

\( f + g \) is continuous at \( a \).
\( f - g \) is continuous at \( a \).
\( f \cdot g \) is continuous at \( a \).
If \( g(x) \neq 0 \) for all \( x \) near \( a \), then \( f / g \) is continuous at \( a \).

Every polynomial is continuous on \( \R \).

Every rational function is continuous on its domain.

The sine, cosine, natural exponential, and natural logarithm functions are continuous on their domains.

The above propositions are extremely valuable. Taken together, they say we can use algebra in the context of limits, provided we know that the function's pieces are continuous. The following is fundamental to many results in the calculus.

If \( f \) is continuous on \( [a, b] \) and either \( f(a) \leq r \leq f(b) \) or \( f(a) \geq r \geq f(b) \), then there is a \( c \in [a, b] \) such that \( f(c) = r \).

Derivatives

The derivative of a function \( f \) at \( x = a \) is \( \displaystyle{f'(a) = \lim_{x \to a} \frac{f(x) - f(a)}{x - a}}\), provided the limit exists. The \( n^{\textrm{th} \) derivative} of \( f \) at \( a \) is denoted \( f^{(n)}(a) \), and is defined recursively by \( f^{(0)}(a) = f(a) \), and \( f^{(n)}(a) = (f^{(n-1)})'(a) \).

A function \( f \) is differentiable at \( x = a \) when \( f'(a) \) exists; moreover, \( f \) is differentiable on \( S \) when \( f \) is differentiable at \( a \) for all \( a \in S \).

The derivative has many associated notations. We often use \( f' \) or \( \leibniz{f}{x} \) for the first derivative as a function; the latter is the Leibniz notation for the derivative. We also denote the higher derivatives in Leibniz notation by \( \leibniz[n]{f}{x} \). Finally, in Leibniz notation, we denote an evaluation of the derivative at \( x = a \) by \( \leibnizEval{f}{x}{a} \).

We favour the Leibniz notation when we denote that we are taking a derivative. For example, we often write \( \derive{\cos(x)}{x} \) to indicate a promise to take the derivative of \( \cos(x) \).

Another common formulation of the derivative is \(\displaystyle{ f'(x) = \lim_{h \to 0} \frac{f(x + h) - f(x)}{h} }\).

Intuitively, the derivative captures instantaneous rate of change.

Table of Common Derivatives

We have the following.

Power Rule: \( \derive{x^n}{x} = n x^{n-1} \)
Exponential: \( \derive{e^x}{x} = e^x \)
Logarithm: \( \derive{\ln|x|}{x} = \frac{1}{x} \)
Cosine: \( \derive{\sin(x)}{x} = \cos(x) \)
Sine: \( \derive{\cos(x)}{x} = -\sin(x) \)

Derivative Rules

This section documents the interactions between function operations and the derivative.

Let \( f \) and \( g \) be differentiable at \( x = a \).

Sum Rule: \( (f + g)'(a) = f'(a) + g'(a) \).
Product Rule: \( (f \cdot g)'(a) = f'(a) g(a) + f(a) g'(a) \).
Chain Rule: \( (f \circ g)'(a) = f'(g(a)) g'(a) \), provided \( f'(g(a)) \) exists.

These three rules are often expressed as follows, in the context of the derivative as a function.

Let \( f \) and \( g \) be differentiable.

Sum Rule: \( \derive{f + g}{x} = \leibniz{f}{x} + \leibniz{g}{x} \).
Product Rule: \( \derive{f \cdot g}{x} = \leibniz{f}{x} \cdot g(x) + f(x) \cdot \leibniz{g}{x} \).
Chain Rule: \( \derive{f \circ g}{x} = \leibnizEval{f}{x}{g(x)} \cdot \leibniz{g}{x} \).

There are many other derivative rules, but all of them follow from the three above. Some common "extra rules" are given below.

Let \( f \) and \( g \) be differentiable, and let \( c \) be a constant.

Constant Rule: \( \derive{c}{x} = 0 \).
Constant Multiple Rule: \( \derive{c f}{x} = c \cdot \leibniz{f}{x} \).
Generalized Power Rule: \( \derive{(f(x))^n}{x} = n (f(x))^{n-1} \cdot \leibniz{f}{x} \).
Quotient Rule: \( \derive{f / g}{x} = \frac{\leibniz{f}{x} \cdot g(x) - f(x) \cdot \leibniz{g}{x}}{\left( \leibniz{g}{x} \right)^2} \).

It's a good idea to know these rules as well; sometimes they make derivatives a little easier.

The next proposition says that differentiability is a stronger version of continuity.

If \( f \) is differentiable at \( a \), then \( f \) is continuous at \( a \).

Another important proposition is given below—this is what makes the chain rule work.

Let \( a < b \). If \( f \) is continuous on \( [a, b] \) and differentiable on \( (a, b) \), then there is \( c \in (a, b) \) for which \( f'(c) = \frac{f(b) - f(a)}{b - a} \).

Shape and Derivative

The derivatives of a function determine its shape. This is something we'll see a lot more clearly at the end of this course when we discuss power series representations of functions. From Calculus I, you should have a handle on the following basics.

First for some definitions. Let \( a < b \); a function \( f \) is…

strictly increasing on interval \( (a, b) \) when for all \( a < x < y < b \) we have \( f(x) < f(y) \).
weakly increasing on interval \( (a, b) \) when for all \( a < x < y < b \) we have \( f(x) \leq f(y) \).
strictly decreasing on interval \( (a, b) \) when for all \( a < x < y < b \) we have \( f(x) > f(y) \).
weakly decreasing on interval \( (a, b) \) when for all \( a < x < y < b \) we have \( f(x) \geq f(y) \).

A critical point of a function \( f \) is an input \( x = a \) for which either \( f'(a) = 0 \) or \( f'(a) \) does not exist. Suppose \( a < b \) and \( (a, b) \) does not contain any critical points for the function \( f \). Then \( f \) is differentiable on \( (a, b) \), and the sign of \( f'(c) \) is constant for all \( c \in (a, b) \) by the Intermediate Value Theorem. The sign of the derivative on that interval then tells #+attr_html :result-name Increasing-Decreasing Test

Assume \( a < b \) there are no critical points in \( (a, b) \).

If \( f'(c) > 0 \) for some \( c \in (a, b) \), then \( f \) is strictly increasing on \( (a, b) \).
If \( f'(c) < 0 \) for some \( c \in (a, b) \), then \( f \) is strictly decreasing on \( (a, b) \).

This allows us to determine the rough shape of the curve from first derivative information. If we improve to use the second derivative, then we can get a better idea of the general shape of the curve.

A function \( f \) is concave up on \( (a, b) \) when for every \( a < x < y < b \), the line segment connecting the points \( (x, f(x)) \) and \( (y, f(y)) \) is strictly above the curve. Formally speaking, this can be expressed as the inequality \( (c - x) f(y) - (c - y) f(x) \geq f(c) (y - x) \) for all \( a < x < c < y < b \).

Similarly, \( f \) is concave down on \( (a, b) \) when for every \( a < x < y < b \), the line segment connecting the points \( (x, f(x)) \) and \( (y, f(y)) \) is strictly below the curve. Formally speaking, this can be expressed as the inequality \( (c - x) f(y) - (c - y) f(x) \leq f(c) (y - x) \) for all \( a < x < c < y < b \).

While the concept is a bit difficult to write out, the intuition is relatively easy to convey in words: a function is concave up if it is increasing at an increasing rate. Because the derivative measures the instantaneous rate of change, this says that the function's rate of change is increasing. Combining these observations with the Increasing-Decreasing Test yields the following.

Assume \( f' \) is differentiable and has no critical points in \( (a, b) \).

If \( f''(c) > 0 \) for some \( c \in (a, b) \), then \( f \) is concave up on \( (a, b) \).
If \( f''(c) < 0 \) for some \( c \in (a, b) \), then \( f \) is concave down on \( (a, b) \).

Optimization

Beyond just giving us an idea of what the graph of a function looks like, the derivative can be used to solve problems of optimization. To discuss optima, we need a bit more terminology.

Let \( f \) be a function and let \( a \in \dom(f) \). We say \( a \) is…

a global maximum point of \( f \) if \( f(x) \leq f(a) \) for all \( x \in \dom(f) \).
a local maximum point of \( f \) if \( f(x) \leq f(a) \) for all \( x \) near \( a \).
a global minimum point of \( f \) if \( f(x) \geq f(a) \) for all \( x \in \dom(f) \).
a local minimum point of \( f \) if \( f(x) \geq f(a) \) for all \( a \) near \( a \).

A (local or global) extremum is either a (local or global) maximum or minimum. The (local or global) extreme values of a function are the outputs \( f(a) \) at extrema.

An optimization problem is a problem of the form "find the extreme values of \( f \)", possibly with extra restrictions in context (e.g., maybe we need to ensure the extreme points are positive because they represent lengths).

When hunting for extrema, the following is a useful result.

The extrema of function \( f \) occur only at critical points of \( f \).

In particular, this result tells us where to look for extrema. However, this result does not guarantee the existence of extrema; for that, we need further results.

If \( f \) is continuous on \( [a, b] \), then \( f \) attains its global extrema on \( [a, b] \).

In particular, the previous two theorems combine to tell us how to hunt for global extreme values for a continuous function \( f \) defined on \( [a, b] \). First, find all critical points of \( f \) on \( [a, b] \), and then compute \( f(c) \) for all the critical points; the biggest is the global maximum, and the smallest is the global minimum.

Note that the preceding discussion had little to do with calculus beside. However, if our functions are differentiable, then their calculus can tell us quite a lot about their extrema.

Suppose \( f \) is continuous on \( [a, b] \) and \( c \) is the only critical point on \( (a, b) \).

If \( f \) is increasing on \( (a, c) \) and decreasing on \( (c, b) \), then \( c \) is a local maximum point.
If \( f \) is decreasing on \( (a, c) \) and increasing on \( (c, b) \), then \( c \) is a local minimum point.

In light of the Increasing-Decreasing Test, this can be reduced to a computation of the derivative on points between the critical points of our function. An alternative method is given below. #+html_attr :result-name Second Derivative Test

Suppose \( f \) has a critical point \( c \), and \( f' \) is differentiable on \( (a, b) \) where \( a < b < c \).

If \( f''(c) < 0 \), then \( c \) is a local maximum point.
If \( f''(c) > 0 \), then \( c \) is a local minimum point.