Differential equation

A differential equation is an equation that relates an unknown function to its own derivatives.

Two features are common to all differential equations:

• the unknown is a function, not a number;
• a solution is rarely unique, but instead often a family of functions containing arbitrary constants. A particular member of the solution family can be singled out by extra conditions, such as the state of the system at some time;

Types of differential equations

Ordinary and partial differential equations

Ordinary differential equations

An ordinary differential equation (ODE) describes an unknown function which depends on a single independent variable.

Writing the unknown as y(x), an ODE can be expressed in its implicit form:

F\bigl(x,y,y',y'',\ldots,y^{(n)}\bigr)=0

where F is a given function.

If the equation can be solved for the highest derivative, it takes an explicit form.

For a first-order equation:

\frac{dy}{dx}=f(x,y)

The right-hand side f(x, y) prescribes the slope that any solution function must have at the point (x, y).

For an equation of order n that is solved for its highest derivative:

y^{(n)}=f\bigl(x,y,y',y'',\ldots,y^{(n-1)}\bigr)

Integrating the equation once removes one derivative and introduces one arbitrary constant. Therefore, a problem of order n generally needs n conditions to single out one particular solution of the solution family.

Partial differential equations

A partial differential equation describes an unknown function that depends on two or more independent variables. Its general form is:

F\Bigl(x_1,x_2,\ldots,x_n,\ u,\ \frac{\partial u}{\partial x_1},\ldots,\ \frac{\partial^{2}u}{\partial x_i\,\partial x_j},\ldots\Bigr)=0

Order and linearity

The order of a differential equation is the order of the highest derivative that appears in it.

A differential equation is linear if the unknown function and its derivatives appear only to the first power and are never multiplied together (they may be multiplied by functions of the independent variables).

An n-th order linear ODE therefore has the form

a_n(x)\,y^{(n)}+a_{n-1}(x)\,y^{(n-1)}+\cdots+a_1(x)\,y'+a_0(x)\,y=g(x)

We consider this equation homogeneous when g(x) = 0.

Linear homogeneous equations obey the superposition principle: any linear combination of solutions is again a solution.

Differential equations as mathematical models

Mechanics

Force and acceleration

Newton's second law of motion states that the acceleration of a body is proportional to the net force acting on it:

m\frac{d^{2}x}{dt^{2}}=F

where x(t) is the position of the body, m its mass, and the force F may itself depend on time, position and velocity.

Harmonic oscillations

For a mass attached to a spring whose restoring force is linear in the displacement, this becomes the harmonic oscillator equation:

\frac{d^{2}x}{dt^{2}}+\omega_{0}^{2}x=0

whose solutions are sinusoidal oscillations at the natural frequency ω₀; such motion is called simple harmonic motion.

A mass on a spring executing simple harmonic motion, the solution of the harmonic oscillator equation. Credit: Evil saltine (public domain).

Exponential growth and decay

A quantity that changes at a rate proportional to its current size satisfies the exponential growth or decay equation:

\frac{dy}{dt}=k\,y

Its solutions are exponentials:

y(t)=y_{0}\,e^{kt}

When k > 0, it can describe a population with unlimited resources that grows exponentially, whereas k < 0 describes a radioactive substance that decays exponentially towards zero.

Heat transfer

Newton's law of cooling describes a body whose temperature T differs from a constant ambient temperature Tₐ:

\frac{dT}{dt}=-k\bigl(T-T_{a}\bigr)

The temperature difference decays exponentially, so the body approaches the ambient temperature but never quite reaches it.

Geometry of first-order equations

Direction fields

As described above, a first-order equation can be written explicitly as dy/dx = f(x, y), which assigns to every point (x, y) the slope that a solution curve must have there.

Drawing a short line segment of that slope at many points produces a direction field (or slope field). Every solution curve must stay tangent to the field directions everywhere.

Direction field of the equation dy/dx = y, with several solution curves. Each segment shows the local slope; the solutions are exponential curves. Credit: jjbeard (public domain).

As such, direction fields make the qualitative behaviour of solutions visible even when no closed-form solution exists.

Solving differential equations

Solving a differential equation means finding its solutions. When a solution can be written as a closed-form expression, that is, as a finite combination of elementary functions (polynomials, rational functions, exponentials, logarithms, trigonometric functions and their inverses) built with the usual algebraic operations, it can be evaluated directly at any point; the exponential and sinusoidal solutions met above are of this kind. Most differential equations, however, admit no closed-form solution. Their solutions may still be represented exactly in an open form: as an infinite series (a power series or a Fourier series), as an integral, or through special functions defined by such series. Where even an open form is not available in practice, numerical methods approximate the solution to any desired accuracy.

Analytical methods

The classical methods exploit structure in the equation to produce closed forms where possible, and series or transform representations otherwise:

Separation of variables. When the right-hand side of a first-order equation splits into a function of x times a function of y, the two variables can be separated onto opposite sides and both sides integrated. Applied to the equation dy/dx = −x/y, this gives y dy = −x dx, and integrating yields the family of circles:

\frac{dy}{dx}=-\frac{x}{y}\qquad\Longrightarrow\qquad x^{2}+y^{2}=C

Integrating factors. A linear first-order equation dy/dx + p(x)y = q(x) is multiplied by the integrating factor μ(x) = e^{∫p dx}, chosen so that the left-hand side becomes the derivative of μy; integrating then solves the equation. For instance,

\frac{dy}{dx}+y=e^{-x}\qquad\Longrightarrow\qquad y=(x+C)e^{-x}

The related Bernoulli equation dy/dx + p(x)y = q(x)yⁿ (n ≠ 0, 1) becomes linear after the substitution v = y¹⁻ⁿ.

Characteristic equations. Linear equations with constant coefficients admit exponential trial solutions y = e^rx. Substitution turns the differential equation into an algebraic characteristic equation for r, whose roots build the solution from exponentials, and from sines and cosines when the roots are complex. The second-order equation

y''-3y'+2y=0\qquad\Longrightarrow\qquad y=C_{1}e^{x}+C_{2}e^{2x}

is solved this way, since its characteristic equation r² − 3r + 2 = 0 has roots 1 and 2. For the forced (non-homogeneous) version y″ − 3y′ + 2y = e^x, the method of variation of parameters replaces the constants C₁ and C₂ by functions of x and gives y = C₁e^x + C₂e^2x − xe^x.

Power series. When the coefficients are not constant, a solution is sought as a power series y = Σ aₙxⁿ; substituting it into the equation and equating coefficients yields recurrence relations that determine the aₙ one after another. About a regular singular point, the method of Frobenius extends the same idea to series with a fractional leading power. Even when the equation admits no elementary solution, the series itself is an exact solution in open form. For example, the Airy equation

y''-xy=0\qquad\Longrightarrow\qquad y=C_{1}\operatorname{Ai}(x)+C_{2}\operatorname{Bi}(x)

has no closed-form solution in elementary functions; its solutions are the Airy functions, which are defined by such series (or equivalent integrals).

Laplace transforms. Under the Laplace transform, differentiation becomes multiplication by the transform variable, so an initial value problem becomes an algebraic equation for the transformed solution, which is inverted to give the answer. The method is especially convenient for problems with prescribed initial values and with forcing terms that jump. For example,

y''+y=1,\qquad y(0)=0,\quad y'(0)=0\qquad\Longrightarrow\qquad y(t)=1-\cos t

Fourier series for partial differential equations. For linear PDEs, separation of variables looks for product solutions u(x, t) = X(x)T(t); by the superposition principle, sums of such products solve the equation, and the coefficients are fixed by the initial data. For the heat equation on a rod of length L whose ends are held at temperature zero, this gives the Fourier sine series

u(x,t)=\sum_{n=1}^{\infty}b_{n}\sin\Bigl(\frac{n\pi x}{L}\Bigr)\,e^{-\alpha (n\pi/L)^{2}t}

an exact solution in open form whose terms decay at rates set by the thermal diffusivity α.

The theory of these methods is treated in standard textbooks on ordinary and partial differential equations. Still, only a minority of differential equations, almost all of them linear, can be solved explicitly; most equations that arise in applications are treated by numerical or qualitative methods.

Numerical methods

When no closed-form solution is available, solutions are approximated by numerical integration. The simplest scheme, Euler's method, advances an approximate solution yₙ at points xₙ = x₀ + nh by repeatedly applying one step:

y_{n+1}=y_{n}+h\,f(x_{n},y_{n})

Euler's method approximates a solution curve by following the local slope over short steps; smaller steps follow the exact curve more closely. Credit: Oleg Alexandrov (public domain).

The error introduced in a single step is proportional to the square of the step size h, so over a fixed interval the accumulated error shrinks in proportion to h; the polygonal approximation therefore converges to the exact solution as h tends to zero. More accurate and more stable methods, such as the Runge–Kutta family, are built on the same idea of using the differential equation to extrapolate from known values, and are implemented in essentially every numerical computing environment.

Qualitative methods

For nonlinear equations, and especially for systems of two or more equations, the geometry of solutions matters as much as their formulas. Qualitative analysis studies the equilibrium points at which the system does not change and asks whether each is stable: a small push away from a stable equilibrium is followed by a return towards it, whereas the system leaves an unstable one. A pendulum hanging downwards is a stable equilibrium of its equation of motion; balanced exactly upright, it is unstable, and the slightest disturbance topples it. Solutions are then understood through the way they flow between equilibria in the phase plane, which for a pendulum consists of closed loops around the stable equilibrium, corresponding to periodic swinging.

As a parameter changes, the behaviour can reorganise itself in a bifurcation. A familiar example is a fish population that is harvested: while the harvest is mild the population settles to a stable equilibrium, but as the harvesting effort passes a critical value the surviving equilibria collide and vanish, and the population collapses.

In three or more dimensions a system can settle onto a chaotic attractor: the solutions remain bounded, deterministic and aperiodic, and they are extremely sensitive to initial conditions, so that two nearly identical starting states diverge quickly. Edward Lorenz first observed such behaviour in 1963 in a simple system of three differential equations modelling atmospheric convection, which is why long-term weather prediction is impossible in practice; the phenomenon has since been found throughout physics, biology, chemistry and economics.

History

Portrait of Isaac Newton (after Godfrey Kneller, 1689). Newton's laws of motion and of universal gravitation, published in the Principia (1687), are differential equations. Credit: James Thronill after Godfrey Kneller (public domain).

Differential equations arose with the calculus that Isaac Newton and Gottfried Wilhelm Leibniz developed independently in the late 17th century. Newton's laws of motion and of universal gravitation, published in his Principia of 1687, are differential equations. However, it was Leibniz who introduced the notation dy/dx still used today.

In the 18th century Leonhard Euler turned a collection of ad hoc tricks into a systematic theory, developing the general treatment of linear equations with constant coefficients, series methods, and the first numerical integration scheme, which still bears his name.

Leonhard Euler (portrait by Jakob Emanuel Handmann, 1753). Euler created much of the systematic theory of differential equations and gave his name to Euler's method. Credit: Jakob Emanuel Handmann (public domain).

Alongside the theory of ordinary equations, the 18th and 19th centuries saw the development of partial differential equations of physics. Jean le Rond d'Alembert wrote down and solved the one-dimensional wave equation for the vibrating string in the 1740s, and Joseph Fourier derived the heat equation from the physics of conduction and solved it with trigonometric series in his Théorie analytique de la chaleur of 1822, founding Fourier analysis. Augustin-Louis Cauchy and, later, Rudolf Lipschitz and Émile Picard placed the existence and uniqueness of solutions on a rigorous footing.

The modern theory of dynamical systems grew out of a qualitative, geometric way of studying differential equations pioneered at the end of the 19th century by Henri Poincaré, while he worked on the three-body problem of celestial mechanics, and by Aleksandr Lyapunov, whose 1892 thesis gave a general definition of stability. Their viewpoint, combined with the growing power of numerical computing, matured during the 20th century into the theory of dynamical systems that now pervades the sciences. In 1963, meteorologist Edward Lorenz found chaotic behaviour in a simple system of three differential equations modelling atmospheric convection, a landmark in the study of nonlinear dynamics: although the system was deterministic, its solutions were aperiodic and so sensitive to initial conditions that nearby states rapidly diverged, an effect he popularised as the butterfly effect.

Further reading

• Differential equation, Wikipedia
• Differential Equation, Wolfram MathWorld
• 18.03SC Differential Equations, MIT OpenCourseWare
• Leonhard Euler, MacTutor History of Mathematics