Secant method

Derivation of the method

Starting with initial values x₀ and x₁, we construct a line through the points (x₀, f(x₀)) and (x₁, f(x₁)), as shown in the picture above. In point–point form, the equation of this line is

${\displaystyle y=\frac{f(x_1)-f(x_0)}{x_1-x_0}(x-x_1)+f(x_1).}$

The root of this linear function, that is the value of x such that y = 0 is

${\displaystyle x=x_1-f(x_1)\frac{x_1-x_0}{f(x_1)-f(x_0)}.}$

We then use this new value of x as x₂ and repeat the process, using x₁ and x₂ instead of x₀ and x₁. We continue this process, solving for x₃, x₄, etc., until we reach a sufficiently high level of precision (a sufficiently small difference between x_n and x_n−1):

${\displaystyle \begin{array}{cc}{x}_2& =x_1-f(x_1)\frac{x_1-x_0}{f(x_1)-f(x_0)},\\ x_3& =x_2-f(x_2)\frac{x_2-x_1}{f(x_2)-f(x_1)},\\ & ⋮\\ x_n& =x_{n-1}-f(x_{n-1})\frac{x_{n-1}-x_{n-2}}{f(x_{n-1})-f(x_{n-2})}.\end{array}}$

Convergence

The iterates ${\displaystyle x_n}$ of the secant method converge to a root of ${\displaystyle f}$ if the initial values ${\displaystyle x_0}$ and ${\displaystyle x_1}$ are sufficiently close to the root and ${\displaystyle f}$ is well-behaved. When ${\displaystyle f}$ is twice continuously differentiable and the root in question is a simple root, i.e., it has multiplicity 1, the order of convergence is the golden ratio ${\displaystyle \phi =(1+\sqrt{5})/2\approx \mathrm{1.618.}}$ This convergence is superlinear but subquadratic.

If the initial values are not close enough to the root or ${\displaystyle f}$ is not well-behaved, then there is no guarantee that the secant method converges at all. There is no general definition of "close enough", but the criterion for convergence has to do with how "wiggly" the function is on the interval between the initial values. For example, if ${\displaystyle f}$ is differentiable on that interval and there is a point where ${\displaystyle f^{\prime }=0}$ on the interval, then the algorithm may not converge.

Comparison with other root-finding methods

The secant method does not require or guarantee that the root remains bracketed by sequential iterates, like the bisection method does, and hence it does not always converge. The false position method (or regula falsi) uses the same formula as the secant method. However, it does not apply the formula on ${\displaystyle x_{n-1}}$ and ${\displaystyle x_{n-2}}$, like the secant method, but on ${\displaystyle x_{n-1}}$ and on the last iterate ${\displaystyle x_k}$ such that ${\displaystyle f(x_k)}$ and ${\displaystyle f(x_{n-1})}$ have a different sign. This means that the false position method always converges; however, only with a linear order of convergence. Bracketing with a super-linear order of convergence as the secant method can be attained with improvements to the false position method (see Regula falsi § Improvements in regula falsi) such as the ITP method or the Illinois method.

The recurrence formula of the secant method can be derived from the formula for Newton's method

${\displaystyle x_n=x_{n-1}-\frac{f(x_{n-1})}{f^{\prime }(x_{n-1})}}$

by using the finite-difference approximation, for a small ${\displaystyle ϵ=x_{n-1}-x_{n-2}}$:

${\displaystyle f^{\prime }(x_{n-1})=\underset{ϵ\to 0}{lim}\frac{f(x_{n-1})-f(x_{n-1}-ϵ)}{ϵ}\approx \frac{f(x_{n-1})-f(x_{n-2})}{x_{n-1}-x_{n-2}}}$

The secant method can be interpreted as a method in which the derivative is replaced by an approximation and is thus a quasi-Newton method.

If we compare Newton's method with the secant method, we see that Newton's method converges faster (order 2 against order the golden ratio φ ≈ 1.6). However, Newton's method requires the evaluation of both ${\displaystyle f}$ and its derivative ${\displaystyle f^{\prime }}$ at every step, while the secant method only requires the evaluation of ${\displaystyle f}$. Therefore, the secant method may sometimes be faster in practice. For instance, if we assume that evaluating ${\displaystyle f}$ takes as much time as evaluating its derivative and we neglect all other costs, we can do two steps of the secant method (decreasing the logarithm of the error by a factor φ² ≈ 2.6) for the same cost as one step of Newton's method (decreasing the logarithm of the error by a factor of 2), so the secant method is faster. In higher dimensions, the full set of partial derivatives required for Newton's method, that is, the Jacobian matrix, may become much more expensive to calculate than the function itself. If, however, we consider parallel processing for the evaluation of the derivative or derivatives, Newton's method can be faster in clock time though still costing more computational operations overall.

Practical considerations

Mathematically, the following two forms of the Secant method are equivalent: $${\displaystyle x_{n+1}=x_n-f(x_n)\left[\frac{x_n-x_{n-1}}{f(x_n)-f(x_{n-1})}\right]}$$ and $${\displaystyle x_{n+1}=\frac{x_{n-1}f(x_n)-x_{n}f(x_{n-1})}{f(x_n)-f(x_{n-1})}.}$$

When approximate arithmetic is used, for example pen-and-paper arithmetic carried out to a fixed number of decimal places or the floating-point binary arithmetic available on computers, the first version is preferable for two reasons:

• It is of the form ${\displaystyle x_{n+1}=x_n-f(x_n)\mathrm{\Delta }}$, for some number ${\displaystyle \mathrm{\Delta }}$. Even if ${\displaystyle \mathrm{\Delta }}$ is inaccurate, the change in the estimate of the root will be small at convergence because ${\displaystyle f(x_n)}$ will also be small.
• The second form is susceptible to catastrophic cancellation, since ${\displaystyle f(x_n)\to f(x_{n-1})}$ as convergence approaches, so cancellation error in the denominator can be large.

Generalization

Broyden's method is a generalization of the secant method to more than one dimension.

The following graph shows the function f in red and the last secant line in bold blue. In the graph, the x intercept of the secant line seems to be a good approximation of the root of f.

Example

Suppose we seek a solution of the polynomial

${\displaystyle f(x)=2x^3-x-3.}$

Applying the Secant method with initial guesses ${\displaystyle x_0=1}$ and ${\displaystyle x_1=2}$ gives the following approximations:

Algorithm

The following pseudocode describes the Secant method for approximating a root of the equation ${\displaystyle f(x)=0}$. It returns an estimate ${\displaystyle x_2}$ that satisfies ${\displaystyle |x_2-x_1|<\epsilon }$, where ${\displaystyle \epsilon }$ is a user-specified tolerance.

<b id="mw5Q">input:</b> Real valued function, <i id="mw5g">f</i> Initial approximations, <i id="mw5w">x0</i> and <i id="mw6A">x1</i> Tolerance, <i id="mw6Q">ε</i> Maximum iterations, <i id="mw6g">Nmax</i> <b id="mw6w">output:</b> A value <i id="mw7A">x</i> such that <i id="mw7Q">f</i>(<i id="mw7g">x</i>) ≈ 0 <b id="mw7w">for</b> <i id="mw8A">n</i> = 0 to <i id="mw8Q">Nmax</i> <b id="mw8g">do</b> <i id="mw8w">x2</i> ← <i id="mw9A">x1</i> - <i id="mw9Q">f</i>(<i id="mw9g">x1</i>)·(<i id="mw9w">x1</i> - <i id="mw-A">x0</i>) / (<i id="mw-Q">f</i>(<i id="mw-g">x1</i>) - <i id="mw-w">f</i>(<i id="mw_A">x0</i>)) <b id="mw_Q">if</b> |<i id="mw_g">x2</i> − <i id="mw_w">x1</i>| < <i id="mwAQA">ε</i> <b id="mwAQE">then</b> <b id="mwAQI">return</b> <i id="mwAQM">x2</i> <b id="mwAQQ">end if</b> <i id="mwAQU">x0</i>, <i id="mwAQY">x1</i> ← <i id="mwAQc">x1</i>, <i id="mwAQg">x2</i> <b id="mwAQk">end for</b> <b id="mwAQo">return</b> <i id="mwAQs">x2</i>