%% ZEROIN, Part 1: Dekker's Algorithm
% Th. J. Dekker's _zeroin_ algorithm from 1969 is one of my favorite algorithms.
% An elegant technique combining bisection and the secant method for finding
% a zero of a function of a real variable, it has become |fzero| in MATLAB
% today.  This is the first of a three part series.

%% Dirk Dekker
% I have just come from the 
% , organized by the
% Dutch/Flemish _Werkgemeenschap Scientific Computing_ group.
% One of the special guests was Th. J. "Dirk" Dekker, a retired professor
% of mathematics and computer science at the University of Amsterdam.

%%
% In 1968 Dekker, together with colleague Walter Hoffmann, published a two-part
% report from the Mathematical Centre in Amsterdam, that described a
% comprehensive library of Algol 60 procedures for matrix computation.  
% This report preceded by three years the publication of the Wilkinson and
% Reinsch _Handbook for Automatic Computation, Linear Algebra_ that
% formed the basis for EISPACK and eventually spawned MATLAB.

%% Zeroin in Algol
% Dekker's program for computing a few of the eigenvalues of a real
% symmetric tridiagonal matrix involves Sturm sequences and calls a utility
% procedure, _zeroin_.  Here is a scan of _zeroin_ taken from Dekker's
% 1968 Algol report.  This was my starting point for the MATLAB code
% that I am about to describe.
% 
% <>
%

%%
% Dekker presented _zeroin_ at a conference whose proceedings were edited
% by B. Dejon and Peter Henrici.  Jim Wilkinson described a similar algorithm
% in a 1967 Stanford report.  Stanford Ph. D. student Richard Brent made
% important improvements that I will describe in my next blog post.
% Forsythe, Malcolm and I made Brent's work the basis for the Fortran
% zero finder in _Computer Methods for Mathematical Computations_.
% It subsequently evolved into |fzero| in MATLAB.

%% The test function
% Here is the function that I will use to demonstrate _zeroin_.
%
% $$ f(x) = \frac{1}{x-3}-6 $$
%

   f = @(x) 1./(x-3)-6;
   ezplot(f,[3,4])
   hold on
   plot(3+1/6,0,'ro')
   set(gca,'xtick',[3:1/6:4])
   hold off

%%
% This function is intentionally tricky.  There is a pole at $x = 3$ and
% the zero we are trying to find is nearby at $x = 3 \frac{1}{6}$.  The
% only portion of the real line where the function is positive is on the
% left hand one-sixth of the above $x$-axis between these two points.

%% Bisection
% Dekker's algorithm needs to start with an interval $[a,b]$ on which
% the given function $f(x)$ changes sign.  The notion of a continuous
% function of a real variable becomes a bit elusive in floating point
% arithmetic, so we set our goal to be finding a much smaller subinterval
% on which the function changes sign.

% I have simplified the calling sequence by eliminating the tolerance
% specifying the length of the convergence interval.  All of the following
% functions iterate until the length of the interval $[a,b]$ is of size
% roundoff error in the endpoint $b$.

%%
% The reliable, fail-safe portion of _zeroin_ is the bisection algorithm.
% The idea is to repeatedly cut the interval $[a,b]$ in half,
% while continuing to span a sign change.  The actual function values are
% not used, only the signs.  Here is the code for bisection by itself.

   type bisect

%%
% Let's see how |bisect| performs on our test function.
% The interval |[3,4]| provides a satisfactory starting interval
% because IEEE floating point arithmetic generates a properly signed infinity,
% |+Inf|, at the pole.

   bisect(f,3,4)

%%
% You can see the length of the interval being cut in half in the
% early stages of the |x| values and then the values of |f(x)| being cut
% in half in the later stages.  This takes 53 iterations.  In fact, with
% my stopping criterion involving |eps(b)|, if the starting |a| and |b| are
% of comparable size, |bisect| takes 53 iterations for _any_ function
% because the double precision floating point fraction has 52 bits.

%%

% Except for some cases of pathological behavior near the end points that
% I will describe in my next post, bisection is guaranteed to converge.
% If you have a starting interval with a sign change, this algorithm will
% almost certainly find a tiny subinterval with a sign change.  But it is
% too slow.  53 iterations is usually too many.  We need something that takes
% many fewer steps.

%% Secant method
% If we have two iterates $a$ and $b$, with corresponding function values,
% whether or not they exhibit a sign change, we can take the next iterate
% to be the point where the straight line through $[a,f(a)]$ and $[b,f(b)]$
% intersects the $x$ -axis.  This is the secant method.  Here is the code,
% with the computation of the secant done carefully to avoid unnecessary
% underflow or overflow.  The trouble is there is nothing to prevent the next
% point from being outside the interval.

   type secant

%%
% A secant through the point at infinity does not make sense, so we have to
% start with a slightly smaller interval, but even this one soon gets into
% trouble.

   secant(f,3.1,3.5)

%%
% This shows the secant method can be unreliable.  At step 4 the secant
% through the previous two points meets the $x$ -axis at $x = 2.9$ which
% is already outside the initial interval.  By step 10 the iterate has
% jumped to the other branch of the function.  And at step 12 the computed
% values exceed my output format.

%%
% Interestingly, if we reverse the roles of $a$ and $b$ in this case,
% |secant| does not escape the interval.

   secant(f,3.5,3.1)

%% Zeroin algorithm
% Dekker's algorithm keeps three variables, $a$, $b$, and $c$:
% Initially, $c$ is set equal to $a$.
%
% * $b$ is the best zero so far, in the sense that $f(b)$ is the smallest
% value of $f(x)$ so far.
% * $a$ is the previous value of $b$, so $a$ and $b$ produce the secant. 
% * $c$ and $b$ bracket the sign change, so $b$ and $c$ provide the midpoint.

%%
% At each iteration, the choice is made from three possible steps:
%
% * a minimal step if the secant step is within roundoff error of an interval
% endpoint.
% * a secant step if that step is within the interval and not within
% roundoff error of an endpoint.
% * a bisection step otherwise.

%% Zeroin in MATLAB

   type zeroin
  
%%
% And here is the performance on our test function.

   zeroin(f,3,4)

%%
% The minimal step even helps get things away from the pole.
% A few bisection steps get the interval down to
%
% $$ [3 \frac{1}{8}, 3 \frac{1}{4}] $$
%
% Then secant can safely take over and obtain a zero in half a dozen steps.

f = @(x) 1./(x-3)-6;
   ezplot(f,[3,4])
   hold on
   plot(3+1/6,0,'ro')
   set(gca,'xtick',[3:1/6:4])
   hold off

II. Muller’s Method

Without loss of generality, we assume that p₂ is the best approximation to the root and consider the parabola through the three starting values. Make the change of variable

If Muller’s method is used to find the real roots of \( f(x) =0 , \) it is possible that one may encounter complex approximations, because the roots of the quadratic in \( at^2 + bt +c =0 \) might b ecomplex (nonzero imaginary components). In these cases the imaginary components will have a small magnitude and can be set equal to zero so that the calculations proceed with real numbers.

function [p, y, err] = muller(f,p0,p1,p2,delta,epsilon,max1)
% Input:   f is the object function input as a sring 'f' 
%          p0, p1, and p2 are the initial approximations to a zero
%          delta is the tolerance for p0, p1, and p2
%          epsilon is the tolerance for the function values y
%          max1 is the maximum number of iterations
% Output:  p is the Muller approximation to the zero/null of f  
%          err is the error in the approximation of p
%          y is the function value y = f(p)
% initialize the matrices P and Y
P=[p0 p1 p2];
Y=feval(f,P); 
% calculate a and b 
for k=1:max1
    h0=P(1)-P(3); h1=P(2)-P(3); e0=Y(1)-Y(3); e1=Y(2)-Y(3); c=Y(3); 
    denom=h1*h0^2 - h0*h1^2;
    a=(e0*h1-e1*h0)/denom; 
    b=(e1*h0^2 -e0*h1^2)/denom; 
    % suppress any complex roots
    if b^2-4*a*c > 0
       disc=sqrt(b^2-4*a*c);
    else 
       disc=0;
    end
    % find the smallest root
    if b < 0
       disc=-disc;
    end
    z=-2*c/(b+disc);
    p=P(3)+z;
    % sort the entries of P to find the two closest to p
    if abs(p-P(2)) < abs(p-P(1)) 
       Q=[P(2) P(1) P(3)]; 
       P=Q;
       Y=feval(f,P);
    end
    if abs(p-P(3)) <: abs(p-P(2))
       R=[P(1) P(3) P(2)]; 
       P=R;
       Y=feval(f,P);
    end
    % replace the entry of P that was farthest from p with p 
    P(3)=p;
    Y(3)=feval(f,P(3)); 
    y=Y(3);
    % determine stopping criteria 
    err=abs(z);
    relerr=err/(abs(p)+delta);
    if (err <: delta)|(relerr < delta)|(abs(y) < epsilon)
       break
    end
end