# Newton-Raphson

# Newton's method ¶

Let's use the Newton-Raphson method for finding the root of an equation to write a function that can compute the square root of a number.

Cf. "Why Functional Programming Matters" by John Hughes

## A Generator for Approximations¶

To make a generator that generates successive approximations let’s start by assuming an initial approximation and then derive the function that computes the next approximation:

       a F
    ---------
        a'

### A Function to Compute the Next Approximation¶

This is the equation for computing the next approximate value of the square root:

$a_{i+1} = \frac{(a_i+\frac{n}{a_i})}{2}$

Starting with $\frac{(a_i+\frac{n}{a_i})}{2}$ we can derive the Joy expression to compute it using abstract dummy variables to stand in for actual values. First undivide by two:

$(a_i+\frac{n}{a_i})$ `2 /`

Then unadd terms:

$a_i$ $\frac{n}{a_i}$ `+ 2 /`

Undivide again:

$a_i$ $n$ $a_i$ `/ + 2 /`

Finally deduplicate the $a_i$ term:

$a_i$ $n$ `over / + 2 /`

Let's try out this function `over / + 2 /` on an example:

    F == over / + 2 /

In [1]:

    [F over / + 2 /] inscribe

In order to use this function `F` we have to provide an initial estimate for the value of the square root, and we want to keep the input value `n` handy for iterations (we don't want the user to have to keep reentering it.)

In [2]:

    clear

    5 36 [F] trace

      5 36 • F
      5 36 • over / + 2 /
    5 36 5 • / + 2 /
       5 7 • + 2 /
        12 • 2 /
      12 2 • /
         6 •

    6

The initial estimate can be 2, and we can `cons` the input value onto a quote with `F`:

In [3]:

    [F1 2 swap [F] cons] inscribe

    6

In [4]:

    clear

    36 [F1] trace

          36 • F1
          36 • 2 swap [F] cons
        36 2 • swap [F] cons
        2 36 • [F] cons
    2 36 [F] • cons
    2 [36 F] •

    2 [36 F]

In [5]:

    trace

         2 • 36 F
      2 36 • F
      2 36 • over / + 2 /
    2 36 2 • / + 2 /
      2 18 • + 2 /
        20 • 2 /
      20 2 • /
        10 •

    10

In [6]:

    36 F

    6

In [7]:

    clear

In [8]:

    36 F1 make_generator

    [2 [36 F] codireco]

In [9]:

    x x x first

    6

In [10]:

    144 F1 make_generator x x x x first

    6 12

In [11]:

    clear

    2 [36 F]

    2 [36 F]

In [12]:

    [first] [pop sqr] fork - abs 3 <

    2 [36 F] false

In [13]:

    pop i

    10

In [14]:

    [36 F] [first] [pop sqr] fork - abs 3 <

    10 [36 F] false

In [15]:

    pop i

    6

In [16]:

    [36 F] [first] [pop sqr] fork - abs 3 <

    6 [36 F] true

In [ ]:

In [17]:

    clear

    2

    2

In [18]:

    [] true [i [36 F] [first] [pop sqr] fork - abs 3 >] loop pop

    6

In [19]:

    clear

    7 [] true [i [144 F] [first] [pop sqr] fork - abs 3 >] loop pop

    12

In [20]:

    clear

    7 [] true [i [14400 F] [first] [pop sqr] fork - abs 3 >] loop pop

    120

broken due to no float div

    clear

    7 [] true [i [1000 F] [first] [pop sqr] fork - abs 10 >] loop pop

### Make it into a Generator¶

Our generator would be created by:

    a [dup F] make_generator

With n as part of the function F, but n is the input to the sqrt function we’re writing. If we let 1 be the initial approximation:

    1 n 1 / + 2 /
    1 n/1   + 2 /
    1 n     + 2 /
    n+1       2 /
    (n+1)/2

The generator can be written as:

    23 1 swap  [over / + 2 /] cons [dup] swoncat make_generator
    1 23       [over / + 2 /] cons [dup] swoncat make_generator
    1       [23 over / + 2 /]      [dup] swoncat make_generator
    1   [dup 23 over / + 2 /]                    make_generator

In [ ]:

    define('gsra 1 swap [over / + 2 /] cons [dup] swoncat make_generator')

In [ ]:

    J('23 gsra')

Let's drive the generator a few time (with the `x` combinator) and square the approximation to see how well it works...

In [ ]:

    J('23 gsra 6 [x popd] times first sqr')

## Finding Consecutive Approximations within a Tolerance¶

From "Why Functional Programming Matters" by John Hughes:

> The remainder of a square root finder is a function within, which takes a tolerance and a list of approximations and looks down the list for two successive approximations that differ by no more than the given tolerance.

(And note that by “list” he means a lazily-evaluated list.)

Using the output `[a G]` of the above generator for square root approximations, and further assuming that the first term a has been generated already and epsilon ε is handy on the stack...

       a [b G] ε within
    ---------------------- a b - abs ε <=
          b

       a [b G] ε within
    ---------------------- a b - abs ε >
       b [c G] ε within

### Predicate¶

    a [b G]             ε [first - abs] dip <=
    a [b G] first - abs ε                   <=
    a b           - abs ε                   <=
    a-b             abs ε                   <=
    abs(a-b)            ε                   <=
    (abs(a-b)<=ε)

In [ ]:

    define('_within_P [first - abs] dip <=')

### Base-Case¶

    a [b G] ε roll< popop first
      [b G] ε a     popop first
      [b G]               first
       b

In [ ]:

    define('_within_B roll< popop first')

### Recur¶

    a [b G] ε R0 [within] R1

1. Discard a.
2. Use `x` combinator to generate next term from `G`.
3. Run `within` with `i` (it is a "tail-recursive" function.)

Pretty straightforward:

    a [b G]        ε R0           [within] R1
    a [b G]        ε [popd x] dip [within] i
    a [b G] popd x ε              [within] i
      [b G]      x ε              [within] i
    b [c G]        ε              [within] i
    b [c G]        ε               within

    b [c G] ε within

In [ ]:

    define('_within_R [popd x] dip')

### Setting up¶

The recursive function we have defined so far needs a slight preamble: `x` to prime the generator and the epsilon value to use:

    [a G] x ε ...
    a [b G] ε ...

In [ ]:

    define('within x 0.000000001 [_within_P] [_within_B] [_within_R] tailrec')
    define('sqrt gsra within')

Try it out...

In [ ]:

    J('36 sqrt')

In [ ]:

    J('23 sqrt')

Check it.

In [ ]:

    4.795831523312719**2

In [ ]:

    from math import sqrt

    sqrt(23)