Thun/docs/Newton-Raphson.ipynb

Newton's method

Let's use the Newton-Raphson method for finding the root of an equation to write a function that can compute the square root of a number.

Cf. "Why Functional Programming Matters" by John Hughes

A Generator for Approximations

To make a generator that generates successive approximations let’s start by assuming an initial approximation and then derive the function that computes the next approximation:

   a F
---------
    a'

A Function to Compute the Next Approximation

This is the equation for computing the next approximate value of the square root:

a_{i+1} = \frac{(a_i+\frac{n}{a_i})}{2}

Starting with \frac{(a_i+\frac{n}{a_i})}{2} we can derive the Joy expression to compute it using abstract dummy variables to stand in for actual values. First undivide by two:

(a_i+\frac{n}{a_i}) 2 /

Then unadd terms:

a_i \frac{n}{a_i} + 2 /

Undivide again:

a_i n a_i / + 2 /

Finally deduplicate the a_i term:

a_i n over / + 2 /

Let's try out this function over / + 2 / on an example:

F == over / + 2 /

In [1]:

[F over / + 2 /] inscribe

In order to use this function F we have to provide an initial estimate for the value of the square root, and we want to keep the input value n handy for iterations (we don't want the user to have to keep reentering it.)

In [2]:

clear

5 36 [F] trace

  5 36 • F
  5 36 • over / + 2 /
5 36 5 • / + 2 /
   5 7 • + 2 /
    12 • 2 /
  12 2 • /
     6 • 

6

The initial estimate can be 2, and we can cons the input value onto a quote with F:

In [3]:

[F1 2 swap [F] cons] inscribe

6

In [4]:

clear

36 [F1] trace

      36 • F1
      36 • 2 swap [F] cons
    36 2 • swap [F] cons
    2 36 • [F] cons
2 36 [F] • cons
2 [36 F] • 

2 [36 F]

In [5]:

trace

     2 • 36 F
  2 36 • F
  2 36 • over / + 2 /
2 36 2 • / + 2 /
  2 18 • + 2 /
    20 • 2 /
  20 2 • /
    10 • 

10

In [6]:

36 F

6

In [7]:

clear

In [8]:

36 F1 make_generator

[2 [36 F] codireco]

In [9]:

x x x first

6

In [10]:

144 F1 make_generator x x x x first

6 12

In [11]:

clear

2 [36 F]

2 [36 F]

In [12]:

[first] [pop sqr] fork - abs 3 <

2 [36 F] false

In [13]:

pop i

10

In [14]:

[36 F] [first] [pop sqr] fork - abs 3 <

10 [36 F] false

In [15]:

pop i

6

In [16]:

[36 F] [first] [pop sqr] fork - abs 3 <

6 [36 F] true

In [ ]:

In [17]:

clear

2

2

In [18]:

[] true [i [36 F] [first] [pop sqr] fork - abs 3 >] loop pop

6

In [19]:

clear

7 [] true [i [144 F] [first] [pop sqr] fork - abs 3 >] loop pop

12

In [20]:

clear

7 [] true [i [14400 F] [first] [pop sqr] fork - abs 3 >] loop pop

120

broken due to no float div

clear

7 [] true [i [1000 F] [first] [pop sqr] fork - abs 10 >] loop pop

Make it into a Generator

Our generator would be created by:

a [dup F] make_generator

With n as part of the function F, but n is the input to the sqrt function we’re writing. If we let 1 be the initial approximation:

1 n 1 / + 2 /
1 n/1   + 2 /
1 n     + 2 /
n+1       2 /
(n+1)/2

The generator can be written as:

23 1 swap  [over / + 2 /] cons [dup] swoncat make_generator
1 23       [over / + 2 /] cons [dup] swoncat make_generator
1       [23 over / + 2 /]      [dup] swoncat make_generator
1   [dup 23 over / + 2 /]                    make_generator

In [ ]:

define('gsra 1 swap [over / + 2 /] cons [dup] swoncat make_generator')

In [ ]:

J('23 gsra')

Let's drive the generator a few time (with the x combinator) and square the approximation to see how well it works...

In [ ]:

J('23 gsra 6 [x popd] times first sqr')

Finding Consecutive Approximations within a Tolerance

From "Why Functional Programming Matters" by John Hughes:

The remainder of a square root finder is a function within, which takes a tolerance and a list of approximations and looks down the list for two successive approximations that differ by no more than the given tolerance.

(And note that by “list” he means a lazily-evaluated list.)

Using the output [a G] of the above generator for square root approximations, and further assuming that the first term a has been generated already and epsilon ε is handy on the stack...

   a [b G] ε within
---------------------- a b - abs ε <=
      b


   a [b G] ε within
---------------------- a b - abs ε >
   b [c G] ε within

Predicate

a [b G]             ε [first - abs] dip <=
a [b G] first - abs ε                   <=
a b           - abs ε                   <=
a-b             abs ε                   <=
abs(a-b)            ε                   <=
(abs(a-b)<=ε)

In [ ]:

define('_within_P [first - abs] dip <=')

Base-Case

a [b G] ε roll< popop first
  [b G] ε a     popop first
  [b G]               first
   b

In [ ]:

define('_within_B roll< popop first')

Recur

a [b G] ε R0 [within] R1

1. Discard a.
2. Use x combinator to generate next term from G.
3. Run within with i (it is a "tail-recursive" function.)

Pretty straightforward:

a [b G]        ε R0           [within] R1
a [b G]        ε [popd x] dip [within] i
a [b G] popd x ε              [within] i
  [b G]      x ε              [within] i
b [c G]        ε              [within] i
b [c G]        ε               within

b [c G] ε within

In [ ]:

define('_within_R [popd x] dip')

Setting up

The recursive function we have defined so far needs a slight preamble: x to prime the generator and the epsilon value to use:

[a G] x ε ...
a [b G] ε ...

In [ ]:

define('within x 0.000000001 [_within_P] [_within_B] [_within_R] tailrec')
define('sqrt gsra within')

Try it out...

In [ ]:

J('36 sqrt')

In [ ]:

J('23 sqrt')

Check it.

In [ ]:

4.795831523312719**2

In [ ]:

from math import sqrt

sqrt(23)