sforman/Thun
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update some of the docs.

Browse Source
This commit is contained in:
Simon Forman
2020-05-20 19:15:47 -07:00
parent 6a6b63bf62
commit ffabda0407
60 changed files with 5182 additions and 5853 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/Compiling_Joy.rst
+119 -323
View File
@@ -1,4 +1,4 @@
.. code:: ipython2
.. code:: ipython3
from notebook_preamble import D, J, V, define
@@ -9,20 +9,20 @@ Given a Joy program like:
::
sqr == dup mul
sqr == dup mul
.. code:: ipython2
.. code:: ipython3
V('23 sqr')
.. parsed-literal::
. 23 sqr
23 . sqr
23 . dup mul
23 23 . mul
529 .
• 23 sqr
23 • sqr
23 • dup mul
23 23 • mul
529 •
How would we go about compiling this code (to Python for now)?
@@ -32,36 +32,36 @@ Naive Call Chaining
The simplest thing would be to compose the functions from the library:
.. code:: ipython2
.. code:: ipython3
dup, mul = D['dup'], D['mul']
.. code:: ipython2
.. code:: ipython3
def sqr(stack, expression, dictionary):
return mul(*dup(stack, expression, dictionary))
.. code:: ipython2
.. code:: ipython3
old_sqr = D['sqr']
D['sqr'] = sqr
.. code:: ipython2
.. code:: ipython3
V('23 sqr')
.. parsed-literal::
. 23 sqr
23 . sqr
529 .
• 23 sqr
23 • sqr
529 •
It’s simple to write a function to emit this kind of crude “compiled”
It's simple to write a function to emit this kind of crude "compiled"
code.
.. code:: ipython2
.. code:: ipython3
def compile_joy(name, expression):
term, expression = expression
@@ -80,9 +80,9 @@ code.
return compile_joy(defi.name, defi.body)
.. code:: ipython2
.. code:: ipython3
print compile_joy_definition(old_sqr)
print(compile_joy_definition(old_sqr))
.. parsed-literal::
@@ -96,13 +96,13 @@ But what about literals?
::
quoted == [unit] dip
quoted == [unit] dip
.. code:: ipython2
.. code:: ipython3
unit, dip = D['unit'], D['dip']
.. code:: ipython2
.. code:: ipython3
# print compile_joy_definition(D['quoted'])
# raises
@@ -111,7 +111,7 @@ But what about literals?
For a program like ``foo == bar baz 23 99 baq lerp barp`` we would want
something like:
.. code:: ipython2
.. code:: ipython3
def foo(stack, expression, dictionary):
stack, expression, dictionary = baz(*bar(stack, expression, dictionary))
@@ -126,95 +126,93 @@ Compiling Yin Functions
Call-chaining results in code that does too much work. For functions
that operate on stacks and only rearrange values, what I like to call
“Yin Functions”, we can do better.
"Yin Functions", we can do better.
We can infer the stack effects of these functions (or “expressions” or
“programs”) automatically, and the stack effects completely define the
We can infer the stack effects of these functions (or "expressions" or
"programs") automatically, and the stack effects completely define the
semantics of the functions, so we can directly write out a two-line
Python function for them. This is already implemented in the
``joy.utils.types.compile_()`` function.
.. code:: ipython2
.. code:: ipython3
from joy.utils.types import compile_, doc_from_stack_effect, infer_string
from joy.library import SimpleFunctionWrapper
.. code:: ipython2
::
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-14-d5ef3c7560be> in <module>
----> 1 from joy.utils.types import compile_, doc_from_stack_effect, infer_string
2 from joy.library import SimpleFunctionWrapper
ModuleNotFoundError: No module named 'joy.utils.types'
.. code:: ipython3
stack_effects = infer_string('tuck over dup')
Yin functions have only a single stack effect, they do not branch or
loop.
.. code:: ipython2
.. code:: ipython3
for fi, fo in stack_effects:
print doc_from_stack_effect(fi, fo)
.. parsed-literal::
(a2 a1 -- a1 a2 a1 a2 a2)
.. code:: ipython2
.. code:: ipython3
source = compile_('foo', stack_effects[0])
All Yin functions can be described in Python as a tuple-unpacking (or
“-destructuring”) of the stack datastructure followed by building up the
"-destructuring") of the stack datastructure followed by building up the
new stack structure.
.. code:: ipython2
.. code:: ipython3
print source
.. parsed-literal::
def foo(stack):
"""
::
(a2 a1 -- a1 a2 a1 a2 a2)
"""
(a1, (a2, s1)) = stack
return (a2, (a2, (a1, (a2, (a1, s1)))))
.. code:: ipython2
.. code:: ipython3
exec compile(source, '__main__', 'single')
D['foo'] = SimpleFunctionWrapper(foo)
.. code:: ipython2
::
File "<ipython-input-9-1a7e90bf2d7b>", line 1
exec compile(source, '__main__', 'single')
^
SyntaxError: invalid syntax
.. code:: ipython3
V('23 18 foo')
.. parsed-literal::
. 23 18 foo
23 . 18 foo
23 18 . foo
18 23 18 23 23 .
Compiling from Stack Effects
----------------------------
There are times when you’re deriving a Joy program when you have a stack
There are times when you're deriving a Joy program when you have a stack
effect for a Yin function and you need to define it. For example, in the
Ordered Binary Trees notebook there is a point where we must derive a
function ``Ee``:
::
[key old_value left right] new_value key [Tree-add] Ee
------------------------------------------------------------
[key new_value left right]
[key old_value left right] new_value key [Tree-add] Ee
------------------------------------------------------------
[key new_value left right]
While it is not hard to come up with this function manually, there is no
necessity. This function can be defined (in Python) directly from its
@@ -222,18 +220,18 @@ stack effect:
::
[a b c d] e a [f] Ee
--------------------------
[a e c d]
[a b c d] e a [f] Ee
--------------------------
[a e c d]
(I haven’t yet implemented a simple interface for this yet. What follow
(I haven't yet implemented a simple interface for this yet. What follow
is an exploration of how to do it.)
.. code:: ipython2
.. code:: ipython3
from joy.parser import text_to_expression
.. code:: ipython2
.. code:: ipython3
Ein = '[a b c d] e a [f]' # The terms should be reversed here but I don't realize that until later.
Eout = '[a e c d]'
@@ -241,31 +239,15 @@ is an exploration of how to do it.)
print E
.. parsed-literal::
[[a b c d] e a [f]] [[a e c d]]
.. code:: ipython2
.. code:: ipython3
(fi, (fo, _)) = text_to_expression(E)
.. code:: ipython2
.. code:: ipython3
fi, fo
.. parsed-literal::
(((a, (b, (c, (d, ())))), (e, (a, ((f, ()), ())))),
((a, (e, (c, (d, ())))), ()))
.. code:: ipython2
.. code:: ipython3
Ein = '[a1 a2 a3 a4] a5 a6 a7'
Eout = '[a1 a5 a3 a4]'
@@ -273,31 +255,15 @@ is an exploration of how to do it.)
print E
.. parsed-literal::
[[a1 a2 a3 a4] a5 a6 a7] [[a1 a5 a3 a4]]
.. code:: ipython2
.. code:: ipython3
(fi, (fo, _)) = text_to_expression(E)
.. code:: ipython2
.. code:: ipython3
fi, fo
.. parsed-literal::
(((a1, (a2, (a3, (a4, ())))), (a5, (a6, (a7, ())))),
((a1, (a5, (a3, (a4, ())))), ()))
.. code:: ipython2
.. code:: ipython3
def type_vars():
from joy.library import a1, a2, a3, a4, a5, a6, a7, s0, s1
@@ -306,174 +272,88 @@ is an exploration of how to do it.)
tv = type_vars()
tv
.. parsed-literal::
{'a1': a1,
'a2': a2,
'a3': a3,
'a4': a4,
'a5': a5,
'a6': a6,
'a7': a7,
's0': s0,
's1': s1}
.. code:: ipython2
.. code:: ipython3
from joy.utils.types import reify
.. code:: ipython2
.. code:: ipython3
stack_effect = reify(tv, (fi, fo))
print doc_from_stack_effect(*stack_effect)
.. parsed-literal::
(... a7 a6 a5 [a1 a2 a3 a4 ] -- ... [a1 a5 a3 a4 ])
.. code:: ipython2
.. code:: ipython3
print stack_effect
.. parsed-literal::
(((a1, (a2, (a3, (a4, ())))), (a5, (a6, (a7, ())))), ((a1, (a5, (a3, (a4, ())))), ()))
Almost, but what we really want is something like this:
.. code:: ipython2
.. code:: ipython3
stack_effect = eval('(((a1, (a2, (a3, (a4, s1)))), (a5, (a6, (a7, s0)))), ((a1, (a5, (a3, (a4, s1)))), s0))', tv)
Note the change of ``()`` to ``JoyStackType`` type variables.
.. code:: ipython2
.. code:: ipython3
print doc_from_stack_effect(*stack_effect)
.. parsed-literal::
(a7 a6 a5 [a1 a2 a3 a4 ...1] -- [a1 a5 a3 a4 ...1])
Now we can omit ``a3`` and ``a4`` if we like:
.. code:: ipython2
.. code:: ipython3
stack_effect = eval('(((a1, (a2, s1)), (a5, (a6, (a7, s0)))), ((a1, (a5, s1)), s0))', tv)
The ``right`` and ``left`` parts of the ordered binary tree node are
subsumed in the tail of the node’s stack/list.
subsumed in the tail of the node's stack/list.
.. code:: ipython2
.. code:: ipython3
print doc_from_stack_effect(*stack_effect)
.. parsed-literal::
(a7 a6 a5 [a1 a2 ...1] -- [a1 a5 ...1])
.. code:: ipython2
.. code:: ipython3
source = compile_('Ee', stack_effect)
print source
Oops! The input stack is backwards...
.. parsed-literal::
def Ee(stack):
"""
::
(a7 a6 a5 [a1 a2 ...1] -- [a1 a5 ...1])
"""
((a1, (a2, s1)), (a5, (a6, (a7, s0)))) = stack
return ((a1, (a5, s1)), s0)
Oops! The input stack is backwards…
.. code:: ipython2
.. code:: ipython3
stack_effect = eval('((a7, (a6, (a5, ((a1, (a2, s1)), s0)))), ((a1, (a5, s1)), s0))', tv)
.. code:: ipython2
.. code:: ipython3
print doc_from_stack_effect(*stack_effect)
.. parsed-literal::
([a1 a2 ...1] a5 a6 a7 -- [a1 a5 ...1])
.. code:: ipython2
.. code:: ipython3
source = compile_('Ee', stack_effect)
print source
.. parsed-literal::
def Ee(stack):
"""
::
([a1 a2 ...1] a5 a6 a7 -- [a1 a5 ...1])
"""
(a7, (a6, (a5, ((a1, (a2, s1)), s0)))) = stack
return ((a1, (a5, s1)), s0)
Compare:
::
[key old_value left right] new_value key [Tree-add] Ee
------------------------------------------------------------
[key new_value left right]
[key old_value left right] new_value key [Tree-add] Ee
------------------------------------------------------------
[key new_value left right]
.. code:: ipython2
.. code:: ipython3
eval(compile(source, '__main__', 'single'))
D['Ee'] = SimpleFunctionWrapper(Ee)
.. code:: ipython2
.. code:: ipython3
V('[a b c d] 1 2 [f] Ee')
.. parsed-literal::
. [a b c d] 1 2 [f] Ee
[a b c d] . 1 2 [f] Ee
[a b c d] 1 . 2 [f] Ee
[a b c d] 1 2 . [f] Ee
[a b c d] 1 2 [f] . Ee
[a 1 c d] .
Working with Yang Functions
---------------------------
Consider the compiled code of ``dup``:
.. code:: ipython2
.. code:: ipython3
def dup(stack):
@@ -484,21 +364,15 @@ Consider the compiled code of ``dup``:
To compile ``sqr == dup mul`` we can compute the stack effect:
.. code:: ipython2
.. code:: ipython3
stack_effects = infer_string('dup mul')
for fi, fo in stack_effects:
print doc_from_stack_effect(fi, fo)
.. parsed-literal::
(n1 -- n2)
Then we would want something like this:
.. code:: ipython2
.. code:: ipython3
def sqr(stack):
@@ -510,21 +384,15 @@ Then we would want something like this:
How about…
How about...
.. code:: ipython2
.. code:: ipython3
stack_effects = infer_string('mul mul sub')
for fi, fo in stack_effects:
print doc_from_stack_effect(fi, fo)
.. parsed-literal::
(n4 n3 n2 n1 -- n5)
.. code:: ipython2
.. code:: ipython3
def foo(stack):
@@ -545,30 +413,24 @@ How about…
.. code:: ipython2
.. code:: ipython3
stack_effects = infer_string('tuck')
for fi, fo in stack_effects:
print doc_from_stack_effect(fi, fo)
.. parsed-literal::
(a2 a1 -- a1 a2 a1)
Compiling Yin~Yang Functions
----------------------------
First, we need a source of Python identifiers. I’m going to reuse
First, we need a source of Python identifiers. I'm going to reuse
``Symbol`` class for this.
.. code:: ipython2
.. code:: ipython3
from joy.parser import Symbol
.. code:: ipython2
.. code:: ipython3
def _names():
n = 0
@@ -579,9 +441,9 @@ First, we need a source of Python identifiers. I’m going to reuse
names = _names().next
Now we need an object that represents a Yang function that accepts two
args and return one result (we’ll implement other kinds a little later.)
args and return one result (we'll implement other kinds a little later.)
.. code:: ipython2
.. code:: ipython3
class Foo(object):
@@ -594,10 +456,10 @@ args and return one result (we’ll implement other kinds a little later.)
code.append(('call', out, self.name, (in0, in1)))
return (out, stack), expression, code
A crude “interpreter” that translates expressions of args and Yin and
A crude "interpreter" that translates expressions of args and Yin and
Yang functions into a kind of simple dataflow graph.
.. code:: ipython2
.. code:: ipython3
def I(stack, expression, code):
while expression:
@@ -618,7 +480,7 @@ Yang functions into a kind of simple dataflow graph.
Something to convert the graph into Python code.
.. code:: ipython2
.. code:: ipython3
strtup = lambda a, b: '(%s, %s)' % (b, a)
strstk = lambda rest: reduce(strtup, rest, 'stack')
@@ -676,14 +538,14 @@ Something to convert the graph into Python code.
''' % (name, code_gen(I((), expression, [])))
A few functions to try it with…
A few functions to try it with...
.. code:: ipython2
.. code:: ipython3
mul = Foo('mul')
sub = Foo('sub')
.. code:: ipython2
.. code:: ipython3
def import_yin():
from joy.utils.generated_library import *
@@ -699,98 +561,32 @@ A few functions to try it with…
# n, stack = stack
# return (n, (n, stack)), expression
... and there we are.
.. parsed-literal::
<ipython-input-74-a6ea700b09d9>:1: SyntaxWarning: import * only allowed at module level
def import_yin():
… and there we are.
.. code:: ipython2
.. code:: ipython3
print compile_yinyang('mul_', (names(), (names(), (mul, ()))))
.. parsed-literal::
def mul_(stack):
(a31, (a32, stack)) = stack
a33 = mul(a32, a31)
stack = (a33, stack)
return stack
.. code:: ipython2
.. code:: ipython3
e = (names(), (dup, (mul, ())))
print compile_yinyang('sqr', e)
.. parsed-literal::
def sqr(stack):
(a34, stack) = stack
a35 = mul(a34, a34)
stack = (a35, stack)
return stack
.. code:: ipython2
.. code:: ipython3
e = (names(), (dup, (names(), (sub, (mul, ())))))
print compile_yinyang('foo', e)
.. parsed-literal::
def foo(stack):
(a36, (a37, stack)) = stack
a38 = sub(a37, a36)
a39 = mul(a38, a36)
stack = (a39, stack)
return stack
.. code:: ipython2
.. code:: ipython3
e = (names(), (names(), (mul, (dup, (sub, (dup, ()))))))
print compile_yinyang('bar', e)
.. parsed-literal::
def bar(stack):
(a40, (a41, stack)) = stack
a42 = mul(a41, a40)
a43 = sub(a42, a42)
stack = (a43, (a43, stack))
return stack
.. code:: ipython2
.. code:: ipython3
e = (names(), (dup, (dup, (mul, (dup, (mul, (mul, ())))))))
print compile_yinyang('to_the_fifth_power', e)
.. parsed-literal::
def to_the_fifth_power(stack):
(a44, stack) = stack
a45 = mul(a44, a44)
a46 = mul(a45, a45)
a47 = mul(a46, a44)
stack = (a47, stack)
return stack