Mercurial > hg > Members > tobaru > cbc > CbC_llvm
annotate docs/tutorial/OCamlLangImpl2.rst @ 15:66597e792247
minor fix
author | Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp> |
---|---|
date | Tue, 17 Sep 2013 21:32:34 +0900 |
parents | 9ad51c7bc036 |
children |
rev | line source |
---|---|
3
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
1 =========================================== |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
2 Kaleidoscope: Implementing a Parser and AST |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
3 =========================================== |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
4 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
5 .. contents:: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
6 :local: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
7 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
8 Chapter 2 Introduction |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
9 ====================== |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
10 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
11 Welcome to Chapter 2 of the "`Implementing a language with LLVM in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
12 Objective Caml <index.html>`_" tutorial. This chapter shows you how to |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
13 use the lexer, built in `Chapter 1 <OCamlLangImpl1.html>`_, to build a |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
14 full `parser <http://en.wikipedia.org/wiki/Parsing>`_ for our |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
15 Kaleidoscope language. Once we have a parser, we'll define and build an |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
16 `Abstract Syntax |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
17 Tree <http://en.wikipedia.org/wiki/Abstract_syntax_tree>`_ (AST). |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
18 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
19 The parser we will build uses a combination of `Recursive Descent |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
20 Parsing <http://en.wikipedia.org/wiki/Recursive_descent_parser>`_ and |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
21 `Operator-Precedence |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
22 Parsing <http://en.wikipedia.org/wiki/Operator-precedence_parser>`_ to |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
23 parse the Kaleidoscope language (the latter for binary expressions and |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
24 the former for everything else). Before we get to parsing though, lets |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
25 talk about the output of the parser: the Abstract Syntax Tree. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
26 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
27 The Abstract Syntax Tree (AST) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
28 ============================== |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
29 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
30 The AST for a program captures its behavior in such a way that it is |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
31 easy for later stages of the compiler (e.g. code generation) to |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
32 interpret. We basically want one object for each construct in the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
33 language, and the AST should closely model the language. In |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
34 Kaleidoscope, we have expressions, a prototype, and a function object. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
35 We'll start with expressions first: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
36 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
37 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
38 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
39 (* expr - Base type for all expression nodes. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
40 type expr = |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
41 (* variant for numeric literals like "1.0". *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
42 | Number of float |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
43 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
44 The code above shows the definition of the base ExprAST class and one |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
45 subclass which we use for numeric literals. The important thing to note |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
46 about this code is that the Number variant captures the numeric value of |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
47 the literal as an instance variable. This allows later phases of the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
48 compiler to know what the stored numeric value is. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
49 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
50 Right now we only create the AST, so there are no useful functions on |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
51 them. It would be very easy to add a function to pretty print the code, |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
52 for example. Here are the other expression AST node definitions that |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
53 we'll use in the basic form of the Kaleidoscope language: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
54 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
55 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
56 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
57 (* variant for referencing a variable, like "a". *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
58 | Variable of string |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
59 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
60 (* variant for a binary operator. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
61 | Binary of char * expr * expr |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
62 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
63 (* variant for function calls. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
64 | Call of string * expr array |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
65 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
66 This is all (intentionally) rather straight-forward: variables capture |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
67 the variable name, binary operators capture their opcode (e.g. '+'), and |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
68 calls capture a function name as well as a list of any argument |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
69 expressions. One thing that is nice about our AST is that it captures |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
70 the language features without talking about the syntax of the language. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
71 Note that there is no discussion about precedence of binary operators, |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
72 lexical structure, etc. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
73 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
74 For our basic language, these are all of the expression nodes we'll |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
75 define. Because it doesn't have conditional control flow, it isn't |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
76 Turing-complete; we'll fix that in a later installment. The two things |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
77 we need next are a way to talk about the interface to a function, and a |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
78 way to talk about functions themselves: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
79 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
80 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
81 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
82 (* proto - This type represents the "prototype" for a function, which captures |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
83 * its name, and its argument names (thus implicitly the number of arguments the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
84 * function takes). *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
85 type proto = Prototype of string * string array |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
86 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
87 (* func - This type represents a function definition itself. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
88 type func = Function of proto * expr |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
89 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
90 In Kaleidoscope, functions are typed with just a count of their |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
91 arguments. Since all values are double precision floating point, the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
92 type of each argument doesn't need to be stored anywhere. In a more |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
93 aggressive and realistic language, the "expr" variants would probably |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
94 have a type field. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
95 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
96 With this scaffolding, we can now talk about parsing expressions and |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
97 function bodies in Kaleidoscope. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
98 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
99 Parser Basics |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
100 ============= |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
101 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
102 Now that we have an AST to build, we need to define the parser code to |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
103 build it. The idea here is that we want to parse something like "x+y" |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
104 (which is returned as three tokens by the lexer) into an AST that could |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
105 be generated with calls like this: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
106 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
107 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
108 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
109 let x = Variable "x" in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
110 let y = Variable "y" in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
111 let result = Binary ('+', x, y) in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
112 ... |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
113 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
114 The error handling routines make use of the builtin ``Stream.Failure`` |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
115 and ``Stream.Error``s. ``Stream.Failure`` is raised when the parser is |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
116 unable to find any matching token in the first position of a pattern. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
117 ``Stream.Error`` is raised when the first token matches, but the rest do |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
118 not. The error recovery in our parser will not be the best and is not |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
119 particular user-friendly, but it will be enough for our tutorial. These |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
120 exceptions make it easier to handle errors in routines that have various |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
121 return types. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
122 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
123 With these basic types and exceptions, we can implement the first piece |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
124 of our grammar: numeric literals. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
125 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
126 Basic Expression Parsing |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
127 ======================== |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
128 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
129 We start with numeric literals, because they are the simplest to |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
130 process. For each production in our grammar, we'll define a function |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
131 which parses that production. We call this class of expressions |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
132 "primary" expressions, for reasons that will become more clear `later in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
133 the tutorial <OCamlLangImpl6.html#unary>`_. In order to parse an |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
134 arbitrary primary expression, we need to determine what sort of |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
135 expression it is. For numeric literals, we have: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
136 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
137 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
138 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
139 (* primary |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
140 * ::= identifier |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
141 * ::= numberexpr |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
142 * ::= parenexpr *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
143 parse_primary = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
144 (* numberexpr ::= number *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
145 | [< 'Token.Number n >] -> Ast.Number n |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
146 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
147 This routine is very simple: it expects to be called when the current |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
148 token is a ``Token.Number`` token. It takes the current number value, |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
149 creates a ``Ast.Number`` node, advances the lexer to the next token, and |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
150 finally returns. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
151 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
152 There are some interesting aspects to this. The most important one is |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
153 that this routine eats all of the tokens that correspond to the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
154 production and returns the lexer buffer with the next token (which is |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
155 not part of the grammar production) ready to go. This is a fairly |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
156 standard way to go for recursive descent parsers. For a better example, |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
157 the parenthesis operator is defined like this: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
158 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
159 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
160 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
161 (* parenexpr ::= '(' expression ')' *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
162 | [< 'Token.Kwd '('; e=parse_expr; 'Token.Kwd ')' ?? "expected ')'" >] -> e |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
163 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
164 This function illustrates a number of interesting things about the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
165 parser: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
166 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
167 1) It shows how we use the ``Stream.Error`` exception. When called, this |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
168 function expects that the current token is a '(' token, but after |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
169 parsing the subexpression, it is possible that there is no ')' waiting. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
170 For example, if the user types in "(4 x" instead of "(4)", the parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
171 should emit an error. Because errors can occur, the parser needs a way |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
172 to indicate that they happened. In our parser, we use the camlp4 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
173 shortcut syntax ``token ?? "parse error"``, where if the token before |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
174 the ``??`` does not match, then ``Stream.Error "parse error"`` will be |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
175 raised. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
176 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
177 2) Another interesting aspect of this function is that it uses recursion |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
178 by calling ``Parser.parse_primary`` (we will soon see that |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
179 ``Parser.parse_primary`` can call ``Parser.parse_primary``). This is |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
180 powerful because it allows us to handle recursive grammars, and keeps |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
181 each production very simple. Note that parentheses do not cause |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
182 construction of AST nodes themselves. While we could do it this way, the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
183 most important role of parentheses are to guide the parser and provide |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
184 grouping. Once the parser constructs the AST, parentheses are not |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
185 needed. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
186 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
187 The next simple production is for handling variable references and |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
188 function calls: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
189 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
190 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
191 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
192 (* identifierexpr |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
193 * ::= identifier |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
194 * ::= identifier '(' argumentexpr ')' *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
195 | [< 'Token.Ident id; stream >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
196 let rec parse_args accumulator = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
197 | [< e=parse_expr; stream >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
198 begin parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
199 | [< 'Token.Kwd ','; e=parse_args (e :: accumulator) >] -> e |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
200 | [< >] -> e :: accumulator |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
201 end stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
202 | [< >] -> accumulator |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
203 in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
204 let rec parse_ident id = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
205 (* Call. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
206 | [< 'Token.Kwd '('; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
207 args=parse_args []; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
208 'Token.Kwd ')' ?? "expected ')'">] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
209 Ast.Call (id, Array.of_list (List.rev args)) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
210 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
211 (* Simple variable ref. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
212 | [< >] -> Ast.Variable id |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
213 in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
214 parse_ident id stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
215 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
216 This routine follows the same style as the other routines. (It expects |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
217 to be called if the current token is a ``Token.Ident`` token). It also |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
218 has recursion and error handling. One interesting aspect of this is that |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
219 it uses *look-ahead* to determine if the current identifier is a stand |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
220 alone variable reference or if it is a function call expression. It |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
221 handles this by checking to see if the token after the identifier is a |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
222 '(' token, constructing either a ``Ast.Variable`` or ``Ast.Call`` node |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
223 as appropriate. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
224 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
225 We finish up by raising an exception if we received a token we didn't |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
226 expect: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
227 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
228 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
229 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
230 | [< >] -> raise (Stream.Error "unknown token when expecting an expression.") |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
231 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
232 Now that basic expressions are handled, we need to handle binary |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
233 expressions. They are a bit more complex. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
234 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
235 Binary Expression Parsing |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
236 ========================= |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
237 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
238 Binary expressions are significantly harder to parse because they are |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
239 often ambiguous. For example, when given the string "x+y\*z", the parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
240 can choose to parse it as either "(x+y)\*z" or "x+(y\*z)". With common |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
241 definitions from mathematics, we expect the later parse, because "\*" |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
242 (multiplication) has higher *precedence* than "+" (addition). |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
243 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
244 There are many ways to handle this, but an elegant and efficient way is |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
245 to use `Operator-Precedence |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
246 Parsing <http://en.wikipedia.org/wiki/Operator-precedence_parser>`_. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
247 This parsing technique uses the precedence of binary operators to guide |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
248 recursion. To start with, we need a table of precedences: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
249 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
250 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
251 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
252 (* binop_precedence - This holds the precedence for each binary operator that is |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
253 * defined *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
254 let binop_precedence:(char, int) Hashtbl.t = Hashtbl.create 10 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
255 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
256 (* precedence - Get the precedence of the pending binary operator token. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
257 let precedence c = try Hashtbl.find binop_precedence c with Not_found -> -1 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
258 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
259 ... |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
260 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
261 let main () = |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
262 (* Install standard binary operators. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
263 * 1 is the lowest precedence. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
264 Hashtbl.add Parser.binop_precedence '<' 10; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
265 Hashtbl.add Parser.binop_precedence '+' 20; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
266 Hashtbl.add Parser.binop_precedence '-' 20; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
267 Hashtbl.add Parser.binop_precedence '*' 40; (* highest. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
268 ... |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
269 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
270 For the basic form of Kaleidoscope, we will only support 4 binary |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
271 operators (this can obviously be extended by you, our brave and intrepid |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
272 reader). The ``Parser.precedence`` function returns the precedence for |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
273 the current token, or -1 if the token is not a binary operator. Having a |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
274 ``Hashtbl.t`` makes it easy to add new operators and makes it clear that |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
275 the algorithm doesn't depend on the specific operators involved, but it |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
276 would be easy enough to eliminate the ``Hashtbl.t`` and do the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
277 comparisons in the ``Parser.precedence`` function. (Or just use a |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
278 fixed-size array). |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
279 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
280 With the helper above defined, we can now start parsing binary |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
281 expressions. The basic idea of operator precedence parsing is to break |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
282 down an expression with potentially ambiguous binary operators into |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
283 pieces. Consider ,for example, the expression "a+b+(c+d)\*e\*f+g". |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
284 Operator precedence parsing considers this as a stream of primary |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
285 expressions separated by binary operators. As such, it will first parse |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
286 the leading primary expression "a", then it will see the pairs [+, b] |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
287 [+, (c+d)] [\*, e] [\*, f] and [+, g]. Note that because parentheses are |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
288 primary expressions, the binary expression parser doesn't need to worry |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
289 about nested subexpressions like (c+d) at all. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
290 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
291 To start, an expression is a primary expression potentially followed by |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
292 a sequence of [binop,primaryexpr] pairs: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
293 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
294 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
295 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
296 (* expression |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
297 * ::= primary binoprhs *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
298 and parse_expr = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
299 | [< lhs=parse_primary; stream >] -> parse_bin_rhs 0 lhs stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
300 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
301 ``Parser.parse_bin_rhs`` is the function that parses the sequence of |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
302 pairs for us. It takes a precedence and a pointer to an expression for |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
303 the part that has been parsed so far. Note that "x" is a perfectly valid |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
304 expression: As such, "binoprhs" is allowed to be empty, in which case it |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
305 returns the expression that is passed into it. In our example above, the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
306 code passes the expression for "a" into ``Parser.parse_bin_rhs`` and the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
307 current token is "+". |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
308 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
309 The precedence value passed into ``Parser.parse_bin_rhs`` indicates the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
310 *minimal operator precedence* that the function is allowed to eat. For |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
311 example, if the current pair stream is [+, x] and |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
312 ``Parser.parse_bin_rhs`` is passed in a precedence of 40, it will not |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
313 consume any tokens (because the precedence of '+' is only 20). With this |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
314 in mind, ``Parser.parse_bin_rhs`` starts with: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
315 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
316 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
317 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
318 (* binoprhs |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
319 * ::= ('+' primary)* *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
320 and parse_bin_rhs expr_prec lhs stream = |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
321 match Stream.peek stream with |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
322 (* If this is a binop, find its precedence. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
323 | Some (Token.Kwd c) when Hashtbl.mem binop_precedence c -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
324 let token_prec = precedence c in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
325 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
326 (* If this is a binop that binds at least as tightly as the current binop, |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
327 * consume it, otherwise we are done. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
328 if token_prec < expr_prec then lhs else begin |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
329 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
330 This code gets the precedence of the current token and checks to see if |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
331 if is too low. Because we defined invalid tokens to have a precedence of |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
332 -1, this check implicitly knows that the pair-stream ends when the token |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
333 stream runs out of binary operators. If this check succeeds, we know |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
334 that the token is a binary operator and that it will be included in this |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
335 expression: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
336 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
337 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
338 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
339 (* Eat the binop. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
340 Stream.junk stream; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
341 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
342 (* Okay, we know this is a binop. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
343 let rhs = |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
344 match Stream.peek stream with |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
345 | Some (Token.Kwd c2) -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
346 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
347 As such, this code eats (and remembers) the binary operator and then |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
348 parses the primary expression that follows. This builds up the whole |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
349 pair, the first of which is [+, b] for the running example. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
350 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
351 Now that we parsed the left-hand side of an expression and one pair of |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
352 the RHS sequence, we have to decide which way the expression associates. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
353 In particular, we could have "(a+b) binop unparsed" or "a + (b binop |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
354 unparsed)". To determine this, we look ahead at "binop" to determine its |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
355 precedence and compare it to BinOp's precedence (which is '+' in this |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
356 case): |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
357 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
358 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
359 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
360 (* If BinOp binds less tightly with rhs than the operator after |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
361 * rhs, let the pending operator take rhs as its lhs. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
362 let next_prec = precedence c2 in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
363 if token_prec < next_prec |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
364 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
365 If the precedence of the binop to the right of "RHS" is lower or equal |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
366 to the precedence of our current operator, then we know that the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
367 parentheses associate as "(a+b) binop ...". In our example, the current |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
368 operator is "+" and the next operator is "+", we know that they have the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
369 same precedence. In this case we'll create the AST node for "a+b", and |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
370 then continue parsing: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
371 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
372 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
373 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
374 ... if body omitted ... |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
375 in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
376 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
377 (* Merge lhs/rhs. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
378 let lhs = Ast.Binary (c, lhs, rhs) in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
379 parse_bin_rhs expr_prec lhs stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
380 end |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
381 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
382 In our example above, this will turn "a+b+" into "(a+b)" and execute the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
383 next iteration of the loop, with "+" as the current token. The code |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
384 above will eat, remember, and parse "(c+d)" as the primary expression, |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
385 which makes the current pair equal to [+, (c+d)]. It will then evaluate |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
386 the 'if' conditional above with "\*" as the binop to the right of the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
387 primary. In this case, the precedence of "\*" is higher than the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
388 precedence of "+" so the if condition will be entered. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
389 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
390 The critical question left here is "how can the if condition parse the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
391 right hand side in full"? In particular, to build the AST correctly for |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
392 our example, it needs to get all of "(c+d)\*e\*f" as the RHS expression |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
393 variable. The code to do this is surprisingly simple (code from the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
394 above two blocks duplicated for context): |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
395 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
396 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
397 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
398 match Stream.peek stream with |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
399 | Some (Token.Kwd c2) -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
400 (* If BinOp binds less tightly with rhs than the operator after |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
401 * rhs, let the pending operator take rhs as its lhs. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
402 if token_prec < precedence c2 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
403 then parse_bin_rhs (token_prec + 1) rhs stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
404 else rhs |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
405 | _ -> rhs |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
406 in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
407 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
408 (* Merge lhs/rhs. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
409 let lhs = Ast.Binary (c, lhs, rhs) in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
410 parse_bin_rhs expr_prec lhs stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
411 end |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
412 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
413 At this point, we know that the binary operator to the RHS of our |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
414 primary has higher precedence than the binop we are currently parsing. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
415 As such, we know that any sequence of pairs whose operators are all |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
416 higher precedence than "+" should be parsed together and returned as |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
417 "RHS". To do this, we recursively invoke the ``Parser.parse_bin_rhs`` |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
418 function specifying "token\_prec+1" as the minimum precedence required |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
419 for it to continue. In our example above, this will cause it to return |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
420 the AST node for "(c+d)\*e\*f" as RHS, which is then set as the RHS of |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
421 the '+' expression. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
422 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
423 Finally, on the next iteration of the while loop, the "+g" piece is |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
424 parsed and added to the AST. With this little bit of code (14 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
425 non-trivial lines), we correctly handle fully general binary expression |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
426 parsing in a very elegant way. This was a whirlwind tour of this code, |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
427 and it is somewhat subtle. I recommend running through it with a few |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
428 tough examples to see how it works. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
429 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
430 This wraps up handling of expressions. At this point, we can point the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
431 parser at an arbitrary token stream and build an expression from it, |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
432 stopping at the first token that is not part of the expression. Next up |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
433 we need to handle function definitions, etc. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
434 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
435 Parsing the Rest |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
436 ================ |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
437 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
438 The next thing missing is handling of function prototypes. In |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
439 Kaleidoscope, these are used both for 'extern' function declarations as |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
440 well as function body definitions. The code to do this is |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
441 straight-forward and not very interesting (once you've survived |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
442 expressions): |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
443 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
444 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
445 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
446 (* prototype |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
447 * ::= id '(' id* ')' *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
448 let parse_prototype = |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
449 let rec parse_args accumulator = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
450 | [< 'Token.Ident id; e=parse_args (id::accumulator) >] -> e |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
451 | [< >] -> accumulator |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
452 in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
453 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
454 parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
455 | [< 'Token.Ident id; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
456 'Token.Kwd '(' ?? "expected '(' in prototype"; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
457 args=parse_args []; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
458 'Token.Kwd ')' ?? "expected ')' in prototype" >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
459 (* success. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
460 Ast.Prototype (id, Array.of_list (List.rev args)) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
461 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
462 | [< >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
463 raise (Stream.Error "expected function name in prototype") |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
464 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
465 Given this, a function definition is very simple, just a prototype plus |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
466 an expression to implement the body: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
467 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
468 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
469 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
470 (* definition ::= 'def' prototype expression *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
471 let parse_definition = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
472 | [< 'Token.Def; p=parse_prototype; e=parse_expr >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
473 Ast.Function (p, e) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
474 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
475 In addition, we support 'extern' to declare functions like 'sin' and |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
476 'cos' as well as to support forward declaration of user functions. These |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
477 'extern's are just prototypes with no body: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
478 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
479 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
480 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
481 (* external ::= 'extern' prototype *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
482 let parse_extern = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
483 | [< 'Token.Extern; e=parse_prototype >] -> e |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
484 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
485 Finally, we'll also let the user type in arbitrary top-level expressions |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
486 and evaluate them on the fly. We will handle this by defining anonymous |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
487 nullary (zero argument) functions for them: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
488 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
489 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
490 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
491 (* toplevelexpr ::= expression *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
492 let parse_toplevel = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
493 | [< e=parse_expr >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
494 (* Make an anonymous proto. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
495 Ast.Function (Ast.Prototype ("", [||]), e) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
496 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
497 Now that we have all the pieces, let's build a little driver that will |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
498 let us actually *execute* this code we've built! |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
499 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
500 The Driver |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
501 ========== |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
502 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
503 The driver for this simply invokes all of the parsing pieces with a |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
504 top-level dispatch loop. There isn't much interesting here, so I'll just |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
505 include the top-level loop. See `below <#code>`_ for full code in the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
506 "Top-Level Parsing" section. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
507 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
508 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
509 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
510 (* top ::= definition | external | expression | ';' *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
511 let rec main_loop stream = |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
512 match Stream.peek stream with |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
513 | None -> () |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
514 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
515 (* ignore top-level semicolons. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
516 | Some (Token.Kwd ';') -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
517 Stream.junk stream; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
518 main_loop stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
519 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
520 | Some token -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
521 begin |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
522 try match token with |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
523 | Token.Def -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
524 ignore(Parser.parse_definition stream); |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
525 print_endline "parsed a function definition."; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
526 | Token.Extern -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
527 ignore(Parser.parse_extern stream); |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
528 print_endline "parsed an extern."; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
529 | _ -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
530 (* Evaluate a top-level expression into an anonymous function. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
531 ignore(Parser.parse_toplevel stream); |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
532 print_endline "parsed a top-level expr"; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
533 with Stream.Error s -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
534 (* Skip token for error recovery. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
535 Stream.junk stream; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
536 print_endline s; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
537 end; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
538 print_string "ready> "; flush stdout; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
539 main_loop stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
540 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
541 The most interesting part of this is that we ignore top-level |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
542 semicolons. Why is this, you ask? The basic reason is that if you type |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
543 "4 + 5" at the command line, the parser doesn't know whether that is the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
544 end of what you will type or not. For example, on the next line you |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
545 could type "def foo..." in which case 4+5 is the end of a top-level |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
546 expression. Alternatively you could type "\* 6", which would continue |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
547 the expression. Having top-level semicolons allows you to type "4+5;", |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
548 and the parser will know you are done. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
549 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
550 Conclusions |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
551 =========== |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
552 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
553 With just under 300 lines of commented code (240 lines of non-comment, |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
554 non-blank code), we fully defined our minimal language, including a |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
555 lexer, parser, and AST builder. With this done, the executable will |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
556 validate Kaleidoscope code and tell us if it is grammatically invalid. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
557 For example, here is a sample interaction: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
558 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
559 .. code-block:: bash |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
560 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
561 $ ./toy.byte |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
562 ready> def foo(x y) x+foo(y, 4.0); |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
563 Parsed a function definition. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
564 ready> def foo(x y) x+y y; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
565 Parsed a function definition. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
566 Parsed a top-level expr |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
567 ready> def foo(x y) x+y ); |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
568 Parsed a function definition. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
569 Error: unknown token when expecting an expression |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
570 ready> extern sin(a); |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
571 ready> Parsed an extern |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
572 ready> ^D |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
573 $ |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
574 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
575 There is a lot of room for extension here. You can define new AST nodes, |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
576 extend the language in many ways, etc. In the `next |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
577 installment <OCamlLangImpl3.html>`_, we will describe how to generate |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
578 LLVM Intermediate Representation (IR) from the AST. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
579 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
580 Full Code Listing |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
581 ================= |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
582 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
583 Here is the complete code listing for this and the previous chapter. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
584 Note that it is fully self-contained: you don't need LLVM or any |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
585 external libraries at all for this. (Besides the ocaml standard |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
586 libraries, of course.) To build this, just compile with: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
587 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
588 .. code-block:: bash |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
589 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
590 # Compile |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
591 ocamlbuild toy.byte |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
592 # Run |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
593 ./toy.byte |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
594 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
595 Here is the code: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
596 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
597 \_tags: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
598 :: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
599 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
600 <{lexer,parser}.ml>: use_camlp4, pp(camlp4of) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
601 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
602 token.ml: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
603 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
604 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
605 (*===----------------------------------------------------------------------=== |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
606 * Lexer Tokens |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
607 *===----------------------------------------------------------------------===*) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
608 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
609 (* The lexer returns these 'Kwd' if it is an unknown character, otherwise one of |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
610 * these others for known things. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
611 type token = |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
612 (* commands *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
613 | Def | Extern |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
614 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
615 (* primary *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
616 | Ident of string | Number of float |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
617 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
618 (* unknown *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
619 | Kwd of char |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
620 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
621 lexer.ml: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
622 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
623 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
624 (*===----------------------------------------------------------------------=== |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
625 * Lexer |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
626 *===----------------------------------------------------------------------===*) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
627 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
628 let rec lex = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
629 (* Skip any whitespace. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
630 | [< ' (' ' | '\n' | '\r' | '\t'); stream >] -> lex stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
631 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
632 (* identifier: [a-zA-Z][a-zA-Z0-9] *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
633 | [< ' ('A' .. 'Z' | 'a' .. 'z' as c); stream >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
634 let buffer = Buffer.create 1 in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
635 Buffer.add_char buffer c; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
636 lex_ident buffer stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
637 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
638 (* number: [0-9.]+ *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
639 | [< ' ('0' .. '9' as c); stream >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
640 let buffer = Buffer.create 1 in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
641 Buffer.add_char buffer c; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
642 lex_number buffer stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
643 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
644 (* Comment until end of line. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
645 | [< ' ('#'); stream >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
646 lex_comment stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
647 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
648 (* Otherwise, just return the character as its ascii value. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
649 | [< 'c; stream >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
650 [< 'Token.Kwd c; lex stream >] |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
651 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
652 (* end of stream. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
653 | [< >] -> [< >] |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
654 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
655 and lex_number buffer = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
656 | [< ' ('0' .. '9' | '.' as c); stream >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
657 Buffer.add_char buffer c; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
658 lex_number buffer stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
659 | [< stream=lex >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
660 [< 'Token.Number (float_of_string (Buffer.contents buffer)); stream >] |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
661 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
662 and lex_ident buffer = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
663 | [< ' ('A' .. 'Z' | 'a' .. 'z' | '0' .. '9' as c); stream >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
664 Buffer.add_char buffer c; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
665 lex_ident buffer stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
666 | [< stream=lex >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
667 match Buffer.contents buffer with |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
668 | "def" -> [< 'Token.Def; stream >] |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
669 | "extern" -> [< 'Token.Extern; stream >] |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
670 | id -> [< 'Token.Ident id; stream >] |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
671 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
672 and lex_comment = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
673 | [< ' ('\n'); stream=lex >] -> stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
674 | [< 'c; e=lex_comment >] -> e |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
675 | [< >] -> [< >] |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
676 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
677 ast.ml: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
678 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
679 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
680 (*===----------------------------------------------------------------------=== |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
681 * Abstract Syntax Tree (aka Parse Tree) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
682 *===----------------------------------------------------------------------===*) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
683 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
684 (* expr - Base type for all expression nodes. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
685 type expr = |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
686 (* variant for numeric literals like "1.0". *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
687 | Number of float |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
688 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
689 (* variant for referencing a variable, like "a". *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
690 | Variable of string |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
691 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
692 (* variant for a binary operator. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
693 | Binary of char * expr * expr |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
694 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
695 (* variant for function calls. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
696 | Call of string * expr array |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
697 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
698 (* proto - This type represents the "prototype" for a function, which captures |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
699 * its name, and its argument names (thus implicitly the number of arguments the |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
700 * function takes). *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
701 type proto = Prototype of string * string array |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
702 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
703 (* func - This type represents a function definition itself. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
704 type func = Function of proto * expr |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
705 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
706 parser.ml: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
707 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
708 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
709 (*===---------------------------------------------------------------------=== |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
710 * Parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
711 *===---------------------------------------------------------------------===*) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
712 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
713 (* binop_precedence - This holds the precedence for each binary operator that is |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
714 * defined *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
715 let binop_precedence:(char, int) Hashtbl.t = Hashtbl.create 10 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
716 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
717 (* precedence - Get the precedence of the pending binary operator token. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
718 let precedence c = try Hashtbl.find binop_precedence c with Not_found -> -1 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
719 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
720 (* primary |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
721 * ::= identifier |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
722 * ::= numberexpr |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
723 * ::= parenexpr *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
724 let rec parse_primary = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
725 (* numberexpr ::= number *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
726 | [< 'Token.Number n >] -> Ast.Number n |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
727 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
728 (* parenexpr ::= '(' expression ')' *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
729 | [< 'Token.Kwd '('; e=parse_expr; 'Token.Kwd ')' ?? "expected ')'" >] -> e |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
730 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
731 (* identifierexpr |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
732 * ::= identifier |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
733 * ::= identifier '(' argumentexpr ')' *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
734 | [< 'Token.Ident id; stream >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
735 let rec parse_args accumulator = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
736 | [< e=parse_expr; stream >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
737 begin parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
738 | [< 'Token.Kwd ','; e=parse_args (e :: accumulator) >] -> e |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
739 | [< >] -> e :: accumulator |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
740 end stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
741 | [< >] -> accumulator |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
742 in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
743 let rec parse_ident id = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
744 (* Call. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
745 | [< 'Token.Kwd '('; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
746 args=parse_args []; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
747 'Token.Kwd ')' ?? "expected ')'">] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
748 Ast.Call (id, Array.of_list (List.rev args)) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
749 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
750 (* Simple variable ref. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
751 | [< >] -> Ast.Variable id |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
752 in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
753 parse_ident id stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
754 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
755 | [< >] -> raise (Stream.Error "unknown token when expecting an expression.") |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
756 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
757 (* binoprhs |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
758 * ::= ('+' primary)* *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
759 and parse_bin_rhs expr_prec lhs stream = |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
760 match Stream.peek stream with |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
761 (* If this is a binop, find its precedence. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
762 | Some (Token.Kwd c) when Hashtbl.mem binop_precedence c -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
763 let token_prec = precedence c in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
764 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
765 (* If this is a binop that binds at least as tightly as the current binop, |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
766 * consume it, otherwise we are done. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
767 if token_prec < expr_prec then lhs else begin |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
768 (* Eat the binop. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
769 Stream.junk stream; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
770 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
771 (* Parse the primary expression after the binary operator. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
772 let rhs = parse_primary stream in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
773 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
774 (* Okay, we know this is a binop. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
775 let rhs = |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
776 match Stream.peek stream with |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
777 | Some (Token.Kwd c2) -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
778 (* If BinOp binds less tightly with rhs than the operator after |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
779 * rhs, let the pending operator take rhs as its lhs. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
780 let next_prec = precedence c2 in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
781 if token_prec < next_prec |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
782 then parse_bin_rhs (token_prec + 1) rhs stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
783 else rhs |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
784 | _ -> rhs |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
785 in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
786 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
787 (* Merge lhs/rhs. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
788 let lhs = Ast.Binary (c, lhs, rhs) in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
789 parse_bin_rhs expr_prec lhs stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
790 end |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
791 | _ -> lhs |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
792 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
793 (* expression |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
794 * ::= primary binoprhs *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
795 and parse_expr = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
796 | [< lhs=parse_primary; stream >] -> parse_bin_rhs 0 lhs stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
797 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
798 (* prototype |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
799 * ::= id '(' id* ')' *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
800 let parse_prototype = |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
801 let rec parse_args accumulator = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
802 | [< 'Token.Ident id; e=parse_args (id::accumulator) >] -> e |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
803 | [< >] -> accumulator |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
804 in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
805 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
806 parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
807 | [< 'Token.Ident id; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
808 'Token.Kwd '(' ?? "expected '(' in prototype"; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
809 args=parse_args []; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
810 'Token.Kwd ')' ?? "expected ')' in prototype" >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
811 (* success. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
812 Ast.Prototype (id, Array.of_list (List.rev args)) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
813 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
814 | [< >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
815 raise (Stream.Error "expected function name in prototype") |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
816 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
817 (* definition ::= 'def' prototype expression *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
818 let parse_definition = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
819 | [< 'Token.Def; p=parse_prototype; e=parse_expr >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
820 Ast.Function (p, e) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
821 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
822 (* toplevelexpr ::= expression *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
823 let parse_toplevel = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
824 | [< e=parse_expr >] -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
825 (* Make an anonymous proto. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
826 Ast.Function (Ast.Prototype ("", [||]), e) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
827 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
828 (* external ::= 'extern' prototype *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
829 let parse_extern = parser |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
830 | [< 'Token.Extern; e=parse_prototype >] -> e |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
831 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
832 toplevel.ml: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
833 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
834 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
835 (*===----------------------------------------------------------------------=== |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
836 * Top-Level parsing and JIT Driver |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
837 *===----------------------------------------------------------------------===*) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
838 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
839 (* top ::= definition | external | expression | ';' *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
840 let rec main_loop stream = |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
841 match Stream.peek stream with |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
842 | None -> () |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
843 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
844 (* ignore top-level semicolons. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
845 | Some (Token.Kwd ';') -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
846 Stream.junk stream; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
847 main_loop stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
848 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
849 | Some token -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
850 begin |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
851 try match token with |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
852 | Token.Def -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
853 ignore(Parser.parse_definition stream); |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
854 print_endline "parsed a function definition."; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
855 | Token.Extern -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
856 ignore(Parser.parse_extern stream); |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
857 print_endline "parsed an extern."; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
858 | _ -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
859 (* Evaluate a top-level expression into an anonymous function. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
860 ignore(Parser.parse_toplevel stream); |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
861 print_endline "parsed a top-level expr"; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
862 with Stream.Error s -> |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
863 (* Skip token for error recovery. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
864 Stream.junk stream; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
865 print_endline s; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
866 end; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
867 print_string "ready> "; flush stdout; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
868 main_loop stream |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
869 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
870 toy.ml: |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
871 .. code-block:: ocaml |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
872 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
873 (*===----------------------------------------------------------------------=== |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
874 * Main driver code. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
875 *===----------------------------------------------------------------------===*) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
876 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
877 let main () = |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
878 (* Install standard binary operators. |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
879 * 1 is the lowest precedence. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
880 Hashtbl.add Parser.binop_precedence '<' 10; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
881 Hashtbl.add Parser.binop_precedence '+' 20; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
882 Hashtbl.add Parser.binop_precedence '-' 20; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
883 Hashtbl.add Parser.binop_precedence '*' 40; (* highest. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
884 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
885 (* Prime the first token. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
886 print_string "ready> "; flush stdout; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
887 let stream = Lexer.lex (Stream.of_channel stdin) in |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
888 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
889 (* Run the main "interpreter loop" now. *) |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
890 Toplevel.main_loop stream; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
891 ;; |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
892 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
893 main () |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
894 |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
895 `Next: Implementing Code Generation to LLVM IR <OCamlLangImpl3.html>`_ |
9ad51c7bc036
1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff
changeset
|
896 |