annotate docs/tutorial/OCamlLangImpl2.rst @ 15:66597e792247

minor fix
author Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
date Tue, 17 Sep 2013 21:32:34 +0900
parents 9ad51c7bc036
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
1 ===========================================
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
2 Kaleidoscope: Implementing a Parser and AST
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
3 ===========================================
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
4
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
5 .. contents::
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
6 :local:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
7
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
8 Chapter 2 Introduction
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
9 ======================
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
10
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
11 Welcome to Chapter 2 of the "`Implementing a language with LLVM in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
12 Objective Caml <index.html>`_" tutorial. This chapter shows you how to
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
13 use the lexer, built in `Chapter 1 <OCamlLangImpl1.html>`_, to build a
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
14 full `parser <http://en.wikipedia.org/wiki/Parsing>`_ for our
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
15 Kaleidoscope language. Once we have a parser, we'll define and build an
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
16 `Abstract Syntax
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
17 Tree <http://en.wikipedia.org/wiki/Abstract_syntax_tree>`_ (AST).
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
18
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
19 The parser we will build uses a combination of `Recursive Descent
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
20 Parsing <http://en.wikipedia.org/wiki/Recursive_descent_parser>`_ and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
21 `Operator-Precedence
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
22 Parsing <http://en.wikipedia.org/wiki/Operator-precedence_parser>`_ to
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
23 parse the Kaleidoscope language (the latter for binary expressions and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
24 the former for everything else). Before we get to parsing though, lets
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
25 talk about the output of the parser: the Abstract Syntax Tree.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
26
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
27 The Abstract Syntax Tree (AST)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
28 ==============================
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
29
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
30 The AST for a program captures its behavior in such a way that it is
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
31 easy for later stages of the compiler (e.g. code generation) to
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
32 interpret. We basically want one object for each construct in the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
33 language, and the AST should closely model the language. In
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
34 Kaleidoscope, we have expressions, a prototype, and a function object.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
35 We'll start with expressions first:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
36
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
37 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
38
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
39 (* expr - Base type for all expression nodes. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
40 type expr =
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
41 (* variant for numeric literals like "1.0". *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
42 | Number of float
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
43
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
44 The code above shows the definition of the base ExprAST class and one
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
45 subclass which we use for numeric literals. The important thing to note
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
46 about this code is that the Number variant captures the numeric value of
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
47 the literal as an instance variable. This allows later phases of the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
48 compiler to know what the stored numeric value is.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
49
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
50 Right now we only create the AST, so there are no useful functions on
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
51 them. It would be very easy to add a function to pretty print the code,
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
52 for example. Here are the other expression AST node definitions that
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
53 we'll use in the basic form of the Kaleidoscope language:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
54
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
55 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
56
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
57 (* variant for referencing a variable, like "a". *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
58 | Variable of string
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
59
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
60 (* variant for a binary operator. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
61 | Binary of char * expr * expr
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
62
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
63 (* variant for function calls. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
64 | Call of string * expr array
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
65
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
66 This is all (intentionally) rather straight-forward: variables capture
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
67 the variable name, binary operators capture their opcode (e.g. '+'), and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
68 calls capture a function name as well as a list of any argument
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
69 expressions. One thing that is nice about our AST is that it captures
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
70 the language features without talking about the syntax of the language.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
71 Note that there is no discussion about precedence of binary operators,
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
72 lexical structure, etc.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
73
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
74 For our basic language, these are all of the expression nodes we'll
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
75 define. Because it doesn't have conditional control flow, it isn't
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
76 Turing-complete; we'll fix that in a later installment. The two things
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
77 we need next are a way to talk about the interface to a function, and a
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
78 way to talk about functions themselves:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
79
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
80 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
81
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
82 (* proto - This type represents the "prototype" for a function, which captures
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
83 * its name, and its argument names (thus implicitly the number of arguments the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
84 * function takes). *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
85 type proto = Prototype of string * string array
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
86
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
87 (* func - This type represents a function definition itself. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
88 type func = Function of proto * expr
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
89
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
90 In Kaleidoscope, functions are typed with just a count of their
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
91 arguments. Since all values are double precision floating point, the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
92 type of each argument doesn't need to be stored anywhere. In a more
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
93 aggressive and realistic language, the "expr" variants would probably
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
94 have a type field.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
95
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
96 With this scaffolding, we can now talk about parsing expressions and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
97 function bodies in Kaleidoscope.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
98
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
99 Parser Basics
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
100 =============
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
101
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
102 Now that we have an AST to build, we need to define the parser code to
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
103 build it. The idea here is that we want to parse something like "x+y"
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
104 (which is returned as three tokens by the lexer) into an AST that could
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
105 be generated with calls like this:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
106
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
107 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
108
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
109 let x = Variable "x" in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
110 let y = Variable "y" in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
111 let result = Binary ('+', x, y) in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
112 ...
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
113
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
114 The error handling routines make use of the builtin ``Stream.Failure``
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
115 and ``Stream.Error``s. ``Stream.Failure`` is raised when the parser is
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
116 unable to find any matching token in the first position of a pattern.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
117 ``Stream.Error`` is raised when the first token matches, but the rest do
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
118 not. The error recovery in our parser will not be the best and is not
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
119 particular user-friendly, but it will be enough for our tutorial. These
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
120 exceptions make it easier to handle errors in routines that have various
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
121 return types.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
122
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
123 With these basic types and exceptions, we can implement the first piece
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
124 of our grammar: numeric literals.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
125
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
126 Basic Expression Parsing
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
127 ========================
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
128
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
129 We start with numeric literals, because they are the simplest to
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
130 process. For each production in our grammar, we'll define a function
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
131 which parses that production. We call this class of expressions
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
132 "primary" expressions, for reasons that will become more clear `later in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
133 the tutorial <OCamlLangImpl6.html#unary>`_. In order to parse an
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
134 arbitrary primary expression, we need to determine what sort of
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
135 expression it is. For numeric literals, we have:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
136
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
137 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
138
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
139 (* primary
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
140 * ::= identifier
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
141 * ::= numberexpr
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
142 * ::= parenexpr *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
143 parse_primary = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
144 (* numberexpr ::= number *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
145 | [< 'Token.Number n >] -> Ast.Number n
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
146
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
147 This routine is very simple: it expects to be called when the current
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
148 token is a ``Token.Number`` token. It takes the current number value,
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
149 creates a ``Ast.Number`` node, advances the lexer to the next token, and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
150 finally returns.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
151
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
152 There are some interesting aspects to this. The most important one is
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
153 that this routine eats all of the tokens that correspond to the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
154 production and returns the lexer buffer with the next token (which is
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
155 not part of the grammar production) ready to go. This is a fairly
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
156 standard way to go for recursive descent parsers. For a better example,
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
157 the parenthesis operator is defined like this:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
158
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
159 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
160
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
161 (* parenexpr ::= '(' expression ')' *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
162 | [< 'Token.Kwd '('; e=parse_expr; 'Token.Kwd ')' ?? "expected ')'" >] -> e
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
163
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
164 This function illustrates a number of interesting things about the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
165 parser:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
166
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
167 1) It shows how we use the ``Stream.Error`` exception. When called, this
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
168 function expects that the current token is a '(' token, but after
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
169 parsing the subexpression, it is possible that there is no ')' waiting.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
170 For example, if the user types in "(4 x" instead of "(4)", the parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
171 should emit an error. Because errors can occur, the parser needs a way
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
172 to indicate that they happened. In our parser, we use the camlp4
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
173 shortcut syntax ``token ?? "parse error"``, where if the token before
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
174 the ``??`` does not match, then ``Stream.Error "parse error"`` will be
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
175 raised.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
176
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
177 2) Another interesting aspect of this function is that it uses recursion
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
178 by calling ``Parser.parse_primary`` (we will soon see that
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
179 ``Parser.parse_primary`` can call ``Parser.parse_primary``). This is
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
180 powerful because it allows us to handle recursive grammars, and keeps
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
181 each production very simple. Note that parentheses do not cause
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
182 construction of AST nodes themselves. While we could do it this way, the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
183 most important role of parentheses are to guide the parser and provide
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
184 grouping. Once the parser constructs the AST, parentheses are not
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
185 needed.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
186
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
187 The next simple production is for handling variable references and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
188 function calls:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
189
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
190 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
191
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
192 (* identifierexpr
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
193 * ::= identifier
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
194 * ::= identifier '(' argumentexpr ')' *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
195 | [< 'Token.Ident id; stream >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
196 let rec parse_args accumulator = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
197 | [< e=parse_expr; stream >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
198 begin parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
199 | [< 'Token.Kwd ','; e=parse_args (e :: accumulator) >] -> e
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
200 | [< >] -> e :: accumulator
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
201 end stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
202 | [< >] -> accumulator
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
203 in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
204 let rec parse_ident id = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
205 (* Call. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
206 | [< 'Token.Kwd '(';
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
207 args=parse_args [];
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
208 'Token.Kwd ')' ?? "expected ')'">] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
209 Ast.Call (id, Array.of_list (List.rev args))
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
210
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
211 (* Simple variable ref. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
212 | [< >] -> Ast.Variable id
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
213 in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
214 parse_ident id stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
215
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
216 This routine follows the same style as the other routines. (It expects
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
217 to be called if the current token is a ``Token.Ident`` token). It also
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
218 has recursion and error handling. One interesting aspect of this is that
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
219 it uses *look-ahead* to determine if the current identifier is a stand
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
220 alone variable reference or if it is a function call expression. It
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
221 handles this by checking to see if the token after the identifier is a
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
222 '(' token, constructing either a ``Ast.Variable`` or ``Ast.Call`` node
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
223 as appropriate.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
224
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
225 We finish up by raising an exception if we received a token we didn't
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
226 expect:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
227
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
228 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
229
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
230 | [< >] -> raise (Stream.Error "unknown token when expecting an expression.")
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
231
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
232 Now that basic expressions are handled, we need to handle binary
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
233 expressions. They are a bit more complex.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
234
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
235 Binary Expression Parsing
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
236 =========================
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
237
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
238 Binary expressions are significantly harder to parse because they are
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
239 often ambiguous. For example, when given the string "x+y\*z", the parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
240 can choose to parse it as either "(x+y)\*z" or "x+(y\*z)". With common
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
241 definitions from mathematics, we expect the later parse, because "\*"
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
242 (multiplication) has higher *precedence* than "+" (addition).
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
243
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
244 There are many ways to handle this, but an elegant and efficient way is
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
245 to use `Operator-Precedence
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
246 Parsing <http://en.wikipedia.org/wiki/Operator-precedence_parser>`_.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
247 This parsing technique uses the precedence of binary operators to guide
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
248 recursion. To start with, we need a table of precedences:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
249
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
250 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
251
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
252 (* binop_precedence - This holds the precedence for each binary operator that is
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
253 * defined *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
254 let binop_precedence:(char, int) Hashtbl.t = Hashtbl.create 10
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
255
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
256 (* precedence - Get the precedence of the pending binary operator token. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
257 let precedence c = try Hashtbl.find binop_precedence c with Not_found -> -1
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
258
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
259 ...
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
260
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
261 let main () =
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
262 (* Install standard binary operators.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
263 * 1 is the lowest precedence. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
264 Hashtbl.add Parser.binop_precedence '<' 10;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
265 Hashtbl.add Parser.binop_precedence '+' 20;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
266 Hashtbl.add Parser.binop_precedence '-' 20;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
267 Hashtbl.add Parser.binop_precedence '*' 40; (* highest. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
268 ...
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
269
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
270 For the basic form of Kaleidoscope, we will only support 4 binary
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
271 operators (this can obviously be extended by you, our brave and intrepid
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
272 reader). The ``Parser.precedence`` function returns the precedence for
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
273 the current token, or -1 if the token is not a binary operator. Having a
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
274 ``Hashtbl.t`` makes it easy to add new operators and makes it clear that
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
275 the algorithm doesn't depend on the specific operators involved, but it
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
276 would be easy enough to eliminate the ``Hashtbl.t`` and do the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
277 comparisons in the ``Parser.precedence`` function. (Or just use a
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
278 fixed-size array).
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
279
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
280 With the helper above defined, we can now start parsing binary
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
281 expressions. The basic idea of operator precedence parsing is to break
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
282 down an expression with potentially ambiguous binary operators into
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
283 pieces. Consider ,for example, the expression "a+b+(c+d)\*e\*f+g".
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
284 Operator precedence parsing considers this as a stream of primary
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
285 expressions separated by binary operators. As such, it will first parse
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
286 the leading primary expression "a", then it will see the pairs [+, b]
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
287 [+, (c+d)] [\*, e] [\*, f] and [+, g]. Note that because parentheses are
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
288 primary expressions, the binary expression parser doesn't need to worry
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
289 about nested subexpressions like (c+d) at all.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
290
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
291 To start, an expression is a primary expression potentially followed by
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
292 a sequence of [binop,primaryexpr] pairs:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
293
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
294 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
295
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
296 (* expression
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
297 * ::= primary binoprhs *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
298 and parse_expr = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
299 | [< lhs=parse_primary; stream >] -> parse_bin_rhs 0 lhs stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
300
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
301 ``Parser.parse_bin_rhs`` is the function that parses the sequence of
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
302 pairs for us. It takes a precedence and a pointer to an expression for
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
303 the part that has been parsed so far. Note that "x" is a perfectly valid
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
304 expression: As such, "binoprhs" is allowed to be empty, in which case it
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
305 returns the expression that is passed into it. In our example above, the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
306 code passes the expression for "a" into ``Parser.parse_bin_rhs`` and the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
307 current token is "+".
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
308
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
309 The precedence value passed into ``Parser.parse_bin_rhs`` indicates the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
310 *minimal operator precedence* that the function is allowed to eat. For
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
311 example, if the current pair stream is [+, x] and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
312 ``Parser.parse_bin_rhs`` is passed in a precedence of 40, it will not
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
313 consume any tokens (because the precedence of '+' is only 20). With this
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
314 in mind, ``Parser.parse_bin_rhs`` starts with:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
315
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
316 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
317
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
318 (* binoprhs
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
319 * ::= ('+' primary)* *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
320 and parse_bin_rhs expr_prec lhs stream =
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
321 match Stream.peek stream with
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
322 (* If this is a binop, find its precedence. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
323 | Some (Token.Kwd c) when Hashtbl.mem binop_precedence c ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
324 let token_prec = precedence c in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
325
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
326 (* If this is a binop that binds at least as tightly as the current binop,
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
327 * consume it, otherwise we are done. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
328 if token_prec < expr_prec then lhs else begin
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
329
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
330 This code gets the precedence of the current token and checks to see if
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
331 if is too low. Because we defined invalid tokens to have a precedence of
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
332 -1, this check implicitly knows that the pair-stream ends when the token
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
333 stream runs out of binary operators. If this check succeeds, we know
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
334 that the token is a binary operator and that it will be included in this
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
335 expression:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
336
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
337 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
338
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
339 (* Eat the binop. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
340 Stream.junk stream;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
341
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
342 (* Okay, we know this is a binop. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
343 let rhs =
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
344 match Stream.peek stream with
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
345 | Some (Token.Kwd c2) ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
346
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
347 As such, this code eats (and remembers) the binary operator and then
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
348 parses the primary expression that follows. This builds up the whole
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
349 pair, the first of which is [+, b] for the running example.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
350
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
351 Now that we parsed the left-hand side of an expression and one pair of
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
352 the RHS sequence, we have to decide which way the expression associates.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
353 In particular, we could have "(a+b) binop unparsed" or "a + (b binop
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
354 unparsed)". To determine this, we look ahead at "binop" to determine its
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
355 precedence and compare it to BinOp's precedence (which is '+' in this
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
356 case):
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
357
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
358 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
359
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
360 (* If BinOp binds less tightly with rhs than the operator after
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
361 * rhs, let the pending operator take rhs as its lhs. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
362 let next_prec = precedence c2 in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
363 if token_prec < next_prec
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
364
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
365 If the precedence of the binop to the right of "RHS" is lower or equal
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
366 to the precedence of our current operator, then we know that the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
367 parentheses associate as "(a+b) binop ...". In our example, the current
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
368 operator is "+" and the next operator is "+", we know that they have the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
369 same precedence. In this case we'll create the AST node for "a+b", and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
370 then continue parsing:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
371
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
372 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
373
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
374 ... if body omitted ...
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
375 in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
376
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
377 (* Merge lhs/rhs. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
378 let lhs = Ast.Binary (c, lhs, rhs) in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
379 parse_bin_rhs expr_prec lhs stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
380 end
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
381
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
382 In our example above, this will turn "a+b+" into "(a+b)" and execute the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
383 next iteration of the loop, with "+" as the current token. The code
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
384 above will eat, remember, and parse "(c+d)" as the primary expression,
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
385 which makes the current pair equal to [+, (c+d)]. It will then evaluate
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
386 the 'if' conditional above with "\*" as the binop to the right of the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
387 primary. In this case, the precedence of "\*" is higher than the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
388 precedence of "+" so the if condition will be entered.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
389
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
390 The critical question left here is "how can the if condition parse the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
391 right hand side in full"? In particular, to build the AST correctly for
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
392 our example, it needs to get all of "(c+d)\*e\*f" as the RHS expression
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
393 variable. The code to do this is surprisingly simple (code from the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
394 above two blocks duplicated for context):
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
395
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
396 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
397
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
398 match Stream.peek stream with
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
399 | Some (Token.Kwd c2) ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
400 (* If BinOp binds less tightly with rhs than the operator after
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
401 * rhs, let the pending operator take rhs as its lhs. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
402 if token_prec < precedence c2
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
403 then parse_bin_rhs (token_prec + 1) rhs stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
404 else rhs
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
405 | _ -> rhs
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
406 in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
407
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
408 (* Merge lhs/rhs. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
409 let lhs = Ast.Binary (c, lhs, rhs) in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
410 parse_bin_rhs expr_prec lhs stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
411 end
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
412
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
413 At this point, we know that the binary operator to the RHS of our
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
414 primary has higher precedence than the binop we are currently parsing.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
415 As such, we know that any sequence of pairs whose operators are all
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
416 higher precedence than "+" should be parsed together and returned as
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
417 "RHS". To do this, we recursively invoke the ``Parser.parse_bin_rhs``
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
418 function specifying "token\_prec+1" as the minimum precedence required
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
419 for it to continue. In our example above, this will cause it to return
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
420 the AST node for "(c+d)\*e\*f" as RHS, which is then set as the RHS of
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
421 the '+' expression.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
422
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
423 Finally, on the next iteration of the while loop, the "+g" piece is
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
424 parsed and added to the AST. With this little bit of code (14
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
425 non-trivial lines), we correctly handle fully general binary expression
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
426 parsing in a very elegant way. This was a whirlwind tour of this code,
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
427 and it is somewhat subtle. I recommend running through it with a few
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
428 tough examples to see how it works.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
429
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
430 This wraps up handling of expressions. At this point, we can point the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
431 parser at an arbitrary token stream and build an expression from it,
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
432 stopping at the first token that is not part of the expression. Next up
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
433 we need to handle function definitions, etc.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
434
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
435 Parsing the Rest
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
436 ================
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
437
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
438 The next thing missing is handling of function prototypes. In
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
439 Kaleidoscope, these are used both for 'extern' function declarations as
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
440 well as function body definitions. The code to do this is
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
441 straight-forward and not very interesting (once you've survived
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
442 expressions):
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
443
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
444 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
445
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
446 (* prototype
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
447 * ::= id '(' id* ')' *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
448 let parse_prototype =
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
449 let rec parse_args accumulator = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
450 | [< 'Token.Ident id; e=parse_args (id::accumulator) >] -> e
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
451 | [< >] -> accumulator
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
452 in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
453
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
454 parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
455 | [< 'Token.Ident id;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
456 'Token.Kwd '(' ?? "expected '(' in prototype";
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
457 args=parse_args [];
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
458 'Token.Kwd ')' ?? "expected ')' in prototype" >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
459 (* success. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
460 Ast.Prototype (id, Array.of_list (List.rev args))
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
461
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
462 | [< >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
463 raise (Stream.Error "expected function name in prototype")
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
464
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
465 Given this, a function definition is very simple, just a prototype plus
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
466 an expression to implement the body:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
467
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
468 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
469
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
470 (* definition ::= 'def' prototype expression *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
471 let parse_definition = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
472 | [< 'Token.Def; p=parse_prototype; e=parse_expr >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
473 Ast.Function (p, e)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
474
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
475 In addition, we support 'extern' to declare functions like 'sin' and
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
476 'cos' as well as to support forward declaration of user functions. These
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
477 'extern's are just prototypes with no body:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
478
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
479 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
480
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
481 (* external ::= 'extern' prototype *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
482 let parse_extern = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
483 | [< 'Token.Extern; e=parse_prototype >] -> e
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
484
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
485 Finally, we'll also let the user type in arbitrary top-level expressions
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
486 and evaluate them on the fly. We will handle this by defining anonymous
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
487 nullary (zero argument) functions for them:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
488
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
489 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
490
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
491 (* toplevelexpr ::= expression *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
492 let parse_toplevel = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
493 | [< e=parse_expr >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
494 (* Make an anonymous proto. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
495 Ast.Function (Ast.Prototype ("", [||]), e)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
496
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
497 Now that we have all the pieces, let's build a little driver that will
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
498 let us actually *execute* this code we've built!
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
499
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
500 The Driver
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
501 ==========
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
502
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
503 The driver for this simply invokes all of the parsing pieces with a
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
504 top-level dispatch loop. There isn't much interesting here, so I'll just
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
505 include the top-level loop. See `below <#code>`_ for full code in the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
506 "Top-Level Parsing" section.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
507
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
508 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
509
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
510 (* top ::= definition | external | expression | ';' *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
511 let rec main_loop stream =
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
512 match Stream.peek stream with
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
513 | None -> ()
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
514
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
515 (* ignore top-level semicolons. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
516 | Some (Token.Kwd ';') ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
517 Stream.junk stream;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
518 main_loop stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
519
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
520 | Some token ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
521 begin
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
522 try match token with
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
523 | Token.Def ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
524 ignore(Parser.parse_definition stream);
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
525 print_endline "parsed a function definition.";
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
526 | Token.Extern ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
527 ignore(Parser.parse_extern stream);
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
528 print_endline "parsed an extern.";
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
529 | _ ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
530 (* Evaluate a top-level expression into an anonymous function. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
531 ignore(Parser.parse_toplevel stream);
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
532 print_endline "parsed a top-level expr";
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
533 with Stream.Error s ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
534 (* Skip token for error recovery. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
535 Stream.junk stream;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
536 print_endline s;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
537 end;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
538 print_string "ready> "; flush stdout;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
539 main_loop stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
540
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
541 The most interesting part of this is that we ignore top-level
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
542 semicolons. Why is this, you ask? The basic reason is that if you type
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
543 "4 + 5" at the command line, the parser doesn't know whether that is the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
544 end of what you will type or not. For example, on the next line you
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
545 could type "def foo..." in which case 4+5 is the end of a top-level
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
546 expression. Alternatively you could type "\* 6", which would continue
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
547 the expression. Having top-level semicolons allows you to type "4+5;",
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
548 and the parser will know you are done.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
549
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
550 Conclusions
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
551 ===========
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
552
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
553 With just under 300 lines of commented code (240 lines of non-comment,
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
554 non-blank code), we fully defined our minimal language, including a
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
555 lexer, parser, and AST builder. With this done, the executable will
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
556 validate Kaleidoscope code and tell us if it is grammatically invalid.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
557 For example, here is a sample interaction:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
558
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
559 .. code-block:: bash
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
560
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
561 $ ./toy.byte
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
562 ready> def foo(x y) x+foo(y, 4.0);
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
563 Parsed a function definition.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
564 ready> def foo(x y) x+y y;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
565 Parsed a function definition.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
566 Parsed a top-level expr
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
567 ready> def foo(x y) x+y );
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
568 Parsed a function definition.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
569 Error: unknown token when expecting an expression
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
570 ready> extern sin(a);
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
571 ready> Parsed an extern
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
572 ready> ^D
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
573 $
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
574
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
575 There is a lot of room for extension here. You can define new AST nodes,
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
576 extend the language in many ways, etc. In the `next
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
577 installment <OCamlLangImpl3.html>`_, we will describe how to generate
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
578 LLVM Intermediate Representation (IR) from the AST.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
579
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
580 Full Code Listing
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
581 =================
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
582
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
583 Here is the complete code listing for this and the previous chapter.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
584 Note that it is fully self-contained: you don't need LLVM or any
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
585 external libraries at all for this. (Besides the ocaml standard
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
586 libraries, of course.) To build this, just compile with:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
587
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
588 .. code-block:: bash
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
589
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
590 # Compile
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
591 ocamlbuild toy.byte
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
592 # Run
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
593 ./toy.byte
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
594
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
595 Here is the code:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
596
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
597 \_tags:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
598 ::
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
599
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
600 <{lexer,parser}.ml>: use_camlp4, pp(camlp4of)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
601
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
602 token.ml:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
603 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
604
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
605 (*===----------------------------------------------------------------------===
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
606 * Lexer Tokens
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
607 *===----------------------------------------------------------------------===*)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
608
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
609 (* The lexer returns these 'Kwd' if it is an unknown character, otherwise one of
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
610 * these others for known things. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
611 type token =
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
612 (* commands *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
613 | Def | Extern
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
614
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
615 (* primary *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
616 | Ident of string | Number of float
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
617
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
618 (* unknown *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
619 | Kwd of char
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
620
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
621 lexer.ml:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
622 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
623
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
624 (*===----------------------------------------------------------------------===
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
625 * Lexer
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
626 *===----------------------------------------------------------------------===*)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
627
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
628 let rec lex = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
629 (* Skip any whitespace. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
630 | [< ' (' ' | '\n' | '\r' | '\t'); stream >] -> lex stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
631
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
632 (* identifier: [a-zA-Z][a-zA-Z0-9] *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
633 | [< ' ('A' .. 'Z' | 'a' .. 'z' as c); stream >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
634 let buffer = Buffer.create 1 in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
635 Buffer.add_char buffer c;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
636 lex_ident buffer stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
637
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
638 (* number: [0-9.]+ *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
639 | [< ' ('0' .. '9' as c); stream >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
640 let buffer = Buffer.create 1 in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
641 Buffer.add_char buffer c;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
642 lex_number buffer stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
643
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
644 (* Comment until end of line. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
645 | [< ' ('#'); stream >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
646 lex_comment stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
647
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
648 (* Otherwise, just return the character as its ascii value. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
649 | [< 'c; stream >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
650 [< 'Token.Kwd c; lex stream >]
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
651
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
652 (* end of stream. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
653 | [< >] -> [< >]
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
654
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
655 and lex_number buffer = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
656 | [< ' ('0' .. '9' | '.' as c); stream >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
657 Buffer.add_char buffer c;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
658 lex_number buffer stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
659 | [< stream=lex >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
660 [< 'Token.Number (float_of_string (Buffer.contents buffer)); stream >]
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
661
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
662 and lex_ident buffer = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
663 | [< ' ('A' .. 'Z' | 'a' .. 'z' | '0' .. '9' as c); stream >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
664 Buffer.add_char buffer c;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
665 lex_ident buffer stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
666 | [< stream=lex >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
667 match Buffer.contents buffer with
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
668 | "def" -> [< 'Token.Def; stream >]
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
669 | "extern" -> [< 'Token.Extern; stream >]
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
670 | id -> [< 'Token.Ident id; stream >]
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
671
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
672 and lex_comment = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
673 | [< ' ('\n'); stream=lex >] -> stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
674 | [< 'c; e=lex_comment >] -> e
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
675 | [< >] -> [< >]
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
676
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
677 ast.ml:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
678 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
679
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
680 (*===----------------------------------------------------------------------===
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
681 * Abstract Syntax Tree (aka Parse Tree)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
682 *===----------------------------------------------------------------------===*)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
683
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
684 (* expr - Base type for all expression nodes. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
685 type expr =
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
686 (* variant for numeric literals like "1.0". *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
687 | Number of float
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
688
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
689 (* variant for referencing a variable, like "a". *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
690 | Variable of string
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
691
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
692 (* variant for a binary operator. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
693 | Binary of char * expr * expr
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
694
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
695 (* variant for function calls. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
696 | Call of string * expr array
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
697
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
698 (* proto - This type represents the "prototype" for a function, which captures
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
699 * its name, and its argument names (thus implicitly the number of arguments the
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
700 * function takes). *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
701 type proto = Prototype of string * string array
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
702
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
703 (* func - This type represents a function definition itself. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
704 type func = Function of proto * expr
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
705
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
706 parser.ml:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
707 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
708
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
709 (*===---------------------------------------------------------------------===
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
710 * Parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
711 *===---------------------------------------------------------------------===*)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
712
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
713 (* binop_precedence - This holds the precedence for each binary operator that is
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
714 * defined *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
715 let binop_precedence:(char, int) Hashtbl.t = Hashtbl.create 10
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
716
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
717 (* precedence - Get the precedence of the pending binary operator token. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
718 let precedence c = try Hashtbl.find binop_precedence c with Not_found -> -1
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
719
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
720 (* primary
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
721 * ::= identifier
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
722 * ::= numberexpr
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
723 * ::= parenexpr *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
724 let rec parse_primary = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
725 (* numberexpr ::= number *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
726 | [< 'Token.Number n >] -> Ast.Number n
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
727
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
728 (* parenexpr ::= '(' expression ')' *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
729 | [< 'Token.Kwd '('; e=parse_expr; 'Token.Kwd ')' ?? "expected ')'" >] -> e
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
730
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
731 (* identifierexpr
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
732 * ::= identifier
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
733 * ::= identifier '(' argumentexpr ')' *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
734 | [< 'Token.Ident id; stream >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
735 let rec parse_args accumulator = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
736 | [< e=parse_expr; stream >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
737 begin parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
738 | [< 'Token.Kwd ','; e=parse_args (e :: accumulator) >] -> e
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
739 | [< >] -> e :: accumulator
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
740 end stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
741 | [< >] -> accumulator
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
742 in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
743 let rec parse_ident id = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
744 (* Call. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
745 | [< 'Token.Kwd '(';
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
746 args=parse_args [];
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
747 'Token.Kwd ')' ?? "expected ')'">] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
748 Ast.Call (id, Array.of_list (List.rev args))
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
749
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
750 (* Simple variable ref. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
751 | [< >] -> Ast.Variable id
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
752 in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
753 parse_ident id stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
754
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
755 | [< >] -> raise (Stream.Error "unknown token when expecting an expression.")
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
756
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
757 (* binoprhs
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
758 * ::= ('+' primary)* *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
759 and parse_bin_rhs expr_prec lhs stream =
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
760 match Stream.peek stream with
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
761 (* If this is a binop, find its precedence. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
762 | Some (Token.Kwd c) when Hashtbl.mem binop_precedence c ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
763 let token_prec = precedence c in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
764
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
765 (* If this is a binop that binds at least as tightly as the current binop,
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
766 * consume it, otherwise we are done. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
767 if token_prec < expr_prec then lhs else begin
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
768 (* Eat the binop. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
769 Stream.junk stream;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
770
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
771 (* Parse the primary expression after the binary operator. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
772 let rhs = parse_primary stream in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
773
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
774 (* Okay, we know this is a binop. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
775 let rhs =
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
776 match Stream.peek stream with
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
777 | Some (Token.Kwd c2) ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
778 (* If BinOp binds less tightly with rhs than the operator after
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
779 * rhs, let the pending operator take rhs as its lhs. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
780 let next_prec = precedence c2 in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
781 if token_prec < next_prec
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
782 then parse_bin_rhs (token_prec + 1) rhs stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
783 else rhs
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
784 | _ -> rhs
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
785 in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
786
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
787 (* Merge lhs/rhs. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
788 let lhs = Ast.Binary (c, lhs, rhs) in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
789 parse_bin_rhs expr_prec lhs stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
790 end
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
791 | _ -> lhs
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
792
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
793 (* expression
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
794 * ::= primary binoprhs *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
795 and parse_expr = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
796 | [< lhs=parse_primary; stream >] -> parse_bin_rhs 0 lhs stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
797
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
798 (* prototype
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
799 * ::= id '(' id* ')' *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
800 let parse_prototype =
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
801 let rec parse_args accumulator = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
802 | [< 'Token.Ident id; e=parse_args (id::accumulator) >] -> e
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
803 | [< >] -> accumulator
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
804 in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
805
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
806 parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
807 | [< 'Token.Ident id;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
808 'Token.Kwd '(' ?? "expected '(' in prototype";
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
809 args=parse_args [];
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
810 'Token.Kwd ')' ?? "expected ')' in prototype" >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
811 (* success. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
812 Ast.Prototype (id, Array.of_list (List.rev args))
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
813
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
814 | [< >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
815 raise (Stream.Error "expected function name in prototype")
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
816
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
817 (* definition ::= 'def' prototype expression *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
818 let parse_definition = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
819 | [< 'Token.Def; p=parse_prototype; e=parse_expr >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
820 Ast.Function (p, e)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
821
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
822 (* toplevelexpr ::= expression *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
823 let parse_toplevel = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
824 | [< e=parse_expr >] ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
825 (* Make an anonymous proto. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
826 Ast.Function (Ast.Prototype ("", [||]), e)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
827
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
828 (* external ::= 'extern' prototype *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
829 let parse_extern = parser
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
830 | [< 'Token.Extern; e=parse_prototype >] -> e
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
831
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
832 toplevel.ml:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
833 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
834
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
835 (*===----------------------------------------------------------------------===
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
836 * Top-Level parsing and JIT Driver
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
837 *===----------------------------------------------------------------------===*)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
838
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
839 (* top ::= definition | external | expression | ';' *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
840 let rec main_loop stream =
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
841 match Stream.peek stream with
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
842 | None -> ()
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
843
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
844 (* ignore top-level semicolons. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
845 | Some (Token.Kwd ';') ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
846 Stream.junk stream;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
847 main_loop stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
848
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
849 | Some token ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
850 begin
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
851 try match token with
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
852 | Token.Def ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
853 ignore(Parser.parse_definition stream);
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
854 print_endline "parsed a function definition.";
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
855 | Token.Extern ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
856 ignore(Parser.parse_extern stream);
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
857 print_endline "parsed an extern.";
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
858 | _ ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
859 (* Evaluate a top-level expression into an anonymous function. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
860 ignore(Parser.parse_toplevel stream);
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
861 print_endline "parsed a top-level expr";
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
862 with Stream.Error s ->
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
863 (* Skip token for error recovery. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
864 Stream.junk stream;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
865 print_endline s;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
866 end;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
867 print_string "ready> "; flush stdout;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
868 main_loop stream
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
869
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
870 toy.ml:
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
871 .. code-block:: ocaml
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
872
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
873 (*===----------------------------------------------------------------------===
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
874 * Main driver code.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
875 *===----------------------------------------------------------------------===*)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
876
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
877 let main () =
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
878 (* Install standard binary operators.
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
879 * 1 is the lowest precedence. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
880 Hashtbl.add Parser.binop_precedence '<' 10;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
881 Hashtbl.add Parser.binop_precedence '+' 20;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
882 Hashtbl.add Parser.binop_precedence '-' 20;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
883 Hashtbl.add Parser.binop_precedence '*' 40; (* highest. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
884
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
885 (* Prime the first token. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
886 print_string "ready> "; flush stdout;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
887 let stream = Lexer.lex (Stream.of_channel stdin) in
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
888
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
889 (* Run the main "interpreter loop" now. *)
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
890 Toplevel.main_loop stream;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
891 ;;
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
892
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
893 main ()
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
894
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
895 `Next: Implementing Code Generation to LLVM IR <OCamlLangImpl3.html>`_
9ad51c7bc036 1st commit. remove git dir and add all files.
Kaito Tokumori <e105711@ie.u-ryukyu.ac.jp>
parents:
diff changeset
896