Mercurial > hg > Members > shinya > pyrect

annotate pyrect/jitgrep.py @ 69:4de11d799dee

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
add boost algorithm. but it's buggy, not work.
author Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
date Sun, 07 Nov 2010 09:32:25 +0900
parents 020ba001c58a
children e06786b3c2dc
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
12
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
1 #!/usr/bin/env python
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
2
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
3 import sys
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
4 import os
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
5 import re
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
6 import time
14
55684cb51347 add LICENSE
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 12
diff changeset
7 from optparse import OptionParser
47
701beabd7d97 add input-rules, Range, CharacterClass, Anchor and MultiByte-Char(but not work)\nand more simplify NFA (is global improvement).
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 43
diff changeset
8 from pyrect.translator import *
69
4de11d799dee add boost algorithm. but it's buggy, not work.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 63
diff changeset
9 from pyrect.regexp import Regexp, CharCollector
12
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
10
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
11 def main(argv):
34
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
12 myusage = """%prog [--buf-size=size] [--dump]
52
abb0691e792a bug fix. remove unnecessarily files.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 47
diff changeset
13 [--time] [--debug] [--cc=compiler] [-c]
56
ee9945561f80 add parallel I/O grep (per line) with pthread. but it's very slow. really slow..
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 54
diff changeset
14 [-Olevel] regexp [file..] [--out=file]
69
4de11d799dee add boost algorithm. but it's buggy, not work.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 63
diff changeset
15 [--thread=thread_num] [--disable-filter]
4de11d799dee add boost algorithm. but it's buggy, not work.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 63
diff changeset
16 [--disable-booster]"""
14
55684cb51347 add LICENSE
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 12
diff changeset
17 psr = OptionParser(usage=myusage)
55684cb51347 add LICENSE
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 12
diff changeset
18
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
19 redirect = ""
27
3db85244784b modify jitgrep, pre-compile grep main routine to libgrep.so. so JIT-compile only required DFA-transition.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 25
diff changeset
20 srcpath = "/tmp/jitgrep_dfa.c"
3db85244784b modify jitgrep, pre-compile grep main routine to libgrep.so. so JIT-compile only required DFA-transition.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 25
diff changeset
21 binpath = "/tmp/jitgrep"
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
22
33
e9e90c006760 simplify grep.c, correnspod syntax '^'.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 32
diff changeset
23 psr.add_option("--cc", action="store", type="string", dest="cc", default="gcc", metavar="FILE",
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
24 help="Choose compiler (default is gcc).")
52
abb0691e792a bug fix. remove unnecessarily files.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 47
diff changeset
25 psr.add_option("-c", action="store_true", dest="compile", default=False , help="compile only.")
34
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
26 psr.add_option("--buf-size=size", action="store", type="string", dest="bufsize", default="1M" , help="Set read-buffer size (e.x. 1024, 1024K, 2M)")
57
81b44ae1cd73 add fixed-string filter(Boyer-Moore), and add option '--disable-filter'.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 56
diff changeset
27 psr.add_option("--cflags", action="store", type="string", dest="cflags", default="-O3", help="Print compile/matching time.")
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
28 psr.add_option("--time", action="store_true", dest="time", default=False, help="Print compile/matching time.")
56
ee9945561f80 add parallel I/O grep (per line) with pthread. but it's very slow. really slow..
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 54
diff changeset
29 psr.add_option("--thread", action="store", type="string", dest="thread", default="0", metavar="FILE", help="number of thread.")
57
81b44ae1cd73 add fixed-string filter(Boyer-Moore), and add option '--disable-filter'.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 56
diff changeset
30 psr.add_option("--disable-filter", action="store_true", dest="no_filter", default=False, help="disable fixed-string filter(Boyer-Moore).")
69
4de11d799dee add boost algorithm. but it's buggy, not work.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 63
diff changeset
31 psr.add_option("--disable-booster", action="store_true", dest="no_booster", default=False, help="disable booster.")
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
32 psr.add_option("--debug", action="store_true", dest="debug", default=False, help="Dump commands, not evalute matching (except interactive mode).")
42
84059ea1a2db add goto_grep_translator.py, this is label-based grep implimentation.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 40
diff changeset
33 psr.add_option("--label", action="store_true", dest="label", default=False, help="label implimentation in C.")
34
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
34 psr.add_option("--dump", action="store_true", dest="dump", default=False, help="Dump generated grep-source.")
25
9efed6ff6088 bug-fix jitgrep.py: duplicate keywords 'help' in OptionParser.add_option() 's argument.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 15
diff changeset
35 psr.add_option("--out", action="store", type="string", dest="out", default="", metavar="FILE", help="Output file.")
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
36
14
55684cb51347 add LICENSE
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 12
diff changeset
37 (opts, args) = psr.parse_args(argv)
55684cb51347 add LICENSE
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 12
diff changeset
38
27
3db85244784b modify jitgrep, pre-compile grep main routine to libgrep.so. so JIT-compile only required DFA-transition.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 25
diff changeset
39 if len(args) < 2:
3db85244784b modify jitgrep, pre-compile grep main routine to libgrep.so. so JIT-compile only required DFA-transition.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 25
diff changeset
40 psr.print_usage()
52
abb0691e792a bug fix. remove unnecessarily files.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 47
diff changeset
41 return
27
3db85244784b modify jitgrep, pre-compile grep main routine to libgrep.so. so JIT-compile only required DFA-transition.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 25
diff changeset
42
33
e9e90c006760 simplify grep.c, correnspod syntax '^'.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 32
diff changeset
43 if opts.cc == "cbc":
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
44 cbc = True
40
962ae4154724 add CbCGREPTranslator.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 34
diff changeset
45 opts.cc = "$CBCROOT/INSTALL_DIR/bin/gcc"
52
abb0691e792a bug fix. remove unnecessarily files.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 47
diff changeset
46 opts.cflags += " -L$CBCROOT/gcc -w"
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
47 else:
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
48 cbc = False
12
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
49
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
50 if opts.debug: print("option", opts)
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
51 if opts.debug: print("args", args)
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
52
33
e9e90c006760 simplify grep.c, correnspod syntax '^'.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 32
diff changeset
53 string = args[1]
e9e90c006760 simplify grep.c, correnspod syntax '^'.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 32
diff changeset
54
34
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
55 try:
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
56 if opts.bufsize[-1] == 'K':
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
57 bufsize = int(opts.bufsize[:-1]) * 2**10
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
58 elif opts.bufsize[-1] == 'M':
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
59 bufsize = int(opts.bufsize[:-1]) * 2**20
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
60 else:
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
61 bufsize = int(opts.bufsize)
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
62 except ValueError:
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
63 psr.print_usage()
52
abb0691e792a bug fix. remove unnecessarily files.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 47
diff changeset
64 return
34
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
65
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
66 if opts.time : start_time = time.time()
69
4de11d799dee add boost algorithm. but it's buggy, not work.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 63
diff changeset
67
63
020ba001c58a modify I/O routine. use mmap. it's really faster than fgets ;-)
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 59
diff changeset
68 reg = Regexp(".*"+string)
69
4de11d799dee add boost algorithm. but it's buggy, not work.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 63
diff changeset
69 reg.chars = Regexp.get_chars(string)
4de11d799dee add boost algorithm. but it's buggy, not work.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 63
diff changeset
70 (reg.max_len, _, _) = Regexp.get_analyze(string)
4de11d799dee add boost algorithm. but it's buggy, not work.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 63
diff changeset
71
40
962ae4154724 add CbCGREPTranslator.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 34
diff changeset
72 if cbc:
43
83c69d42faa8 replace converting-flow, module dfareg with module regexp. it's is substantial changing in implimentation.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 42
diff changeset
73 grept = CbCGREPTranslator(reg)
42
84059ea1a2db add goto_grep_translator.py, this is label-based grep implimentation.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 40
diff changeset
74 elif opts.label:
43
83c69d42faa8 replace converting-flow, module dfareg with module regexp. it's is substantial changing in implimentation.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 42
diff changeset
75 grept = GoToGREPTranslator(reg)
40
962ae4154724 add CbCGREPTranslator.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 34
diff changeset
76 else:
43
83c69d42faa8 replace converting-flow, module dfareg with module regexp. it's is substantial changing in implimentation.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 42
diff changeset
77 grept = GREPTranslator(reg)
57
81b44ae1cd73 add fixed-string filter(Boyer-Moore), and add option '--disable-filter'.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 56
diff changeset
78 grept.filter = not opts.no_filter
69
4de11d799dee add boost algorithm. but it's buggy, not work.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 63
diff changeset
79 grept.skip_boost = not opts.no_booster
59
fd3d0b8326fe implement regexp-syntax any-char ('.').
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 57
diff changeset
80 grept.thread_line = int(opts.thread)
69
4de11d799dee add boost algorithm. but it's buggy, not work.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 63
diff changeset
81
34
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
82 grept.bufsize = bufsize
12
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
83
34
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
84 if opts.dump:
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
85 grept.translate()
52
abb0691e792a bug fix. remove unnecessarily files.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 47
diff changeset
86 return
34
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
87 else:
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
88 tmpsrc = open(srcpath, 'w')
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
89 grept.translate(tmpsrc)
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
90 tmpsrc.close()
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
91
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
92 if (opts.time):
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
93 end_time = time.time()
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
94 print("Translation: " + str(end_time - start_time) + " Sec.")
12
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
95
34
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
96 cmd = " ".join([opts.cc, opts.cflags, srcpath, "-o", binpath])
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
97 if opts.debug:
14
55684cb51347 add LICENSE
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 12
diff changeset
98 print("compile command", cmd)
55684cb51347 add LICENSE
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 12
diff changeset
99 else:
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
100 if (opts.time): start_time = time.time()
14
55684cb51347 add LICENSE
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 12
diff changeset
101 os.system(cmd)
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
102 if (opts.time):
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
103 end_time = time.time()
27
3db85244784b modify jitgrep, pre-compile grep main routine to libgrep.so. so JIT-compile only required DFA-transition.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 25
diff changeset
104 print("Compiling : " + str(end_time - start_time) + " Sec.")
12
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
105
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
106 if opts.debug:
32
8d5ab8036fac buf-fix: forgot adding CFLAGS to compile-option(jitgrep.py) .
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 30
diff changeset
107 print("argv=", argv)
8d5ab8036fac buf-fix: forgot adding CFLAGS to compile-option(jitgrep.py) .
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 30
diff changeset
108 print("args=", args)
8d5ab8036fac buf-fix: forgot adding CFLAGS to compile-option(jitgrep.py) .
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 30
diff changeset
109 print("opts=", opts)
12
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
110
52
abb0691e792a bug fix. remove unnecessarily files.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 47
diff changeset
111 if opts.compile:
abb0691e792a bug fix. remove unnecessarily files.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 47
diff changeset
112 return
abb0691e792a bug fix. remove unnecessarily files.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 47
diff changeset
113
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
114 if len(args) == 2 and not opts.debug:
12
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
115 while True:
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
116 try:
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
117 os.system(binpath + ' ' + raw_input())
14
55684cb51347 add LICENSE
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 12
diff changeset
118 except (KeyboardInterrupt, EOFError):
12
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
119 break
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
120 else:
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
121 if (opts.time): redirect = "> /dev/null"
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
122 if (opts.out): redirect = ">" + opts.out
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
123 cmd = ' '.join([binpath, "dummy"] + args[2:] + [redirect])
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
124 if opts.debug:
14
55684cb51347 add LICENSE
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 12
diff changeset
125 print("exec command", cmd)
55684cb51347 add LICENSE
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 12
diff changeset
126 else:
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
127 if (opts.time): start_time = time.time()
14
55684cb51347 add LICENSE
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 12
diff changeset
128 os.system(cmd)
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
129 if (opts.time):
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
130 end_time = time.time()
27
3db85244784b modify jitgrep, pre-compile grep main routine to libgrep.so. so JIT-compile only required DFA-transition.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 25
diff changeset
131 print("Matching : " + str(end_time - start_time) + " Sec.")
12
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
132
15
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
133 if not opts.debug:
104ee9208b17 add options -Olevel --time --debug --cc=compiler to jitgrep. add agrep to benchgrep.sh.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 14
diff changeset
134 #os.remove(srcpath)
34
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
135 #os.remove(binpath)
50b10929be29 change compile-method to full-source-compile.
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents: 33
diff changeset
136 pass
12
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
137
41391400fe68 add GREPTranslator(Translator) and implement jit-compile-grep,
Ryoma SHINYA <shinya@firefly.cr.ie.u-ryukyu.ac.jp>
parents:
diff changeset
138 if __name__ == '__main__': main(sys.argv)