A simple transform which maps text in the AST to its identity using a map function, therefor making all references in the heap unique, would get rid of all that double character data too.
I feel like I am back to my C64 where, in order to get at -or change- that extra RAM, one needed to 'peek' all values of a memory region and 'poke' them to their addresses just before disabling the ROM shadowed memory region.
Monday, August 30, 2010
Sunday, August 29, 2010
Not Doing a Lot
As the title says, severe lack of motivation to program. Anyhow, thought a bit more about a trivial typing scheme, it should be enough if I just reverse/extend type variables with their assumed implemented interfaces, unify on that, and get rid of generalization to derive an AST based type checker. Not sure where all the assumptions and stuff should go then in the context, but think about that later.
Saturday, August 28, 2010
Uncool
I'ld think implementing strings would help somewhat, but I didn't manage to convince myself, and biggest reason: I just don't feel like, having strings as lists of characters works for Haskell too and gives nice algorithms. The real problem is 1GB of character data, which are probably mostly copies of themselves. Other stuff I could do:
- Make the data unique by storing strings in a map and only extract unique copies.
- Don't retain specific character data, it's probably all positional information anyway.
- Change the runtime such that all shareable data is shared.
I like 1 the most, but that would best be solved by having a constant pool referred to as a variable, and I don't have variables at the moment.
(Did I mention using a symbol table? A simple transform which just visits all AST nodes once would do it too, of course.)
(Did I mention using a symbol table? A simple transform which just visits all AST nodes once would do it too, of course.)
Thursday, August 26, 2010
Log 082610
The heap profiler confirmed what I knew. Two basic things which slow down my compiler are: a) way too many chars, so I'll need a native text type, b) comparisons are just too slow, which may also be the result of having way too many chars.
There are some other optimizations, like dropping positional information in the linking phase and implementing a condensed and more intelligible representation for identifiers, which would help too.
First move forward, implement all those basic string primitives (length, pos_of, index, etc.) on strings.
(A cell on a 64 bit system is 64 bits. A char is placed in an array consisting of a length, cell-type, type and value. A cons is placed in an array of a length, cell-type, record-tag, type tag, header and tail. So, ten times eight, is eighty bytes per char. So, eighty times 15,550,128 gives 1,244,010,240 bytes. And most of that is probably the not-shared URI in positional information.)
There are some other optimizations, like dropping positional information in the linking phase and implementing a condensed and more intelligible representation for identifiers, which would help too.
First move forward, implement all those basic string primitives (length, pos_of, index, etc.) on strings.
(A cell on a 64 bit system is 64 bits. A char is placed in an array consisting of a length, cell-type, type and value. A cons is placed in an array of a length, cell-type, record-tag, type tag, header and tail. So, ten times eight, is eighty bytes per char. So, eighty times 15,550,128 gives 1,244,010,240 bytes. And most of that is probably the not-shared URI in positional information.)
Wednesday, August 25, 2010
I should cut back on chars...
So, I wrote a heap profiler, output every 10%:
It starts pretty innocent:
And grows.
And grows.
And grows.
And grows.
And grows.
And grows.
And grows.
And grows.
And grows.
Analyze the following statement: "Nothing is unthinkable, for to think of nothing, is not to think at all."
It starts pretty innocent:
THUNK 160 char 3300 list.cons 3972 int 4 system._txt 256 system.tuple_4 12And grows.
THUNK 248 pointer 4 char 204964 int 20356 position._pos 10176 ast.type_name 1932 system._txt 15728 ast.expr_empty 864 list.cons 206748 ast.decl_instance 32 ast.decl_def 156 ast.type_instance 32 ast.expr_name 1580 ast.type_arrow 568 ast.expr_case 292 ast.decl_bind 32 ast.expr_match 384 ast.decl_space 32 ast.decl_type 16 ast.expr_record 308 ast.decl_using 24 ast.type_case 12 ast.kind_unknown 300 ast.expr_number 88 ast.type_record 20 ast.expr_variable 604 ast.decl_interface 20 ast.type_universal 204 ast.decl_record 12 ast.type_unknown 580 ast.expr_application 676 ast.type_abstraction 60 ast.type_variable 264 ast.kind_star 580 ast.type_interface 20 system.tuple_2 8 ast.type_constrain 196 ast.decl_method 72 ast.type_application 96 ast.expr_select 88 ast.expr_char 12
And grows.
THUNK 356 pointer 4 char 807500 int 79964 position._pos 39976 ast.expr_empty 3104 ast.expr_name 6508 ast.type_name 6468 ast.type_universal 520 system._txt 61664 ast.type_variable 760 ast.decl_record 52 list.cons 816488 ast.kind_unknown 876 ast.expr_case 1472 ast.expr_match 1836 ast.expr_record 1776 ast.decl_space 68 ast.expr_variable 2456 ast.type_arrow 1368 ast.type_unknown 2404 ast.decl_using 84 ast.decl_instance 136 ast.kind_star 2404 ast.type_application 672 ast.decl_type 96 ast.type_case 56 ast.type_instance 136 system.tuple_2 8 ast.type_constrain 400 ast.type_skolem 32 ast.decl_def 504 ast.type_record 64 ast.decl_bind 168 ast.expr_application 3076 ast.comment 16 ast.expr_embed 256 ast.expr_number 400 ast.decl_interface 20 ast.ffi_type_simple 652 ast.expr_char 700 ast.type_abstraction 240 ast.type_interface 20 ast.decl_method 72 ast.expr_select 88 ast.expr_throw 4
And grows.
THUNK 340 ffi.struct 4 pointer 4 char 2809580 int 287476 position._pos 143732 system._txt 223008 ast.expr_match 6116 list.cons 2843552 ast.expr_empty 9960 ast.expr_record 8944 ast.expr_char 3236 ast.expr_name 26248 ast.type_name 23168 ast.type_arrow 4456 ast.decl_def 2364 ast.expr_case 4616 ast.comment 164 ast.expr_application 10448 ast.expr_variable 9752 ast.expr_text 136 ast.type_unknown 9472 ast.kind_star 9472 ast.part 56 ast.type_universal 1952 ast.decl_using 272 ast.type_variable 2260 ast.decl_space 172 ast.kind_unknown 2448 ast.expr_number 836 system.tuple_2 8 ast.decl_type 152 ast.type_case 96 ast.decl_instance 244 ast.decl_interface 36 ast.type_instance 244 ast.type_record 216 ast.type_abstraction 308 ast.type_interface 36 ast.decl_bind 276 ast.decl_record 216 ast.type_constrain 736 ast.type_application 2864 ast.decl_method 104 ast.type_tuple 120 ast.expr_select 112 ast.type_skolem 36 ast.expr_embed 384 ast.ffi_type_simple 980 ast.expr_throw 4
And grows.
THUNK 452 int 957136 pointer 4 char 10596428 list.cons 10718984 position._pos 478560 ast.expr_empty 30196 ast.expr_variable 45388 system._txt 740672 ast.type_unknown 44512 ast.kind_star 44512 ast.expr_application 49172 ast.expr_name 99336 ast.expr_case 16508 ast.type_arrow 7616 ast.type_name 53132 ast.expr_match 23052 ast.decl_def 4696 ast.expr_record 28540 ast.decl_using 840 ast.type_application 4624 ast.expr_text 904 ast.part 144 ast.comment 532 ast.decl_space 280 system.tuple_2 8 ast.expr_number 1280 ast.decl_type 220 ast.type_tuple 216 ast.expr_char 10288 ast.type_case 116 ast.kind_unknown 2724 ast.type_record 504 ast.decl_instance 328 ast.type_instance 328 ast.expr_throw 192 ast.decl_bind 360 ast.type_abstraction 356 ast.type_variable 2460 ast.decl_interface 44 ast.type_universal 2104 ast.type_interface 44 ast.decl_record 504 ast.type_constrain 824 ast.decl_method 120 ast.expr_select 120 ast.expr_try 4 ast.type_skolem 36 ast.expr_embed 384 ast.ffi_type_simple 980
And grows.
THUNK 244 int 1452744 system._txt 1119192 ast.decl_method 120 map.node 7768 ast.decl_def 5964 list.cons 16055612 position._pos 695124 ast.expr_empty 41900 ast.expr_name 148724 ast.type_name 79064 ast.type_abstraction 376 ast.type_arrow 9364 ast.expr_case 24008 ast.decl_type 268 char 15550128 ast.type_variable 2496 ast.type_constrain 824 ast.type_application 5676 ast.type_case 136 ast.kind_unknown 2808 ast.type_universal 2120 ast.expr_match 32504 ast.expr_application 71856 ast.expr_record 48992 ast.type_record 688 ast.expr_embed 384 ast.expr_variable 62796 ast.expr_number 1836 ast.type_unknown 61728 ast.ffi_type_simple 980 ast.part 176 ast.kind_star 61728 ast.comment 672 ast.type_skolem 36 ast.expr_char 21280 ast.decl_interface 44 ast.expr_text 1148 ast.decl_space 488 lambda.lambda_def 4152 ast.type_tuple 288 ast.type_interface 44 lambda.lambda_app 34044 ast.decl_using 1448 ast.decl_record 688 ast.decl_instance 360 system.tuple_2 8 ast.expr_select 120 lambda.lambda_alts 11108 lambda.lambda_record 18484 ast.type_instance 360 lambda.lambda_comb 16652 lambda.lambda_abs 15720 ast.decl_bind 392 ast.expr_try 8 lambda.lambda_throw 11228 lambda.lambda_name 54724 ast.expr_throw 228 lambda.lambda_const 18140 lambda.lambda_method 316 long 18380 ast.expr_type 4 lambda.lambda_select 120 lambda.lambda_embed 384
And grows.
THUNK 5044 system.tuple_2 44 lambda.lambda_record 77532 list.cons 2404732 lambda.lambda_name 256328 system._txt 233196 int 78140 lambda.lambda_app 192680 lambda.lambda_abs 43772 lambda.lambda_alts 33164 lambda.lambda_const 47296 char 1582504 long 47536 lambda.lambda_comb 32128 lambda.lambda_split 22536 lambda.lambda_def 10680 lambda.lambda_test 788 lambda.lambda_call 32
And grows.
THUNK 40940 list.cons 19046368 code.c_mov 587036 code.op_reg 667452 code.op_int 781748 code.c_tst 140168 code.op_nth 317836 int 1180748 long 292384 system._txt 490732 code.op_lbl 79776 code.c_lbl 79776 code.c_return 49960 code.c_comment 119688 code.op_array 149632 char 2048476 code.c_op 28824 code.op_name 33916 code.c_call 14920 code.c_jmp 14908 code.codestate 8 combinator.comb_def 21176 combinator.comb_alts 21176 code.c_function 13912 combinator.comb_var 233048 combinator.comb_split 13696 combinator.comb_stack 33020 combinator.comb_record 50460 system.tuple_2 8 combinator.comb_thunk 53548 combinator.comb_name 53548 combinator.comb_const 26944 combinator.comb_test 396 combinator.comb_call 20
And grows.
THUNK 4692 int 2610712 map.node 392 code.op_int 2057056 list.cons 29897400 code.c_mov 1780684 code.op_nth 1266064 code.op_array 220172 code.op_reg 1702752 system._txt 622820 code.c_comment 192316 char 2500860 code.c_call 121312 code.c_function 25128 code.c_tst 262624 long 510796 code.op_lbl 121324 code.c_return 89752 code.c_lbl 121324 system.tuple_2 8 code.c_op 52676 code.op_name 53548 code.c_jmp 22976
And grows.
THUNK 11764 ffi.struct 4 int 2301420 list.cons 31044276 system._txt 622808 pointer 4 code.c_comment 192316 code.c_mov 1539136 code.c_function 21176 code.op_reg 3341368 code.op_int 2057056 code.c_tst 220540 code.op_lbl 121324 char 2500840 code.op_array 192252 code.c_return 75372 code.c_lbl 121324 code.op_nth 1099972 code.c_op 44152 long 510796 code.c_call 106212 code.op_name 53548 code.c_jmp 22976
And grows.
THUNK 26100 list.cons 33266952 int 2301420 system._txt 630724 code.op_nth 1099972 code.op_reg 3341368 code.op_int 2057056 code.c_mov 1539136 code.op_array 192252 code.c_comment 192316 pointer 4 code.c_tst 220540 code.c_function 21176 code.op_lbl 121324 long 510796 code.c_return 75372 code.c_lbl 121324 code.c_call 106212 char 2500840 code.c_op 44152 code.op_name 53548 code.c_jmp 22976
Analyze the following statement: "Nothing is unthinkable, for to think of nothing, is not to think at all."
Tuesday, August 24, 2010
A Heap Profiler
Silly thought. I've been looking at how to optimize various around the Hi compiler, but the best manner is always just to get the raw numbers. Now, compiling with -pg gives me great profile information on where time is spend, but it doesn't tell me where all that memory went. I need something which profiles the heap, just a simple tool which counts nodes ordered by kind.
Monday, August 23, 2010
Log 082310
There are times when I can write on the compiler, and there are times when I cannot. Instead, I write on the language specification/standard to get it into a shape that I can express a new typing scheme.
Subscribe to:
Posts (Atom)