Time-Out

Working on the Hi language goes through times where new and old ideas gobble up and I look at them from various angles where actually no code is being written. Ah well, the two, strike that, three things which need to happen, are 1. full conformance (long integers, floats and doubles, native strings, a fast type-checker), 2. a VM with native support for strings, 3. performance.

I'll need to dump the current translation scheme, which is a pity since by keeping it simple I could use the C optimizer to do a lot for free like register allocation and lots of assembly instruction optimizations.

Another of the big drawbacks of going to a VM -which should support strings- is that in the current compilation scheme an allocation is cheap. Functions are translated to combinators, combinators rewrite the heap. The assumption at the moment is that a combinator will always rewrite (to) a DAG, and that there is always enough space in the heap to allocate the intermediates. That way, a) allocation is cheap (just an increment), b) I only garbage collect in between rewrites, which is a fast scheme by itself.

If I am gonna allow string allocations in between, I will open up a severe can of worms, since the assumption that there will always be enough heap space for intermediates must be dropped, thus garbage collections must be traced, thus intermediate pointers from the 'stack' must be tracked and chased also. So, I might as well go for the whole stack of optimizations, hello typed assembly and graph-coloring algorithms.

I use SSA form in the intermediate assembly with an infinite supply of intermediate local registers which is why I also looked at LLVM since that would map, but I might be better of by changing it to stack pushing (also, since the data structure in the heap is a DAG - garbage collection could be done by stack compressing), no idea.

For the bootstrap, I chose the simplest scheme I could get away with for representing records, somewhat with an eye on making it easy for dynamic 'code' generation. It wastes memory like hell; at the moment, I am looking at 1GB compiles, whereas my gut feeling tells me that 20MB should just be enough. Okay, this compiler is pretty naive, and uses list of characters (texts) in places where integers should suffice, but still, without a better scheme for representing data I'll still expect to be in the hundreds of MB region.

I could reuse the ocaml VM, which I kind-of skimmed through a few times, or see what Guile has to give, but still, it looks like I'll need my own VM. (Hmm, CDL3 also has a simple VM, could look there too.)

C Back-End

I looked at various manners of binding LLVM to Hi. It ain't easy. The most direct would be to just use Hi's FFI for an LLVM-Hi binding, but LLVM has a big back-end, and it is written in C++, whereas Hi binds to C. The best route to take looks like to 'double' the code of the internal assembly used in C with a library for that, and -at some point- write an LLVM back-end instead of a C back-end. It would speed up the emitting phase I hope since it wouldn't need (text emitting) functions written in Hi, which are pretty slow at the moment.

LLVM

Yeah, I give in. LLVM looks mature enough as a target. Studying on it. Maybe I can re-use part of the EHC back-end? Problem is, it looks dead, and doesn't GC last I looked. Hmm, GHC has an LLVM back-end now...

A VM

I am restructuring the back-end towards a proper VM. Which actually shouldn't be too hard.

Log 092210

I started again. Just decided to grow the new compiler organically, so work my way past each (performance problem). The basis is sound enough.

Inline Comments

Of course, removing generated comments from the generated code helps a lot too.

Log 090210

I did some light programming and removed the URI from the unserialization steps. It helped somewhat, but the compiler still wastes way too much memory. Everything anew from the ground up? Too much of a hassle. I might just toss the compiler over the edge and be done with it as a nice but failed experiment.

Too many cons-nodes in the heap. Should do a quantitative assessment of that.

Another way forward would actually be setting one step back. Now the code explodes in the translation from combinators to C source code, a better approach might just be to not generate C, but have a combinator language interpreter. It would be somewhat slower, and the compiler would still hog the memory at places, but the generated code would probably be order(s) smaller.