Wednesday, June 30, 2010

Log 063010

Writing serialization code. It ain't too nice, should look at betters ways of doing it, just using a piece of malloc'ed memory for a buffer and some serialization primitives - which I should do better.

Wrote a faculty function on longs, debugged it and it got accepted. Got too much boilerplate code now on various types in the C runtime, thinking about removing references to uint8, .., sint64, hi_char, stuff like that - though some people might just like that approach, not sure. Principally, I only need C types and the Hi internal word size.

I should serialize w.r.t. to a serializer, and make it possible to drop in different ones.

Tuesday, June 29, 2010

Foot Hurts

Shot myself in the foot again. Good news, the system file of stage2 compiles, there don't seem to be bugs. Bad news, I use integers of system width... Which means on the somewhat awkward i86_64 architecture, ints are 32 bits wide, and pointers are 64 bits wide, which means that in my stage2 compiler, at some places I can't use ints to represent bits, but need longs. Which is fine, but hell.. On the good side, it's gonna be robust, since I need to cover all corner cases. (It's just adjusting a type definition and some code.)

(Notice I did change the semantics of the language somewhat the last weeks by not coercing values anymore. I am not sure about that step, but I thought it'll probably work out better if I map types to C types directly since I want to remain very close to C.

Which I am still not that sure of:
For: Type safety. One doesn't coerce that often (unless you link to C....)
Against: Loss in speed in some cases due to explicit conversions, I work with 64 bit word sizes anyway. The need for different sizes in C is not hogging the memory and speed, both I don't care about. Difficult to optimize the conversions away since everything I do is done through FFI anyway.)

XText

I like Eclipse, I guess I might use Xtext. Looks like it has all I need. Problem is, the parser I have is not fully LL, but I think I can make it LL by factoring out some stuff. (I cheated for some instances like 'pattern = expression; expression'.)

Monday, June 28, 2010

Log 062810

I write too much bullshit. Anyway, had a funny bug, somehow, somewhere, sometime, some integer comparison gave the wrong result. For some reason, I thought the ints in the Hi language were 64 bits, but they are not - all types follow the same convention as C, so char is a system char size, int is a system in size, float is a .... (I don't want surprises there, it's supposed to be a glue language.) It's just that internally the compiler uses ints of pointer size, it needs to, for the simple memory model I have. So, changed some routines to 64 bits integer, thought "Whoa, ho there!", changed it back. Bug gone.

Which leaves getting some of the dynamic libraries I link against right, for that I'll just follow the Posix C standard.

How to Implement: Var?

I gave it some more thought. If I ever want Hi to become anything more than a marginal language, I need state... So, for later: How to implement var? And at what level...? (And what to do with multicore, I explicitly want to avoid the need for locking....

Still, monadic programming in some sense is nothing more than programming w.r.t. to an encapsulated environment. There should be a manner to let that environment be mapped to a namespace. For example, each function when invoked keeps around a (fresh?) copy of the variables within that namespace?

Namespace bound variables seem too cumbersome to me, so all variables will just have the same life-time as a heap. Instantiate a new heap, and all variables are set to their initial value.

Integer Code

Hi code is compiled to lambda expression, lambda expressions are compiled to combinators, combinators are compiled to an abstract assembly which only knows how to do one thing: manipulate, inspect and adjust, arrays of integers of pointer size width. Arrays of integers are primitives of the assembly.

In order to do end up with only integer (bit) arrays, all primitives are compiled to them, which, in all respects, works great, and is just a cool thing to do.

In the array language, it is determined which arrays are constants, and which arrays are dynamic, and in one pass the code is generated which explicitly allocates dynamic arrays. (The latter is an easy check, if an array operand is somewhere filled with a register, or is a record array holding a null pointer, it is dynamic.)

The assembly assumes an infinite amount of write-once registers, which means I don't translate to loops, as of yet, and except for dynamically allocated arrays, all registers hold constant values.

I assume C can optimize a lot out of that, but C has some problems handling the static data ('near initializer isn't a constant,' solved for now), and I am not sure C understands all invariants I am trying to communicate to it.

I need my own optimizer.

Code Sizes and Performance

I am looking at my generated code, and yes, it produces a lot. I'll stick with it, for now, but I can but help wonder about the bloat of it. Below, a gross oversimplification.

Say you have n number of constructs, which binary encoded, take N bits. Say your program consists out of m of these constructs. Say for each of these constructs you need E bits to evaluate them. Then the difference in size between interpreted and compiled code is the decision between m*N+n*E (the binary size of m constructs + the size of the interpreter) or m*N*E (the binary size of compiling each construct). This balance is skewed in the sense that N*E can be made small, so I should concentrate on that I think.

It affects speed too. Given caches and all, if the size of the interpreted code is sufficiently smaller than the size of the compiled code, the interpreted code just might run faster.

Second, a change I'll implement anyway. I generate a lot of constants, my assumption is that the C compiler is smart enough to collapse a large number of the same constant to one unique constant. I think I was wrong, the const tag in C denotes a read-only value, not a constant value, so I guess -a minor adoption,- at some point I'ld need to factor constants out myself.

Last thing is locality. If a group of functions define a module, than that group of functions should be placed close together in memory, such that the program doesn't jump 'all over the place.' I think I am good in that respect, the namespaces help there too, but I might look at select statements which are general, and used a lot.