Log 021509

On the rare occasion I am sharp enough to debug: strange errors in the translation of terms, unfortunately they are logical errors. Sigh.

Travelling the C Stack

I use a Cheney style GC which is triggered, and may start a collection, each time a term is rewritten. The collector starts of with one root. This gives a minimal overhead, since in other GCs collections are tried each allocation step and this scheme does without that overhead giving an allocation routine of just a few machine instructions.

A problem with this scheme is that there may not be enough room in the heap for allocating a new term during a rewrite. Normally, less than a hundred bytes will be allocated, so keeping a spill-over region will be quite safe, except for FFI which may introduce a term of unknown size for example while reading files.

The are several ways of handling this: a) adjusting to a normal GC where an allocation may be interrupted with a collection, which also would imply that I would need to travel the C stack to collect other roots, b) explicitly introduce an allocation routine in the language which will grow the heap such that enough room is available for building large terms, c) allocate large terms in another region, d) never allocate large terms, instead pass pointers from C and work on opaque structures which must be deallocated explicitly.

Option d) it is.

Finalizers Considered Questionable

If you glue C to a GC-ed language, you end up with the question what to do with structures passed from C. One way of handling them would be to add finalizers to objects in the GC-ed language. I am figuring out what others did. A post from Python-Dev.

Here's my $0.02.

I agree with the sentiments that use of finalizers 
should be discouraged.  They are extremely helpful
in cases like tempfile.TemporaryFileWrapper, so I
think that they should be supported. I do think that
the language should not promise a high level of service.

Some observations:

  - I spent a little bit of time on the ANSI 
    Smalltalk committee, where I naively advocated
    adding finalizers to the language. I was resoundingly
    told no. :)

  - Most of the Python objects I deal with these days
    are persistent. Their lifetimes are a lot more complicated
    that most Python objects.  They get created once, but they
    get loaded into and out of memory many times.  In fact, they
    can be in memory many times simultaneously. :) A couple
    of years ago I realized that it only made sense to call
    __init__ when an object was first created, not when it is
    subsequently (re)loaded into memory.  This led to a 
    change in Python pickling semantics and the deprecation
    of the loathsome __getinitargs__ protocol. :)

    For me, a similar case can be made against use of __del__
    for persistent objects.  For persistent objects, a __del__
    method should only be used for cleaning up the most volatile
    of resources. A persistent object __del__ should not perform
    any semantically meaningful operations because __del__ has 
    no semantic meaning.

  - Zope has a few uses of __del__. These are all for
    non-persistent objects. Interesting, in grepping for __del__,
    I found a lot of cases where __del__ was used and then commented 
    out.  Finalizers seem to be the sort of thing that people
    want initially and then get over.

I'm inclined to essentially keep the current rules and
simply not promise that __del__ will be able to run correctly.
That is, Python should call __del__ and ignore exceptions raised
(or provide some *optional* logging or other debugging facility).
There is no reason for __del__ to fail unless it depends on
cyclicly-related objects, which should be viewed as a design
mistake.

OTOH, __del__ should never fail because module globals go away. 
IMO, the current circular references involving module globals are
unnecessary, but that's a different topic. ;)

Jim

I think I agree with this: ... GC is about memory management, not resource management .... The gist: since you cannot, in general, rely on finalizers being called when needed, or in the right order, better to avoid them.

Me Wordle Too

From wordle.net.

Compiling out Select

Yet again a change scheduled to the compiler. If I introduce type specialization I can get rid of select statements and, again, simplify the runtime.

Log 020409

Fixing void arguments and results (c).

Log 020309

Still slow. Stack-exception (sigh) (text -> lists of text) (ml).