Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: A simple garbage collector for C (github.com/mkirchner)
192 points by mkirchner on Dec 20, 2019 | hide | past | favorite | 60 comments



Just out of interest for people not familiar with C, latest goings on there etc, what's the usecase where you'd want a garbage collector in C?

My (very surface level) understanding was always the trade off for the increased manual effort of using C - manual memory management being one example - was that you could tailor your solution exactly to your usecase for increased performance / lower resource use.

If you're going for a garbage collector, why not also benefit from some of the increased language power/features of a higher level language?


Typically, new garbage collected languages use the Boehm garbage collector (GC for C) in early versions, and then implement their own once they have time to fully optimize their runtime.

This is what Mono and Golang did. (They both used Boehm until they had time and resources to implement their own runtime-optimized GC.) I suspect Java did too, but I'm not sure.

In this case: "The original motivation for gc is my desire to write my own LISP in C, entirely from scratch - and that required garbage collection."

What basically happens is that gc (and Boehm) are "conservative garbage collectors." They treat all values in a data structure as a potential pointer, because they don't know the contents of the data structure. It's a good "quick and dirty" way to have garbage collection if you can accept the risk that some of your memory will remain uncollected if some of your data happens to have the same value as one of your pointers. In practice, it's a good tradeoff.

(Other tradeoffs are that you can't have things like real-time garbage collection, generational garbage collection, or compacting garbage collection.)

> If you're going for a garbage collector, why not also benefit from some of the increased language power/features of a higher level language?

Ironically, one of Boehm's use cases is looking for memory leaks. You basically #ifdef Boehm into a test build, and if a GC finds garbage, you know that you didn't free something correctly.


> Ironically, one of Boehm's use cases is looking for memory leaks. You basically #ifdef Boehm into a test build, and if a GC finds garbage, you know that you didn't free something correctly.

So a "garbage checker"? :) This is interesting, I never really thought about using it as a checking tool.


Oddly enough, Unity a very popular game development platform ended up sticking with an early version of Mono and Boehm for a very long time (years) and I'm not even sure that it's fully transitioned to something newer yet. (anybody know?)


Think about it the other way around — Java, Python, JavaScript all have garbage collectors. What language are those GCs written in? How does the runtime keep track of the objects in those languages, and how is the lifecycle for that tracking managed?

"Writing a GC in C" (or equivalent language) is not so much a strange thing as it is an inevitability.


Some gcs are indeed written in C, often due to convenience of tooling. But Java's zgc is written in Java. Pypy's gc is written in Python. Go's gc is written in Go.


Writing a GC for C is pretty strange.


It’s really not. If you’re implementing a language runtime for language X, and the runtime is written in C, then writing the X GC actually means “writing a GC in C, for the subset of C allocations that represent X allocations.”

E.g. Python objects are represented as PyObject values in CPython. Saying that Python is garbage collected is equivalent to saying PyObjects are garbage collected. Sure enough, PyObject contains the ref count necessary for its GC.


I have to second GP, writing a GC for C is strange. Not because it is written in C, but for C. State of the art efficient and low latency GC pose requirements on the memory layout of allocated objects and often on the code generated to access those objects, i.e. memory fences of different kind. Some GCs like GHC's or some JVM GCs have special features integrated into the programming language semantics. Since, the semantics of C are basically fixed, the GC misses out on potential improvements and is only ever applicable to a strict subset of C programs.


There's a subtlety here that I think you're missing.

The Hotspot JVM is a C++ program, CPython is a C program that implements the Python language. Both CPython and Hotspot mostly manage their respective memory in the usual styles of their host languages, but they still need some sort of host language representation for their target language objects.

AIUI, this is less obvious with the JVM (where the runtime manages to avoid a lot of host language allocations), but CPython has an explicit PyObject type, and every target language allocation corresponds to a host level allocation in some way. Because the lifecycle of these allocations in C is inextricably linked to the Python-level objects' lifecycle, you don't explicitly allocate/deallocate those manually on an individual level, you let the GC (Refcount in this case) handle that instead. It just turns out that "letting the GC do it" is really just "letting some other logic in my program do it". This means that implementing the Python GC corresponds exactly to implementing GC for PyObject objects at the C level.


In the world of C nothing is strange. (Which is part of why it is future-proof.)


The usecase for this project (according to the README) is:

> The original motivation for gc is my desire to write my own LISP in C, entirely from scratch - and that required garbage collection.

Another reason would be in platforms where you cannot run the language of your choice (because lack of implementations of JVM / Python / whatever), although those would maybe not allow malloc either.


One thing that wasn't mentioned so far is that you may want to use garbage collection only for a portion of your program. You can mix manual malloc/free memory management (plus RAII-style automatic lifetimes) together with GC'd allocation in some parts of your program where you think the tradeoffs make sense.


I could imagine something like this being applicable towards large legacy codebases with a history of memory leaks and segfaults. With legacy codebases you're often only able to tread water and won't be in a position to track everything to it's root cause.


Adding a GC to such a code base is a terribly complex endeavour and would only make things even worse though.


You've answered your own question. For some part of the system you do one type of memory management, for other part you use the GC. If you use a higher level language for everything then writing the performant part will be tricky.


As the author mentioned in the README, it's going to be used for the LISP interpreter he/she is writing from scratch.


Is there a simple GC like this one that uses handles instead of C stack scanning to keep track of local references?


The Boehm GC seems to do both internally.


I just wish C had something like defer in go, That would cover most cases.


If you're willing to use macros, this can do the trick. Return and break will preempt the expression, but continue should work fine.

  #define DEFER(EXPR) for(int _tmp=1; _tmp; _tmp=0,(EXPR))
Example:

  char * data = malloc(32);
  DEFER(free(data))
  {
          // do stuff
  }


Perhaps I'm misunderstanding this, but this doesn't solve the problem at all -- returning within the block statement wouldn't call the deferred expression, which is entirely the point of a defer statement, no? If control is guaranteed to reach the end of the block statement, of course; but requiring such a constraint would make this defer very handicapped. You have tons of function exit points and you most likely want to free memory at every single one.


Yes, the only way I could imagine implementing a "proper" defer in C would be to use (like for most insane C hacks) setjmp/longjmp. I'm fairly sure it's explicitly forbidden by the Geneva Conventions though.

Alternatively you might be able to use nested functions to guard against a stray return but that's not standard.


I guess you could do something like (couldn’t get Xcode’s clang to compile this, so it probably is buggy)

  static unsigned int __deferDepth = 0;

  #define RETURN return
  #define return ((__deferDepth == 0) ? RETURN : break do_return)

  #define DEFER(EXPR) \
    __label__ do_return; \
    for(int _tmp = 1; _tmp; _tmp = (do { \
      __deferDepth += 1; \
      EXPR; \
      __deferDepth -= 1; \
      break; \
      do_return: \
        RETURN; \
    }while(0);))
Nested DEFERs could get hairy, though (do you need a stack of __returnNotBreak values?), and there probably are issues if EXPR contains some nested loops with break or return.

Regardless, this isn’t C. Statement expressions (and __label_) are a gcc* extension, and if you’re willing to go there, __attribute__((cleanup)) seems the wiser route.


How would you implement defer with setjmp/longjmp? I can’t think of anything off the top of my head.


You'd probably have to #define "return" to call longjmp instead. Although I guess at this point you might be better off just have return call the cleanup code. You'd have to be careful to handle the nesting correctly though, which might be slightly easier with setjmp contexts.


Regarding having tons of exit points, I find it actually adds no more cognitive load to use only a single return per function, compared to having multiple returns and keeping resource-freeing up-to-date for each.

Most of the challenge in writing code with a single return seems to come from the fact that I'm not used to doing it. Of course, sometimes it leads to heavily nested control flow, although I'm not sure yet whether or not I mind that. It does certainly simplify reasoning about control flow though, especially in large functions.

That all being said, I'd still rather just have defer.


What you say makes sense but I'd advise beginners reading this comment not to overdo it. I've seen new coders "cargo culting" this one-return-per-function rule and leading to code that was, in my opinion, unnecessarily complicated.

For instance parameter validation is one instance where I think an early return is very much warranted. There you typically have no cleanup to do since it's effectively the prelude of the function.

Regarding nested error handling, I prefer the "goto fail" pattern. Some people are opposed to the use of goto as a matter of principle but I think in this case it's fine. It's effectively the poor man's RAII when you don't have destructors.


Not unreasonable for functions to have guard returns, a 'happy' return. And a goto fail return. That's decent pattern when you have a function that handles modifying state.


> For instance parameter validation is one instance where I think an early return is very much warranted. There you typically have no cleanup to do since it's effectively the prelude of the function.

That's what the assert statement is for; checking pre- and postconditions.


Oh, that's pure evil/genius.


That is really clever, I don't know why I never thought about the for loop being a way to run an expression after a scope ends.


The primary use-case of defer (cleaning up resources) can be done with __attribute__((__cleanup__(<destructor>))). This effectively allows you to do RAII in C, though it does require some careful thought when copying variables (you need to "move" them a-la Rust).

But the really classic way of doing "defer" in C is to do what Linux does:

    int foo(void)
    {
        char *ptr = malloc(13);
        int error = 0;

        error = bar();
        if (error < 0)
           goto out_free;

        // do something else

    out_free:
        free(ptr);
        return error;
    }
Once you get used to it, it feels very natural to write and understand.


Defer can be emulated with the block extensions of Clang and GCC [0], though I'm not sure how much I'd like to see something like that in a codebase.

[0] https://web.archive.org/web/20180426195701/http://fdiv.net/2...


There's also __attribute__ ((__cleanup__(<destructor>))) which can be used to implement RAII in C.


That's exactly what this uses. It just hides it inside a syntax that is more similar to what Go uses.


Ah, apologies, the link wasn't loading at the moment and I assumed it was something else.

Regarding its use in the wild, I've encountered it exactly once, in the lastpass-cli (available on github). I personally wouldn't consider using it for one of my projects because it's too nonstandard, however I suppose that for a security-sensitive application it might reduce the likelihood of messing error handling and leaking things all over the place. That being said using non-standard features might also make it more likely for a contributor to misunderstand what the code does precisely and introduce a problem.


LXC uses it as well, and they have additional macros to "move" pointers and file descriptors to make sure you don't double-free or similar things.


That link seems to be broken? Nothing opens when I click it.


It's a short article, and worth reading if you can work out why the Wayback Machine isn't working for you.

The magic is basically:

    static inline void defer_cleanup(void (^*b)(void)) { (*b)(); }
    #define defer_merge(a,b) a##b
    #define defer_varname(a) defer_merge(defer_scopevar_, a)
    #define defer __attribute__((cleanup(defer_cleanup))) void (^defer_varname(__COUNTER__))(void) =
Which let's you do:

    FILE *a = fopen ("a.txt", "r");
 
      if (!a)
        return EXIT_FAILURE;
      defer
      {
        fclose (a);
      };
It uses cleanup and blocks, both of which are extensions. There may also be some strangeness with the way blocks hold memory. (Block references become const copies).


Works fine for me. It's using __attribute__((cleanup)) to call a function that calls the block you give it.



The original link pops up with a HTTP basic authentication request for me, hence the use of archive.


This seems like a great place to plug C++ - Destructors are what you want. While I'm personally a fan of C++-as-a-whole, there's nothing stopping you using C-with-classes.

You can "emulate" the defer statement with a Scope Guard[0], or if you're willing to use a macro, you can use the __COUNTER__ macro as an "anonymous" variable.

[0] https://github.com/ricab/scope_guard


I know that's not directly what you're asking for, but I found the other day that "C++ Core Guidelines" have a helper library "GSL_util" that provide something similar to Go's defer.

The implementation is very simple, it's just using a class for its destructor:

template <class F>

class final_act

{

public:

    explicit final_act(F f) noexcept : f_(std::move(f)), invoke_(true) {}


    final_act(final_act&& other) noexcept : f_(std::move(other.f_)), invoke_(other.invoke_)
    {
        other.invoke_ = false;
    }


    final_act(const final_act&) = delete;
    final_act& operator=(const final_act&) = delete;


    ~final_act() noexcept
    {
        if (invoke_) f_();
    }

private:

    F f_;
    bool invoke_;
};

Source: https://github.com/microsoft/GSL/blob/ebe7ebfd855a95eb937831...


Of course, in C++ the need for such constructs is reduced due to RAII and scoping running destructors automatically. This is useful when you need to interact with C libraries that you don’t want to wrap in C++ (and when I was doing something like this in the past, I rolled my own as well).


> This is useful when you need to interact with C libraries that you don’t want to wrap in C++

Yep! I really like the defer pattern for those situations :)


If you're willing to use GCC extensions (available also on clang, so should cover most operating systems, even Windows with clang-cl), there's __attribute__((__cleanup__(fn))), as used for instance by systemd: https://gcc.gnu.org/onlinedocs/gcc-9.2.0/gcc/Common-Variable...


There’s boost scope exit but you have to use c++


Check out Zig if you’re looking for a C replacement with a go style defer. It also interops about as well with C as C does.


I thought that depends on llvm though? Not to mention it doesn’t allow horizontal tabs which I dislike pretty intensely, different programming styles and environments work with different tab stops and it makes a lot of sense (IMO) to represent indentation with a single character.


Or D, which integrates with native C libs and has the concept of 'exit scope' for deferred code.


How about efficiency? Looks like the collector has to scan through all allocated memory and the whole stack to check whether there could be a pointer. And if any data looks like such a pointer (even by coincidence) the corresponding memory cannot be collected. Anyway: isn't this just the same concept as the Boehm conservative GC? What's the difference/improvement?


Second paragraph in README:

> The focus of gc is to provide a conceptually clean implementation of a mark-and-sweep GC, without delving into the depths of architecture-specific optimization (see e.g. the Boehm GC for such an undertaking). It should be particularly suitable for learning purposes and is open for all kinds of optimization (PRs welcome!).


I was hoping for a link to the exit() manpage.


I am actually doing this exact same thing right now and will reference this for comparison once i am done with the first version of mine. I am doing stop-and-copy though.


Cool! Thanks for this - and the link to orangeduck’s work (which further points to Cello.)


Can it help with compiling Typescript to WASM? i.e. in https://github.com/AssemblyScript/assemblyscript


AssemblyScript already has tiny hybrid garbage collector (PureRC) which use deferred ARC and GC only for cyclic references: https://github.com/dcodeIO/purerc/blob/master/papers/Bacon03...


What would give you that impression?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: