I learned Common Lisp and "enough" CLOS in high school, was blown away by CLOS's power†, and have spent the entirety of my professional career wishing that I could have a job working on a complex system in CL that used all of CLOS's interesting features. This has not happened (I've been 80% in Ruby-land and 20% doing Clojure), but what's weird is that I haven't even had situations where I really yearned for AOP, multi-methods, the MOP, etc. I can't decide if the systems I've worked in have just been too boring (they certainly solved the business problem though) or if Ruby's object-orientation (and library ecosystem) is good enough that I just don't need the full power of CLOS. (Designing Clojure programs is a totally separate kettle of fish.) I'd be interested to hear from more seasoned Lispers about their experience with CLOS in production.
† I took an FP class during my first semester of college. The professor wrote the book himself, and had an extended sidebar comparing the design considerations of OO systems (easy to add another class; tedious to reopen n subclasses to add another method to each one) and FP systems (easy to add another function dispatching on type; tedious to reopen n functions and make them dispatch on a new type). I pointed out that CLOS handled both situations equally well, and in the final edition of the book he tossed in a footnote saying as much and thanking me.
Not sure how "seasoned" I am, as the biggest program I've written in CLOS that went live was under 2000 LOC, which doesn't sound like much. In other languages, though, it would have been much longer, due to two factors.
One is that CL seems to encourage writing everything in CL, so it's easy for library authors to expose everything (like their intermediate AST) to callers. That helps reduce redundancy. In other languages, what I often need is something that's already computed internally, but not exposed in any interface, so I have to build it myself again. A surprisingly common case is a Python or Ruby library where they wrapped a C library to get good performance, and I need data in a C struct that they didn't think anyone would need.
Two is that CL, more than any other language, makes not just for concise code, but for concise diffs. (This is what you mention in your footnote, but I think it's true in general, not just with CLOS methods.) Whenever I wanted to make a change which I could describe briefly in English, it always ended up being something which I could implement in a similarly brief changeset.
Other HLLs can usually, IME, be just as concise for a given solution, but the nearby solutions in the problem space might require a completely different design. This is where the full power of CL helped me. If you know exactly what you want from the start, then you can implement it easily enough in almost any language. If you change your mind halfway through, the weird features available in that "full power" (CLOS dispatch, macros, special vars, etc) makes it possible to avoid rearchitecting the whole thing. (Then you have the choice to gradually refactor to avoid the need for them, or just leave it.)
Despite all the new programming languages in the past ~30 years, I haven't seen any which made diff size a design priority. I think even Clojure isn't quite as good at this, though I'm still a newbie in that language.
Which database? I'm interested to know what is used in commercial endeavors, since there are a lot of toy object stores littering the software graveyards.
>> I pointed out that CLOS handled both situations equally well, and in the final edition of the book he tossed in a footnote saying as much and thanking me. <<
Clojure's multimethods and protocols also solve both.
I highly recommend playing with CLOS (or one of the Scheme flavored knockoffs like GOOPS in Guile or Swindle in Racket) for anyone who hasn't but works in OO languages (probably most working programmers).
You probably spend all day dealing with OO ideas like method dispatch and inheritance, but to see them laid out explicitly in CLOS, as distinct from the underlying Lisp language, really helps you to internalize them and how arbitrary and malleable they are, as well as how astoundingly general they are when interpreted as broadly as CLOS does (the object systems in workaday languages cut off a lot of the avenues open to systems defined in terms of CLOS).
You can get a similar "aha" moment by looking at how Perl does OO if you already know Perl, or reading the implementations of one of its dozens of different user-implemented object systems. Perl has some language-level support for OO but it is very minimal so these object systems have to do most of the work that a Common Lisp object system would.
I think the main benefit is in seeing OO implemented in a language with no built-in support for it, rather than a language built around the concept of classes and inheritance or some version of it.
Another revelation with CLOS is methods are not namespaced to classes. Before I used CLOS it was not obvious that methods were serving two distinct purposes (specialization and namespaces) within other OO languages.
Interestingly enough, most OO systems for lisp older that CLOS were much more smalltalk like in this sense, so there were several people that found the non-namespaced approach to methods to be superior.
I'm honestly not yet convinced they are superior, but they are clearly more generic, in that it is trivial to implement single-dispatch message-passing style methods in CLOS, but much harder to do the reverse (several patterns in the GoF book exist just to emulate multiple dispatch).
But it's more difficult to redirect all/most messages to a class/instance to another class/instance. Especially if there is no message passing going on ...
Seconded. Also, learning about the MOP (Meta-Object Protocol) is highly recommended to broaden your horizons.
That said, I used to be fascinated with CLOS, and yet these days I write in Clojure and have no more than a few multimethods in my entire code base. OO has many drawbacks and I find I can write better (simpler, more understandable and debuggable) code without it.
What are some, according to you? I mean, I have read a bit about why immutability and functional programming is becoming popular, e.g. it makes concurrent programming and use of multiple cores easier (only a general idea, may be wrong). But what are your specific reasons for saying OO has many drawbacks?
My reasons are very pragmatic: I found that OO doesn't bring me any benefits, while imposing a significant cost.
This realization came as I switched to Clojure: it found that I'd rather use the generic data structures with the excellent functions that come with the standard library. This gave me much better reusability than inheritance. Enforcing contracts can be done in a multitude of ways, and I no longer liked the idea that the only way to enforce data integrity is by keeping the data private and accessible only using "accessors".
Debugging OO code can be a major nightmare. Reading and understanding it is also difficult. How do you know which method implementation will get called at a certain point in your program? You really have no idea until you run it and dynamic dispatch happens. This is also a problem with Clojure multimethods, although it is somewhat mitigated by Clojure's explicit dispatch fn.
As for data modeling, I found that inheritance isn't such a useful tool in practice. It looks great in theoretical problems, but in practice the perfect "is a kind-of" relationship is rare.
The final nail in the coffin was Clojure's STM. Immutable data structures bring so many benefits that after using them for a while it is unthinkable to me right now to go back to mutating data in-place.
>As for data modeling, I found that inheritance isn't such a useful tool in practice.
True. It can lock you into a particular style and hierarchy. Plus, inheritance just to inherit the methods of the superclass seems like overkill or sometimes might be a wrong approach. Personally I think composition can often be more flexible than inheritance.
Seconding the study of either/both CLOS or/and Perl 5. I didn't get what classes were all about in, say, Java or Ruby until I learned how to implement them as part of learning object-oriented Perl. Meanwhile, CLOS is a nice preview of some of the neat things possible with Perl 6's object model.
But when you say “Lua” everyone complains that there is no class system, despite the fact that setmetatable(x, {__index=getfenv(1)}) is equivalent to bless().
I wondered if the author would mention method combinators, which he did. They are super powerful, very useful...and a marvelous opportunity for spaghetti code.
Actually the CLOS combinators show learning from earlier mistakes and allow only :before, :after and :around, the most useful and the most comprehensible combinators.
Lisp Machine Lisp (including the "zetalisp" variant dreamt up by Symbolics' marketing department) had an enormous panoply of method combinators including full control structures: IIRC I had cause to resort at one time or another to :or, :and and :progn combinators (the latter being somewhat like :around, I suppose, but not the same). They made the code extensible, compact, and completely unpredicatable (if you didn't understand the runtime method hierarchy precisely which combined methods would be run in an :or combination?
Nevertheless I sometimes miss them to this day, even in my C++17 codebase!
Method Combinators are still there, slightly improved from Zetalisp's Flavors and New Flavors.
The CLOS Standard Method Combination has primary, :before, :after and :around methods.
There are in CLOS also various 'simple' method combinations like AND (runs all applicable methods until one is false), OR, + (sums up the results of all applicable methods), PROGN (runs all applicable methods and returns the result of the last one). These seem to be rare in CLOS code.
> They made the code extensible, compact, and completely unpredicatable ...
That's kind of the point that one does not care which methods actually run and when - the 'system' takes care of that. Similar to, say, when you call REDUCE on a list - one does not know which elements are in the list at runtime. Or when you call a generic function on a bunch of instances - you know only at runtime which method runs - because of dynamic dispatch - the method combination feature adds a layer of complication to it. Like on a Lisp Machine where you can add a :before method to a window function and it's immediately active, because the method combination is recomputed. Or when one adds a mixin to a window class and the width of a window then has a different result, because the + generic function computes a new value, because a new method is available via the + method combination. You can scare a lot of people with this, though.
One can program full new method combinations in CLOS, like in Flavors. This might even be useful. ;-) An example is the integration of 'Design by Contract' methodology into CLOS. One can add a DBC method combination and then work with preconditions, postconditions and invariants somehow integrated into Common Lisp's condition system.
> That's kind of the point that one does not care which methods actually run and when
What is the right method for debugging? I've been trying to debug one particular issue in CL code (McCLIM + Climacs) and I've hit the wall in every direction that I tried to see which method was responsible with the issue I've seen.
A plethora of around, before and after in both composition and inheritance that I've lost my way. I've tried various depths of reading and runtime debugging with no significant success. It left me with the image that I should have the whole program in my head to be able to understand it.
It's usually the case that the solution is where you're not looking therefore my question is where (and in what way) should I look at the code to understand what it is supposed to do?
The debugging tools need to know about that. Usually you would look into the stack trace and see which methods has been called and editing the method object brings us to the source code. Best when things are NOT optimized for low DEBUG and high SPEED.
When I was working more with those things I wrote some browsers for browsing methods in CLIM and for MCL. The Lisp Machine should already have some tools, IIRC. LispWorks also has a generic function browser. One enters a generic function and argument types -> LispWorks shows which methods are used for those.
Indeed, having good runtime debug tools is absolutely necessary for debugging dynamic objects [1].
I've come to this conclusion during this particular experience and I've confirmed it time and again. I've recently seen an older video of Gregor Kiczales on Aspect Oriented Programming and if I did not read in too much he emphasizes the same issue with proper debugging tools.
Thanks for the tip; it might be what I needed for me to download LispWorks and try again.
[1] Some kernel developers just don't get this even after you show them a call using a function pointer. Read the code they say. Right...
Been working on getting a CLIM codebase (that is older than ANSI CL standard) running under modern open source implementations (SBCL, CCL). This too was my primary problem. CLIM spec was written with CLOS in mind, and implementations tend to absolutely abuse CLOS and MOP. I spent many long hours untangling all the mixins introducing :around methods. Good runtime debugging is absolutely crucial for this kind of code - something which open source CL implementations do not really have. Even prettified by Emacs, the inspector can get you only so far. Hell, this is the only time in my life where SBCL itself crashed completely when I did simple "jump to symbol definition" in Emacs...
Typically your debugger will be a listener, so it's easy for you to write explanatory functions and in-the-moment expressions for unpacking what's going on. The latter is what I miss most whenever I end up in gdb.
In Flavors/Genera one has Zmacs commands: 'List Methods', 'Edit Methods' and 'List Combined Methods'. Similar functionality exists for CLOS - I think there is a 'Show CLOS Generic Function' listener command, or similar.
One of the few eureka moments I’ve had was while reading The Art of the Metaobject Protocol [1]. Some famous computer scientists regard this as the best CS book coming out in the 90’s.
† I took an FP class during my first semester of college. The professor wrote the book himself, and had an extended sidebar comparing the design considerations of OO systems (easy to add another class; tedious to reopen n subclasses to add another method to each one) and FP systems (easy to add another function dispatching on type; tedious to reopen n functions and make them dispatch on a new type). I pointed out that CLOS handled both situations equally well, and in the final edition of the book he tossed in a footnote saying as much and thanking me.