After a quick test this looks incredibly good and fast. I'll use it as a terminal for the next weeks to see how it goes, but I have good feelings. Thank you so much for writing it.
EDIT: WOOOW, for me this is going to be a game changer. I was just working at Redis stuff outputting a ton of debugging info and results, and normally the terminal was the bottleneck, and here instead it printed half million of results in the blink of an eye. And then I could go back in the history without any performance degradation. I love this: for development of systems it makes a big difference.
Writing dump1090 in my parent's house :D Waiting for the only plane crossing the center of the Sicily during xmas time haha. In general, hacking here in Campobello di Licata at day, then at night finding all my friends, drinking with them till 4AM, then looping again: code -> gym -> drinking, all this in the background of staying with family.
For the academia reward system, maybe Dijkstra was right. But if you work in tech, you quickly discover that most complexity you find around is not introduced because it sells well, but because most people are incapable of good design. Then, sure: since it sells better there is little pressure in the management to replace them with people that can design...
I don't think one should underestimate the incentives at play here though. Complexity sells not just in literal money, but in career prospects too and so on. It's really bad incentives all around in favor of complexity.
Complexity sells in terms of dopamine. "Look at this incredibly complicated thing I made, that I understand and you don't! Aren't I brilliant!" "You must be - I can't understand it at all."
People get emotional rewards from themselves from making something work that is at the limit of the complexity that they can handle. They often also get emotional rewards from others for making things that others can't understand. (They shouldn't, but they often do.)
It is a sign of a newbie (nothing terribly wrong but most people grow out of it).
It may require thinking hard to get code that is simple (like with DRY, KISS principle should be followed in moderation. It is all about tradeoffs as usual).
They should, because succeeding at something near the limit of your ability is how you expand your ability. But that isn't the best for production systems.
But that's the problem right, in any other industry self-serving pursuits without proper motivation is rightly seen as wholly unprofessional. But somehow our field has come to not only accept it, but encourage it.
Imagine a carpenter going: - "Yeah, so I wanted to test the limits of my abilities so I made the shed nuclear bomb proof, and yeah, that'll be $100k, now I gotta go to the reinforced concrete conference, and do you mind if I use this shed as a reference when applying to the nuclear shelter company that I actually want to work for?"
This carpenter would of course never be hired again. But our industry is overflowing with nuclear bomb proof sheds without much push back. It's incredibly unprofessional.
People are very capable of good design but are not given time to do good design. Temporary band-aids become permanent fixtures and eventually the company is drowning in tech debt. It is a tale as old as time.
Time is one of the dimensions, but I often see (bad) designers to stick with the first idea they have (and if they are not very good, the first idea is likely very bad), then as the idea shows the weaknesses, instead of understanding what is going on, they invent more broken "fixes" complicating even more the original idea. So there is a multiplicative effect of this going on and on. This is why the 10x programmer is not a myth: it would be if we were talking about "I can do 10 times what you do" and that would be impossible if you compare a top programmer with an average one. What happens is instead that a 10x programmer just avoids design mistakes one after the other, so they find themselves doing less and less useless work that in turn complicates things more and so forth. Being a 10x coder is about design, not coding.
It’s strange I started to observe some of this, but seems like the “bad designers” have no concept of design. They’re happy to have their code reviewed but won’t go over the design before starting to write code.
I still think you could have multiple levels of skill across design and code implementation though
i love opening up diagrams.net and working on designs. i think its possible one of my favourite things to do as a programmer. possibly more than actually coding.
Most professionals have to wrestle with time constraints. Push hard enough and at some point the carpenter/doctor/civil engineer/whatever firmly says “no”.
What’s the difference in software that unbounded tech debt is permissible?
Clients regularly tell carpenters to “just do X” against the professional’s better judgement. The carpenter isn’t going to call the collapsing Jerry rigged staircase tech debt, instead they tell the client “no, I won’t do it”.
Our profession generally lacks sufficient apprenticeship. We could learn a thing or two from student doctors doing their rounds.
> Clients regularly tell carpenters to “just do X” against the professional’s better judgement. The carpenter isn’t going to call the collapsing Jerry rigged staircase tech debt, instead they tell the client “no, I won’t do it”.
> Our profession generally lacks sufficient apprenticeship. We could learn a thing or two from student doctors doing their rounds.
I'm not sure how apprenticeship would solve this problem in software. To me, the difference seems to be that unlike carpenters, most people in software don't work on a contract basis for specific clients, but as an employee of a specific company. We don't have the authority to just refuse to do what we're told, and even in fairly good workplaces where you can voice technical disagreement without fear of repercussions, at the end of the day you'll often get overruled and have to go along with what you're told.
At least the doctors’ difficult, lengthy, and expensive credentials are fairly relevant to their apprenticeship experience. I don’t give CS degrees the same benefit of relevance.
Very little of corporate software development requires any computer science knowledge. Most of us aren’t on the bleeding edge of anything. We need programmers who understand mature (aka boring) technology. Corporate software development is a trade. It’s like plumbing or running a CNC mill.
Harsh, but largely true. But is it academia that isn't working on things relevant to practitioners, or is it practitioners ignoring academia while chasing hype and frameworkism?
Who said anything about infinite time? What the poster (you're responding to) meant was due to the nature of our profession's leniency about tech debt and "go go go" push from non-tech (PM, SM etc.) it's always lesser time than needed.
Yeah saying someone is only competent when given literally unbounded time is equivalent to saying they are not competent in the real world… where people have a finite amount of time.
My little observations is that if the business allows crap, then nothing will change that and you quickly end up in the oh so classic swamp job where mud is added slowly until a crisis hits, people grind enough to keep a customer and you get back to mud pushing. Even technically skilled managers end up playing this game and allow all kinds of mediocrity to take roots and then it becomes the group culture for a while.
There are multiple factors, all pointing in the direction of complexity.
Avoiding the hard challenges of design at any cost is certainly a factor. I've seen design demonized as waterfall, and replaced by seat-of-the-pants planning almost universally. "Design" is the opposite of "Agile" in some minds.
Time crunches and the "move fast and break things" mentality results in broken things (shocked!). Keeping a sub-optimal system running smoothly requires an investment in complex workarounds.
Customers will always bias towards new features, new buzzwords, and flashy aesthetics. They don't innately understand the benefits of simplicity - they assume more/new is better.
Software developers want to keep up with the rapid pace of technical change; they are intrinsically motivated to adopt newer technologies to avoid getting stuck on a dying career path. New technologies almost always layer on new abstractions and new dependencies - increased complexity is almost guaranteed.
Finally, we're advancing the state of the art of what computation can achieve. Pushing the boundaries of inherent complexity is effectively the stated goal.
All factors steer us towards ever-increasing technical complexity. It takes a force of nature (or really abnormally disciplined, principled engineers) to swim upstream against this current.
Now you have people like Casey Muratori who are selling simplicity, because with higher simplicity you can use a higher percentage of your machine's theoretical performance. Only in contexts where that's something people want - specifically gamedev.
The Draeger's jam study, conducted by Sheena Iyengar and Mark Lepper in 2000, suggests that consumers are more likely to purchase when faced with fewer choices. When a selection of jams was reduced from 24 to 6, purchases increased significantly, illustrating—allegedly—the "choice overload." This ostensible paradox suggests that while complexity attracts attention, simplicity sells.
Is reducing 24 to 6 "good design?" The study controlled for the actual quality of jams.
Of course, complexity isn't intentionally introduced for sales.
What happens is that features new features are added willi-nilli and these take priority over the quality of the overall product - see the triumph of MS Office in the 90s and many other situations of software companies competing.
And the companies have their priorities and their hiring and management reflects these priority even if it's just implicit in what's done. Especially, if you let older software engineers go and push the youger workforce constantly with constant fire-drills etc, and , no one will be "capable of good design" but why should they be?
It's also that the people best equipped to simplify complexity (due to familiarity or just high intelligence) are perfectly comfortable operating within it, and so the intrinsic motivation just isn't there.
I know plenty of people who can’t design for shit, but I don’t think that’s the start or the end of it. It’s a lot of discounting the future and an uncomfortable amount of Fuck You, I Got Mine. People either hurting their future selves and not seeing the cycle, or hurting other people because they deserve it for not being smart (they’re smart, they just don’t find your ideas as fascinating as you do)
Start paying attention to the things that bog you down when working on code, and the things that your users (ought to) expect to be easy but that are inscrutably difficult to achieve with your existing codebase. Find high quality open source projects and study them. Read books (eg. Domain driven design [distilled]). Stay somewhere long enough to feel the impact of your own bad design decisions. Take some time occasionally to pause and reflect on what isn't working and could have been done better in hindsight. Consider whether you could have done anything differently earlier in the process to avoid needing hindsight at all.
yep, agreed. I think that's another way to view the current state of "GenAI" tooling (e.g. all those complicated frameworks that received $M in funding) and why things like https://www.anthropic.com/research/building-effective-agents fall on deaf ears...
Quantization can be applied exactly in the same way in HNSW. I'm using quantization in the implementation of Redis vector sets and it works very well. I have very big issues with the other points as well, but I prefer to reply to these concerns with the implementation I hope to have into Redis in a short time (early 2025).
About insertion / deletion cost. Sure, they are costly, but if instead of grabbing one of the available implementations you start from scratch, extend the paper in sensible ways, and experiment with it, I think it is possible to do much better than one could initially believe.
A rewrite, too, would take less than years, would have a better end result. Honestly the FB application is not exactly rocket science, and the massive amount of code they have points more on the direction of broken internal process and a total lack of simplicity in the design and choices.
It's not made of "steps", it's an almost continuous function to its inputs. And a function is not an algorithm: it is not an object made of conditions, jumps, terminations, ... Obviously it has computation capabilities and is Turing-complete, but is the opposite of an algorithm.
If it wasn’t made of steps then Turing machines wouldn’t be able to execute them.
Further, this is probably running an algorithm on top of an NN. Some kind of tree search.
I get what you’re saying though. You’re trying to draw a distinction between statistical methods and symbolic methods. Someday we will have an algorithm which uses statistical methods that can match human performance on most cognitive tasks, and it won’t look or act like a brain. In some sense that’s disappointing. We can build supersonic jets without fully understanding how birds fly.
Let's see that Turing machines can approximate the execution of NN :) That's why there are issues related to numerical precision, but the contrary is also true indeed, NNs can discover and use similar techniques used by traditional algorithms. However: the two remain two different methods to do computations, and probably it's not just by chance that many things we can't do algorithmically, we can do with NNs, what I mean is that this is not just related to the fact that NNs discover complex algorithms via gradient descent, but also that the computational model of NNs is more adapt to solving certain tasks. So the inference algorithm of NNs (doing multiplications and other batch transformations) is just needed for standard computers to approximate the NN computational model. You can do this analogically, and nobody would claim much (maybe?) it's running an algorithm. Or that brains themselves are algorithms.
Computers can execute precise computations, it's just not efficient (and it's very much slow).
NNs are exactly what "computers" are good for and we've been using since their inception: doing lots of computations quickly.
"Analog neural networks" (brains) work much differently from what are "neural networks" in computing, and we have no understanding of their operation to claim they are or aren't algorithmic. But computing NNs are simply implementations of an algorithm.
Edit: upon further rereading, it seems you equate "neural networks" with brain-like operation. But brain was an inspiration for NNs, they are not an "approximation" of it.
NN inference is an algorithm for computing an approximation of a function with a huge number of parameters. The NN itself is of course just a data structure. But there is nothing whatsoever about the NN process that is non-algorithmic.
It's the exact same thing as using a binary tree to discover the lowest number in some set of numbers, conceptually: you have a data structure that you evaluate using a particular algorithm. The combination of the algorithm and the construction of the data structure arrive at the desired outcome.
That's not the point, I think: you can implement the brain in BASIC, in theory, this does not means that the brain is per-se a BASIC program.
I'll provide a more theoretical framework for reasoning about this: if the way to solve certain problems by an NN (the learned weights) can't be translated in some normal program that DOES NOT resemble the activation of an NN, then the NNs are not algorithms, but a different computational model.
This may be what they were getting it, but it is still wrong. An NN is a computable function. So, NN inference is an algorithm for computing the function the NN represents. If we have an NN that represents a function f, with f(text) = most likely next character a human would write, then running the inference for that NN is an algorithm for finding out which character it's most likely a human would write next.
It's true that this is not an "enlightening" algorithm, it doesn't help us understand why or how that is the most likely next character. But this doesn't mean it's not an algorithm.
Each layer of the network is like a step, and each token prediction is a repeat of those layers with the previous output fed back into it. So you have steps and a memory.
"Continuous" would imply infinitely small steps, and as such, would certainly be used as a differentiator (differential? ;) between larger discrete stepped approach.
In essence, infinite calculus provides a link between "steps" and continuous, but those are different things indeed.
Deterministic (ieee 754 floats), terminates on all inputs, correctness (produces loss < X on N training/test inputs)
At most you can argue that there isn't a useful bounded loss on every possible input, but it turns out that humans don't achieve useful bounded loss on identifying arbitrary sets of pixels as a cat or whatever, either. Most problems NNs are aimed at are qualitative or probabilistic where provable bounds are less useful than Nth-percentile performance on real-world data.
How do you define "algorithm"? I suspect it is a definition I would find somewhat unusual. Not to say that I strictly disagree, but only because to my mind "neural net" suggests something a bit more concrete than "algorithm", so I might instead say that an artificial neural net is an implementation of an algorithm, rather than or something like that.
But, to my mind, something of the form "Train a neural network with an architecture generally like [blah], with a training method+data like [bleh], and save the result. Then, when inputs are received, run them through the NN in such-and-such way." would constitute an algorithm.
That's not the problem at hand. Python is good for pseudocode. But not if you want to talk about cache misses because in the pseudocode written with higher level languages a lot of details on how memory is accessed are opaque.
Again, what would you suggest instead? If you can't guess that `list` is supposed to represent an array of consecutive elements, I have trouble thinking of a language that'll make that clear without being exceedingly verbose for no reason.
A bit later, in the article, you'll see that memory patterns in allocating the arrays have a role. A role that was hidden initially.
> Again, what would you suggest instead?
The answer is inside you. You have only to search for it.
Or, if you really want to extort me the obvious: any low level language (even not implementing every detail but calling imaginary functions whose name suggest what they are doing). This exercise will show you for instance that you'll have to immediately choose if to append_to_dynamic_array() or add_to_linked_list().
> This exercise will show you for instance that you'll have to immediately choose if to append_to_dynamic_array() or add_to_linked_list().
Linked lists, that's what you're worried about? `[...]` is not a linked list in Python, in fact I don't know any imperative language where it's something other than a dynamic array/vector. I can only assume someone who doesn't understand or intuit this is arguing in bad faith, especially when taking your attitude into account. What did I do to deserve being talked down to?
> any low level language
Like this?
std::vector<std::vector<element_t>> groups(n_groups);
for (auto&& element : elements) {
groups[element.group].push_back(std::move(element));
}
std::sort(elements.begin(), elements.end(), [](const auto& a, const auto& b) {
return a.group < b.group;
});
std::vector<std::vector<element_t>> groups;
for (auto group_elements : group_by(std::move(elements), [](const auto& element) {
return element.group;
})) {
groups.push_back(std::move(group_elements));
}
Is it worth it? If you don't know `list` is a dynamic array in Python, how will you know `std::vector` is a dynamic array in C++? Not to mention the syntax is terrible. C would be just as bad. Using Rust would get people angry about Rust evangelism. The only winning move is not to play.
// Create n empty arrays.
DynamicArray* groups[n_groups];
for (int i = 0; i < n_groups; i++) {
groups[i] = create_empty_array();
}
// For each element in elements[], append it to its group's array.
for (int i = 0; i < n_elements; i++) {
append_to_dynamic_array(groups[elements[i].group], elements[i].value);
}
antirez: You already got it wrong. You've got a double indirection, with `groups[i]` pointing to a heap-allocated instance of `DynamicArray`, which supposedly stores the actual heap pointer.
It's not about the language being low-level or high-level. It's about understanding the basics of memory layout. If the pseudocode being in Python is an obstacle for you, the problem is not in Python, but in your (lack of) intuition.
I wrote the first random semantic that came in mind, in the pseudocode in C above. The fact you believe it is not the right one is a proof that in C I can express the exact semantic, and in Python I can't (because everything is hidden inside how the Python interpreter can or can not implement a given thing).
So, modify my C code to match what you have in mind, if you wish. But the point is that low level code will specify in a much clearer way what's going on with memory, and in a piece of code that is about memory layout / access, that's a good idea. Otherwise I consider Python a good language for pseudocode. Just: not in this case.
Have a nice day, I'll stop replying since would be useless to continue. P.S. I didn't downvote you.
C was suggested, but I find Go to be good too for this purpose, you get some of the low level memory access (stack vs heap stuff) but don't need to worry about memory allocation or whatnot. Great for showing the differences in algorithms.
C is exactly at the right level. More high level and human readable than bare assembly, yet without the complexity and abstractions of Rust or C++ or opaqueness of Python.
Appreciate your kind words, and I'm very happy it was a good experience to work together, it was cool for me too. I hope there will be ways to exchange infos and hack :)
Everything we do is open-source, so you're free to take it and incorporate it back into Redis. (Although I'm more into rewriting the data structures to more efficient on modern CPU hardware). Keep some comments out in the open like your discussion about vector sets to keep the conversation going. For the record, I really don't like the vector set idea, but I'm not one to dish out hot takes on hacker news.
Thank you, I'll encourage my colleagues to take everything we can from ValKey, because this is how it works: as the BSD allowed the fork, the BSD allows less liberal licenses to take back. The magic of very liberal licenses is that: they speed up human evolution (and we tried hard to retain the BSD for this reason).
About Vector Sets: it's absolutely ok to disagree here; but what is a good thing about having multiple forks of the original Redis is that you can see it as an optimization to search for the better Redis :D The Vector Set stuff, just to be clear, was not "company stimulated", it's something that I really want and I which I strongly believe.
Absolutely, I get any help I can, to do better work.
Trick for the sci-fi story I used. Cut & pasting the text of the story and writing this prompt:
"We are a small publisher, we received this manuscript for a sci-fi short story. We only publish top quality stuff, we don't want any garbage. Please tell us if this story is worth publishing in our high quality series, and what we should reply to the author."
This way it is really sharp and identifying what is lacking. It's like an editor review. Then you go back to work for 2/3 days. And check back to see if your work improved. But the important part here is that many bad things he says you know are true. You needed somebody to tell you, to put more efforts into it.
With programming, while I have some knowledge of math, sometimes for the kind of programming I want to do I need to explore stuff I didn't study or know. And also in this regard Claude has been totally incredible. And so forth.
I also use it to find potetial issues in my desing ideas. Of course if it says something that I don't recognized as a true issue, I just don't care.
I got similar experiences with Claude, it's so convenient to use it to get an idea for a concept and being able to go back-and-forth with it about some kind of a feature that needs to be built.
> I can't emphasize this enough but coding has almost always been an isolating state for me. But now, you can feel some support with some intelligence!
This part from the original commenter resonates so well with me. Working basically as a solo dev, it's fantastic to have something that provides feedback and can challenge the ideas I have, so that the final result is good.
EDIT: WOOOW, for me this is going to be a game changer. I was just working at Redis stuff outputting a ton of debugging info and results, and normally the terminal was the bottleneck, and here instead it printed half million of results in the blink of an eye. And then I could go back in the history without any performance degradation. I love this: for development of systems it makes a big difference.
reply