To any budding game developers, my advice would be to not rush into ECS and try to accomplish your goals with structs and loops first and understand the fundamentals of how a video game 'works' under the hood.
Your game will get done if you work on it, not necessarily if you pick ECS!
I've spent a few months working with ECS/DOTS with the Unity Game Engine, their implementation of this style of architecture end up being very complex in comparison to the old fashion GameObject workflow that made Unity popular in the first place.
Everything is MUCH more difficult to implement, I won't go into details but from my experience the time spent to achieve a similar result is about 5x-10x what it should be.
Weird APIs, poor documentation, limited collective experience, tons of restrictions and limits of the C# language are making the global experience quite nightmarish.
And I have had experience with ECS-like system in different engines in the past, and I am mostly experienced with low-level engine stuff, but the induced complexity there is something I did not expect.
> GameObject workflow that made Unity popular in the first place.
It's a bit much of a over-simplication that just the GameObject architecture Unity uses is what made it popular. For example, if there wasn't a UI editor for Unity and only code, you still think it would be as popular? I'm 99% sure it wouldn't be.
I tried getting into DOTS as well, as ECS is an interesting architecture overall, but Unity kind of dropped the ball on it and made it way more complicated than it has to be, from every side of having to use it.
On the other hand, you have Bevy, which is built with ECS as a core part of the engine, and it makes using ECS a breeze. After getting to know Bevy and ECS outside of DOTS better, I'm not sure I could back to how I wrote my games before. ECS just makes it a lot easier to structure bigger projects than the basic introduction game like flappy bird. The modularity is on a different level and it gives you a much better view of the overall architecture than anything else I've tried before.
Simply put, I'm not sure I'd write the same amount of games if it wasn't ECS, because it makes it fun to make games again, for me.
As a game dev that’s been using Unity ECS in production for over 2 years, it’s a very underwhelming implementation. Sure it’s fast if you design a game with Unitys intentions, but Unity has never made a game and what they made isn’t what game devs need.
This shocks me the most about how Unity operates. That they haven't figured out that dog-fooding their own main product is absolutely essential, especially since every single competitor of theirs does so very successfully as well.
How can you build something great for a specific group of people if you don't know what they are going through?
Whether ECS is complex or not depends a lot on which one you use. I've heard the same thing from a lot of gamedevs that use DOTS, but people in general seem to be pretty happy using Entitas (also a C# ECS that can be used with Unity), Bevy (which is based entirely on ECS) or for that matter Flecs.
Yeah fully agree, ECS is the microservices of gamedev these days.
I've worked on AAA games that used it, and shipped games as a solo dev without it. If you don't know why you need it, then chances are you don't need it. Just keep it in mind as one possible architecture pattern for when your code base starts to experience growing pains is my advice.
I don’t have any intention of writing a game, but my lay understanding of ECS is that it sort of treats your world as an in-memory columnar database for better cache locality: if you’re updating the positions of a collection of entities, you only need to loop over a collection of contiguous `position` structs—you can get a lot more of these in a cache line than you would if the entire entity was a struct containing a position field among many other fields which are irrelevant to the updatePosition operation. How far off am I?
I'm definitely not going to argue that you need ECS to do much of anything, but ECS has come long way in the past 4 years from a tool that lets you have lots of entities in a game, to a more generic system for entity management.
An ECS can also come with a bunch of tools for inspecting entities, monitoring performance or changing components in realtime (https://www.flecs.dev/explorer/). Definitely not something you need to complete a game, but very handy.
The opposite also applies: don't reinvent the wheel if you want to get something done.
Existing game engines like Unity/Unreal already have builtin solutions for entity management, but if you're (for whatever reason) using something that's more custom you can avoid having to reinvent entity management by using an ECS library. Similar to how you'd use libraries like SDL/bgfx/sokol for graphics.
This is true to a point if you need a general purpose entity management framework that will scale to whatever but a lot of games in the "make a game not an engine" vein don't need that complexity. Adding a dependency can save time but can also be its own rabbit hole.
One of those things where discussing theoretical stuff doesn't help because "it depends". :D
How games work is simple. It's a simulation run at some frequency and the simulation can be affected by player input. Sometimes there is a synchronization step with a server and that is more complicated.
ECS is a pattern that results in better performance and more maintainable code. There is no reason not to use it.
Better performance and more maintainable code are never the only concerns. For example, achieving a running prototype quickly with some ported/adapted non-ECS engine from a previous project might be more effective and efficient that starting from scratch with a better ECS architecture.
component systems are useful when you have no design doc, or a design doc written by new-hire monkeys. Or when management has no idea what the goals are.
It just allocates an object per monster, followed by initializing a long list of members that presumably just gets longer as more game systems get added.
The same code in ECS would look a lot simpler, and probably perform better/have better cache locality.
As a programmer not part of this world: what's the go-to alternative in this situation?
Does ECS not naturally arise from using OOP? Trying to think if you are creating instances of players/physics objects/items how that wouldn't turn into a kind of entity system.
No actually the ECS approach is more or less the exact opposite of (naive) OOP. In (naive) OOP you design objects for your domain, like an enemy, or a bullet, or an explosion. In ECS, you define abstract entities and compose data (and potentially behavior) on top of them, so there's no single "enemy" object, there's an entity that composes AI, geometry and sound effects to make it behave as an enemy.
In general, for large systems, this is a great way to achieve decoupling, where the naive OOP approach would get you stuck quickly.
One view on the OOP <=> ECS relationship is that ECS is an exploration of/recreation of/divergence from alternative "natural" OOP systems outside of the "dominant OOP strain" of C++ (and family) (re-)implemented on top of C++-style OOP.
There are hints (and needs) in ECS systems of post-hoc/runtime Prototypical inheritance rather than pre-hoc/design time Class based construction.
There are facets of ECS that resemble concepts like mixins and pattern matching and multi-dispatch and message passing rather than "pure" "C++ VTABLE" dispatch.
Some of these things arise more naturally in other branches of the OOP family tree: the Common Lisp Object System can do some of these things out of the box; Smalltalk and its descendants (even Objective-C) were easier to mold to ECS-like configurations; IO had interesting things to say about this style of OOP; even much-maligned JS' "strange" branch of Prototypical OOP has a bunch of low-level tools that make ECS "less of a need" in JS because there are ways to do them more "naturally".
Certainly, though, in C++ and closer-related derivatives (even Java and C#) ECS isn't as naturally a pattern, which is why it often has need for strong "frameworks" and why there are often multiple, competing ones to pick from.
Do you have any references you could give on this? I think I have a fair idea of all the object systems you listed (and a couple of others). Every time I read about ECS I feel like it should be explainable as some flavour of OOP much as you say here, and I could express something vaguely ECS-shaped in either Self/Io- or CLOS-style OOP, but that’s still a far cry from showing that ECS is such OOP.
In particular, keeping a single object’s state in several separate memory regions seems atypical (prototypes—or Self’s “traits objects”—are usually stateless even if the system doesn’t force them to be); and I can’t think of any way to map iteration over all objects that have parts of a specific kind, often mentioned in expositions of ECS, to a conventional OOP concept (looks relational if anything).
ECS can mean many things. the idea that you put components together (physically, in memory) and try to set up your processing so stuff is processed together may be quite natural.
but actual ECS systems add a lot of stuff on top that's not very natural (in any programming paradigm, not just OOP).
it's about enabling you to add and remove components at runtime, manage complicated relationships between entities and their components in a generic sense. this has all the advantages and disadvantages that those kinds of abstractions bring. bad if you don't need it, good if you do.
In 2D games it is often times some kind of object hierarchy (scene graph, display list, nodes) imposed by the engine/library, which never plays well with ECS IMO. In these instances I think it is better to just inherit from a carefully thought out god object (yeh yeh I know).
If you are using a low level rendering library, or immediate mode library, you can more easily define your state however you want, but still ECS incurs massive overhead compared to basic arrays of structs. If your language allows structural typing then ECS seems to me as having little benefit.
The reason I say ESC incurs more overhead is because I find it involves a whole bunch of access patterns to add, remove, update, and query the data. Comparatively arrays of entities mostly just involves basic loops.
Still a weird take, as the ECS approach makes it much easier to do AoS vs SoA which is actually great for access/loop performance.
I'm sure there's examples where a non-ECS approach beats out an ECS approach. But to say that ECS incurs more overhead generally is just plain false.
ECS are obviously more efficient for traversing a subset of your entity data if that subset resides in one small component, simply because the total span of bytes accessed is lower and the cache hit rate is higher.
I would be more inclined to claim the opposite: that ECS systems are generally MORE performant than arrays of entities, but you can really only be wrong when making general claims like that.
> but still ECS incurs massive overhead compared to basic arrays of structs
I actually know of quite a few projects that use ECS as the foundation for a low level rendering library precisely because it lets you iterate over a bunch of plain arrays, but with more flexibility.
The overhead of an ECS depends on many things, but if you have lots of entities with similar components, the overhead per entity can be quite small.
A collection (e.g. an array or a hash map) that stores things. Maybe more than one collection like that, different for each type of entity: monsters, items, etc. (Separate code paths to handle these different types of things.)
Keeping indexes/keys for references, and loops over these collections can do a lot without any upfront complexity of an entity system.
I'm working with an ECS at $dayjob. But I'm starting to question one aspect of them. Composition of functionality makes sense. And systems acting independently and only on objects with related components makes sense.
But dynamically updating the set of components for an entity in order to "send it" to some other system, and relying on the ECS as a query database to coordinate all this, seems like questionable practice to me. I'm seeing more and more of that in our systems.
I think ECSes are becoming too dynamic, and will start to eat perf, and hide relations. Like some message bus.
I really love ECS. I’ve shipped games with my own spaghetti mess of classes and poorly thought out inheritance (at least it feels that way) but ECS and composition just is much nicer for incrementally adding functionality without thinking about the big picture too much. I love that I can think about just one small feature I want to add instead of the overall grand design.
This might be bad design (no idea) but I’m working on an item system for a game right now. I have Item and ItemHolder components. If it’s a player, they’ll have an InputController component, which lets them use the Item in the ItemHolder, otherwise there is an AIController component that let’s an NPC use the item.
I have a system for handling every piece of functionality that can be defined in its smallest form, which makes it incredibly easy to add or remove functionality to any entity.
Also, performance is great. I have the mindset to not prematurely worry about performance until profiling now.
Inter-entity/system communication without a bunch of tooling to support it is always going to be a bit cludgy and hard to follow as it's by necessity dynamic at runtime. It is really useful though so writing tooling for it is a good idea.
Perf impact could be pretty big if your ECS uses archetypes and needs to do a lot of copying data around as components are added and removed. But in my experience these sort of events tend to be low frequency so probably not terrible. Always good to measure though!
Don't feel hide bound to stick within the confines of the ECS setup. It's a tool not a way of life! Straight forward message passing might be much cleaner and clearer or there might be a better data sharing strategy that can live outside the ECS itself. All frameworks provide fun puzzles of how to fit functionality into them when going round them is sometimes simpler and clearer!
If I understood correctly, you are referring to applying some sort of tag component to a set of entity to be able to query it in a separate system. Am I correct?
For cross system data flow, I think a mechanism similar to what Bevy does is the way to go. A specific event bus where the developer can send any type of struct, which can also allow passing a set of entity ids. I have briefly used it for my project and I think it's clear to reason about. The only catch is that you need to be careful about timing (the time between sending and receiving an event might take one frame to propagate).
When working on game, the control flow must be as easy as possible to follow and modify.
With the ECS implementation in Unity DOTS, this is absolutely not the case, the control flow is mostly managed behind the scene, and multi-threaded, the simplest things are transformed into a puzzle of cascading interactions.
> But dynamically updating the set of components for an entity in order to "send it" to some other system, and relying on the ECS as a query database to coordinate all this, seems like questionable practice to me.
Indeed it is. The idea of components is to decouple systems. Where systems generally operate on few components, and own a disjoint set of components. It is not a communication channel, just like any other database should not (primarily) be used as a communication channel.
You can combine an ECS with any number of different patterns to make your communication work: events, agents, messagebus, you name it.
EnTT is quite nice because it's more than just a ECS (if you want), it also includes libraries for a variety of other features like resource management. It's more of a "game engine" library, which is fantastic for projects that don't need to muck about in a "game studio" like Unity/Unreal (e.g. for games or simulations that don't require traditional level design).
Yes, I think I like the overall EnTT design, but I've only toyed around with the ECS part, to learn how to use it. I still haven't entirely figured out when it's best to use an ECS and when to use a scene graph and if it makes any sense to use both at the same time in a project.
IMHO most games should use both. Think of the game as a 2D table, with Entities on one axis and their Components/Systems aligned on the other axis. Some operations go (densely) along the Component axis, like updating your physics simulation every frame. But many operations go (sparsely) along the orthogonal Entity axis, resolving events and other game rules involving many tiny sets of entities. These are orthogonal operations with different optimizations and I don't think there is one architecture to rule them all. However, I think an ECS like Flecs or EnTT is a good core to build off of.
Agreed. Especially the ability to split an object into multiple components/traits and have an operation run automatically for every component type appeals to me. It just seems elegant and 'good design', similar to 'composition over inheritance'. I think the strength of the scene graph is in the explicit parent-child relationship, where it applies, but I probably need a deeper understanding than I now have to mesh them in the right way.
I read the front page of this site, and I still have a general question about ECSes. For what it's worth, I'm developing a DAW (digital audio workstation) in Rust.
It seems that ECS helps compose "things" (an entity) out of groups of data (each a component), and then operates upon combinations of those components (using systems). My domain is organized around behavior (many musical instruments implementing a Rust trait or interface, such as MIDI), but the data constituting each instrument is totally different. So the behaviors are the same, but the data is different. It seems that ECS isn't a fit, which is a shame because the implementations I've seen offer cool features like easy parallel processing, querying, and so on.
Is there a design pattern like ECS but for common behaviors instead of common data? Or am I being dense and not seeing how ECS actually is a good fit for this situation?
The way you frame the question already suggests a solution, and one that indeed doesn't fit ECS that well. That doesn't mean you couldn't design a DAW using ECS, but you'd have to redesign how it works.
For example:
- create custom components for different instruments
- create systems for specific instruments that match the necessary components
- write the result to a common component (e.g. "ChannelOutput")
You could then add components for EQ, effects, routing, ... and introduce systems that match those. The advantage of this approach would be that you end up with a bunch of loosely coupled systems (for instruments, effects, mixing, ...) and an in-memory representation that's cache & parallelization friendly.
One other datapoint would be that Unreal's animation sequencer uses an ECS design, which has a lot of things in common with a DAW.
Glossing over lots of details here, you can't of course design a DAW in one paragraph.
Thanks (and thanks especially for responding to what should have been a Stack Overflow question). I do see potential for certain ECS systems that span instruments, like DCA, effects wet/dry, routing, and mute. That makes sense. Your suggestion for component/system per instrument was what I felt ECS was asking me to do, but the libraries I investigated seemed to contemplate dozens but not hundreds of components, and that's where I began to think I was putting the cart before the horse.
I had an epiphany a while ago that a modern interactive DAW is a video game, except parts of it run at 44,100 fps. And even the "except" part is misleading, because games have sound as well. That opened up my mind to game development as another domain to steal designs from.
Experienced game developer here: if you are just starting out wanting to learn how to make games then 1000000% focus on gameplay and not technology. Almost all developers I know who starts working on games end up basically just working on engine technology and never get an actual game done. I used to be one of them so I talk from experience! :) Nowadays I just use Unity and focus on the actual hardest part of making games: coming up with fun enjoyable gameplay that isn’t just a clone of somebody else’s game.
I have a sort of off-topic question. The marketing for this claims not using STL containers as a selling point. Does anyone know what makes STL containers unsuitable for this use case? Clearly they're not a one size fits all but I'm intrigued to know why you market an unknown container based API over a tried and tested one?
The biggest reason for me was to reduce compile times, which got cut into approximately half when I replaced STL containers with simpler custom ones. This starts adding up in large projects, where lots of files end up including the ECS.
Any idea what a game loop would look like for an ECS engine that only renders a new frame when some animation is running or when the user is manipulating the camera, but not otherwise?
Let's not argue "why". Most obvious would be saving battery.
I've implemented exactly this at $WORK. We literally just set a global variable, "should render next frame", from input handlers and other important events. It's not pretty but it gets the job done. There are occasional bugs where someone forgets to set the render flag after some significant event. But the bug goes away as soon as the mouse is moved, so it's usually not a big deal. If we wanted we could also schedule a "render once per second" to bound the amount of time that visual issues would persist for.
I'm not sure how this translates to ECS architecture - probably depends on your library, but if you're in charge of the main loop it shouldn't be a problem.
At one company we had a deployment script written in English. That support from India would execute. Except they would occasionally inject their undocumented codes to fix their own issues.
That and the component oriented distributed monolith that ran on single machine. It was monolith architecture but it was made to run on multiple machines. However it was never distributed and starting the components was an arcane science.
One trick I found is to gate mutable access to state behind a "transaction object" which sets a flag for each object mutated, then once the transaction object's destructor is called, recomputes all derived state based on which flags were set. Using a destructor might not be the best idea since I ended up not marking it noexcept, it may be better to add an explicit "recompute" function called at the end, but this breaks the convenience of `state().cursor_mut() = new_cursor;` automatically recomputing derived state once the state() object is destroyed.
For energy efficient game loop, it still runs in a fixed frequency, e.g. in 60fps or 30fps. For each frame, the time is spent on event processing, state update, rendering, and sleep. Some of the steps can be skipped. The goal is to lower the times spent in the non-sleeping steps, and sleeps more for each frame.
Event queues are used to track things in long duration across multiple frames, like user input, animation sequences, sound sequences, pending state update, pending physics update, pending AI update, etc.
In each frame, check the event queues. If they are empty, sleep. Otherwise, process the events in the queues. For user input, do the state update corresponding to the input, like updating the camera position.
For ongoing animations, advance the steps in the sequences (the drawing is done in the rendering step). When an animation sequence completes, remove it from the queue. Same for sound.
For pending state update, run the update. E.g. the ship is moving, do the state update on its velocity and position. For ECS, you might end up calling or not calling the systems for the item type for batch update. E.g. when the bullet queue is not empty, you might just run the bullet related systems over all bullets for simplicity. When all the bullets end and the queue is empty, the bullet related systems can be skipped.
The physics queue has the items being simulated. Run those. Physics could run at a slower rate like 5fps, so the queued items would be skipped most of the times. Same with the AI items in the AI queue.
Whenever there's a state update, mark the screen as outdated. At the rendering step, run rendering if the screen is outdated. It's simpler to just run the whole rendering for any change. Ideally, when there's no event in the queues or the state updates are not triggered, there's no need for rendering.
This applies to using ECS or not. ECS is for tracking states of entities and do batch update with systems.
game loops usually call into simulate() and render() entrypoints.
There's state potentially mutated by simulate().. render() looks at that state and potentially produces a new output to present on-screen.
Typically the render() entrypoint returns a status indicating whether a new output was produced or not to put on-screen. If no output was produced you don't flip pages. So in a quiescent situation with nothing changing, render() returns "don't flip" and you just leave the existing page displayed til the next iteration.
How you track the internal game state / dirty vs. clean state is implementation detail behind those simulate() and render() entrypoints and depends greatly on the type of game.
I wrote a game engine that did that, because it was for a very simple, largely static scene.
You would implement it almost the same as a standard Windows GUI -- using an event handler loop that marks the screen as "dirty" and needing a refresh if it has changed.
Where it would immediately break down is ongoing animations. If you stopped moving your mouse, then animations would stop. Of course, you can set up the event loop to continue triggering if there are animations going on, but then you basically have a standard game engine loop running most of the time.
Personally, I'd be happy if games simply paused rendering when minimised. Some games do this, some don't.
Some games (I'm thinking FFXIV) will run on a lower framerate (15fps) while minimized or, in that game's case, if the player is idle for an X amount of time, that's a pretty decent compromise.
But yeah, games don't need to render any frames while minimized. I do feel like this should be the operating system's responsibility as well, but then, a lot of games still have the graphics rendering and other parts of the game linked tightly together.
You just have to include all the parameters of the animation as part of your "dirtiness" calculation. I reckon in practice this means centralising pretty much _all game state_ in some way that you force the dirtiness state to be set correctly.
I'd say this is something that can be done on a driver / GPU level; if the inputs to render the next frame are the same, it can just reuse the previous one.
That's probably oversimplifying it, I don't know anything about graphics programming.
This doesn't really work. By the time the driver or GPU could "know" that the frame is the same, it's already done a lot of the heavy lifting. As in, it has basically still rendered the next frame by this point. Actually writing the result to the framebuffer is not expensive.
It could be yep. One of the things that makes it a good fit for this use case is that Flecs has a query language that supports things like graph traversal and joins. Combined with either the builtin REST API or some custom protocol you could use it as a realtime in-memory database (which is what I use it for).
Unless I'm mistaken, Unreal hasn't had an official blessed ECS system relying on other things: from just having regular objects to reading data from data tables.
Your game will get done if you work on it, not necessarily if you pick ECS!