I'm maintaining an open source project[0] and I'm struggling with using SemVer because my "app" doesn't have a single API but a few:
At it's core it's a node app. Though I also include a small web server that wraps around it (and a UI frontend).
1. It allows people to write scripts (js) that receives inputs and passes on events based on an API (the strategy API)[1].
2. It has extensive configuration[2] that sometimes changes form (the config API).
3. It talks to a number of external services (crypto exchanges), over a "common" protocol called the "exchange wrapper API"[3] (I am ignoring the version of the exchange API being consumed).
4. The wrapped webserver comes with an API (REST + WS)[4].
5. The "core app" is a chain of plugins, when they change the required config/events also change (usually breaking changes 1 and 4)[5].
I could take all of these components apart (microservice way) and version them separately, but I like the monorepo style I use now where pulling one repo means that everything is working together. Also the fact that (in bug reports) people only have to refer to one version (and when on nightly maybe the git commit if I need more details).
Your project looks cool. Here's some overly harsh criticism from an old dude in no particular order.
The problem I see upon an extremely cursory view of your project is that it's trying very, very hard not to be a sellable product.
I'm a good programmer. Why do I want to learn your API/library instead of calling the exchange API's directly? Is this saving me time? Is it saving me time long term, even when your code changes and breaks things?
If you had to make this into a single web API and sell access to that as your product, what would it look like? There's your versioning and design answer.
Why is the app a "chain of plugins?" If the app breaks when the plugins change then they're not really plugins, are they? Does this design solve a problem or did it just seem like a cool way to do it?
Also, anything that relies heavily on configuration to work is a fundamentally broken design in my opinion. Configuration is global state that is hard to change, and the heavy-handed presence of it in a project is usually an indicator that the abstractions are wrong and most of the code probably relies on some hidden state that's really hard to debug unless you're the code author. If there's a legitimate runtime choice you don't want to make for the user, that's a function parameter. If that looks messy, you probably left too many decisions for the user to make.
An ideal library is stateless so that the user can handle wrapping it with simpler calls and configuration settings. Make building blocks, not skyscrapers.
Sometimes I want to grab all of you young people by the shoulders and shake you until you stop reinventing ever more convoluted ways to do RPC.
Finally, take everything I say with a grain of salt because I'm heavily medicated right now.
Woah great feedback. This is very much appreciated!
I'm not sure if going specifically into all of your points right here is the best way forward. But suffice to say I am very happen to hear them :)
> the abstractions are wrong and most of the code probably relies on some hidden state that's really hard to debug unless you're the code author.
This is very much spot on, definitely something I want to work on.
-----
The main reason that everything is so spread out (plugins, web API, internal API, etc. etc) is because a ton of people are doing different things with it.
99% of the people only touch the basics, and they don't need to touch any config file, they can go through the UI that handles all of it automatically (Gekko is focused on tech savy but not perse professional programmers). They don't know what (my concept of) a plugin is, and they don't care about any API (nor any version for that matter).
It's about the other 1% who are kind of spread out over:
- people who want to hook into certain lifecycles (to push certain data to google spreadsheets[1] for example)
- people who only use subparts of the app, for example to have something that can fetch normalized market data from a number of different websites <- this is a big part of the project, but not the sole, hence it should not dictate versioning.
- Or people who only want to create their own prediction making logic (with AI or whatever) and use the execution logic of my app. <- in the process of pulling this out as a standalone library.
- people building tools on top of the web API that bruteforces a problem space to figure out new solutions[2].
So all the people that care about the versioning (not the 99%) are exactly the hobby DIY hacking people who want to open it up and take it apart. And it feels impossible to steer them into "don't touch this because the interface is not a standardized API".
-----
The main thing I am going to do now is rethink the entire config strategy, because it's a huge mess and I think I am the only one who understands it[3].
At it's core it's a node app. Though I also include a small web server that wraps around it (and a UI frontend).
1. It allows people to write scripts (js) that receives inputs and passes on events based on an API (the strategy API)[1].
2. It has extensive configuration[2] that sometimes changes form (the config API).
3. It talks to a number of external services (crypto exchanges), over a "common" protocol called the "exchange wrapper API"[3] (I am ignoring the version of the exchange API being consumed).
4. The wrapped webserver comes with an API (REST + WS)[4].
5. The "core app" is a chain of plugins, when they change the required config/events also change (usually breaking changes 1 and 4)[5].
I could take all of these components apart (microservice way) and version them separately, but I like the monorepo style I use now where pulling one repo means that everything is working together. Also the fact that (in bug reports) people only have to refer to one version (and when on nightly maybe the git commit if I need more details).
But versioning is a mess.
[0]: https://gekko.wizb.it/
[1]: https://gekko.wizb.it/docs/strategies/creating_a_strategy.ht...
[2]: https://github.com/askmike/gekko/blob/develop/sample-config....
[3]: https://gekko.wizb.it/docs/extending/add_an_exchange.html#Ge...
[4]: https://gekko.wizb.it/docs/internals/server_api.html#REST-AP...
[5]: https://gekko.wizb.it/docs/internals/events.html#List-of-eve...
-------
This is not a criticism, I typed this out in the hope that someone can point me in a sane direction (given the discussion on versioning)