The CI choice is important. There are many features that must come from the CI:
- Support downloading the code from various repo (GIT, SVN...)
- Option to start build from scratch or keep the state from the previous build
- Integrate with your user accounts (LDAP, Google Auth...)
- Good and stable API [jenkins API is a complete mess]
- Storing and retrieving build artifacts
- Bisecting
- Running batch of builds / parametrized builds
- Graphing and statistics (build durations, build failures, ...)
- Backup system for the configuration of the CI master (must be scriptable)
- High availability: Either support a pair of CI master with active/passive, or support scaling horizontally with many active masters (this is critical for > 1000 dev companies with many many projects)
- Build in monitoring in the CI master itself (e.g. manage the space taken by projects and artifacts) with a notifications when things are going critical.
> Support downloading the code from various repo (GIT, SVN...)
Is there a CI server that can't handle this? Worst case you can do a checkout from the command line.
> Option to start build from scratch or keep the state from the previous build
This should be part of the build script. "make clean build" or "make build".
>Integrate with your user accounts (LDAP, Google Auth...)
Not sure whatthe case for this is, but the authorization can be passed to the build script anyway.
> Good and stable API [jenkins API is a complete mess]
What do you need the api for?
> Storing and retrieving build artifacts
Again, better off being a feature of the build script.
> Bisecting
This is a job for humans.
>Running batch of builds / parametrized builds
Doesn't every CI server have this?
> Graphing and statistics (build durations, build failures, ...)
This is really not important.
> Backup system for the configuration of the CI master (must be scriptable)
Why is this part of the CI server and not a general backup system?
> High availability: Either support a pair of CI master with active/passive, or support scaling horizontally with many active masters (this is critical for > 1000 dev companies with many many projects)
This really isn't important. The CI server being out for a few hours will not bring down the company. If you have that many projects then there is no problem with having multiple build servers.
>Build in monitoring in the CI master itself (e.g. manage the space taken by projects and artifacts) with a notifications when things are going critical.
Like the backups, this should be part of a general OS monitoring, no need to complicate the CI server.
>> Support downloading the code from various repo (GIT, SVN...)
>Is there a CI server that can't handle this? Worst case you can do a checkout from the command line.
It's common NOW. It's been a problem for a very long time. The transition to the git era took a while and didn't happen smoothly.
I didn't check, but there may be some of the SaaS services being bound to specific repo services.
>> Option to start build from scratch or keep the state from the previous build
> This should be part of the build script. "make clean build" or "make build".
A script can't "rm -rf ." at the start of the build or it destroys itself ;)
A script could "rm -rf ." at the end of the build... except when the build fails midway, leaving the machine in an inconsistent state.
>>Integrate with your user accounts (LDAP, Google Auth...)
>Not sure whatthe case for this is, but the authorization can be passed to the build script anyway.
I am not talking about build. I am talking about people signing in to the web ui with some sort of user accounts.
>> Good and stable API [jenkins API is a complete mess]
>What do you need the api for?
To add/remove slaves programmatically. To trigger builds and get build status. Among other things.
>> Storing and retrieving build artifacts
>Again, better off being a feature of the build script.
The CI SHOULD have a feature to store artifacts or upload to ftp/ssh/s3. (other)
There is no excuse for not supporting common use case. It's better to have CI's dev program it right once, than having one million user each baking their own buggy scripts.
>> Bisecting
>This is a job for humans.
WTF. The job of a human is not to click 20 times through the interface only to enter the git hash of 20 commits.
Just gimme a button "build last N commits" or "build commit between X date and Y date"
>>Running batch of builds / parametrized builds (had matrix build in mind specifically)
>Doesn't every CI server have this?
As a matter of fact: No, they don't.
(I was also surprised to discover that in the wild).
>> Graphing and statistics (build durations, build failures, ...)
>This is really not important.
It is important. We NEED to check the evolution of the build duration.
>> Backup system for the configuration of the CI master (must be scriptable)
>Why is this part of the CI server and not a general backup system?
It is the responsibility of one's system that the system is backable.
If all the configuration of the CI is a "conf.xml". It's okay, I can just copy that file.
If all CI master is a mess of thousands of directories and files, where the master node configuration and the build configurations and the past build info are intertwined. I am forced to backup everything together, which may be 100 GB of data.
If the past builds and current configurations are referencing each other with unique GUID and numbers, in a way that ANY file missing might cause the jenkins server to not boot... I am fucked, especially so because there is no way to backup atomically 100 GB of data across 100000 files and directories while builds are running.
> This really isn't important. The CI server being out for a few hours will not bring down the company. If you have that many projects then there is no problem with having multiple build servers.
Given your arguments of all things being useless. You seem to be working in a very small company or not have worked professionally at all.
Please try to envision a real company: When the CI system breaks, there are 100 people who are interrupted and they will be partially unable to work until it is fixed.
By The Way: It's easier to maintain and manage one big system. By splitting one CI pipeline into 10 CI, you just created 9 SPOF and you just increased the failure rate dramatically.
>>Build in monitoring in the CI master itself (e.g. manage the space taken by projects and artifacts) with a notifications when things are going critical.
>Like the backups, this should be part of a general OS monitoring, no need to complicate the CI server
It is the responsibility of one's system that the system is reliable.
First) CI servers have disk space issues. I expect a sensible way to find out what projects/builds are taking and the ability to clean it up. [a monitoring system will give full disk alert, it doesn't understand the CI internal organization]
Second) If the dev are not retarded, they can put a notification for everyone to see when the disk is at 99% instead of writing the last % and crashing... only to never reboot again because it corrupted itself in the process.
