GitLab Enterprise user here. GitLab's open-source policy is one of the major reasons we chose it (along with self-hosting and fairly modern, built-in continuous integration), but the GitLab community could use some additional design and documentation effort.
Things that almost make me want to switch back to GitHub:
- GitLab is really hard to look at. All of our devs are having trouble visually locating comments in diffs. We're always scrolling through the left-side menu looking for the issues tab. It's hard to visually differentiate between items like comments in merge requests. Our eyes just can't find and follow things. Even BitBucket looks better.
- GitLab-CI has poor documentation on runners, and a rather buggy test runner registration process that doesn't provide any debugging output and produces false positives quite often.
- GitLab-CI has no unconditional cleanup directive, like CircleCI does... but if you're re-using a test runner instead of tearing down the server each time, unconditional cleanup is the one thing you need. And no, the Docker executor doesn't solve this because we don't want to test our Docker containers inside Docker.
- CI is based on commit IDs. You can't run particular CI events on merge requests. For example, this means no way to run integration tests on merge requests.
- Typography could be improved. When you're looking at the issues page all day, this is quite noticeable.
- Slack integration is not great; it dumps links to "GitLab Enterprise Edition" instead of useful commit/changeset stuff like GitHub does.
If the community/GitLab can focus on putting on some of the design polish and service integration polish that GitHub has, GitLab would be a total winner for us, long-term.
The quick reply to your comments is: we're working on almost all of it.
> - GitLab is really hard to look at. All of our devs are having trouble visually locating comments in diffs. We're always scrolling through the left-side menu looking for the issues tab. It's hard to visually differentiate between items like comments in merge requests. Our eyes just can't find and follow things. Even BitBucket looks better.
We're working hard on this and focus more and more of our development capacity to this.
> - GitLab-CI has poor documentation on runners, and a rather buggy test runner registration process that doesn't provide any debugging output and produces false positives quite often.
There are a lot of improvements coming in this field. Especially the integration and setup of runners is going to be easier. We're hoping to make this a zero-step setup thing.
Same with documentation. Our new GitLab CI maintainer Kamil has been spending a lot of time on writing documentation.
> - GitLab-CI has no unconditional cleanup directive, like CircleCI does... but if you're re-using a test runner instead of tearing down the server each time, unconditional cleanup is the one thing you need. And no, the Docker executor doesn't solve this because we don't want to test our Docker containers inside Docker.
This is interesting, I'm passing it onto Kamil.
> - CI is based on commit IDs. You can't run particular CI events on merge requests. For example, this means no way to run integration tests on merge requests.
This is be interesting to look at. I'm not sure whether we have something like this in the planning, but I'm taking note.
> - Typography could be improved. When you're looking at the issues page all day, this is quite noticeable.
We're worried about adding page load by including typefaces. We might change our minds on this. What OS are you using?
Would Roboto work, for instance?
> - Slack integration is not great; it dumps links to "GitLab Enterprise Edition" instead of useful commit/changeset stuff like GitHub does.
We welcome contributions to improve it. This is not a focus at the moment, but considering the popularity of Slack, we can definitely have a look at this.
Super.. good to hear! Ideally the community can get a little more involved in design/UX too. I'm not sure if GitLab plans to set the tone there first - perhaps that's also on the roadmap.
> Would Roboto work, for instance?
Maybe the typeface isn't so much the issue as the layout? On the issue view, comments and descriptions have different positioning and styling. Also, important information lives above the issue title, which is unexpected.
Being unable to trigger CI builds based on merge requests (which can dispatch a build for a commit ID, but you've probably already pushed that commit) is probably the most problematic for us at the moment.
Maybe it would be cool if GitLab came up with some basic design principles/goals and accepted contributions toward that end. I know we'd be interested in contributions here and there to improve workflow/UX.
> We're working hard on this and focus more and more of our development capacity to this.
No, no, no. :) It's not a development thing, it's a design thing. The good part is that your users are well-defined - developers; you could do a great job there, but don't allocate your developers to it. Hire or contract someone experienced in UX.
P.S. With a good UX work, you might end up not only bettering GitLab's layout but also discovering new, differentiating and truly useful (user-wise) features.
> GitLab is really hard to look at. All of our devs are having trouble visually locating comments in diffs. We're always scrolling through the left-side menu looking for the issues tab. It's hard to visually differentiate between items like comments in merge requests. Our eyes just can't find and follow things. Even BitBucket looks better.
I agree out interface can be a lot better. Our first interaction engineer will start September 1 to improve it.
>GitLab-CI has poor documentation on runners, and a rather buggy test runner registration process that doesn't provide any debugging output and produces false positives quite often.
There’s a lot of going on in this field. We are listening to people’s user cases and trying hard with the rest of community to extend documentation that will cover them. I agree the test runner registration should give better feedback if there are failures. False positives shouldn't happen, please create an issue if they do.
> GitLab-CI has no unconditional cleanup directive, like CircleCI does... but if you're re-using a test runner instead of tearing down the server each time, unconditional cleanup is the one thing you need. And no, the Docker executor doesn't solve this because we don't want to test our Docker containers inside Docker.
Agree, we have that on our roadmap: allow to run commands in case of script failure.
> CI is based on commit IDs. You can't run particular CI events on merge requests. For example, this means no way to run integration tests on merge requests.
I don't know what you mean exactly. Do you mean you want to test the code that will result after the merge instead of the commit in the feature branch? It would be a nice option to test the merge of the feature branch with master instead of the feature branch itself (you will have to retest all MR’s every time master is updated). BTW The new build trigger API (to be released in a month) is one of the upcoming features that will allow more build customisation.
> I don't know what you mean exactly. Do you mean you want to test the code that will result after the merge instead of the commit in the feature branch? It would be a nice option to test the merge of the feature branch with master instead of the feature branch itself (you will have to retest all MR’s every time master is updated). BTW The new build trigger API (to be released in a month) is one of the upcoming features that will allow more build customisation.
No, I want to trigger additional types of tests under two circumstances:
- When a merge request is created
- When additional commits are added to a branch for which a merge request exists
The utility is: I don't want to run integration tests if someone is just pushing to a feature branch - that takes up worker time. But I do want to run integration tests before that feature branch gets merged upstream. Achieving this by having everyone submit merge requests to an intermediate branch doesn't work, because we don't know the integration test result until after the merge to the intermediate branch is done, meaning developers might be trying to push a bunch of failing code to the same branch, just to get integration tests to run.
> But I do want to run integration tests before that feature branch gets merged upstream.
Specifically, the integration tests should run against the version of the source code in the feature branch that the merge request is about. The target branch can be merged into the feature branch to have full coverage of the post-merge status, so the target branch's importance is really minimal here.
> Yes, it would be a nice option to test the merge of the feature branch with master instead of the feature branch itself (you will have to retest all MR’s every time master is updated).
Actually no, that's not what I was trying to get at. The tests just have to run against the version of the source code in the feature branch. If the integration tests need to cover anything else, it should be inserted into the feature branch manually. Stale branches are their own problem and are quite common, independent of CI-related testing. Running unit tests all the time is fine, but it would be nice if special additional tests can be triggered only for new merge requests and for updated merge request source branches but not feature branch pushes unassociated with merge requests.
Yes, it would be a nice option to test the merge of the feature branch with master instead of the feature branch itself (you will have to retest all MR’s every time master is updated).
Question for you: as a GitLab Enterprise user, knwoing that GitLab has CI built-in, is there a situation in which you would buy CircleCI to go alongside it? Obviously we (we being CircleCI, founder here) have GitHub Enterprise support, and I wonder if we should build GitLab Enterprise support too.
> is there a situation in which you would buy CircleCI to go alongside it?
Well, we were using circleci for a while until the switch to gitlab-ci just recently. Biggest problem was that the circleci runners lose the local docker cache, which meant that unnecessary large chunks of time were spent redoing docker container builds, slowing everything down in our workflow. Note that we run unit tests in docker containers, and to speed things up it would be useful to run unit tests in parallel on multiple separate machines, in addition to preserving the local docker cache on each runner machine.
> So assuming we fixed that, would you buy it (considering esp that GitLab CI exists and you already use GitLab)?
Yes but no, we already switched and we're on to other things. We reported the docker cache issues to you guys a looong time ago, and we get it- most of your customers are probably focusing on webapp unit testing, and probably not on building docker containers and then running unit tests in those containers. There's a lot of hammering to get each CI solution to work just right, but maybe we could switch back more easily if you guys made a gitlab-ci compatibility layer..... (big project, I know).
Oh also we have somewhat-malleable self-hosting requirements, but I mean for the right solution we've been able to forego self-hosting in the past.
That was true in the early days. Now we have toooooons of Docker customers, so this thing is right at the top of our TODO list (and would be done already if it wasnt architecturally hard).
Things that almost make me want to switch back to GitHub:
- GitLab is really hard to look at. All of our devs are having trouble visually locating comments in diffs. We're always scrolling through the left-side menu looking for the issues tab. It's hard to visually differentiate between items like comments in merge requests. Our eyes just can't find and follow things. Even BitBucket looks better.
- GitLab-CI has poor documentation on runners, and a rather buggy test runner registration process that doesn't provide any debugging output and produces false positives quite often.
- GitLab-CI has no unconditional cleanup directive, like CircleCI does... but if you're re-using a test runner instead of tearing down the server each time, unconditional cleanup is the one thing you need. And no, the Docker executor doesn't solve this because we don't want to test our Docker containers inside Docker.
- CI is based on commit IDs. You can't run particular CI events on merge requests. For example, this means no way to run integration tests on merge requests.
- Typography could be improved. When you're looking at the issues page all day, this is quite noticeable.
- Slack integration is not great; it dumps links to "GitLab Enterprise Edition" instead of useful commit/changeset stuff like GitHub does.
If the community/GitLab can focus on putting on some of the design polish and service integration polish that GitHub has, GitLab would be a total winner for us, long-term.