It's interesting to me how much effort is being spent on adding tools like issue tracking, build pipelines, deploy management, etc. vs just providing good hooks for standalone tools that do all those things.
We have discussions about Jenkins vs Concourse, where to keep ansible vault passwords, should documentation live in Github wiki or in Confluence (apparently "tech" documentation in the form, "business" documentation in the latter - what if it's both? Who decides?), and so on.
There is something nice about being able to go to a single place and saying "OK, it's all here in this box". Github has made inroads with some of this stuff, but not quite enough. Gitlab could try and do all this, but then people will moan ("I prefer JIRA/Trello/whatever").
Most of the pain around developer/business workflow around us at the moment actually comes down to the fact that nobody has _really_ thought about providing a great unified UX for all of this.
Part of the concern is people want to be "flexible". No, dictate, just make sure what you dictate is a better solution to what people have.
If GitLab get it right, github could be a minor player (unless they keep up) in a few years time.
I used to think that a collection of standalone tools was better and that we should focus on code hosting and review. When Dmitriy started with GitLab CI I was dismissive but of course as the author he could do as he pleased. It turned out that having a CI tool that closely works together with the main product was a better experience, especially behind the firewall/single-tenant/on-premises.
Then Dmitriy and Kamil (our CI lead) proposed integrating the CI coordinated into GitLab the main program instead of a standalone Ruby on Rails app. Again I was sceptical. But this has been a great success.
We have all the API's (webhooks, commit status API, etc.) to integrate well with Jenkins (and we'll make sure that we'll always play well with that, JIRA, Slack, and others). But having everything in one interface and one datastore makes for a much better user experience.
We've since doubled-down on this strategy. Bringing the entire software development lifecycle under one roof https://about.gitlab.com/direction/#scope During the webinar we'll show a demo video of how these components will soon work together. We think it is an amazing experience and everyone on our team is focussed to get it right.
Replacing Jenkins has been... a frustrating experience met with nothing but hollow shells of services. We want to self-host, we want something that can deploy our code, we want to be able to write arbitrary jobs not tied to specific github projects.
The Jenkins ecosystem is atrociously bad. Don't get me wrong, it's incredibly active, but it promotes a "just make a plugin for everything" attitude which results in things like, plugins to echo this or that, plugins to do a million things better done from the shell, etc. On top of that, the Jenkins UI is just... bad. It's ugly and it's a UX disaster. I could go on.
Yet, there's nothing better than Jenkins out there right now. Drone claims to want to replace Jenkins, but doesn't even have support for arbitrary jobs. I'd love more recommendations.
This is why, a while back here on HN, I asked if I could pick up the Gitlab job runner in golang and slap a different web UI on top of it that manages the jobs. This doesn't appear to be possible but I see the value in such a runner existing. Running the jobs is a critical component of such tools.
I just finished setting up a complete pipeline for a new business with my cofounder: Jenkins was a big pain point. Terraform was a pain point. Ansible was a pain point. Getting a vault solution set up and done right was a massive pain point. We've been striving to make things as simple as possible for developers so I wish you the absolute best of luck in what you're doing, I'm continuously impressed with Gitlab.
Email me if you have questions Sytse, I'd love to give more feedback -- email on my profile.
We don't like plugins because they are a bad user experience, make it hard to upgrade, and make it hard to improve the product. We rather see people contribute features to GitLab itself including tests. This way we can make sure they are nicely integrated in the UI and that they still work when we release a new version of GitLab. This allows users to upgrade with confidence and it allows us to make major changes to GitLab.
I'm not sure what you use case it but GitLab CI is pretty extensive and I hope it has everything you need. Please let me know if there is anything missing. If you want to host your code somewhere else you can still use remote mirroring to have GitLab do the deploy.
The GitLab Runner is a simple piece of software on purpose. This allows us to port it to many platforms. All the logic is in GitLab itself so it is easy to provide a lot of feedback about the status in the UI.
The only way to use GitLab CI is as part of GitLab. By doing this we could deeply integrate it into the interface of GitLab and make it a better user experience. If you host your code anywhere else you can use remote mirroring (EE/.com only) to use GitLab CI.
You can't make it replace Jenkins if it can't do arbitrary jobs. I'm talking jobs without even a repository behind them.
The idea of a runner that tests every commit on a repo is super useful. But Jenkins goes one step further by being essentially an arbitrary shell command runner. I use it for example as a CNC for running cleanup jobs, cronjobs etc.
This is shoehorning something into a model that doesn't fit it :/
For example, on Jenkins right now, we have a job which can trigger a bunch of load tests against our website. We can run this job indefinitely, give it arguments, it'll even run a local web server. By default, the job only runs the load test for ~10 mins.
The job is run periodically. When it fails, we get emails. After it completes, regardless of failure, it triggers a "cleanup" job which deletes the loadtest-related data from the website.
This is what I mean with "arbitrary jobs". I think it's absolutely fair to consider them out of scope, but if you want to replace Jenkins, this is what you're going up against.
Thanks for sharing your use case. I think that code of the cleanup job can be in the same repo as the load test itself. Would that be a fit for you? Of course the load test code can have the testing code of the application as a dependency.
Maybe you prefer the load testing code in the same repo as the application. In that case it should be two jobs that are triggered periodically I think.
no, it does not fit the bill. It needs truly arbitrary job running.
I want to be able to just drop in some arbitrary command (e.g. df -h /some/build/agent/dir/ | grep 99%) that would not be worth it to me to write a script, commit it to git, get that deployed and running.
In fact, I AM a fan of this being in git, but the example I'd use is an improvement on TeamCity, where you can configure your job and project configurations to "commit to git" so that you can easily restore when you need to rebuild a server or reload a previous version.
Sooooo many use cases for arbitrary jobs. Many of which may be short lived, but want them automated in some way.
Can Gitlab CI do manual triggers yet? (Drone can't, last time I checked, and that was a huge turnoff)
The "code" in this case really lives in the Jenkins database, as it's a shell snippet. Admittedly I'm not a fan of this. However, since you're a FOSS company as well, you may have hit the case where you want to separate the code for your infrastructure from the code you're releasing in the general repository, but still build off master.
This is where Jenkins' model is quite good. I strongly dislike that it's so hard for Jenkins' metadata to be versioned in proper git repositories, but the separation of the actual code, and the CI-related code is great.
In my taste the shell snippet that is infrastructure specific should like in a small repository. An example of that for us is https://gitlab.com/gitlab-com/runbooks (although that is more process since we try to keep the code generic).
I would love it if Gitlab CI went back to being a separate product. I'll use Gitlab for my own projects but ones that I work on with others are challenging to migrate. And creating mirrors on Gitlab of repositories hosted elsewhere to make use of CI/CD features doesn't sound ideal.
I understand why your different offerings are bundled together, I appreciate how your team tends to reach out to users in public forums such as this one, and I'm a fan of the quality of your work, but quoting from the page you linked to:
"any commits should be pushed to the upstream repository, and will end up in the GitLab repository automatically within an hour, or when a forced update is initiated."
"Wait up to an hour for things to happen" isn't a great story to pitch to team members :/ I'm also talking about at least twenty different repositories. I'll most likely have to continue using Jenkins for those projects but I look forward to using Gitlab CI on weekends.
I thought this time was configurable if you run your own installation of GitLab EE. I agree that for GitLab.com the delay of an hour can be a problem. Maybe we can solve that by adding webhooks?
The new version of drone (0.5) is much better imho, although it is still in beta so can be a bit frustrating at times. They have relaxed a bunch of the constraints and removed alot of the magic. Plus plugins are alot easier to write now.
I have been using buildkite lately, and it has worked rather well. You run the agent on your own hardware, and buildkite itself just acts as a coordinator and log collector.
Give me a rolling upgrade AWS/Rackspace/wherever (I choose) private hosted version of Gitlab with everything just working for under or equal to Github's USD$25/month and you have my business. Do a bulk deal, get some custom instances, lower the overheads. IMHO the pain of Gitlab (which I've used ~2008-now?) has always been unfamiliarity, UI obtuseness (recently vastly improved) and maintainability. Kill those - you're well on the way to doing so - and you'll gain market share quickly. You can even sell stuff like storage space, redundancy and global POPs as real time deployable add-on packages.
Did you know about https://githost.io/ ? It's managed by GitLab Inc and it's the closest you get from "Insert your credit card here, and hit a button" to get your own GitLab instance.
Hey wow that's just what I wanted! Only issue is it's $10 a month more than Github for the base setup, $15 extra for CI, while carrying hardware/storage limitations versus Github. https://githost.io/#pricing
We hope to make it easy to go to a single place for everything needed for development from issue boards, CI, analytics, etc. The unified UX for this tool is key and we are hiring a great team to support that goal.
We hope to create tools and resources to make that pitch and migration easier. Out of curiosity, what tools are you currently using?
We think it'll be a piece by piece migration versus all at once. In a perfect world, we have a tool for an all at once migration but perhaps that will come with time.
The problem with that approach is that you're limited by the quality of your weakest tool. Code review might be a strength, but if your CI tool is weak, that's what you'll be judged on. Integrations are a way to allow you to focus on what you do best and effectively outsource the rest. Ideally, the integration hooks would be well integrated, but I don't see an upside in trying to make a monolithic app in this market.
GitLab allows you to integrate with your existing tools if you please or require. But if you're fine using our tools (which are awesome, if you'd ask me), setup and cost will be zero.
I agree. It is essentially the same vision like the old ALM tools had. They were beaten by better more specialized tools following the unix philosophy: do one thing and do it well.
Apart from GitLab, we're mostly on Atlassian, and pretty strongly committed to JIRA and Confluence.
The CI solution in GitLab is inadequate for our needs; we require the flexibility and extensibility of Jenkins.
The review tool in GitLab has some significant gaps compared with standalone tools like ReviewBoard, and even compared with the one built into Bitbucket. It also has some baffling UX decisions, like the thumb/emoji buttons and the approve/merge buttons which appear in the exact same location as each other. Seriously, just scrap what you have and copy Bitbucket's review interface.
I'm not a gitlab employee or anything, but I'm curious: How is gitlab CI not flexible enough for you?
I mean, it basically just runs arbitrary docker containers. From my mostly amateur (when it comes to this, i'm a professional developer but I don't manage our CI system directly) point of view it seems like the gitlab CI is more flexible than jenkins not less.
The strong coupling between repos and CI jobs is a huge obstacle— for a multi-repo codebase, you need to have triggers that are outside of "a new commit has been pushed".
The reporting is a big problem too— Jenkins has plugins which format and report on JUnit-style reports, including displaying images or other assets generated by tests.
Configuration. The config needing to be committed to the repo is an issue. For many use cases including ours, it's way better to have the CI configuration stored centrally somewhere, and deployed using Jenkins Job Builder (or the various other similar tools which exist for Jenkins).
Executor control. Jenkins lets you do things like put N slots on a buildslave and then have a job occupy M of them, as well as controlling what jobs can run where, and what the priority of different jobs is. This is critical for ensuring overall sanity and efficiency in the pipeline.
Jenkins isn't perfect, but it happens to be a better fit for our needs, and these four factors are all a big part of that.
We think storing the configuration in the repository makes it easier to collaborate on it (new branch is tested in the new way, master in the old way). Can you share your use case for this?
You can specify how many jobs you want GitLab Runner to run. It can even spin up more VMs automatically if you want that. You can bind jobs to runners with labels and specific runners https://docs.gitlab.com/ee/ci/runners/README.html
Our use-case is perhaps a bit unique, but ROS has a federated development model, meaning literally hundreds of repos. See the following file which maps repos to package names and versions:
It would be totally unrealistic to rev job configurations by making commits into the individual repos (and in the public open source case, actually impossible— the maintainers of that community build infrastructure don't have write access to many of the repos— it's equivalent to a Debian distro using DSC files to grab tarballs).
Thanks for the use case and the links. I must admit I never saw a 14234 line yaml configuration file before.
The jobs page is not loading for me right now.
I think that in general developers and the build engineers should both have commit access. But I see your dilemma and think we need to have a conversation to see what the possibilities are.
As one example, GitLab CI does not have a good story with handling unit test failures from e.g. gradle.
You can choose to keep the build output from a failed build, which you can download as a zip file, unpack locally, and look at the reports. This user experience makes test reports mostly useless.
At the same time, they have test coverage badges, where test output is parsed to provide coverage. To me, that's really weird: You don't get proper access to test failures, but lots of work into test coverage.
(I have no knowledge of this, but I suspect that being a Ruby-shop, they have little experience with working with java projects, and therefore don't really understand how the toolchain works. This is all guesswork, though).
In the end, we cannot give up Jenkins. We need to know the result of our tests, and see the test output!
Just running docker containers is basically a show stopper for certain use cases. Not every use case can even fit into a docker container. E.g. testing kernel modules.
We've just opted for GitLab EE, the True-Up Licensing Model won us over, but also the promise of a working Jira Integration. That's still very limited and a pita to setup, even harder when you use atlassians hosted Jira. I don't want to buy GitLab Licenses for non-developers, especially as our Jira is connected to Zendesk, something which doesn't make sense to connect directly to GitLab.
But I agree with your comment that we should extend the scope of the integration and make it work better with hosted JIRA. Our VP of Engineering is assigning someone to this task. Feel free to email stan@ company domain with a link to this comment and with what you need so we can have an engineer look into it.
Many companies use JIRA and it should work great with GitLab.
We're currently moving to the Atlassian stack, from Bitbucket to Jira to Confluence. I really want to switch over to Gitlab reading this but the sunk cost is too huge already...
"Please don't sign comments; they're already signed with your username. If other users want to learn more about you, they can click on it to see your profile. "
Applies to job title, too. Yes, even at companies who are special enough to get millions of dollars from VCs.
Every rule has its exceptions. Sometimes it's nice to be able to quickly verify that the person answering a very specific comment about a specific company works at that company by avoiding the initial confusion or having to click anything. Let's not be overscrupulous.
I think it's important to take the bigger picture into account here. Letting us know who she is was a key part of that comment, since it's immediately relevant to the content.
The market will show that all-in-one box solutions are the best solutions for new companies.
When you're a new startup and you're trying to race to an MVP and being default-alive, what do you really want to do? Do you really want to spend time getting all of your ALM and build pipeline and production environments up and running? Or do you want to start iterating on business features as quickly as possible?
The truth is that providing 80% of the features is going to be good enough for 80% of new companies. When companies figure out that they need one of the 20% of the features that are missing, they can then migrate to get the feature that they positively identified a need for. Putting in much more effort in the beginning for ALM/pipeline/environment features you may not even need is a form of premature optimization.
We both see the value of all-in-one solutions for new companies. I think that for existing companies there is a comparable amount of value. All-in-one solutions make it easier to switch to a new team and be effective immediately. And it allows for company wide processes and tools.
The 20% missing features can be added via API's (we're trying to add API's for everything people need https://docs.gitlab.com/ce/api/ ). Or even better, by having people contribute code to the project itself.
> nobody has _really_ thought about providing a great unified UX for all of this.
Isn't the Stash-Jira-Bamboo-Confluence stack a unified UX for doing all of this (source control, pull requests, deployment, docs, issue tracker, build pipeline, etc)?
It may not be a great UX, but can we at least give Atlassian credit from probably having thought about it having a great UX?
That's funny, because you're right, and Atlassian has an enormous head start. And it's hard to explain why I like GitLab so much more.
I used the entire Atlassian suite for 3 years at my first web development gig, when I started over 7 years ago. I just have all these memories of things being ugly, slow, and hard to customize. Although I had a mostly positive experience with Confluence. Bamboo was just not nice to use. BitBucket does most of the same things, but it just didn't feel nice to use. Maybe it's the UI, or the dark blue theme. Maybe it's purely psychological, like GitHub was the place where all the cool developers hang out, and BitBucket is the corporate nerd who just wants to fit in so they give away all their private repos for free. So I dumped my private, personal stuff on BitBucket, until GitLab came along.
And honestly, up until the last few months, I actually really disliked GitLab. I kind of saw them as just rip-offs who were blatantly copying GitHub, and stealing their customers. But then I realized that GitHub hasn't really done anything interesting for years and years, while GitLab is working on all these awesome features and integrating things that GitHub should have been doing years ago. So now my opinion is that GitHub had their chance, and they blew. GitLab is the new cool place to host your code.
> And it's hard to explain why I like GitLab so much more.
In my case having the Atlassian all tools look the same seems to have backfired; every time I go to BitBucket it makes me think of Jira, which makes me cringe and wish I were on a different web site.
If only they had applied that "forward thinking" to upgrades. It's one of the reasons we're leaving them behind.
We use GitLab for both internal and client project code, including their runners to run limiting on code (for example) so web based edits/commits can be checked with the same rules we may run locally. Love it.
We use GitLab wikis to replace Confluence (which had replaced Evernote because, well, I couldn't trust them any more). We sometimes use Gollum locally because sometimes a UI around editing pages is helpful, though I sure wish prose.io connected to GitLab. I'd also like to see GitLab switch to Kramdown for Markdown rendering (there's an issue open for that already) and actually use the Gollum gem to render the pages, so that we can use the various partials that Gollum supports when it's serving up a wiki.
Once time tracking lands in GitLab, our (admittedly) basic usage of Jira can be replaced, since we're already using guest/reporter accounts for customers to report all issues on any custom dev project we do for them. We'd like to see that on-boarding process (for our clients who we intend to use Issues) go a little more smoothly, and I'm working on a written spec for proposed improvements, and I'm comfortable enough with their code base to take a whack at writing it.
If it's not dev work (e.g. a general SharePoint support contract), we just use a new HelpScout mailbox with a custom email address for the client. That email address is just an alias that forwards to the primary HelpScout mailbox, where a workflow rule moves it to the client's specific HelpScout mailbox. Nice thing is that HelpScout's Slack trigger doesn't fire until after the email is moved, so notifications also go to the channel we use to discuss that customer's issues.
If Mattermost got bundled with their hosted offering, that'd be killer.
We're using their free hosted service, and so I'm reticent to ask for these sorts of changes by opening issues unless I want to at least attempt to contribute the code for it. I'm unsure how some of that would work out in their multi-tenant environment, but I still plan to dig into it.
GitLab is good stuff. I've been deeply impressed by their agility and responsiveness to their customers. If anyone is on the fence, there's no cost to check out early all of their features.
> We're using their free hosted service, and so I'm reticent to ask for these sorts of changes by opening issues unless I want to at least attempt to contribute the code for it.
Please feel free to open issues anyway. We love hearing directly from users and always want to improve the product, even if you don't pay for it or contribute code. :)
We still use TFS at work for issue and code management (mainly due to inertia and it being included with our Visual Studio licenses) but TeamCity easily blows every other CI tool I've used out of the water - we love it and the sane pricing.
The CI choice is important. There are many features that must come from the CI:
- Support downloading the code from various repo (GIT, SVN...)
- Option to start build from scratch or keep the state from the previous build
- Integrate with your user accounts (LDAP, Google Auth...)
- Good and stable API [jenkins API is a complete mess]
- Storing and retrieving build artifacts
- Bisecting
- Running batch of builds / parametrized builds
- Graphing and statistics (build durations, build failures, ...)
- Backup system for the configuration of the CI master (must be scriptable)
- High availability: Either support a pair of CI master with active/passive, or support scaling horizontally with many active masters (this is critical for > 1000 dev companies with many many projects)
- Build in monitoring in the CI master itself (e.g. manage the space taken by projects and artifacts) with a notifications when things are going critical.
> Support downloading the code from various repo (GIT, SVN...)
Is there a CI server that can't handle this? Worst case you can do a checkout from the command line.
> Option to start build from scratch or keep the state from the previous build
This should be part of the build script. "make clean build" or "make build".
>Integrate with your user accounts (LDAP, Google Auth...)
Not sure whatthe case for this is, but the authorization can be passed to the build script anyway.
> Good and stable API [jenkins API is a complete mess]
What do you need the api for?
> Storing and retrieving build artifacts
Again, better off being a feature of the build script.
> Bisecting
This is a job for humans.
>Running batch of builds / parametrized builds
Doesn't every CI server have this?
> Graphing and statistics (build durations, build failures, ...)
This is really not important.
> Backup system for the configuration of the CI master (must be scriptable)
Why is this part of the CI server and not a general backup system?
> High availability: Either support a pair of CI master with active/passive, or support scaling horizontally with many active masters (this is critical for > 1000 dev companies with many many projects)
This really isn't important. The CI server being out for a few hours will not bring down the company. If you have that many projects then there is no problem with having multiple build servers.
>Build in monitoring in the CI master itself (e.g. manage the space taken by projects and artifacts) with a notifications when things are going critical.
Like the backups, this should be part of a general OS monitoring, no need to complicate the CI server.
>> Support downloading the code from various repo (GIT, SVN...)
>Is there a CI server that can't handle this? Worst case you can do a checkout from the command line.
It's common NOW. It's been a problem for a very long time. The transition to the git era took a while and didn't happen smoothly.
I didn't check, but there may be some of the SaaS services being bound to specific repo services.
>> Option to start build from scratch or keep the state from the previous build
> This should be part of the build script. "make clean build" or "make build".
A script can't "rm -rf ." at the start of the build or it destroys itself ;)
A script could "rm -rf ." at the end of the build... except when the build fails midway, leaving the machine in an inconsistent state.
>>Integrate with your user accounts (LDAP, Google Auth...)
>Not sure whatthe case for this is, but the authorization can be passed to the build script anyway.
I am not talking about build. I am talking about people signing in to the web ui with some sort of user accounts.
>> Good and stable API [jenkins API is a complete mess]
>What do you need the api for?
To add/remove slaves programmatically. To trigger builds and get build status. Among other things.
>> Storing and retrieving build artifacts
>Again, better off being a feature of the build script.
The CI SHOULD have a feature to store artifacts or upload to ftp/ssh/s3. (other)
There is no excuse for not supporting common use case. It's better to have CI's dev program it right once, than having one million user each baking their own buggy scripts.
>> Bisecting
>This is a job for humans.
WTF. The job of a human is not to click 20 times through the interface only to enter the git hash of 20 commits.
Just gimme a button "build last N commits" or "build commit between X date and Y date"
>>Running batch of builds / parametrized builds (had matrix build in mind specifically)
>Doesn't every CI server have this?
As a matter of fact: No, they don't.
(I was also surprised to discover that in the wild).
>> Graphing and statistics (build durations, build failures, ...)
>This is really not important.
It is important. We NEED to check the evolution of the build duration.
>> Backup system for the configuration of the CI master (must be scriptable)
>Why is this part of the CI server and not a general backup system?
It is the responsibility of one's system that the system is backable.
If all the configuration of the CI is a "conf.xml". It's okay, I can just copy that file.
If all CI master is a mess of thousands of directories and files, where the master node configuration and the build configurations and the past build info are intertwined. I am forced to backup everything together, which may be 100 GB of data.
If the past builds and current configurations are referencing each other with unique GUID and numbers, in a way that ANY file missing might cause the jenkins server to not boot... I am fucked, especially so because there is no way to backup atomically 100 GB of data across 100000 files and directories while builds are running.
> This really isn't important. The CI server being out for a few hours will not bring down the company. If you have that many projects then there is no problem with having multiple build servers.
Given your arguments of all things being useless. You seem to be working in a very small company or not have worked professionally at all.
Please try to envision a real company: When the CI system breaks, there are 100 people who are interrupted and they will be partially unable to work until it is fixed.
By The Way: It's easier to maintain and manage one big system. By splitting one CI pipeline into 10 CI, you just created 9 SPOF and you just increased the failure rate dramatically.
>>Build in monitoring in the CI master itself (e.g. manage the space taken by projects and artifacts) with a notifications when things are going critical.
>Like the backups, this should be part of a general OS monitoring, no need to complicate the CI server
It is the responsibility of one's system that the system is reliable.
First) CI servers have disk space issues. I expect a sensible way to find out what projects/builds are taking and the ability to clean it up. [a monitoring system will give full disk alert, it doesn't understand the CI internal organization]
Second) If the dev are not retarded, they can put a notification for everyone to see when the disk is at 99% instead of writing the last % and crashing... only to never reboot again because it corrupted itself in the process.
> It's interesting to me how much effort is being spent on adding tools ... vs just providing good hooks for standalone tools that do all those things.
This is very important. From the GL's PoV an integrated suite sounds great because the integration is easier and you can capture more of the value chain. But not only is it a barrier to adoption, if successful it squeezes out innovation. The whole point of open source is user benefit, right?
Why is it a barrier to adoption? Our intend is to make it easy and possible to use only part of GitLab, whilst integrating with your other tools.
The integration with our own tools will always be better, because we can do things that'd be hard with external tools, but if you want to use something else for anything, it'll work with GitLab.
> Our intend is to make it easy and possible to use only part of GitLab, whilst integrating with your other tools. ... if you want to use something else for anything, it'll work with GitLab.
If that's your intent then my statement is not operative, great. But your other public statements imply (to my reading) the opposite of what you say in your comment. For example, from the announcement:
> We prefer to offer an integrated set of tools instead of a network of services or offering plugins for the following reasons:
and some comments from your colleague theunquietone:
> We believe in a single solution from idea to production.
and when someone commented "Makes for a tough pitch at an established shop"
he replied: "We hope to create tools and resources to make that pitch and migration easier."
It's early days so you have time to clarify/refine your story.
There are hooks, services, and OAuth2 support, as well as the API, which internally we leverage to extend GitLab or integrate with other products, kind of the way GitLab CI did before it was merged.
Come to think of it, services like Travis CI could integrate with gitlab.com easily (if they wanted to do so).
Our goal is to ship a single tool that will work for (almost) everyone. We believe that this will be a better experience and allow for deeper integration, besides cost, setup and maintenance advantages.
That said, we don't expect to be able to replace every single tool one might use -such as internal tools-, therefore we make it possible to integrate with GitLab easily and have an API that covers everything.
Examples of this are plenty. For instance, we build our commit status API to support our own CI system, but immediately also published the public API that allows any other CI to make use of this and offer similar integration in GitLab (in that case, you'll still have to switch between apps, which we believe can be better - hence our goal).
At work, the worst thing we've done this year from a product management standpoint is migrate from Github Issues to JIRA.
It was done for the right reason: to have a single place for all our work (JIRA for feature planning/sprint planning/dev+qa tracking, Confluence for documentation) but it's been hobbled because we still need github to coordinate code review + merges.
The better solution would have been to stay with Github and use Huboard or something to layer on the swimlane sprint planny stuff, but keep _all_ our info in GH Issues.
Moving entirely to Gitlab in a year or so once things have matured would be a great win for us, I'm sure.
Is not whole Atlassians business a unified toolset for all of this? I would not say that nobody actually tried to be that one place... I only seeing GitLab beeing the best open source solution without making a big risk with vendor lock in. And they are also cheaper than their competitors (github, Atlassian etc.) when you want to do private team development.
I wouldn't say Atlassian has worked to make a great unified UX. I mean, they tried, but using their products makes it painfully clear that they just glued togeather a bunch of disparate projects. For example, each product supports different markup languages (or none at all) with different editor buttons and shortcuts and so on. Even the github ecosystem feels more integrated, because everyone just follows the broad strokes of what github does with its UX and integrates with it.
As someone who has been doing product development their entire career, I have found there to be an economy of focus in a software company when it comes to product development. I think it's a great vision for Gitlab but it's risky because it splits their focus into like 6 beams. The major players in the market (Atlassian, GitHub) already have this vision and they are there already pretty much.
The market wants flexibility in their tools in spite of what people on HN tell you. Atlassian has tools for every lifecycle stage but people still mix and match with other tools. Github is missing a few of these stages but I'm sure they are hard at work filling in the blanks. I know people who use GitHub and JIRA together even though GitHub has it's own issue tracker.
I agree it would be risky to go on this path. So when Dmitriy wanted to make CI years ago I told him to keep the main thing the main thing. Why reinvent the CI wheel, our efforts were better spent making our Jenkins integration better. But I didn't stop him.
And what I learned was that we were able to do CI and make it a better product that what was on the market. That was counterintuitive for me.
But we've since seen the benefits of integrating CI, CD, issue boards, and a docker container registry. We still have lots of work to do integrating Mattermost, Koding, Cog, Prometheus. But we're convinced we can do a better job.
> Most of the pain around developer/business workflow around us at the moment actually comes down to the fact that nobody has _really_ thought about providing a great unified UX for all of this.
Microsoft did with TFS. That was enough to convince me it's a terrible idea.
File editing/locking/merging isn't great, particularly if we've got Excel in our Source Control. We can either have everyone lock files or everyone merge. You can't merge Excel files so it means everyone has to lock or we store Excel elsewhere.
Unix Client is flaky and inconsistent and doesn't get much love.
I can't put a version number into a file so I can see in Prod what version the code is.
We are currently on Github and are using CoreOS' Quay.io private Docker registry with Github hooks for automated build creation. Total cost: $50/month for Github for private repos, and $100/month for CoreOS' managed service that includes Quay.io.
Years ago, I did set up our own private Docker registry and build server, but it was a lot of work to setup and maintain, so I killed it. Hopefully and probably, that's become easier to do today.
However, last night, I decided that I had had enough with our current setup (it's slow, expensive, and cumbersome), and moved to GitLab. Here were the steps:
1. Create an account.
2. Create a repository, and select the option to import from Github.
3. Connect my Github account, and import all of our private repositories.
4. Ensure Container Registry was enabled for the repositories.
9. Start a build to ensure Docker image gets built.
In less than 10 steps, I was able to migrate all of our code and CI to GitLab, in less than 2 hours. With the repository mirroring, Github can remain synced to GitLab, so that I have time to modify our deployment scripts to use the GitLab URLs instead. I'll be cancelling our GitHub and CoreOS subscriptions this week.
As our team grows, we will likely host GitLab on our own servers, and I expect that will go smoothly. I'll be happy to pay them at that time. Right now, I'm really happy with this migration.
I would love to hear how this is running for you after a month -- when some of the edge cases have been poked. It would be good to know if you run into problems or if it is still going as smoothly as the initial setup.
I switched over to GitLab for a static blog when I realized I could use the free shared runners to compile any static site generator out there. Such a beautiful feature. I would be interested to hear how it holds up in a production environment.
We've been using it in production for months, and it's as great as it was the first day. I've also made a static blog with the Gitlab runner, and my experience has been similarly positive.
Do either of you mind linking to your static blogs or an article discussing your process here? I've only ever used a wordpress install for blogging but have been thinking about just building a static blog from scratch.
Building a static blog from scratch is just writing HTML files for your posts, and then linking them them to a list of pages, placing it at the top. The classic example of this is JWZ's "gruntle," or jwzrants, which actually predates the term blog. You can find it at https://www.jwz.org/gruntle/.
However, most of us prefer to generate such things from templates and markup files. The most popular system for doing this, by far, is Jekyll (https://jekyllrb.com/), probably because Github recommends it. Metalsmith (http://www.metalsmith.io/) and Wintersmith (http://wintersmith.io/), while not as popular, are certainly options if you're a JS fan, and don't mind cutting code to write your configurations, and have some very nice features for customization. However, if you're coming from Wordpress, I'd reccomend Lektor (https://www.getlektor.com/). It's Python based, well documented, and uses a simple file tree to build the website like all other options discussed. However, it also provides a simple admin panel, which can be used to write your posts, manage your posts, and publish the generated site to your server.
As for distributing a static site, you just have to stick it onto some kind of hosting. S3, Apache, Github pages, whatever. Once again, Lektor can do this via the admin panel. For the rest of the generators, you're on your own: you'll have to write the scripts yourself, but this is fairly trivial.
We have a bunch of working example sites in a bunch of different static site generators at the "pages" group: https://gitlab.com/groups/pages
Jekyll, Middleman, Pelican, Metalsmith, Octopress, Nanoc, etc. etc. All working examples. We also have gitlab-ci.yml templates if you create the YML file via the web editor, so for many of these you can get a simple site deploying pretty quick.
That's exactly my experience. I used the docker omnibus containers to get quickly up and running. Running, supervising and updating is a pleasure in combination with docker-compose.
As for CI, I utilized the docker-machine runner so we could autoscale based on demand.
Big plus: With Google Cloud Platform, we can leverage preemptible VMs for the runners (not for the coordinator of course), so the cost is really low.
Awesome, if you're using Google Cloud you'll be happy to hear that we just merged Kubernetes support for Runner creation https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_r... Giving you the option to use Google Container engine to spin up new Runners.
This is fantastic. Right now I use GCE autoscaling groups to spin up runners while I've got several kubernetes clusters running 24/7. It'll be nice to utilize those for builds (especially with how fast the scheduler is) going forward. Saves money, as well.
I'm working on an open source project that does just that: deploy container updates. It's not available just yet, but you can follow me for when I announce it.
https://twitter.com/jadbox
Gitlab-ci can do this easily with the gitlab-ci-multirunner docker image. My CI/CD spins up multirunners in autoscaling groups and those said multirunners (it's basically a docker container with privilege mode to spin up the Dockerfiles of whatever you're building) handle the builds, whether they're node or ruby or anything else I've made into docker images.
Right now, Gerrit and Phabricator give an almost unrivalled code review experience - the ability to queue multiple comments up and submit them at once, mark previous things as fixed, keep track of a patch as it evolves along with the comments that were made against it. However the Github/Gitlab PR flow is easier for most occasional contributors.
My question to Gitlabbers following this thread: do you have anything in the works for improving code review to better match some of the use cases which are handled so well by the Gerrit/Phabricator approach?
Phabricator is a really solid GitHub alternative. Some of the largest open source projects are using it and lots of companies.
I believe that the GitHub pull request workflow and issue tracker does not scale to large projects and very much prefer Gerrit/Phabricator. Phabricator was created at Facebook for a large development team (mobile apps) and it shows. They have solutions for issues which only appear at scale (code ownership in monorepos, for example - the owners/herald mechanism). It's also very performant, has almost no dependencies and is extremely easy to deploy.
GitLab is a great GitHub alternative, but it's also copying
its flaws.
> As far as code reviews go, this is pretty spot on. I was part of a a startup that was using GitHub pull requests for code reviews. As the team grew, it became more and more intractable, although not simply because of notifications. Side-by-side diffs and checkpointed diffs (so that you can see what changed since the last round of review and whether/how your comments were addressed) are handled very poorly by GitHub. We ultimately switched to Phabricator, and while there was a little friction as folks got acquainted with the new tool, it made code reviews a much more pleasant process.
Recently, I had to go through a full code review back on GH pull requests, and it felt like pulling teeth in comparison. They're fine for interacting with contributors to an open source project, but compared to working with a tool like Phabricator that's built for a code reviewer's workflow (and for teams of engineers working together on a project), they just don't hold a candle, in my opinion.
We have many requests to improve our code review and make it more like Phabricator and Gerrit.
We want to strike a balance between light-weight and powerful, but fully intend to give you all the power of a Phabricator or Gerrit in GitLab. We're working on several [0] initiatives towards this goal.
If you are interested in seeing these improvements in GitLab, please let us know what is important and what we're not seeing!
I'm not a GitHub user myself, only have experience with GitHub. The article I linked outlined it very well.
Essentially, what I need is code ownership (like Phabricator's "Owner" feature or Chromium's OWNER file: https://www.chromium.org/developers/owners-files) and essentially what's outlined in the article I linked (diffs instead of heavyweight merge requests).
Absolutely! We want GitLab code review to be the best there is.
It's hard to find a good balance between light-weight and powerful code review tools, but I think we're finding the balance by making small iterative improvements.
I collect most improvements here [0], I just added your request under 'transaction 'batch' comments'.
I miss this functionality when reviewing large Github PR's.
A PR can quickly contain thousands of changed (added/removed) lines of code. When reviewing a PR, I want to keep track of which lines I've already approved. I would love to see a way to mark code blocks or entire commits as "reviewed". This is especially handy because PR diff's are shown alphabetically by file path, but the code paths obviously aren't, so it's really hard to keep track of review progress.
Because then the reviews for the connected PRs ends up being hideous to track.
Either you can:
• Submit the PRs one at a time... which is awful. Why not work on them in parallel?
• Submit your series of PRs all at once, then you have to set the PR base of each one to the following one to make the github diff view actually reasonable for reviews. THEN, when you have a code change based on feedback on the first PR, you have to merge it into all the later dependent PRs. THEN, you have to have to jiggle the PR bases very carefully when you're landing the PRs, otherwise it will merge them all into each other, and you end up with one ginormous commit landing—defeating the point of the small commits. AND unless you're using a squash based workflow, you'll end up with that giant merge mess in the middle of your commit history.
Posting multiple comments and submit them at once would be awesome, I can't find an issue for it right now, but Ill create one so you can track it.
Merge Request versions is something we plan on shipping this release[1]. Resolving a discussion on a diff has been shipped with 8.11[2]. We'd love to hear your feedback on it, as we know we how good Gerrit and Phabricator are in this regard.
If there is another feature you'd like in GitLab too, we would love to hear more on it.
We (NVIDIA) recently moved away from Quay/Github/Jenkins to Gitlab for our deep learning automation and the experience so far has been truly amazing. We were able to automate our most complex DL container pipeline in a matter of days. We still have to workaround some Gitlab limitations (e.g. issues [CE]17069, [CE]18994, [CE]18106, [EE]224) but overall it's great to see everything working in harmony (i.e. Docker registry, CI pipelines, Git repositories, Runners on-premises). On a personal note, I would like to see more storage on Githost.io instances considering the fact that you can't easily delete pipeline traces and that Docker images can quickly add up.
Thank you so much for commenting. It is great to hear that the deep learning automation department of Nvidia is using GitLab and is happy with everything working in harmony.
Please comment in the issues if you have additional details about the use case or questions.
The costs of GitHost.io correlate with the storage since they are Digital Ocean instances. Not sure how to solve. Maybe by allowing to use their networked storage, but this seems complex. Consider emailing support@gitlab.com if you have any questions or suggestions.
You can also switch your container registry to use S3, which might be more cost-effective. I'm not positive if GitHost.io supports that, but it likely does.
We use heavily gitlab-CE for VideoLAN (VLC, x264) on our hosted infra, and so far we're very happy about it.
The only part blocking us from moving completely to gitlab and deprecate everything else is mostly the limited issues tracker (compared to trac, for example).
Well, mostly, I believe we need support for custom fields, so we have for each ticket, the Platform, Component, Branch, Type, etc. And those fields can be queried or made mandatory.
Not OP, but I'm on the same boat. We've been using a self-hosted GitLab CE instance for more than three years now, but we can't use the bare-bones approach of GitLab for issues tracking.
Trac allows us to fully customize the issues workflow, even for individual projects. We can have custom fields. We use several plugins (the extensibility of Trac is second to none!), even a few developed in-house, implementing custom validations and/or custom functionality.
Trac is very lightweight (as GitLab issues), but much, much more powerful and flexible. When you need the extra functionality, you can enable it, or install some 3rd party plugin that provides it, or develop your own plugin or component (which is really easy), without turning the whole thing in a bloated mess (like Jira).
The only thing I don't like about Trac is its glacial development speed, and its UI, which turns older and more outdated every year...
Sure. Please note I'm referring to Trac's UI, not GitLab's.
* it sucks on mobile. It's no responsive. At all;
* it's 2016 and I can't drag&drop a file to upload an attachment. Yes, there are a couple of 3rd party plugins that solve this, but this is one of those components that should have been added to the core ages ago;
* visualizing the work to do is harder than it should. A "visual" panel (like the one introduced by GitLab in its latest version) with drag&drop would be awesome.
* it's hard to customize. Things like changing colors, icons, padding, alignment, fonts are not at all straightforward unless you're willing to overwrite quite a lot of Trac's core CSS, and deal with the problems that might arise when you update. Trac rocks in extensibility and flexibility in almost every aspect, except for its UI and layout;
Looks nice indeed. I've used wiki pages with predefined ticket queries in different "panels", but they are static and non-interactive. I've also used a drag-and-drop "kanban cards" plugin, but without ticket queries/updates. I'd actually be a bit scared that ticket updates by drag and drop would lead to a lot of clutter in the ticket history. In the end I now mostly use the custom ticket query UI, although it sure could be improved.
Just a few years ago I was in love with Github and felt nothing could ever displace it. But as time went on, they just fell asleep at the wheel. I've been using Gitlab for the last six months and there is no chance I would ever go back to being a paying customer of Github. I'm so glad to hear GitLab raised more money to keep going!
As a customer, I love the integrated CI and how easily I can now run tests for each commit and show pass/fail and coverage information right on the commit. We used to use Stash and Jenkins, but it was pretty hard to get the integration to work reliably.
Another thing I love is the gitlab-runner, which lets me spin up a machine and have another server to run tests on.
What I really don't love is how limited issues feel compared to JIRA. Namely, the following workflow is so hard it might as well be impossible:
* Issue gets opened by a user/assigned to a developer.
* Developer has a view that's only issues that they haven't worked on or that they're working on.
* After the issue has been worked on, the developer can assign it back to QA.
* QA retests and closes.
This was very well served with JIRA's open -> in progress -> resolved -> closed pipeline, as well as with the views that could filter things like "show me issues assigned to me or nobody, that are not resolved or closed, ordered by priority".
We tried to do something similar with labels and the board, but it's really inconvenient, because we can't filter for "show me issues that are NOT <something>". We used a "new" label instead, but the board doesn't know that "new" is the backlog and that that label should be removed when the issue moves out of the former.
In general, we've been having problems with issues, although the milestone view is pretty fantastic.
> show me issues assigned to me or nobody, that are not resolved or closed, ordered by priority
Right now, GitLab orders by priority out of the box on the board. You can filter for yourself or assigned to nobody. Not both, but we're working on improving the filters (including the option to do "NOT X OR Y". [0]
> We used a "new" label instead, but the board doesn't know that "new" is the backlog and that that label should be removed when the issue moves out of the former.
Just create a list with the 'new' label and you should be good, right?
We're thinking about having different types of lists in Boards, such as 'Milestone boards' and 'Status boards'. Something like that could solve some of your requests as well.
> Just create a list with the 'new' label and you should be good, right?
Yes, but then you have a "backlog" and a "new" list, which are no different, and yet distinct :/
> Not both, but we're working on improving the filters (including the option to do "NOT X OR Y". [0]
That's fantastic, logical operators would be a great feature.
> We're thinking about having different types of lists in Boards, such as 'Milestone boards' and 'Status boards'. Something like that could solve some of your requests as well.
I think that, if you have powerful filtering functionality, then everything could just be a filter. E.g. a board would be a list of filters with two actions or an associated label (i.e. "show me all issues that are X AND Y (OR NOT Z)", and assign label B when an issue is added here and unassign it when removed). Issue lists could similarly just be a filter view ("show me the issues with assignee=me or assignee=none and status=new or not status=closed"), etc.
There are so many good things that it would be time consuming to make a list, but your general attitude and philosophy percolates through the product.
What could be better? Time management and planning (which is different from deadlines).
We're dreaming of some sort of "automatic milestones" (we use weeks) that allow us to put issues in "next time unit of work" (and eventually "subsequent time units of work") at the beginning of the week. Currently we create (and close) "%year-%week"-numbered milestones each week, for each relevant project, and that's extremely heavyweight to manage.
Such a workflow where you fill buckets by eyeballing tasks allows us to have a coarse grained estimate of when will things be done (weeks are quite a natural unit for that esp. when communicating to upper management or customers in a language they speak: "we don't have room for week 36 so this'll be done by week 37") as well as a bird's eye view with the cross-project milestones.
A very interesting idea! Feel free to suggest a minimal implementation for this and we'd be happy to look into building it. I shy away from introducing too many new variables, so something that could leverage milestones would be probably the best.
Not closely related, but you hit the keyword 'Time', we're planning to build time tracking for GitLab soon [0].
> Feel free to suggest a minimal implementation for this and we'd be happy to look into building it. I shy away from introducing too many new variables, so something that could leverage milestones would be probably the best.
Agreed. I may look into it but I've got an unending backlog of FOSS things (with a couple pertaining to GitLab[0]) and while this is of high interest internally work on this is restricted by policy, so that's solely on my spare time.
> Not closely related, but you hit the keyword 'Time', we're planning to build time tracking for GitLab soon
The fact that I can use the CI in one tab to build docker images in another tab and host them right next to the repository in another tab was absolutely killer.
Also, GitLab community is free (!) and imports very well from GitHub.
I love that I can just to this in .gitlab-ci.yml before pushing:
One middle-to-major annoyance I still have with gitlab though, is that merge requests across forks aren't visible from any automation via the destination repo. That is, if a bot account has access to the prod repo to perform CI builds, and somebody opens a MR from a private fork, the bot can't see it.
This has made us adjust our workflow for pretty much all prod repos to push branches to be reviewed under a review/* branch on the destination repo, which is cumbersome and requires additional permissions.
GitHub makes all pull requests available as branches on the destination repo via the refs/pulls/* namespace, and that's been really useful. It'd be awesome if gitlab had this too!
~~Ah this is a good improvement that we should definitely pick up on the short term. I've added it to [0] for now, but I'll look into this more later, I'm sure that we've thought / planned about this before.~~
I can only echo the comments here - switching to GL has been fantastic all round. I welcome the addition of other tools into the product. The closer the tools are the more effective they become.
Honestly, installing and maintaining GitLab is still not a pleasure. The Omnibus package was too unsafe for us because we wanted greater control about versions and stuff; and the bundled pre-configured applications in it (e.g. nginx) cause a lot of problems with existing installations.
We were basically not able to create a stable deployment on a machine also running other services. But none of use really likes Ruby and rvm, maybe that's also one of the reasons why we struggled.
I'd love for you to explain your motives behind not using Omnibus further.
Nowadays we provide images / containers for most major platforms, so there should definitely be a good solution for you there.
That said, we make the Omnibus packages as that's the easiest and best way to maintain GitLab. It takes all the work away from you. The vast majority of our customers use Omnibus, even secure instances with thousands of developers.
Not OP, but I dislike many things about your omnibus packages. I hate packages that install into /opt (thanks for the extra tld). I hate that it's 900+MB of completely redundant junk—nginx? check, postgres, check, and entire ruby? git? curl? All of those are on my system already.
That said, I switched to it after maintaining a hand installed version and I love how I don't have to run rails schema upgrades by hand any more. I used to put off upgrading gitlab for months because it was such a pain and now I don't have to think about it at all. Rails devs could learn a lot from php devs in that regard (upgrading wordpress is absolutely trivial to upgrading gitlab).
So yeah, I love the omnibus package since it means I don't have to think about it any more; I hate the omnibus package because it's a huge redundant monolith and pure crap from the point of view of a Debian package purist.
This is a nuisance for me as a CentOS admin as well. I absolutely dislike software that decides to just plop a big bag of crap in /opt. I don't want something administering it's own PostgreSQL or nginx instance, let those choices and the administration of these services up to system administrators and alternatively provide a quick setup tool for the less inclined.
I work in a mostly .Net shop, but the couple Python+Flask apps I maintain and administer integrate with everything else the way I expect as a sysadmin. As a Linux guy I have scoffed at Windows shops for ages for the sin of "xcopy deploy" but omnibus packages are no better.
quick question - did you consider using jruby ? I see on your docs you mention that you use certain libs with native extensions... but 20 mil should go a long way towards resolving those.
how about building out a single jar based deployment based on jruby ? For example, mingle is built on jruby (ruby on rails) and packaged as a easy to install jar file [1]
jruby will be higher performing and will use overall lesser memory in general (yes.. the jvm takes up quite a bit of RAM during startup).
Running a jar in production using jetty is a breeze.
Not sure where your problem is. I run gitlab with mattermost and docker registry with my own nginx that also servers a lot of other stuff. You can disable the internal nginx for all application in gitlab.rb and you find the nginx.conf files in the gitlab-omnibus repository. It's not one-click but but it's pretty much more than I have hoped for.
It's also pretty self-contained you don't have to care about ruby and rvm and stuff like that, daemons are supervised with runit. I don't see why you have the need to switch versions on that.
Even if you want to modify it, it's probably easier to maintain a fork of the chef-cookbooks and apply local changes to that.
I'm not saying it couldn't be better but for opensource software it's quite well managed and packaged and documented, at least in my opinion.
Yup, the documentation sufficiently covered our case- we needed the front-end to be Apache httpd for our SSO, and there were steps (and example httpd configs) to substitute it in for the nginx in the omnibus package.
> You can disable the internal nginx for all application
Yeah, we have tried that but that didn't work for us.
Instead, the internal nginx flooded our hard drive with log files, that was a bit annoying, haha.
This is also the biggest pain point for me. I had it running on Ubuntu and didn't like the 'we ship nginx' philosophy so tried to install it on arch. After fixing a few obscure problems, I went with GOGS and it installed flawlessly in under five minutes. It's unfortunate because gitlab has a lot to offer.
These days I run all the various different services as different VM's, then have a single reverse proxy that points at the various instances. This way I can make sure that I don't accidentally need a newer version of some library than some other app causing conflicts.
At that point the internal shipped NGINX can be there and be happy without issues.
Yeah, that's something we learned from it.
At that time, though, we just wanted to replace Redmine and only had a single machine available which was already running a bunch of things.
I've been running the Omnibus for over a year and only ran into an issue with updating a few times and they were typically 500 errors caused by a gitlab bug, and in one case an issue with the database migration needing some tweaks. It's (their omnibus) been far and away one of the most seamless and impressive third-party installation and update systems I've used.
Conclusion from reading: Gitlab is just another open-source company that raised money and is now desperate to monetize it's products.
Unfortunately for them. The market is already saturated with CI tools, including good ones.
- If you want good self-hosted CI, you use teamcity (jetbrains) or bamboo (atlassian). Side note: They cost money, you get what you pay for.
- If you want good SaaS CI, you use travis-ci (linux), circle-ci (linux) or appveyor (windows).
- If you want to suffer endlessly, you use Jenkins (previously hudson). It's shit, it has a Bad UI, it's an aggregation of poorly maintained plugins, it lack even the most simple features, the list goes on...
- If you want to go exotic, you can find dozens of other [partial] CI tools.
There is no room for gitlab. Teamcity already has a free edition offering 20 projects and 3 slaves. All the aforementioned tools are free for open-source projects.
Disclaimer: I have used all the tools mentioned above.
-----
The reason the good tools are not popular is:
1) they cost money and people are bitches when it comes to spending even $10
2) most people start with the old well-known shitty tools and then they're locked in... and the efforts required to move away just increase over time (sadly, nobody got fired for choosing Jenkins in the first place :( )
I've used just about all of the tools listed above as well and today happily I use gitlab for nearly all of them as opposed to miserably using some of them in piecework together.
> - If you want good self-hosted CI, you use teamcity (jetbrains) or bamboo (atlassian). Side note: They cost money, you get what you pay for.
Those are proprietary.
> - If you want good SaaS CI, you use travis-ci (linux), circle-ci (linux) or appveyor (windows).
I believe all of those are proprietary.
> There is no room for gitlab. Teamcity already has a free edition offering 20 projects and 3 slaves. All the aforementioned tools are free for open-source projects.
As far as I can tell, every project you mentioned except Jenkins is proprietary. And as much as I like Jenkins, you're right that it can be a proper pain to deal with. So you're just showing that there is a need for a good, free software, CI system. And GitLab is working on providing one.
It's really hard to charge money, especially against a truckload of both free AND paid tools which do the job.
They decided to NOT charge money on the number of projects or the number of runners. They've made a product with no incentive whatsoever to pay for it.
We could say it's great, we get get another free tool! but how are they going to make this work on the long term?
They have no business model and the investor funds will eventually run out at some point. Then, we'll be left with another free open source abandoned projects, just like jenkins?
Their business model is the enterprise edition which comes with better support and newer "enterprisey" features first. It seems to be doing really well given their comments here. Gitlab.com is a loss leader to get their potential users excited about gitlab so they convince the potential buyers to pay for it.
Perhaps I'm in the minority but I have actually setup my home system to use SSO (backed via LDAP).
It annoys me no end that all these services seem to think that charging to maintain my userbase is a way to get me to think they care.
What it means is it invariably the SSO support is barely tested and poorly documented. For a number of products, I've tried out the "enterprise" version and realised that their features (e.g. SSO, audit, etc.) are just not useful.
Largely because there are not enough people looking at the thing or attempting to use it.
You seem to be focusing only on their public instance gitlab.com which is comparable to github.com. They essentially have the same model as GitHub(free for the community, payed for enterprise/companies). From where I am standing they seem to be giving GitHub some healthy competition in both areas.
I've been using Gitlab CI for more than two months now, and it's incredibly simple to set up and really powerful specially with the integration with docker, also, self-hosted gitlab is free.
I also have Teamcity where I work and the comparison is not even fair, I set up gitlab in one day to serve as a continuous integration/deployment server
Also as the frontend lead, our focus will always be the user experience first and this is getting better every single day. Obviously you can use tons of products with a "bad UI", but you'll find that using GitLab will increase your life expectancy by lowering your stress level.
Congrats guys!
I'm curious if you are going to change directions in technology - ruby is fairly unsuited for on-premise deployment.
Before I get flamed, of course you guys have made a great build and deployment system... but nothing can beat a " java -jar start.jar" or "./golang" . And I think it comes a fair bit of performance for free.
Wonder what are your thoughts around that ? I keep thinking that Gitlab could be the "killer app" for a new fangled java framework like SparkJava or something.
Ruby, compared to some compiled languages, is indeed harder to ship, which is why we've got the gitlab-omnibus package.[1] Thanks to our packaging team the install and getting GitLab up and running shouldn't take over 10 minutes. Maybe not as easy as compile and run like Go offers, but I urge you to try it, it is really simple.
Now, we use more and more Go within GitLab. For example, gitlab-workhorse[2] and the gitlab runner[3] are written in it. But converting our main app, gitlab-ce and gitlab-ee, to another framework and language would makes us unable to ship new features for at least a year. Even when we would gradually rewrite this would hurt our ability to consistently ship new great features.
Also, please don't forget that Ruby and Ruby on Rails are very mature, stable, and so far have served us very well and I expect it will for the next years.
We might, as we've been doing for some time, let workhorse handle more compute expensive operations, but again, other than that I don't see it happening any time soon.
Anything can beat a java -jar start.jar! This is my main quarrel with loading anything like Confluence or JIRA myself instead of using their cloud-based tools. Huge memory/cpu hogs, requires additional configurations for connecting to Postgres or MariaDB, etc.
Not to mention, once you have `java -jar start.jar`, you're only 10% of the way to discovering all the command line options you need to pass this particular java app for it to behave itself properly.
On a related note, I'd like to host GitLab on my own server, but it's the cheapest Linode and I've heard that GitLab requires a somewhat beefy server. I'm currently using Gogs which is really light but also very simple.
We would seriously consider GitLab if they supported Mercurial. I am so annoyed with atlassian (another company that perhaps could pay attention more to developers). I like git for OSS but we love hg for our internal projects.
It would severely hurt our velocity if we'd decided to support Mercurial, whilst the benefit is only for a small group of potential users. We're not planning to support hg.
A git migration tool would be terrific: I'm on a svn project with a 20Gb history, 3Gb working copy, with TONS of versioned binary assets. The migration story to git is a nightmare with lots of different tools and utilities that reportedly do some part of the migration, but don't play well together.
I'd like to see a simple conversion tool that lets me configure everything (branch filters, user conversion, wildcards for what assets must go to LFS) that lets me convert a repo in TFS/SVN/HG into git with LFS.
The key here is integrated LFS. Without it, git is a complete no go for any shops that does lots of large versioned binaries. Git+LFS still seems like black magic to me, so it would be nice to see someone make a friendlier front for it.
> Without it, git is a complete no go for any shops that does lots of large versioned binaries.
Git is source control. It's for versioning source, not binaries. Built dependencies or artifacts or whatever else should be stored somewhere else like FTP.
For some of us, binaries are source. For example, anyone developing in Unity or Unreal Engine has large binary files that represent the game levels, blueprints, and other assets. These aren't generated files, logically they are source code that should be versioned along with the textual source code.
(Tip for Unity users: The very first thing to do when you start a project is to go into the project settings and change it to Text Assets. That way the scene files and such will be text instead of binary. It's still a fairly opaque YAML format, but at least they will be more diffable.)
Build artifacts aren't necessary to bring over to git in a conversion from hg/svn. I archive installers and other artifacts in svn, but if I moved to git I'd just archive those somewhere else. The binaries that are actually part of the build (a few binary dependencies, lots of graphics, test in/out data etc) must be next to the source however because those are needed to reproduce a build.
That's why I (and many others) need Git+LFS - because I need a lot of versioned artifacts such as images and binary application documents. One example is test inputs and outputs if you have an app that produces and consumes binary data. Every revision of the source tree needs the right revision of the corresponding binaries.
You are right that this is solvable with ftp, with one folder per revision and some scripting to manage committing source changes to git and binary changes to ftp but that is pretty much what git+LFS is!
Right, which is why I replied in the first place. The parent seemed to think LFS was black magic and didn't want to adopt git without it. In reality LFS is just another versioned file storage scheme similar to a Nexus repo or Linux distro package repo. Any FTP server and a script to push artifacts at the end of a build would get the job done.
I meant it seems like black magic to get it set up properly. Especially on windows.
> Any FTP server and a script to push artifacts at the end of a build
The binaries in this case aren't produced by the build, they are test input data, test output data, graphics files etc., and without them a developers' build or tests will likely fail.
So any time a developer updates his working copy (git pull, svn update, whatever) the the binaries need to be in sync, preferably automatically, since otherwise the source code itself is useless. It seems to me that LFS was designed to allow the user to not download all the binaries, but instead leave the small pointer files. That might be handy, but in my scenario I need ALL the binaries in place and up to date all the time. Basically I need to alias any syncing to also do the git lfs fetch.
Any time the user commits changes (svn commit, git push etc) the binaries must also be pushed.
I'd be very happy to adopt git but for acceptance within the organization it needs that seamless integration of binary versioning without extra steps, and also file locking.
I'm not the original poster but some of us prefer Mercurial's workflow to git's, so a migration tool, while nice to have as an option, isn't what would help win customers like us. (I do like the direction that GitLab is taking, though.)
As an aside to the general HN readership, are there any good Mercurial hosts other than BitBucket?
RhodeCode is self-hosted, but you can run it in Azure/Amazon cloud. For large companies, that setup might prove even more stable than relying on someone else's cloud [1]
Marcin from RhodeCode here. You should consider checking out RhodeCode, it's openSource and supports Mercurial, along with Git and Subversion. It's built explicitly for internal usage behind the firewall.
It allows you to use hg as a git client. It supports git branches (mapped to hg bookmarks), tags, etc and is mostly pretty seamless, with a few rough edges where the semantics don't quite match up --- e.g., hg supports moving tags, which git doesn't like doing.
I use it with github, and have done some pretty complex stuff with it, and nearly everything has Just Worked. The biggest problem is with the bookmark-to-branch mapping, so I tend to create/delete branches via the github web UI, then pull the repository to update the client, but I'm totally willing to admit that I've never really bothered to learn much about how that works so I could just not be driving it properly.
Big file support that works transparently without ever giving the feeling that it's tacked on. Real file locking support for those binaries is also critical.
I think git LFS is having same issues as Mercurial Large Files had in the beginning. It's going to get there, but it takes time. We're considering that Mercurial LargeFiles support is much more mature.
In regards to Git vs HG. We see that most of the users take advantage of plugin system that Mercurial has, eg Facebook with their plugins for handling huge monorepos.
It also adds some advanced workflows, eg doing a code-review using phases and secret commits makes it much more pleasant to work compared to Git.
FileLocking is quite easy with Mercurial actually.
We're working on some other approach on this actually and want to work on a better system that emulates what SVN/Perforce have and bring it to Git and Mercurial with RhodeCode as the tool to have at least basic file permissions.
Not OP, but I tried Annex with Unreal 4 and the symlinks caused problems with the editor. Obviously that's a UE4 issue rather than an Annex issue, but it meant LFS was the only reasonable option at the time when I tested (~12 months ago).
Maybe I just haven't been following the news, but since when does Y Combinator do Series B rounds? I thought they (you?) were exclusively seed funders.
I've been using the GitLab Community Edition for about ~1 yr. It's an amazing experience so far. Even the upgrade is really simple (just a apt-get upgrade!). Best of luck to you guys.
GitLab has made a lot of great inroads and will continue to do so. It's all about momentum. Of course people are going the tools that are familar to themselves. However as more and more start to adopt GitLab (probably as a github replacement or small personal projects), they are going to ask for that more in the corporate world.
It's also a matter of developing the maturity of the integrated tools. CI for a example. Does jenkins have a shit load more features? Yes. Does Jenkins do scheduled tasks? Yes. But jenkins is also a massive ugly, unwieldy behemoth. Some issues like the Cronjob portion can be worked around in GitLab (webcron to build trigger APIs) but it's not as nice... yet. The CI feature is barely an infant compared to the age of Jenkins. However for 90% of what I need? Works great. Well worth it to me.
Every month they've been releasing improvements on numerous fronts and it's been amazing to see. The product has gone from "eh" to "wow" in about a year. I look forward to seeing if they can sustain this product growth (I'm guessing the 20mil will help).
Gitlab's design is fine. Hiring new designers now will only lead them to change everything to prove themselves and make things worse and unpractical in the process, Google style.
I kind of dislike the change that happened a couple of months ago, where instead of going to CI straight from the homepage to see if the build in the only project that is actually using CI is successful, I now have to go to the project (they are sorted by the recent activity, so I have to chase the one with the CI), then into Pipelines, and, from that screen, I can't see which branch was built, so I need one more click to get into the Builds tab.
Other than that, I'm perfectly satisfied with the design.
Glad to hear you like the design overall. Would it help for your use case to add pipelines to the left side menu (global menu) so you can quickly go there?
The navigation bar on the left was much better then on the top. Most current displays are widescreen, so vertical space is more valuable then horizontal space. Unfortunately I can't easily find a "before" screenshot to make a fair comparison, but I feel that the current navigation menu wastes more space and looks worse then the previous one.
We definitely understand the need for serious design. That's exactly why we've raised funding. Currently we have open roles for designer, content design, UX lead, UX researcher, etc. Please let us know if you have any referrals for design candidates.
We have quite a good UI/UX team in my opinion, but there's a lot of surface area to cover. If there's anything specific you think could be improved upon feel free to open an issue. :)
If anything the bottleneck right now is with frontend developers being able to implement features/design improvements, not with designers.
A few years ago I tried really, really hard to get an interview at GitHub, because I had the same vision and wanted to see them expand into new products like CI, and all the stuff that Gitlab is doing now.
Why has GitHub been so stagnant, even though they have so many employees?
It will be great if they add communication tool like slack/hipchat in their master plan.
Reason, I am asking for this, it that I see such tools have critical to do development these days. gitter has already shown initial success/value of integrating with SCM.
We're not planning on building a chat client in GitLab. We think Mattermost is awesome and it is part of our Omnibus package. We're working on better chatops/chatbot integration with Cog https://gitlab.com/gitlab-org/omnibus-gitlab/issues/1412
GitLab plan is "embrace, extend, and extinguish". And I hope they will fail on that, and then, they'll start providing standard integration APIs for third party.
Indeed our company strategy is modelled after MSFT. GitLab 9.0 will be renamed XP and we already have a Vista team in place. They keep demanding a higher expense budget to achieve a Ballmer peak, not sure what that is about.
Anyway, we want to make sure GitLab plays nice with others. there is an extensive API https://docs.gitlab.com/ce/api/ and it includes a commit status api to play well with other CI solutions. Please let us know if there is anything missing in our standard integration APIs for third parties.
Gitlab sounds better day after day. Still we are doing github, Jenkins and AWS ECR (as a container registry
Just a quick idea of mine was:"So github is pretty centralized right now,so is gitlab as well.But I can run gitlab on my own machine. So would it be possible for gitlab to offer a sync service,where I could sync my gitlab server with gitlabs' server? " So if gitlab(github) is down I can ~~let~~ rely on my local copy?
Part of the discussion for the downside of centralization on GitHub is that it is also the issue tracker and wiki. Does GitLab offer mirroring for these services too?
If I understand OP's question, it seems syncing would involve more than just `git push` from the local gitlab server to remote.
I'd pay the guy that has my github/twitter/gmail/hn username on GitLab $500 to give it to me. I don't think he's actively using it, but I can't figure out how to get in touch with him. :(
I really want to switch to GitLab given the focus on tooling and workflow. edit: and also your super nifty tanuki logo, which would make a hip vinyl laptop sticker.
> Does this mean if the account has private repositories, even if unused / dormant, that a username cannot be freed?
Correct. We understand that sometimes users will be dormant for a while and when they have data of any kind, we have to give them the benefit of the doubt.
Please send us an email at support at gitlab to start the process.
Just tried to set up the 2 factor authentication. Easy. But, would it be possible to provide a phone number fallback? The set of printed codes is nice, but the chances of losing them are far higher than losing your phone or having to deal with a corrupt authenticator app.
I like that they aren't providing a phone number as a backup. There has been a lot of discussion recently questioning SMS-based 2FA as a target for social engineering (and other attacks).
Thanks. Phone number backups are hard to set up and there are concerns about the security. That is why we have not build support for it. BTW I recommend using 1password so that you don't have to reset 2FA when you move phones (like with Google Authenticator).
With the addition of trello like board, gitlab has come to a point where I just can't name a new feature I would love to see in it. But then again, like most, I would not have put so many things in it (good ol' unix philosophy) and have been pleasantly surprised about how everything added just fit in.
My only deception about the announcement: after the teasing last week, I would have expected more concrete features to be announced in the "master plan" article (what's the next feature?), but well, I'll wait and see.
Btw, this is certainly not magic. Would love to hear about your dev / product design tips to make something having so many features not being a bloated software.
I don't see anything about Documentation in there? I would really like something better than Confluence woven throughout. Is there something that I'm missing?
Wikis do exist, and you can always have a `docs` directory and then use GitLab CI to package the docs up and/or deploy that directory as a static site.
Please allow me to link to a obligatory XKCD to lighten the subject https://xkcd.com/261/
The master plan reference is inspired by Elon Musk. We can't compare us or our plans with him. But we're inspired that his master plans are public https://www.tesla.com/blog/master-plan-part-deux just like ours.
Haha that XKCD was hilarious. I'm was totally surprised my comment even made it through the spam filters. I was seriously considering spelling Hitler, H*tler or something.
Yeah, I should setup pushing to a remote repo, that could be really helpful. Phabricator can also be configured to push to remotes automatically I think. Either way, thanks for the great product!
I'm really excited for this master plan. I would love to have a just one tool to do most of the job (c9.io would be a better alternative to Koding btw). As of now, Gitlab seems to come the closest when it comes to that vision.
That said, Issue Tracker and Issue Board could be developed further to be more in line with Jira and Trello/clubhouse.io
Apart from that, I love gitlab and keep up the great work!
> Tools such as GitHub and Bitbucket choose the name pull request since the first manual action would be to pull the feature branch. Tools such as GitLab and Gitorious choose the name merge request since that is the final action that is requested of the assignee
I think GitLab are right to choose 'merge request', it has an immediately obvious meaning compared to 'pull request'
Gitlab's terminology is actually closer to the underlying VCS terminology.
git pull <branchname> // fetch <branchname> from a remote host AND merge it into the current branch.
git merge <branchname> // merge <branchname> into the current branch
I suspect it's someone's pet peeve that "Pull Requests" are named incorrectly and they got an opportunity to rename them when building Gitlab.
GL seems great and I'm glad it's evolving. About a year ago I tried GL and it was painfully slow. This was the main reason I chose Phabricator. Have there been any progress on this performance issue? I acknowledge I'm quite outdated here.
We moved to gitlab a couple of months ago (the public one) and it's really slow so we are thinking of having gitlab only for CI and do everything else on github.
Nice to see it's going well. One thing that interests me as an NL-based entrepreneur (well, technically a freelancer for now): are you guys still based in Utrecht or did you have to move to the Valley in order to profit from the investment scene?
During raising the A round I promised I would be in San Francisco the majority of my time to spend time with customers, investors, partners, and press. Being remote only allows me to travel back to the Netherlands 3 times a year for 3 weeks each apart from my vacation time.
Just started working on a sort of getting started mega blog post on CI/CD/CT—is there anyone at GitLab who would be up to review once ready and make sure I'm covering the GitLab offerings completely?
Does GitHub own any intellectual property that might require GitLab to licensing? or did GitHub totally miss the intellectual property piece of the puzzle?
We have discussions about Jenkins vs Concourse, where to keep ansible vault passwords, should documentation live in Github wiki or in Confluence (apparently "tech" documentation in the form, "business" documentation in the latter - what if it's both? Who decides?), and so on.
There is something nice about being able to go to a single place and saying "OK, it's all here in this box". Github has made inroads with some of this stuff, but not quite enough. Gitlab could try and do all this, but then people will moan ("I prefer JIRA/Trello/whatever").
Most of the pain around developer/business workflow around us at the moment actually comes down to the fact that nobody has _really_ thought about providing a great unified UX for all of this.
Part of the concern is people want to be "flexible". No, dictate, just make sure what you dictate is a better solution to what people have.
If GitLab get it right, github could be a minor player (unless they keep up) in a few years time.