If it's a startup knocking up an MVP, I hardly see compile times being an issue. It's only when there's a massive, usually legacy, and therefore largely Objective-C based codebase that compile times start to hurt.
It shouldn't be a factor in the decision about using Swift or not.
When choosing a language, framework or “platform” there are only three things that matter:
(1) Time to market: how fast can you implement a “ship-able” feature, app
(2) Resource cost/availability: can you find delivery resources easily and cost effectively. Sometimes this resource is you, metric appropriately.
(3) Scalability: as you grow and features change, how do the other two things change. Do they change for the better or the worse.
The weighting of these depends on longitivtiy expectations. If your short sighted for immediate reward weigh (1) and (2) higher if your confidence is high on the outcome of what your doing then weigh (2) and (3) higher.
For this discussion Swift wins on almost all fronts. The exception might be legacy maintenance , and even then it might be a knife fight of when and how.
It shouldn't be a factor in the decision about using Swift or not.