Julian interfaces are less formal than swift protocols. We use sub-typing and/or traits together with multiple dispatch to define generic pluggable interfaces.
There’s a lot of discussion around making a more formal protocol-like system but so far what we have works surprisingly well, so we’re not in a huge hurry to implement something and want to slowly explore the design space.
To some degree it can, but there are a number of problems with using Swift for this:
(1) Swift does not support multiple dispatch. That limits the way you can glue together unrelated libraries.
(2) Swift does not use abstract type hierarchies very much. E.g. in Julia one frequently define functions to use arguments of type Number, AbstractArray, AbstractString etc. This means it is easy for somebody in the future and define an array that works on a GPU or which is statically allocated and all existing functions operating on arrays work just fine.
I can invent a whole new number type and make it a subtype of Number and all my existing algorithms operating on numbers work just fine. One example of this would be Dual Numbers, which allow automatic differentiation. This was fairly easy to accomplish in Julia, but has been a major undertaking on Swift, which I don't think is done yet. I think they actually have to change the whole compiler. For Julia this is just a library thing.
(3) Swift function and method syntax is a pain to work with. For purely object oriented code it is very nice to read with parameter names. But once you get into functional programming and composition I find that it just creates a mess. I have to fiddle way too much with my Swift code to get function composition working as I desire. With Julia it is straightforward.
I would say composition is easier when everything is just based on the same function syntax.
That makes sense. Swift is also way too complex and syntatically noisy imo. I like that Julia has a smaller set of very powerful abstractions.
Though isn't point (2) just a convention thing? Protocols can refine other protocols. So in S4TF there's a layer protocol and an RNN protocol which extends that, IIRC.
I don't think so, I may be wrong, but I am quite sure you would get a significant performance penalty in Swift if you used protocols all over the place.
For instance if `func foo(bar: Number)` in Swift would give bad performance I believe as the number object would have to be boxed.
Julia can work with abstract types in a lot of instance without getting any performance penalty due to how the Julia type system works and Just in Time compilation. I don't quite see how a statically typed AOT compiled language could achieve the same.
Must confess I am a bit too tired to parse that text effectively at the moment, but I don't think that is a solution. It is basically a solution to deal with generics across libraries. In C++ this is a big problem right. Templates cannot really be put in libraries. You put them in header files.
So if you put generics in a library and link it, how are you going to know what to specialize and what not to specialize? That is the problem it seems to be they are solving here.
But this is still a compile time issue they are solving. What I am talking about is an issue that happens at runtime.
If I call a function f(x, y, z) and don't know the exact types of x, y, z at compile time, then Swift has no way of generating an efficient implementation of f. Julia OTOH due to its support for multiple dispatch CAN create an efficient implementation of f(x, y, z) for all possible types of x, y and z.
Actually on further reflection I cannot see any way an AOT compiler can solve this problem. Say you got this definition:
f(x: Number, y: Number, z: Number)
A Swift library could in theory compile all sorts of concrete variations of this function for concrete number types. However there is no way it can provide all number types. The user could provide new subtypes of Number not known when the library containing f was created.
I was a big Swift fan before and did not like JITs but Julia really convinced me how absolutely amazing Just in Time compilation is, especially combined with a dynamic language. You can just do so much crazy stuff that you have no way of achieving in a sane way in a statically type ahead of time compiled language.
You're right that in general an AOT compiler cannot solve this problem. Swift does specialize generic functions within a module. Across module boundaries, you can declare a function with the `@inlinable` attribute, which makes its body available as part of the module's binary interface. Of course this hinders future evolution of the function -- you can swap in a more efficient implementation of an algorithm for instance, but you have to contend with existing binaries that compiled an older version of the function.
The standard library makes extensive use of `@inlinable` for algorithms on generic collection protocols and so on.
Basically, because Swift has to support separate compilation of shared libraries.
The @inlinable attribute is in fact implemented by serializing the high-level IR of a function as part of the module's interface; but doing this for all functions would be a non-starter, because it would place unacceptable restrictions on binary framework evolution.
You can think of @inlinable as being somewhat similar to defining a function in a C header file. Unlike C++ templates, Swift generics don't require all definitions to be available at compile time, because dictionary passing is used to compile generic code when monomorphization cannot be performed.
I am a big fan of Julia, but Swift is perhaps the only statically typed object-oriented language (apart from Objective-C) which I have found offers some similarity in flexibility to Swift's way of dealing with types.
With the ability in Swift of adding extensions to conforming to a particular protocol to a class, you gain some of the same flexibility in Swift as in Julia.
It means you can take an existing class which was not designed in particular for some kind of abstraction and add conformance to an abstraction (protocol) you later added.
That is kind of what Julia gives you, with the ability to easily add functions dispatching on an existing type.
Say you got a `Polygon` and `Circle` type in Swift and Julia which you want to add serialization to without either one having been designed for it originally. In Swift I would define a `Serializable` protocol with a `serialize` method taking an `IO` object to serialize to. Then I would extend `Polygon` and `Circle` to implement this protocol.
The challenge in Julia is that I might want to define that only objects of type `Shape` can be serialized, but if `Polygon` and `Circle` was not already defined as subtypes of `Shape` I cannot do anything about that without changing source code. Swift has an advantage in his case.
My only alternative in Swift would be to create a Union type of all tye types I want to be serializable.
> The challenge in Julia is that I might want to define that only objects of type `Shape` can be serialized, but if `Polygon` and `Circle` was not already defined as subtypes of `Shape` I cannot do anything about that without changing source code. Swift has an advantage in his case.
You can do this with traits. One pattern is that you can define
struct Shape{T} end
struct Not{T} end
has_shape_trait(::T) where {T} = Not{Shape}
has_shape_trait(::Circle) = IsShape{Circle}
has_shape_trait(::Polygon) = IsShape{Polygon}
and then you can write
serialize(io::IO, x) = serialize(io, has_shape_trait(x), x)
serialize(io::IO, ::Shape, x) = # shape serialization here
serialize(io::IO, ::Not{Shape}, x) = # Fallback code, or an error here (or just leave it undefined)
This requires a bit more boiler-plate than regular abstract types but it's a pretty powerful technique (and has no runtime overhead). I do dream of having built in traits someday though to remove some of the boiler plate.
__________
By the way, is this a typo in your first paragraph?
> I am a big fan of Julia, but Swift is perhaps the only statically typed object-oriented language (apart from Objective-C) which I have found offers some similarity in flexibility to Swift's way of dealing with types.
You seem to be saying Swift is the only language which is similar in flexibility to Swift. Maybe you meant to reference another language?
> With the ability in Swift of adding extensions to conforming to a particular protocol to a class, you gain some of the same flexibility in Swift as in Julia.
You can add methods or computed properties, but that's about it. That's only one axis of flexibility, and it's really only syntactic sugar for writing and calling your own functions. You can't add any other kinds of features, unless they chose to use protocols in their interfaces -- which they usually didn't.
For example, that page gives the example of adding precision to numbers in Julia. I'm not sure how you could do something analogous in Swift, short of writing your own numeric tower from scratch. In Swift 4 they did add a Numeric protocol, but it's not used much. It's probably hard to retcon this sort of interface onto a framework which was built around concrete structs from the start.
> You can add methods or computed properties, but that's about it. That's only one axis of flexibility, and it's really only syntactic sugar for writing and calling your own functions. You can't add any other kinds of features, unless they chose to use protocols in their interfaces -- which they usually didn't.
I don't fully agree with this. I can add an extension implementing a protocol. That allows me to use classes I did not create in some kind of new subsystem I have just made, which requires objects adhering to a particular interface.
For instance I can take a library X somebody else made in Swift and made my own serialization library Y. Then I can add a serialization protocol to all classes in X. I can then have object graphs consisting of X objects which can now be serialized by my serialization library Y. This is beyond just adding syntax sugar for function calls. You are dispatching on the object type.
> For example, that page gives the example of adding precision to numbers in Julia. I'm not sure how you could do something analogous in Swift, short of writing your own numeric tower from scratch.
Yes this is the limitation of Swift which I have tried to articulate elsewhere in this discussion. In Julia I can make a subtype of AbstractArray or Number and this type can be used in all sorts of existing Julia libraries. That possibility does not exist Swift and I am uncertain if it ever can be made to exist.
If a Swift function took an abstract number type as argument, then the value I believe would have to be boxed. I don't see how an AOT compiler could avoid boxing. I mean inside a library perhaps, but across library/framework boundaries I don't see how you could avoid it.
Unless Swift is fundamentally redesigned as a language, I don't think it can ever match Julia in numerical computing and composability. Although I find it far easier to work with than C++ with respect to composability. My language preference is probably Julia, Go and then Swift. Go is somewhat primitive but it is kind of fun to work with. I like that they dialed back the static typing a bit. Swift feels a bit too Nazi at times.
Thanks. I mean in julia you can use traits for that, but it's not built in (yet). Though there's no speed penalty, as you probably know.
So this is about extending types, but it sounds like swift is strictly "better" then, since it's also statically checked? Or is there something that multiple dispatch gives that substantively better?
I'm trying to get a feel for if the Swift for Tensorflow project will afford the same kind of composability, while keeping static type checking, modules etc (assuming they work out cross module code specialization, which I think is happening).
