> How useful is Scala if you don't need to do functional programming with a strong type system?

Functional programming with a strong type system, IMO, has much wider applicability than just the esoteric mathematical abstractions (e.g. Haskell/scalaz) that you're asserting is a niche for Scala.

I can think of many non-scalaz uses of Scala, but even for non-Scala uses, look at Jane Street's evangelism of OCaml (functional + strong type system) for financial systems.

(Note that, personally, while I like the benefits of FP, I admittedly still think in/like aspects of OO, even if just out of habit, so I find Scala's blend particularly nice.)

I think the issue is not that there aren't applications, it's that people just aren't used to looking for them. As an example, let's think of classical geometry. They figured out the area of circles, the volumes of cones, etc without any calculus. If you know the tricks you can figure it out too, or you can just apply a little calculus and it just falls out. I feel that people dismiss the math as unnecessary because they can get by without it, but in reality the math is always there and sometimes provides much simpler solutions.

I've found over the last few years that my thinking has morphed from primarily OO to primarily functional. It isn't for lack of writing OO code, which I do professionally. I find that my brain loves to think in terms of functional transformations on classes of data types.

Suddenly I find myself noticing when things should be expressed as a monad or monoid. I also find myself thinking in type signatures for dynamic languages and silently cursing the language designer who decided a function can return null without warning. I see patterns in my code and find myself wishing I could overload the semi-colon to implement monadic behavior.

I get and have built Probability monads often enough. They seem really cool since keeping normalization around is theoretically nice, but they seem broken in "real" applications.

Everything I do with probability ends up looking mathematically dissimilar to probability. Most algorithms go far out of their way to avoid on-the-fly normalization because it's so expensive. An EM update monad is just the State monad.

I love Haskell (never used Scala) but I find the very things you're marketing it with abhorrent to use.

I don't see how Haskell or Scala are better in mathematical abstractions. I guess it would be equally easier (or even much easier) to abstract away this kind of thing in clojure, you could generate all the checking that you would get in Haskell/Scala. The only real diffrence I see is the compiletime checking, right?

Yeah, the big advantage of Haskell (and Scala) for abstract mathematical code is the compile-time type checking. Normally, I'm extremely happy with dynamic languages, because I make maybe 2 type bugs a year. I mean, it's not that hard to remember whether a variable contains an Employee or a String, and that's all you need for some programs.

But when you're designing a library, and a variable contains the free module over nullable real numbers, then the type checker is your friend. At least I can't keep that stuff straight without lots of help. And Haskell's ability to do compile-time dispatch on the desired return type of a function makes monads cleaner, too.

If you want to do this in Clojure, see "Typing Haskell in Haskell" http://web.cecs.pdx.edu/~mpj/thih/ , and whip up some macros that do Hindley-Milner type inference with type classes. And if you do, please e-mail me so I can play with it. :-)

Scala is not Haskell and in Scala you can only check if a variable contains an Employee or a String. The problem is that such types do not actually contain anything useful about the content within that object, it just says how the object behaves (the messages it responds to).

For instance, say you have a String ... well, a String containing what? A name? A regular expression? A number waiting to be converted to floating point? A bank account number? A phone number? An email?

I don't see compile-time type-checking in Scala as something useful, as most often than not it stands in my way. It does help to alleviate some bugs related to incorrect usage of types, but on the other hand you need tests with good code coverage anyway.

The whole point of having an expressive type system is to encode useful properties in it. If a type of String isn't any use to you, give the value a more expressive type. Make a Name type, or a RegEx type, or whatever. Use unboxed types:

  http://www.chuusai.com/2011/06/09/scala-union-types-curry-howard/

if you don't want to pay an abstraction tax.

Indeed. I think many people seem to miss the fact that types are a language for describing the data in your program.

I also think many people do not understand the benefits of type abstraction, perhaps because many languages do not even offer it without requiring boxing. Even if your RegEx type is a String, not revealing that to client code is wise engineering.

My favorite example is dealing with user input on the web. Everything from the data passed by the user's browser to the data stored in the database is just a string of bytes. You could go about representing this as a string, but really there are different types of strings, specifically there are sanitized and unsanitized strings. The database should only ever receive sanitized strings and all input from the user is an unsanitized string. Then you have two simple functions that map from one to the other, say, escapeString and unescapeString.

If you don't represent these as different data types it's very easy to accidentally lose track of if you sanitized a string or not. Did you sanitize it before you placed it in your model, or do you need to sanitize it before to display it? Did we unsanitize it when we pulled it from the DB? Should we be displaying it unsanitized in the HTML?

If you encode these differences in opaque types it suddenly becomes impossible to make a mistake. It suddenly becomes very easy to reason about a program. Based on the type you can immediately know what type of string you are dealing with and not have to every ask yourself if it has been sanitized yet.

For me, that reduces the complexity of the application and makes reasoning about the program easier. Wise application of types to semantic differences in data can really make a program easier to build.

That is a poor example since you should not normally have to code like this when handling user input on the web.

You must escape all user input you send to the database, and this is normally done by your database driver by using parametrized queries (be they prepared or not). And when the data is retrieved from the database you generally do not have to unescape anything, especially not since the returned data is not SQL strings but in whatever binary protocol is used and is unescaped (if necessary) by your driver.

So what your code should look like is something like:

  db.execute("INSERT INTO foo (a, b) VALUES ($1, $2)", a, b)

So for the database you just have two types. Queries and data. Data should always be treated as unsafe while queries is the special class that could help out if you do a lot of manual query building.

Not so. What if something fails database validation so you want to display it in the form with an error? This can lead to attacks where a malicious user gives the victim a specially crafted URL that inserts a script into their page, then when they access the URL the script executes inside their browser with their session.

For this reason any user input needs to be escaped whenever it is rendered in HTML as well. If you got it from the DB it should be safe, but if you got it from the url parameters it isn't.

Data from the database is just as unsafe as data from URL parameters which is why escaping of data must be done in the presentation layer. This is basically the same thing as SQL queries. You have a template like this (most of the HTML omitted).

  <li class="<%= user_type %>"><%= username %></li>

Where username and user_type are variables containing data fetched from the database. This template is then compiled to something like the blow. Strings literals of the HTML-safe type start with s", and concatetnation to the output buffer is <<.

  out << s"<li class=\"" << user_type << s"\">" << username << s"</li>"

So to implement your own templating language and and template helper functions this is a useful concept. But query building should be in the database layer and HTML building in the presentation layer. People using the database should not have to know of or care about that the data might at some point be displayed in a HTML page.

This makes sense. You're totally right about the database. I've run into that issue before, but I had forgotten.

Your example is similar to the approach that Yesod, a Haskell web-framework, takes. The templating language, Hamlet, will accept many types, such as Text or String, but ultimately everything gets translated into RepHTML before being rendered to a real HTML string that is sent to the client. Whenever you give the template a value it automatically runs the appropriate conversion function based on type to generate something safe for display on the page.

The only way to get from a String to RepHTML is either to use the proper escaping function (usually done automatically) or use a special function to do an unsafe conversion, forcing you to explicitly state that a conversion is safe.

The type system is one of the tools used to guarantee that these things get sanitized. You can generate RepHTML in other places, such as through a widget, or other helper functions, and it knows at that point that you properly escaped it. Without the type system you could run into issues when a function generates HTML (sanitized) and then the final template re-sanitizes it. It would be very awkward to do things like layout templates if things are getting sanitized many times.

Yeah, that is how I expect a templating system to work. The one in Rails (modified ERB)works in the same way. It has a SafeBuffer (name taken from memory) class which is a subclass of String. Strings can be converted into the safe class either by escaping or through unsafe conversion which emans that we say the string is safe.

It's tricky. Note activity this week around getting XSS protection right

http://weblog.rubyonrails.org/

I'm not that familiar with lift and Yesod, but it seems like they're both able to use compile-time checks as additional layers of protection.

https://github.com/dpp/liftweb/wiki/lifts-security

Maybe my example wasn't clear enough.

How about if you want to do stuff only with Employees with a salary greater than X? Consider this case ...

   a_employees = employees.filter{|x| x.salary > A}
   b_employees = employees.filter{|x| x.salary > B}

   # where A > X, B > X
   # so both should work with:

   def do_stuff(employees) where employees.salary > X
       ...

In other words, creating new subclasses is a poor substitute for code contracts, especially since many contracts could be checked at compile time.

I'm genuinely surprised that you make only two typing bugs per year. I make about 2 per minute, and I consider myself a good programmer. My higher frequency may be in part because my style of coding relies heavily on the typing system: I recompile after about every line of code I type, and make sure my code always type-checks, so maybe I'm not as carefully planning my programming as somebody who's accustomed to dynamically typed languages. Nevertheless, I find it hard to believe that you don't trip up when you create deeply nested loops/recursion. If you don't, what's the trick?

It is possible to build all kinds of crazy type stuff into any lisp. Look at Qi or Typed Racket they build all kinds of type-stuff into the language (Im not an expert in stuff like this so I don't know how far you could go). There are allready people working on this in Clojure.

A Typesystem in this still combind with some tools to check this befor you run the programm should provide almost everything you get from Haskell or Scala.

This kind of approach does yield the additional benefits:

1. Much easier to change

2. You can add another static analysing system ontop of it

3. Less restrictive

I am curious, what kind of programming problem, aside from a math theorem prover, requires free module in a variable?

I think the reason why Scala is often used for these niche applications is that:

A) The language doesn't fall apart as soon as you try something not imagined by the original language creators.

B) Pushing the boundaries actually returns useful results.

C) Java isn't able to solve the given task or at least not productively.

That doesn't make Scala a non-general-purpose language. Probably the majority out there is using Scala as a better Java and is perfectly happy with it.