Tuesday, October 03, 2006

Closures and Java: The Horror Story Begins

Previously I posted how it is too late for Java to include closures and it seems my fears have duly been confirmed. In a post on his blog Neal Gafter has already confirmed what we all knew already: that existing APIs won't be able to be changed to accomodate closures.

This is exactly what I said in my my previous post would be the critical problem with the closure proposal. So what do we get instead if existing APIs can't be changed? This is where it starts to become terrifying. Yes ladies and gentlemen we get the methods added to Collections accessible for a static import. I mean since when did Java become a functional programming language?

Neal presents the example as follows with the initial Java code being:


Droid seekDroid(Map map) {
for (Map.Entry entry : map.entrySet()) {
String droidName = entry.getKey();
Droid droid = entry.getValue();
if (droid.isSought()) {
System.out.printf("'%s' is the droid we seek.%n",
droidName);
return droid;
}
}
}


This is fairly standard stuff made slightly better by the Java 5 for loop. Now to "optimize" this with the proposed closures we have the introduction of a new keyword specifically for looping in the for keyword:


Droid seekDroid(Map map) {
for eachEntry(String droidName, Droid droid : map) {
if (droid.isSought()) {
System.out.printf("'%s' is the droid we seek.%n",
droidName);
return droid;
}
}
}


I mean how is this this better? There is hardly any difference in terms of LOC or code verbosity to the previous example. Not only that whoever thought closures were specially for looping has clearly lost the point. If you have to modify the language specifically to support one use case of closures that should be big ugly red flag waving.


Clearly supporting backwards compatability is going to be a huge stumbling block here and I fear what we will end up with is a botch job. Now just for comparison sake contrast the above example to the equivalent Groovy code:


def seekDroid ( map ) {
def droid = map.find { droid -> droid.value?.isSought () }
if (droid)
printf ( "'%s' is the droid we seek.%n",
droid.value.name )
return droid
}


Enough said really. Languages like Groovy and Ruby have closures at their core and they are not bolted on as an after thought. If closures are not embraced this way they will become more of an additional burden for the programmer to learn than something core to the language.

Needless to say the horror story doesn't end here, however, as let's take a look at the method that provides this feature:


public interface Block2 {
void invoke(K k, V v) throws E;
}
public static
void for eachEntry(Map map, Block2 block) throws E {
for (Map.Entry entry : map.entrySet()) {
block.invoke(entry.getKey(), entry.getValue());
}
}


What can I say? That's about as clear as mud. Generics are bad enough as it is, but that is about as readable as a newspaper delivered in the rain. Implementing this method in Groovy would go something like this:


static eachEntry(Map map, Closure closure) {
for(entry in map.entrySet()) {
closure.call(entry.key, entry.value)
}
}


In my opinion, Sun need to carefully consider what they are doing here and whether it will cause more harm than good. It would be of more value for them to embrace dynamc languages like Ruby (which they are doing with JRuby) and Groovy than to bolt on something that has the bad smell about it that the current proposal does. They should be focusing on more important issues to Java such as decent desktop integration with Swing (which still fails to look and behave natively), shared VM on windows and other such well documented problems that never seem to get solved.

My 2c.

12 comments:

Charles Oliver Nutter said...

I tend to agree; however I *am* interested in potential JVM changes that make it easier to compile closures in JRuby and Groovy. I think evolving the Java language is unnecessary...don't buy into the C# myth that every new release has to be completely freaking different from the previous one. Java is fine for what it's good for...let's not try to make it something it's not.

Ricky Clarkson said...

"existing APIs won't be able to be changed to accomodate closures."

Not true, he mentioned adding methods to Collections to support looping etc.

I already have my own Collections class that has 'all', 'any', and other such algorithms, in Java 5.0.

Java might have done alright so far without closures, but it might lose people to more sophisticated languages. Comparing Java syntax to Ruby and Python is a little unfair, as those are dynamically typed languages (read: can't be refactored).

Try comparing it to Haskell instead, a much stronger typed language than even Java is.

What's wrong with static methods? If you want to be able to swap out your static 'Collections.all' implementation, you can, and guess what, closures can help with this too!

interface All
{
boolean all(Iterable iterable,Test test);
}

All all=(Iterable iterable,Test test){Collections.all(iterable,test);};

All all2=some other implementation.

If existing APIs can't be retrofitted to support closures, then autoboxing can do it (as I just demonstrated with All all=something).

I think the syntax for the custom loop looks awful, but that's not the fault of closures. That looks like a bit of a fringe idea, maybe Neal is putting a pet thing out there to see whether many people would like it.

Personally, without tail recursion optimisation, I think Iterable is the best that we'll get.

Pure functional iteration (recursion for loops) isn't really doable in Java until there is a guarantee that it will be optimised to iteration.

Given Neal's example, I'd rather compose functions.

return Collections.find(map,isSought);

Graeme Rocher said...

@Charles
Now this I agree with. Hooks into the JVM - YES! Features to the Java language - no way.

@Ricky
You hit the nail on the head in your post really. Java is strongly typed and hence maybe it is not suited to closures at all?

Closures are being added to C# because they are adding dynamic typing as well. Without this I fear the syntax and user experience will be horrific.

What's wrong with static methods? Well in Java they aren't polymorphic for one which means you have to use all that horrific generics crap to code them.

And I would hope that we have left the world of functional global methods decades ago.

Anonymous said...

I agree that the problem of how to use closures with collections is key to whether they should be added to Java. I wrote about extension methods a while back - http://jroller.com/page/scolebourne?entry=extension_methods_and_closures - and I believe a solution along those lines is an essential complement to closures.

Ricky Clarkson said...

"You hit the nail on the head in your post really. Java is strongly typed and hence maybe it is not suited to closures at all?"

I'd love to take that as a compliment, but I can't. Haskell supports closures, and is much more strongly-typed than Java. Haskell's syntax is certainly 'different'. In some cases it looks a lot cleaner than Java, intuitively, in others it doesn't.

Some of the syntax proposed for closures is quite attractive; some isn't. I hope the simplest cases remain attractive, e.g.:

invokeLater{frame.setVisible(true);}

"What's wrong with static methods? Well in Java they aren't polymorphic for one which means you have to use all that horrific generics crap to code them."

I'm not sure why you would need generics to write a static method; I think I must be missing something here.

Of course static methods aren't polymorphic, that's fairly obvious. That shows their simplicity.

If you only ever want one implementation of, say, Math.abs, then Math.abs(5) is pretty much the simplest way of doing it. Defining an interface for it and providing a factory is certainly possible, but in that case it gains you nothing, and, most importantly, you still end up with something static, e.g., MathFactory.createBogStandardAbsImplementation().

If you provide something simple, such as Math.abs, and client code wants to be able to use alternative implementations, then they can wrap it themselves, or possibly the language can help with that with a form of autoboxing (perhaps this is what C# delegates are; I should look sometime).

Perhaps I could do something like:

Function[Double,Double] abs=Math.abs;

rather than:

Function[Double,Double] abs=new Function[Double,Double]()
{
public Double run(Double input)
{
return Math.abs(input);
}
};

In other words, there's nothing wrong with static methods, they are just very simple. If you need substitutability, then you have to go the extra half-mile, but that really isn't a problem.

Don't be afraid of static, it is the simplest thing that could possibly work, in many situations.

Graeme Rocher said...

Ricky, yes there is nothing wrong with static in certain cases, but how can you compare the Math class with its simple methods to a collections library?

With the Math class not having polymorphism is a good thing because you always want these methods to behave the same given how fundamental they are.

Unfortunately a collections library is very different. It makes absolutely no sense that collections methods are static at all and has nothing to do with being afraid of static.

With collections we have several implementations provided outside JDK collections such as the commons-collections library, the way Hibernate uses Persistent versions to do lazy loading and so on. These collections implementations differ in implementation, but are transparent to the user. This is the essence of OO.

I mean, how can you provide a collections feature that uses closures without it being polymorphic? I'm sorry, but Java is an OO language, if you're using static methods in this way you havnt "got" OO. Static methods although useful in many, many cases are not useful in this case and it will cause more harm than good.

tug said...

This:

static eachEntry(Map map, Closure closure) {
for(entry in map.entrySet()) {
closure.call(entry.key, entry.value)
}
}

can be written more simply as:
static eachEntry(map, closure) {
map.each{key, value -> closure(key, value)}
}

:)

Ricky Clarkson said...

A static method doesn't prevent substitutability, as you can always wrap (interfacify - look, new jargon!).

Suppose that Collections.only (a method that filters out everything that fails a test) was not static.

You would need something like CollectionsUtility.createTheUsualCollectionsImplementation().only(iterable,closure)

Of course, that could go down in size:

Collections.stdOnly().only(iterable,closure)

I don't see what you gain with that, except that there would be an Only/CanOnly/Onlyable interface too (something declaring an 'only' method), which would serve as an automatic starting point when you wanted to write your own.

Otherwise you would be polluting the actual collection interfaces with methods that don't vary between implementations.

Of course, they could vary between implementations, and in that case, interfacifying 'only' would be useful, but that doesn't mean that it should be disallowed to have a static implementation of only somewhere.

"These collections implementations differ in implementation, but are transparent to the user. This is the essence of OO."

Not really. Encapsulation has been around since before objects.

OO is about grouping related capabilities.

"I mean, how can you provide a collections feature that uses closures without it being polymorphic? I'm sorry, but Java is an OO language, if you're using static methods in this way you havnt "got" OO."

I almost assume this question is not a technical one. I could implement a Collections.only as a static method, fairly simply.

Java is an OO language, but not everything needs to be an object. What you're talking about suggests that a single method such as Collections.only needs to be its own object.

What I'm saying is that you can have an algorithm as a static method, that is the simplest thing that could possibly work, then if you find that you need to be able to swap in other implementations, you can wrap the static method, or, yes, even refactor it so that it isn't static anymore (if clients aren't using it, and you ALWAYS need substitutability).

At some point the non-static breaks down and you need to get at a real implementation.

Suppose I have a method that uses a static method, that is a hard coupling. Interfacify it, provide it through IoC (someContainer.getImplementationOfOnly()), and you don't have the hard coupling to the implementation anymore. Brilliant!

Or is it?

If the implementation is the only conceivable/existing implementation, then you haven't gained anything. You still have the hard coupling, it is just spread out, until someone comes up with a useful alternative implementation, which is when the IoC approach becomes useful.

Graeme Rocher said...

I'm not sure if it is even worth responding to this last comment, but here we go.

"A static method doesn't prevent substitutability, as you can always wrap (interfacify - look, new jargon!)."

Yes you could do this, in fact it is called the factory pattern. However, we're not talking about something a user implements. We're talking about core APIs here.

This is not an appropriate or elegant solution for core APIs, which should always be interface and object based

"Java is an OO language, but not everything needs to be an object. What you're talking about suggests that a single method such as Collections.only needs to be its own object."

Absolutely, but core APIs do.

"what I'm saying is that you can have an algorithm as a static method, that is the simplest thing that could possibly work, then if you find that you need to be able to swap in other implementations, you can wrap the static method, or, yes, even refactor it so that it isn't static anymore"


Errr.. this just clarifies my point that you clearly havn't got it Ricky. You can't practise agile on core APIs. Waving the magic "refactor" wand does'nt work when dealing with JDK APIs otherwise the Calendar classes would have died a horrible death years ago.

Once a JDK API has been introduced the whole backward compatability conundrum comes into play.

We're discussing the implementation and provision of new JDK methods to support closures. My point is that these cannot be statc as we're not living in "agile la la land" where core APIs can be refactored whenever you choose.

Core APIs like java.util.* have to be interface based and they have to be objects simple as that. Otherwise, how are framework developers supposed to provide alternative implementations? The factory pattern is not a solution. In fact we've already seen that with JNDI being the failure that it is and the development of the @Resource annotation.

Dependency Injection (as opposed to IoC) is the way forward and this relies in interfaces and polymorphism. I'm not understanding how you don't comprehend this simple statement.

The current proposal in Neils blog proposes adding static methods to core APIs. My argument is not that a usr can do this, but that they should not be implemented this way in core APIs. Otherwise there is absolutely no point in adding closures.

Ricky Clarkson said...

I realised you are tiring of this being on your blog, so I posted a 'reply' on mine.

Where does static really fit?

"Dependency Injection (as opposed to IoC)"

I wondered about that, I've not read a whole lot about either term, but I believe I use dependency injection, where I want substitutability. I looked at the wikipedia entry for both terms, and it seems to suggest that DI is a subset of IoC (contradicting you).

Also, the IoC page says "This article or section is in need of attention from an expert on the subject." - maybe you can help.

Slava Pestov said...

Graeme:

"You hit the nail on the head in your post really. Java is strongly typed and hence maybe it is not suited to closures at all?"

Many statically typed languages offer closures. There is nothing incompatible between closures and static typing.

"Closures are being added to C# because they are adding dynamic typing as well. Without this I fear the syntax and user experience will be horrific."

C# is not adding dynamic typing, they're adding static type inference. Totally different animal.

Please check your facts next time, otherwise you come off as just another ignorant Java programmer.

Graeme Rocher said...

Thanks for the clarification regarding C# slava. Admittedly, C# is not my strong point :-)