"show your working"—a plea for Scala coding quality

The recent debate about the complexity of Scala over Java and other languages has lead me to bemoan the quality of some significant chunks of Scala code currently circulating in the public domain.

I probably should just be patient as the cycle of programming language maturity is a common one. In the early days of a language, in the absence of coding standards and common practice, people grab at the new features of a language and enthusiastically use them with little awareness of the consequences. As a language matures, the community of developers adopts and polices standards to the extent that a newby developer soon finds themselves instructed on the basics of what to do or not to do.

There's a lot of Scala code out there that feels like it's been put together by spotty adolescents with too many hormones and not enough common sense. Okay, I'm being harsh, I'm a Scala newby too, but I think it really is time we started to get a bit more focused on developing some good coding practice and mentoring each other towards maturity.

I'm going to start with a simple issue which is close to the recent complexity debate...

Extremes of pure functional style encourage code obscurity

Functional programming is great, but taken to an extreme functional programming can:

• encourage short meaningless function names.
• discourage temporary named variables which help to explain what is going on.
• encourage the creation of large numbers of small functions which obscure the modular interfaces.

Take this function as a slightly extreme example:

  def ^!!||^(implicit f: Foldable[IN]) :
      Kleisli[Option, String, NonEmptyList[List[Char]]] =
          kleisli((p : String) => {(this !!|| p).toNel })

I'm not trying to pick on anyone here, so apologies to the author, but this method is one of over 50 public methods in a single Scala trait of an open source Scala library. Is it any surprise that people think Scala is complicated?

So, here are my Scala programming tips for the day:

1. Minimise your modular interface: in other words, keep your publicly accessible functions, properties and classes, etc. to a minimum. If it doesn't need to be public make it private. Why? ... because developers reading your code shouldn't be left swimming around trying to find what's important and what isn't. If you create a function that is only used by one other function, define it within the function that requires it. This makes it absolutely clear that it is only there to support the other.
2. Use meaningful names and avoid symbols: if your readers can't figure out what a method is supposed to do then chances are neither will you in a few months time! The use of symbols for function names is a curse of the current trend in DSLs (Domain Specific Languages). Yes, do define a + method to add two objects together, but don't write functions called, ~>, ~~> and ~%> and think you're clever!
3. Don't be afraid to use vals to store partial results in your function rather than trying to string everything together into one long expression. Not just does this simplify the code by breaking it down into smaller parts, it also provides you with the opportunity to "show your working" by assigning partial results to values with meaningful names. I doubt an assignment to a val has any impact on performance. But I'll leave it someone more qualified in bytecode to back me up on that.

Okay. That's me done for today. Don't miss out on this great Scala community style guide. A really valuable contribution to Scala maturity.

The official way to bypass data modification on O2 mobile networks

Last week I received details from O2 of the official way of bypassing the 'optimisation platform' that O2 use on their mobile networks. They were particularly concerned that I pass on the following comment on the use of this and so I include it below verbatim:

when using this “bypass” function you must consider page impressions will take longer, on average, and thereby detract from the user experience, a slower experience. Also, a greater volume of data will be downloaded , on average, and those customers who do not have unlimited data will use their data bundle faster or incur high bills.

They then then went on to reference section 14.9.5 of the W3C HTTP 1.1 Protocol specification: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.5 which refers to the use of the HTTP Header "Cache-Control: no-transform" response directive. By setting up your server to return this response header, O2 indicate that they will not modify the data.

I should add that I haven't had a chance to test this yet and in particular, to test whether this stops the compression of images as well as the modification of HTTP source code.

Whilst this appears to offer a way for web developers to prevent their site content from being modified, it does not resolve issues for developers of web applications which utilise web data feeds that are not under their own control. For example, the developer of the great iPad Viewfinder app which provides Flickr photo search and download cannot prevent O2 reducing the quality of the images when using it over their mobile network.

O2's response to data modification queries

Just over a week ago I had a conference call with Tim Fielden (Data Access) and Gavin Sheldon (Head of Networks O2 UK).

1. They indicated that they had looked into the legal issues I had raised and were advised that the data modifications they are making do not cause web site developers to infringe any copyright or licensing terms. As I said to them at the time, I still find it hard to believe that O2 themselves can modify other peoples data without informing them.
2. They recognised a need for developers to be able to bypass their data modifications in more exceptional circumstances and said that they would send details of how to do this. I'll explain this next in a separate entry so it's easier to search for in future.
3. They expressed confidence in their extensive testing of their network and data compression/modification systems and said that, in their testing, they found that these network optimisations resulted in the best customer experience. I specifically queried whether or not they tested accessing multiple pages in a row to which they reiterated their confidence in the extensiveness of their testing, but welcomed any specific cased of web sites that were found to be underperforming on their network.

All in all they were very polite and clearly expressed their desire to be helpful but I was left feeling that it was going take more than a few customer complaints before they would do anything to stop modifying data without telling anyone. No-doubt they feel that that, as not the only operator to be doing this in Europe, they don't need to rush to change the status quo.

Operators modifying mobile data content (a table)

Following my request for people to test their own mobile provider, here is an up-to-date summary of responses so far. The number under "Modifies" or "Doesn't appear to" is the number of individual confirmations I've received. I won't update this after I've received 4 consistent responses.

Please note that "modifies" means that the simple javascript test appears to show that the provider is inlining CSS style sheets. The "Doesn't appear to" means that the test didn't detect CSS inlining, but doesn't exclude the possibility that the operator is still modifying content in a different way.

Country	Operator	Modifies	Doesn't appear to
Australia	Optus	-	1
Canada	Fido	-	1
Germany	Vodafone	2	-
Netherlands	T-Mobile	*	* (Note 1)
UK	O2	4	-
UK	Orange	-	1
UK	T-Mobile	2	-
UK	Three	-	2
UK	Vodafone	-	1
USA	AT&T	-	1
USA	Speakeasy	-	1

1. T-Mobile in the Netherlands apparently gives users a choice when they subscribe to the service.

O2 Data Modification Update

A number of people have asked for an update on the O2 data modification situation. Here is a quick list in no particular order:

1. The Head of O2 Networks has responded to my communications and has arranged a meeting later this week to discuss the issues. He has also asked the O2 legal team to reassess the licensing implications of the modification of licensed source code including the removal of copyright messages.
2. The data modification technology used by O2 appears to be provided by the company ByteMobile which appears to suggest that some of its technologies are also in use by Orange, Vodafone and T-Mobile in the UK.
3. A 'back of an envelope' calculation appears to indicate that the data modification taking place will indeed significantly increase the amount of data being sent to your mobile device when you access two or more pages of a standard web site like the BBC news or Guardian news web sites. This is on the assumption that your mobile device would normally locally cache CSS and JavaScript files. It's quite possible that some older mobile devices do not do this.
4. The Register (online technology publication) has responded to me about this issue but to date I don't believe that this issue has been published in any press article. I think people generally find it a little too technical to write about. Everyone I have spoken to has been completely unaware that this practice is going on.
5. A comment making reference to this issue that I added to a BBC news article was removed as being "potentially defamatory".
6. Following issues arising with another web site, this blog has followed up on the same issue: O2 UK mobile users - your operator is breaking this site for you..
7. StartupCafe kindly posted an article on my investigations: O2 Compression on mobile devices.
8. I have contacted Ofcom who are assessing my complaint.