My NetBeans Favorite Five

NetBeans 8.2 is out, multiple cursors and all, so it’s a fitting occasion to post my five favorite features. First, some background to avoid making a just-downloaded-an-went-a-bloggin’ impression: I’ve been working daily with NetBeans since version 6 (so this post has been brewing for a while). I mostly work with Java (core, Java EE and some Swing) and with HTML/Javascript. I’ve also used NetBeans for Markdown, Scala, PlantUml, Sphinx, Python, Ruby, and making coffee. OK, forget the Ruby part.

When joining the data science team at the Institute of Quantitative Social Science at Harvard, I was happy to see they were using NetBeans as well. Dataverse, our institutional data repository system, is based on Java EE and is developed with NetBeans. It’s currently more than 97K lines of open-source code*. So you can see I have some milage using this tool.

BTW, I’ve worked with the common alternatives (both paid and free) but somehow I always come back to NetBeans. I guess I just don’t like my code being eclipsed.

OK, OK, no need to throw a BadPunException. Let’s get to it:

Little Comforts/Going the Extra Mile

Such as the suggested variable names, auto-pasting traces to the stack trace analyzer, the fact that the stack trace analyzer exists, “view differences” in the Unit testing result pane, get + code-completion generates the proper getter (there’s also a setter, of course).
And then there are the top-of-block popup and the file-too-long label. Love these.

top-of-block extended-label

Additionally, NetBeans includes the required stuff out of the proverbial box. JUnit, Git etc. No need to install manually and decide between various conflicting versions of third party plugins.

Project Groups

Switch quickly between groups of projects. Or, really, working contexts. Since I have a few separate contexts I’m working on in parallel, this feature is a huge timesaver.

Great Java (+EE) and Javascript Support

Great language support for Java (including the next version). Very good Javascript support as well. The code warnings are useful, and the “convert to functional operation” refactoring had taught me some new Java 8 features I was not aware of.

Good support for Java EE features, such as integration and plugins (JPQ modeler etc.). Integration with application servers etc. is easy. There’s also Docker integration, I hear. I’m not using Docker currently. I hope I can still keep my developer license.

Not reinventing the wheel

For example, using Apace ant at the core of the native project type make these projects useful outside of NetBeans too (an anti-vendor-lock-in vendor!).

It Just Works

It does. Srsly. It’s a very dependable tool. And I’m looking forward to seeing it graduating from its Apache incubator.

 

 

*  Count generated using David A. Wheeler’s ‘SLOCCount’.

Some remarks on The Repeated Deaths of OOP

I like people that heartily dislike a programing language paradigm. First, this means they think about how programming is done, rather than about programs only. Second, as far as hate go, hating a paradigm is much better than, say, hating people for being born with the wrong skin color, faith or sexual preference. You can trust me on this one – I’m from the middle east.

But if one is to hate a programing language paradigm, one should do it in an informed way. Case in point: @Loup’s recent post, “the repeated deaths of OOP” (@Greptilian sent me a link to this post not long ago, which was quite nice of him). While this post has some interesting ideas, presented with an envy-inducing eloquence, I’d like to constructively disagree on a few issues. Before I do, let me stress that some of the mistakes in the post are very common, which might have misinformed the fact-checking stage of it. This is the reason I write a post of my own, rather than just an internal answer email. And thanks @Greptilian for suggesting that I write one. Quite nice of you.

OK. Let’s start. (Or, to quote Pascal language, BEGIN)

The post begins by stating that “Current mainstream programming languages all support OOP — except maybe C, and it is no longer that mainstream”. There’s some echo chamber effect in the PL blogosphere, and we should be aware of it. C is very mainstream. It’s currently #2 in the TIOBE index. It has more than double the rating of language #3 – C++. It is true, however, that it is not mainstream while writing web applications. I guess it’s just harder to blog when you have a daytime job (note to self: revisit this statement once I get a daytime job).

The post continues by stating that the it is unclear what “OOP” means, and then describes the eight “fundamental building blocks”, constructed from a survey done by Deborah J. Armstrong. That’s a classic straw-man argument. Why would a survey of a population that can’t agree on anything end up in a definition coherent enough to base a very successful programming paradigm on?

When I learned OOP – Based on Timothy Budd’s “An Introduction to Object Oriented Programming” (current edition), we were presented with four blocks that make sense, instead of eight that don’t. It was argued that an object is a 4-tuple:

  • structure – what the object looks like in memory
  • state – the values the object currently holds
  • protocol – the set of messages a given object can “understand”
  • behavior – what said object does when it gets a message from its protocol

Note the lack of classes and inheritance, which are a way of code reuse within an object oriented context. Abstraction, encapsulation and polymorphism naturally flow from these four principles. Budd’s book is pretty popular and I assume I’m not the only one that remembers his presentation on the subject (and the cute Platypus on the cover). At any event, when looking for a definition, it’s better to go to the books rather than the allegedly confused public. I mean, imagine how monads would look if we applied this approach to them (here’s Gilad Bracha on this subject).

“Then Java came. It was presented as an “easier C++”. Same syntax, similar abstractions, garbage collection… At that point Smalltalk was hardly used any more, and “OOP” basically meant Java and C++.”

One does not simply conflate C++’s version of OOP – multiple inheritance, supermarket principle, no GC – with Java’s. Java has a direct influence from SmallTalk and Objective-C, and the original staff hated C++. And played hockey with Scott McNealy. Here’s Patrick Naughton, a member of the original team that worked on Java (“Oak”, at the time):

… I thought Objective-C was the coolest thing since sliced bread, and I hated C++.
So, naturally when I stayed to start the (eventually) Java project, Obj-C
had a big influence. James Gosling, being much older than I was, he had
lots of experience with SmallTalk and Simula68, which we also borrowed
from liberally.

source

It is important to stress that the garbage collection is key in an object-oriented language, rather than a utility that makes life easier. With no garbage collector, any object creation has to be matched with precisely one object destruction (and of the same object as well). This means objects are not “first class citizens” in the language – the programmer cannot create anonymous ones. Java has to have garbage collection, because of the exact same reasons LISP has to have it when it allows programmers to create anonymous functions.

C++, on the other hand, is governed by the “supermarket principle” – the language accommodates many programming styles (including OOP, modular, procedural, and to some extent, functional. See Bjarne Stroustrup’s introduction to “The C++ Programming Language“), but you only pay for what you use. Garbage collection runs against this principle, as it has a heavy price tag.

Onward, to the second death.

“suddenly we hardly needed Object any more.”

Dude, you realize where toString(), hashcode() equals() and come from, right?

Onward – I’m skipping the “third death”, as I disagree that the grim reaper operates on a programming paradigm by adding more languages that support it.

“Lambdas are everywhere…”

A lambda is an object with a single method. Since it’s a single method, it does not need to have a name. When the need to name it rises, it is commonly called “apply”. In fact, OOP is a generalization of FP – FP only allows objects with a single method; OOP allows many methods. FP objects – “lambdas” or, more accurately, “closures” – can even have state: captured values from the internal calculations that created said closure. That state may even be mutable, e.g when it contains box values. So, yay FP, but FP and OOP are not mutually exclusive. You can do both. At the same time. In the same language. We’ll get to that.

Yet from a popular perspective, first class functions are not OO at all: they come from functional programming. Yet they managed to invade every single mainstream OOP language.

They didn’t invade. They where there at the beginning. SmallTalk had them. And again, FP was one of the two main influences on SmallTalk. Here’s Alan Kay, reflecting on his work (from ACM Queue)

We had two ideas, really. One of them we got from *Lisp*: late binding. The other one was the idea of objects.
source

By the way – this is a really good read. Long – but good.

“There is an easier way: don’t mutate state. Always construct new stuff, never throw away anything. Then let your smart pointers or your garbage collector take care of unreachable references.”

And I assume the garbage collector does everything in O(1), and was sent by to us from the heavens? More to the point – I think we can turn the hubris knob down ever so slightly, and construct the following argument from the above quote (at the risk of being accused of constructing a straw man myself):

“Application programming can be done with no mutable state. Code that mutates state is inherently low-level, and can be left to system programmers, the same way we treat assembly code today”

Now that’s a statement I see the merit of. I don’t agree, though – garbage collection has a real cost, and sometimes it does make sense to minimize it. There are programming models that accommodate mutability, while being safe and simple even in concurrent contexts. The actor model is probably the most popular, but there are others (shameless plug – my ongoing PhD thesis works with behavioral programming, developed by Harel, Marron and Weiss, which also offers that, in its very experimental way).

I agree that Entity-component-systems (ECS) is an interesting approach. Also, indeed, relational databases and OOP don’t mesh well, once OOP starts to use stateful inheritance. I guess this impedance contributes to the growing popularity of NoSQL databases (should ECS support that?).

Functional Reactive Programming is very interesting too. But do note that, at least in the web context (Elm, React) it is used to change the DOM, which is one of the poster boys of OOP (inheritance, composite design pattern, the works). This may actually be a good case for the aforementioned statement, where higher level programming is done in FP, while the underlying layer is done in OOP.

So is OOP not dead yet?

Prophecy, as the old proverb goes, was given to the fools. Well, I’ve been called a fool numerous times, and thus am happy to offer my opinion of the future of OOP: It’s not going anywhere. It will mutates (or rather, has mutated) to include FP. We already see this in Scala and Javascript – these languages model functions as objects, and take it from there. Martin Odersky calls this “postfunctional programming“. The term seems to be taking off. I think in the forthcoming years we’ll see OOP gobbling up logic programming as well. Some sort of merging between Prolog and Java/Scala. LP has been quiet for too long now.

P.S.

Oh, and regarding the game industry leaving OOP – I have no idea if the game industry is indeed leaving OOP. Some hard numbers are in order. But even if that’s true, and even if we accept that “games are simulations” – they are simulations that have to be done fast enough to be rendered, smoke, explosions, blood, gore, background music and all. In real time. And at least partially on GPUs, rather than CPUs. I’m not sure how strong a classifier the game industry is, given these traits. But hey, interesting development nonetheless (if true. Braaaaains! I mean, Nuuuuuuuumbers!).

Print a filled rectangle in Scala

@puppies , over at StackOverflow, had a question about how to print a filled rectangle in Scala. I guess s/he has an imperative background, because a) we all do, and b) s/he made an attempt to do this using loops. But in Scala, you can also do this with mapping and list concatenation.

Here’s my initial take (all code is printed on the scala REPL):


val ec="@"
val cc="X"
val cols=8
val rows=5

((ec*(cols+2)) +: Range(0,rows).map( _ => ec+cc*cols+ec) :+ (ec*(cols+2)) ).foreach( println(_) )

Which results in

@@@@@@@@@@
@XXXXXXXX@
@XXXXXXXX@
@XXXXXXXX@
@XXXXXXXX@
@XXXXXXXX@
@@@@@@@@@@

So then, Ben Reich suggested using until in stead of Range. Also, we can replace the side-effect in foreach with mkString:


((ec*(cols+2)) +: (0 until rows).map( _ => ec+cc*cols+ec) :+ (ec*(cols+2)) ).mkString("\n")

Can you print this rectangle using a shorter snippet? Post your version in the comments.

Uninheritance Using Undefined in Javascript

Javascript has a value called undefined. It’s an interesting little fella, which is the value of everything that is not defined. One common use of it is trolling your javascript console:


> defined
ReferenceError: defined is not defined
> var o = {}
undefined
> o.defined == undefined
true

Ha ha, silly javascript, not can’t even tell defined from it’s opposite. Except that it can. And it’s also very elegant.

See, unlike other dynamic languages that requires programs to test the symbol table for symbol presence (e.g. ruby’s defined?() method), javascript does not have to test the table, since every valid expression (such as a variable reference) has a value. It just may be that this is undefined.

Common inheritance models allows programmers to refine behavior (when overriding methods) or to extend them (when adding new methods). But javascript, with it’s undefined and its prototype-based inheritance, has a new type of inheritance – inheritance with method deletion.

Suppose we want to model vehicles. Vehicles can go, er, I mean go(), and they also have a number of doors. Easy enough, here’s the vehicle object, and a car and a truck.


var vehicle = Object.create({});
vehicle.go = function() { return this.name + " is going"; };
vehicle.doors = function() { return this.doorCount; };
// make a truck
var truck = Object.create( vehicle );
truck.doorCount = 2;
truck.name="Truck";
// make a car
var car = Object.create( vehicle );
car.doorCount = 5;
car.name="Car";

This works fine (all examples here were executed using Firefox Developer Edition):

> var aCar = Object.create(car);
undefined
> aCar.name="MyLittlePony";
"MyLittlePony"
> aCar.go()
"MyLittlePony is going"

Nothing new so far. But how do you go about modeling a broken car? When using languages that use class-based inheritance, such as Java, one might result to throwing an IllegalStateException, since brokenCar is a vehicle and thus must have a go() method. In javascript, however, there’s another option:


> var brokenCar = Object.create( car )
undefined
> brokenCar.go = undefined;

And now:

> brokenCar.doors()
5

But

> brokenCar.go()
TypeError: brokenCar.go is not a function

In other words, brokenCar is a car that can’t go, just like in real life, where a broken car is a car that can’t go.

Achievement “modeling bliss” unlocked.

Anorm and Duplicate Column Names

A bit of a technical post, just in case someone else hits this problem, e.g. while working through Play for Scala book (which I heartily recommend). That’s how I stumbled on this issue.

Anorm is an interesting alternative to ORMs. This is pretty much going against the flow of “let us write your SQL for you”, so they have some explainin’ to do. Anorm sticks to native SQL, and make it easier to parse result rows into objects. One way of making it easy are SQL row parser combinators.

The idea is this: parsers take column values from an SQL result row, and return a Scala object. Combining the parsers yields tuples, that can later be parsed into Scala objects (normally, some case class).

So that’s all fine and dandy, but the code below produces unwanted results:


val prodsAndItems =  SQL("""
    SELECT p.*, s.* FROM products p INNER JOIN stock_items s 
    ON p.id = s.product_id
    ORDER BY p.id
    """)

  val productParser: RowParser[Product] = {
    long("id")~long("ean")~str("name")~str("description") map {
      case id~ean~name~description => Product(id, ean, name, description)
    }
  }
  
  val productsParser: ResultSetParser[List[Product]] = { 
    productParser *
  }

  val stockItemParser: RowParser[StockItem] = {
    long("id")~long("product_id")~long("warehouse_id")~long("quantity") map {
      case id~prodId~wareHouseId~quantity => StockItem(id,prodId,wareHouseId,quantity)
    }
  }

  val stockItemsParser: ResultSetParser[List[StockItem]] = {
    stockItemParser*
  }
  
  val productStockItemParser: RowParser[(Product, StockItem)] = {
    productParser ~ stockItemParser map (flatten)
  }

  def selectAllProductsAndItems: Map[Product,Seq[StockItem]] = DB.withConnection{ implicit c =>
    val res:List[(Product, StockItem)] = prodsAndItems.as( productStockItemParser * )
    res.groupBy( _._1 ).mapValues( _.map(_._2) )
  }

This is a classic case of products, warehouses and stock items (product quantities that are stocked in warehouses). You would expect the call to selectAllProductsAndItems to return a map object mapping products to their stock items. Instead, it returns a set of new products that don’t exist in the database, and maps a single stock item to each one.

Now, I love creating new products. I just don’t want my data access layer to do it for me. Enough of this “machines replacing people” zeitgeist.

Why was Anorm doing this? (spoiler alert from here on)
Both stock_items and products have a column called “id”. The basic parsers (defined in stockItemParser and productParser) call the columns by their names, and so get ambiguous results.

The solution turns out to be pretty simple (as always) – reference the column by its full name: “products.id” instead of “id”. Over to you, Peg:

P.S commit link on github

A Playful Eye for the JEE Guy

Here’s a link for a presentation I gave at the IQSS (an awesome place I’m lucky to be part of). It’s yet another “intro to Play” talk, geared towards people with background in JEE.

There’s a twist, though – the presentation is done in Play.

It’s available at github, and there’s also a PDF transcript available at the IQSS site, in case you don’t want to install the software needed for it to run.

As always, big thanks to IQSS’ data science team for making this possible.

Lessons learned by a frugal hacktivist

There’s a compulsory biometric database being built in Israel. It’s bad, but it’s not the point of this post. I’m taking part in some civilian effort to stop it, and part of this is having a website. So we built one. For free. Here it is. And here is what I learned, divided to non-technical and rather technical points.

Non-technical:

  • We wanted a good looking, fast website. As the government employes paid commenters, we didn’t want any talkback options.
  • We have some people that can build websites (me, but not just me). However, we’re all busy.
  • The website had to be updated periodically, but not too often.
  • We are a voluntary organization, and we don’t have any money.

While we were aware of standard options, such as opening a wordpress blog or a Facebook page, we decided to go with a static web site, hosted as a github project page. The motivation was that this would generate a very fast website, and we could get all the design flexibility we need (canned solutions are not that flexible, really).

However, keep in mind that github project pages come with some strings attached. The website has to be open-source, or pay for the hosting, and no server-side processing can occur. This means any updates to the site require some technical skills.

For hacktivism, the open source string may be a real issue. If you spend a lot of time creating a web site, and you don’t want the other side to just fork it and create their own version. You may want to keep your source closed. We didn’t have this issue, as the biometric database already has web presence, as well as TV commercials, posters, etc.

So far, which is not very far really, the choice for static website seems good. Here are the technicalities. The main goal was to get something done fast, but at a high quality.

Site generator

We went with harp.js. While jekyll looks like a more natural choice for github, it is not as flexible as harp. Jekyll assume you are writing a blog. Harp has a more holistic approach, and also supports lesscss (and many other languages). I already had experience with jekyll (earned while doing this site) but decided to try harp and have jekyll as a “plan b”. All in all this worked.

Front end

Bootstrap3. You can’t beat the king.

Content Generation

Here’s where Jekyll is better than Harp – we wanted to generate a list of links, display a list of them and give each one a page. Jekyll would have done this out of the box, if we would have treated each link as a post. Harp requires having a file for each page in the site, which meant we had to maintain the json map and a set of files, and make sure each key in the map has a file named after it. Not fun and error prone. Well, bash to the rescue:


cat _data.json | jq keys | grep \\w | cut -d\" -f2 | xargs -I{} touch {}.md

This line ensures that each key has a file with corresponding name. If the file already exists, it is not overridden, which is important if someone wanted to add additional data to the to a link’s page.

Issues

  • Contact forms require a back-end to process the input, and static sites don’t have any back-end. There are ways of doing them, e.g. using google forms. We address this soon, but currently we’re happy to use github issues, which also works. Also, you can use a mailto link, but you’ll be facing another set of issues there.
  • As always, collaborating using github is a pleasure.
  • When other people fork your site, they shouldn’t call their branch “gh-pages” – it makes the CNAME files collide. Only have a single branch with this name.

Conclusion

When needing a website that does not take data from users, static websites may be a good choice. If you can keep your source open, hosting them as github project pages is a fast solution that’s free and easy, as long as you have people with technical skills available. If you have some cause you’re trying to push, you can fork us on gihub.

Oh, and if you want to learn about the biometric data base in Israel and you don’t read Hebrew – we have a site for this.

Draw More, Work Less

Below are the slides for a talk I gave at IQSS about software tools that generate diagrams. Two examples are shown – GraphViz and PlantUML.
I was hoping to show that focusing on the semantic level of the diagram (e.g. structure, relations) and letting the computer decide on the actual layout can make life easier. I know I use these tool quite often.

Anyway, the slides are below. The files for most diagrams can be found here.

Geek Dad #12FFA: Cowsay

The summer vacation is upon us, and with it, the need for kids to not be bored (or is it the need for the parents to have kids that are not bored?). Coloring pictures is a nice pastime, and you can always find a lot of coloring images by going to google images and limiting the search to line art only.

But, does it educate your kids to be better geeks? Does it tickle their imagination? Not really, no. Well, what is a geek dad to do?

 _________________________ 
< Lets color ascii art!!  >
 ------------------------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Cowsay is a lovely little program written by Tony Monroe. It allows you to quickly create ascii art of cows saying things. And it’s not limited to cows, there is also tux, a bunny, a koala and various other images, which can be listed with cowsay --list. Do not pipe the --list result to a loop and create all the images, as some are NSFW, (or, NSFK, in this case).
Also, it’s not limited to saying things either: there’s also cowthink.

And thus:
10 Kid: Dad, can I color a cow?
20 Dad: cowsay please color me\! | lpr
30 Kid: <coloring happily>
40 GOTO 10

cowsay is available for most platforms. On linux it’s available through the package manager. On OS X, it’s available through brew.

Invitation to Scala

Here is the material for the talk I gave at the Harvard IQSS’s “Tech talk” lecture series. This is a gentle introduction to the Scala language and to functional programming. As this lecture had quite a few demonstrations, you need to follow the lecture notes and try the commands shown there on a scala REPL in order to get the most of it.

Very big “Thank You” for all the IQSS staff that attended – it was great fun*!

Update:

SlideShare featured award
SlideShare’s featured award

This presentation just got featured by SlideShare’s editorial team. Thanks, guys!

 

 

* well, for me, at least.