Tuesday, December 29, 2009

Thrift sounds nice, but installing it is painful

Using Thrift to solve various cross language web services problems appeals to me. Basically you define your service interface in some very simple IDL, and then Thrift generates client and server stubs in various languages. To startup a server, you fire up a main() and use some canned multithreading libraries to build a robust server. Checkout the whitepaper for further details. Those crafty authors took the time to write a whitepaper in tex, just to give it the feel of quality research.

So, you're all ready to install Thrift on a Mac. It's a nuisance, but at least it's reasonably well documented.

But what's a killer is that the Java tutorial code is utterly busted--like it won't even compile for God's sake. To get it to work, you have to:

1. Download some logging junk from slf4j.org. Then add slf4j-simple.jar and slf4j-api.jar to the classpath of the build.xml.
2. Add the same two jars to the JavaServer and JavaClient shell script classpaths.
3. Edit the generated source files. Are you kidding? Edit the freakin' generated source? And this ain't no Java6 problem--this shit won't even compile:

thrift-HEAD/thrift/tutorial/java/src/JavaServer.java:66: duplicate case label
case Operation.DIVIDE:


thrift/tutorial/java/src/JavaServer.java:77: incompatible types
found : tutorial.Operation
required: int
io.what = work.op;

The fixes are straightforward: Remove the qualifying class from the switch statements, and assign some number (I used work.num2) to io.what.

Then things seem to work.

That's one sloppy tutorial. By making such high barriers to even running the tutorial, the Thrift team has really slashed their user base. It's too bad, because I suspect Thrift is really pretty awesome.

Monday, December 14, 2009

Knucklehead Insulation Strikes Again

Shortly after we bought our house, my dad poked around in our side attics and mumbled something about how the insulation was all messed up. At the time, my main priority was on the creature comforts of my 3 month old daughter, which had nothing to do with insulation. So ignored my dad's observation for a few years, until an unrelated roof leak called me up to the side attics. Sure enough, the insulation was a mess.

The top half of the roof is actually properly insulated, with rigid foam, a thermal barrier, and about a 1.5" air gap between the foam and the roof decking. The guys who installed that section knew what they were doing.

Some clowns did a far worse thing on the bottom half of the roof: R-11 insulation jammed all the way up to the roof decking. R-11? In the attic in New England? The attic in my region is supposed to be at least R-38, ideally more like R-60. Why not just hang up a bedsheet and call that insulation? These bozos also didn't know anything about insulation's tricky pal, ventilation. By blocking the airflow at the lowest part of the roof, the job effectively blocked ventilation through the entire roof, rendering the ridge vent worthless. Thus the roof bakes in the summer, which in turn raises the temperature of the top floor, which forces me to shell out way more money for air conditioning.

We have a few knee wall doors which also exhibit a 1980's "who gives a shit?" attitude in regards to heating costs. In the winter time, our precious heat goes right out the knee wall doors, into the side attic, through the R-11 faux insulation, and right up to a pitch change in the roof deck. Nice warm air hitting the lower portion of a roof in the winter is a recipe for something I hate more than the 1980's LA Lakers: ice dams. The escaping warm air contributes to uneven roof temperatures, which, to make a long story short, are the basic cause of ice dams. So the shoddy insulation job leads to more than just inefficient heating. It wears out the roof much faster than it should (hello, $8,000 roof job) and it causes ice dams which--in case you didn't know--lead to water trickling inside your house in ways you thought were impossible.

You can bet I'll be fixing this problem soon with air chutes, rigid foam insulation, and a thermal barrier.

Monday, October 12, 2009

Kettlebells and push mowers

For our small yard I use an old fashioned, completely manual push mower. After working out with kettle bells for a few weeks, the push mower is actually very easy to use. The ballistic kettle bell drills seem to work the same muscles as the push mower.

Saturday, October 10, 2009

My Pavoni died, but I brought it back to life


When I got married, I bought my wife a diamond ring. She bought me a Pavoni. I got the better end of the deal, although she insists that because she gets labor-free espresso, she actually got the better end of the deal.

I pull 1-2 shots each day, almost religiously. For years the Pavoni worked great. The first parts that crapped out were the plastic nuts in the water sight tube. A quick call to Thomas E Cara, whom I met entirely by accident while wandering around San Francisco years before, helped solve the problem. Christopher Cara knew immediately what the problem was and shipped me some brass replacement nuts. Since then the Pavoni had no problems.

A few more years passed until my Pavoni hit the skids again. It started crapping out slowly at first. One day I could only pull one shot; any subsequent shot was dry. No water was coming out. The next day it would be fine. This went on until eventually I couldn't get anything out of the pull. The group head wasn't getting any water.

I took the group head apart and found my problem: the "Group to Boiler Insert", which should be a firm yet supple piece of rubber, had hardened into brittle plastic nastiness. When I went to screw the feed tube back into it, the threads got completely stripped (through no fault of my own). The fact that the makers of a $700 espresso machine saw fit to screw a brass pipe into a cheap piece of plastic that is routinely under 185 degree temperatures is somewhat alarming. Piss poor design if you ask me.

I thus attempted to remove the insert, but because it had become so brittle, it just got shredded. It wasn't coming out.

First I tried to bore my own holes to attempt to re-anchor the removal tool, but these holes too were shredded. Then I just started shredding the whole thing deliberately, piercing it with a drill, turning it into swiss cheese. After enough of that, I could finally chisel the thing out with a flathead screwdriver. Don't chisel it out with a chisel, and if you do chisel it out with a screwdriver, be extremely careful. One slip and you can shred the threads on the inside of the group, which will effectively destroy the entire group head.



Slowly slowly, very slowly, I chipped off all the pieces of the crusted insert. I didn't ding up the threads inside the group too much. Another call to Christopher Cara at Thomas E. Cara and the replacement parts went in the mail. A week later, I screwed the insert back into the group head and very, very carefully screwed the feed pipe back into the insert and voila, good espresso again. Christopher actually sent me two replacements with a hand written note telling me that the plastic threads strip really easily. It took me a good long time before I actually got the feed tube screwed in properly. Shame on Pavoni for not fabricating this piece in brass!

Friday, October 2, 2009

I'm the database guy and I don't know where to find that value in the schema.

Here's an embarrasing story which illustrates just how ridiculously and unnecessarily complicated JPA/Hibernate makes your life. One of my teammates walked into my office with a printout of a web page and asked "Where do I find this value in the schema?" It was a simple concentration value attached to some library in the lab. The person who asked the question is very familiar with our schema. But she couldn't find the value. I looked in the two places I knew of where we store liquid concentration values, both turned up nothing.

No problem, I thought, I'll just look at the code to figure it out. Uh oh.

First I loaded up the HTML for Tapestry 4.0 (which, by the way, I loathe). Here's what it says:

<td align="right">
Concentration (nM):
</td>
<td align="left">
<span jwcid="@Insert" value="ognl:library.quantity.concentration" />
</td>

Okay, so I know ognl:library maps to some method called getLibrary() in the tapestry page, and I know (although the IDE doesn't) that the Library.html file talks to, by convention, LibraryPage.java. So I'll go look in that source code. Lo and behold, here's what I find there:
public abstract Library getLibrary();
Okay, drill into the Library class to find some sort of getQuantity() method.
Oh shit:

@Embedded
public SeqContentQuantity getQuantity()

What the hell does this mean? This is why I hate JPA/Hibernate: it takes a simple, well understood area of software engineering (mapping SQL queries and result sets to java types) and shits all over it. There's some dead-simple SELECT statement running somewhere, and all I want to know is what it is.

At this point I thought it might actually be faster to just start guessing at table and column names by inspecting Oracle's table metadata like so:
select
c.table_name,
c.column_name
from USER_TAB_COLUMNS c
where
c.column_name like '%CONC%'

Eventually I gave up the code analysis route and said fuck it, I'll just turn on hibernate SQL logging like so:

<property name="hibernate.show_sql" value="true"/>
<property name="hibernate.format_sql" value="true" />


in persistence.xml, along with
log4j.logger.org.hibernate=DEBUG
in the webapp's log4j.properties.

This drowned me in SQL, and grepping that much SQL is incredibly tedious, so I set a few breakpoints around the methods that retrieve the data. Only by stepping through the debugger and watching the SQL go by was I able to pinpoint what the actual table and column was. Bleah.

Thursday, October 1, 2009

Random Selenium test failures when using Tapestry 4.0

If you're unlucky/crazy enough to still be using Tapestry 4.0 and you see random flameouts from Selenium with messages like

com.thoughtworks.selenium.SeleniumException: Timed out after ...

Then make sure that Tomcat isn't running with

-Dorg.apache.tapestry.disable-caching=true

Turning off Tapestry's caching, although essential for doing rapid in-place changes to .html files, causes huge flakiness. I've seem PermGen space randomly blow out after a few dozen pages get loaded in a Selenium test. I've also seen random internal Tapestry errors if you attempt concurrent page views while disable-caching=true.

For rapid development, you want disable-caching=true, but to speed up your testing you want disable-caching=false. But waitaminute, when you're testing and your tests fail, you often want to make rapid changes to .html files. That sound you hear? It's me slamming my head against my desk.

Wednesday, September 30, 2009

Uh oh: Tomcat POST too large

After getting a bug report with the attached stack trace, I noticed an obscene amount of network traffic going in and out of my sandbox tomcat. That was my first clue that something had gone horribly wrong. The stack trace from production wasn't helpful at all:

java.lang.NullPointerException
org.apache.catalina.connector.Request.parseParameters(Request.java:2340)
org.apache.catalina.connector.Request.getParameterNames(Request.java:1038)
org.apache.catalina.connector.RequestFacade.getParameterNames(RequestFacade.java:359)
org.apache.tapestry.web.ServletWebRequest.getParameterNames(ServletWebRequest.java:59)
$WebRequest_123a5d0549e.getParameterNames($WebRequest_123a5d0549e.java)
$WebRequest_123a5d05457.getParameterNames($WebRequest_123a5d05457.java)
org.apache.tapestry.services.impl.RequestCycleFactoryImpl.extractParameters(RequestCycleFactoryImpl.java:110)
org.apache.tapestry.services.impl.RequestCycleFactoryImpl.newRequestCycle(RequestCycleFactoryImpl.java:79)
$RequestCycleFactory_123a5d05439.newRequestCycle($RequestCycleFactory_123a5d05439.java)
org.apache.tapestry.engine.AbstractEngine.service(AbstractEngine.java:224)
org.apache.tapestry.services.impl.InvokeEngineTerminator.service(InvokeEngineTerminator.java:60)
$WebRequestServicer_123a5d0548d.service($WebRequestServicer_123a5d0548d.java)
$WebRequestServicer_123a5d05489.service($WebRequestServicer_123a5d05489.java)
org.apache.tapestry.services.impl.WebRequestServicerPipelineBridge.service(WebRequestServicerPipelineBridge.java:56)
$ServletRequestServicer_123a5d0546f.service($ServletRequestServicer_123a5d0546f.java)
org.apache.tapestry.request.DecodedRequestInjector.service(DecodedRequestInjector.java:55)
$ServletRequestServicerFilter_123a5d0546b.service($ServletRequestServicerFilter_123a5d0546b.java)
$ServletRequestServicer_123a5d05471.service($ServletRequestServicer_123a5d05471.java)
org.apache.tapestry.multipart.MultipartDecoderFilter.service(MultipartDecoderFilter.java:52)
$ServletRequestServicerFilter_123a5d05469.service($ServletRequestServicerFilter_123a5d05469.java)
$ServletRequestServicer_123a5d05471.service($ServletRequestServicer_123a5d05471.java)
org.apache.tapestry.services.impl.SetupRequestEncoding.service(SetupRequestEncoding.java:53)
$ServletRequestServicerFilter_123a5d0546d.service($ServletRequestServicerFilter_123a5d0546d.java)
$ServletRequestServicer_123a5d05471.service($ServletRequestServicer_123a5d05471.java)
$ServletRequestServicer_123a5d05463.service($ServletRequestServicer_123a5d05463.java)
org.apache.tapestry.ApplicationServlet.doService(ApplicationServlet.java:123)
org.apache.tapestry.ApplicationServlet.doPost(ApplicationServlet.java:168)
javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
org.springframework.orm.jpa.support.OpenEntityManagerInViewFilter.doFilterInternal(OpenEntityManagerInViewFilter.java:112)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)


But the stack trace from the first time through the bug in my sandbox was revealing:

java.lang.IllegalStateException: Post too large
org.apache.catalina.connector.Request.parseParameters(Request.java:2388)
org.apache.catalina.connector.Request.getParameterNames(Request.java:1047)
org.apache.catalina.connector.RequestFacade.getParameterNames(RequestFacade.java:369)
org.apache.tapestry.web.ServletWebRequest.getParameterNames(ServletWebRequest.java:59)
$WebRequest_124021108bc.getParameterNames($WebRequest_124021108bc.java)
$WebRequest_12402110874.getParameterNames($WebRequest_12402110874.java)
org.apache.tapestry.services.impl.RequestCycleFactoryImpl.extractParameters(RequestCycleFactoryImpl.java:110)
org.apache.tapestry.services.impl.RequestCycleFactoryImpl.newRequestCycle(RequestCycleFactoryImpl.java:79)
$RequestCycleFactory_12402110856.newRequestCycle($RequestCycleFactory_12402110856.java)
org.apache.tapestry.engine.AbstractEngine.service(AbstractEngine.java:224)
org.apache.tapestry.services.impl.InvokeEngineTerminator.service(InvokeEngineTerminator.java:60)
$WebRequestServicer_124021108aa.service($WebRequestServicer_124021108aa.java)
org.apache.tapestry.services.impl.DisableCachingFilter.service(DisableCachingFilter.java:48)
$WebRequestServicerFilter_124021108ac.service($WebRequestServicerFilter_124021108ac.java)
$WebRequestServicer_124021108ae.service($WebRequestServicer_124021108ae.java)
$WebRequestServicer_124021108a6.service($WebRequestServicer_124021108a6.java)
org.apache.tapestry.services.impl.WebRequestServicerPipelineBridge.service(WebRequestServicerPipelineBridge.java:56)
$ServletRequestServicer_1240211088c.service($ServletRequestServicer_1240211088c.java)
org.apache.tapestry.request.DecodedRequestInjector.service(DecodedRequestInjector.java:55)
$ServletRequestServicerFilter_12402110888.service($ServletRequestServicerFilter_12402110888.java)
$ServletRequestServicer_1240211088e.service($ServletRequestServicer_1240211088e.java)
org.apache.tapestry.multipart.MultipartDecoderFilter.service(MultipartDecoderFilter.java:52)
$ServletRequestServicerFilter_12402110886.service($ServletRequestServicerFilter_12402110886.java)
$ServletRequestServicer_1240211088e.service($ServletRequestServicer_1240211088e.java)
org.apache.tapestry.services.impl.SetupRequestEncoding.service(SetupRequestEncoding.java:53)
$ServletRequestServicerFilter_1240211088a.service($ServletRequestServicerFilter_1240211088a.java)
$ServletRequestServicer_1240211088e.service($ServletRequestServicer_1240211088e.java)
$ServletRequestServicer_12402110880.service($ServletRequestServicer_12402110880.java)
org.apache.tapestry.ApplicationServlet.doService(ApplicationServlet.java:123)
org.apache.tapestry.ApplicationServlet.doPost(ApplicationServlet.java:168)
javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
org.springframework.orm.jpa.support.OpenEntityManagerInViewFilter.doFilterInternal(OpenEntityManagerInViewFilter.java:112)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)


Subsequent runs through the app in my sandbox showed the production stack trace, which totally obscured the error. The lesson here: when you're trying to reproduce a bug, do it in the cleanest environment possible.

The real underlying problem was an extremely large chunk of data that was being persisted by Tapestry 4.0. The page code had pulled out an enormous list of Entities and crammed them into the POST. It should have just passed around some database keys or smaller objects more tailored to the UI.

The fast-and-cheap, sloppy, short-term workaround was to tell Tomcat to bump up its max post size. To do this, edit your tomcat's server.xml and set the post size to something larger, like 4Mb or 8Mb by adding this attribute: maxPostSize="8000000".

Saturday, September 26, 2009

Be a real man, dad. Sing.

When my daughter was 18 months old, we took a swim class. There was a pretty even mix of fathers and mothers, boys and girls. Everybody sang the silly songs you sing in toddler swim class. It was great fun.

At 24 months, we joined the slightly older swim class. Same mix of dads and moms, maybe slightly more dads. Maybe slightly more boys than girls.

Nobody sings. It's me and two mothers that seem to carry the group of 15-odd toddlers and their parents. What happened?

Does something happen to fathers of boys between 18 and 24 months that causes them to hesitate to do anything that might smack ever so slightly of being girly? I wedged myself between two non-singing dads and their 24 month old sons and proceeded to sing along with the instructor, hoping I might be able to pressure them into actually enjoying themselves. All I got were dirty looks. I imagined these dads were thinking to themselves, "Watch out for that dad, Johnny, he sings silly songs with his kids--he might be a homo." I'm fairly certain that just be singing along I have lost any hope of befriending these ManDads.

It takes a real man to sing "Wheels on the Bus" and "Pop Goes the Weasel" in his swim trunks while surrounded by slightly more muscular men. (That said, I am very proud of the fact that I can now do all of 3 pull-ups and 3 chin-ups). I resisted singing with my daughter until I realized just how powerful a drug it is for her. Although my instinct tells me that the best hope for solving problems involving screaming infants and toddlers at 2:00 AM is to slam my face through a plaster wall, a far better thing to do is sing some silly song. Once I embraced this fact, life got a lot better.

My guess is that if my first kid were a boy, I'd fit in just great with the ManDads. I'd refuse sing-a-longs. I wouldn't hug my kids very much or say things like "I love you" to my children as often as I do. Maybe I would have caved and smashed my fists through the wall with some frequency in response to middle-of-the-night scream fests. But having a girl first has definitely softened me up, and it'll be interesting to see how ManDads react to me when I sing silly songs with my yet-to-be-born 2 year old son. I'm guessing they'll just figure I'm a pussy.

Friday, September 25, 2009

Quick visual diff on a Mac? FileMerge.

Thank you, stackoverflow.com. I have enjoyed reading Jeff Atwood's posts about stackoverflow, but I've always been a bit dubious. I googled "mac diff" just now and a link to stackoverflow came up as the top hit, directing me to use FileMerge. Perfect.

I needed to do a diff on some dog assembly AGP files to make sure I hadn't totally mangled the dog genome. FileMerge was perfect for the job!

Wednesday, September 23, 2009

Disabling screen saver password protection in Windows XP


I use a Mac. I run Parallels to run Windows. My Mac turns on its password-protected screensaver after about 30 minutes. My virtual PC turns its on after about 15 minutes. So I ended up having to enter my password fairly often. This was a real nuisance. I thought I couldn't change this because my IT group had locked the option out of the display properties, which seems odd because I have administrative rights on the VM.

But then I discovered that I could solve the problem by tweaking the registry:

HK_CURRENT_USER -> Software -> Policies -> Microsoft -> Windows -> Control Panel -> Desktop

Set ScreenSaverIsSecure = 0.

Sunday, September 20, 2009

Great pegboard hooks

For about a year, we've been using pegboard to hold our pots and pans a la Julia Child. We haven't been using very stable hooks. Sometimes you go to grab a pan and the hook ends up on the floor. I installed the pegboard when I was sleep deprived from having a baby and in the doldrums of a 6 month case of mono. In other words, I wasn't very with it, so the hooks I got for the pegboard didn't work out so well.

Then I got some of these amazing pegboard hooks at Amazon. Problem solved. You screw them in a bit and they're rock solid. You can unscrew them very easily if you need to.

Amazing Mexican in Arlington

Zocalo, in Arlington on Broadway just east of the center, is amazingly yummy. We went there for lunch. Best fajitas and enchiladas I've ever had. Not overly cheesy, greasy stuff. Good, fresh ingredients, well balanced.

On the other hand, Ixtapa, just over the Lexington border, is a much better place to plow through a few drinks in the company of your (young) children. Great spicy cheese dip most of the time, but not entirely consistent.

Ixtapa is almost as good as Los Loros in Atlanta. Cheap, cheesy Mexican. Entirely different category from Zocalo.

Friday, September 18, 2009

Debugging Unit Tests in Maven2 (mvn)

Recently I got fouled up by this issue in Intellij's IDEA. Basically IDEA loads up the Spring context one way, maven another. IDEA's trying to be helpful by doing this, but it's causing me headache. Instead of running and debugging my unit tests in IDEA, I had to resort to launching mvn on the command line and attaching a remote debugger to it via IDEA. How do you do this? Comme ca:

mvn -Dmaven.surefire.debug="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=62905 -Xnoagent -Djava.compiler=NONE" -Dtest=YourTestClass test


Then create a remote debugging session in IDEA for port 62905.

It's pretty tedious because you have to fire up mvn, then wait for it to wait, and then start the debugging session from IDEA.

While you're at it, you might want to set -Dhibernate.show_sql=true and change your log4j.properties to include log4j.logger.org.hibernate.SQL=DEBUG, SQL_APPENDER to see what Hibernate is sending to the database.

Tuesday, September 8, 2009

Mail.app is eating Bamboo

We upgraded Bamboo recently. The latest version sends out much smarter email summaries of build failures. Unfortunately, Mail.app eats the HTML so I get an email that has a subject and zero visible content...unless I go to View -> Message -> Best Alternative.

I have to hit a few button to tell Mail.app "You're HTML display is busted. Try something else"?! No other email client I've ever used has ever had trouble dealing with HTML, even horribly malformed HTML. Mail.app mostly seems to be able to handle HTML, but these corner cases are really annoying, especially when you have to deal with them on every message. I can't even set this property on a folder or a smart folder. That's really bad design.

Here's how you get Mail.app to fix itself--also terrible design. Open up terminal and type:

defaults write com.apple.mail PreferPlainText -bool FALSE

Yikes.

Tip 'o the hat to this post.

Tuesday, September 1, 2009

Kettlebells are kicking my butt

Baby number two is due in about 6 months. When we were at T-minus 6 months for our first child, I was in great shape. I swam about 50 laps of crawl 5 days a week at the most amazing pool at MIT's Z center. Best shape of my life.

Then our daughter arrived, I never slept, and I ate junk food to keep myself sane for the better part of the first two years of her life. After the first year, I got mono, which knocked me on my ass for about 6 straight months. I packed on some pounds and am now generally a pretty huge weakling. I strain muscles really easily now that I have so little muscle mass. The latest episode involved straining my wrist while unloading a lawn mower to hack my out-of-control-overgrown lawn. The lawn had gotten so overgrown that our little manual push mower wasn't so much cutting grass as combing it, so I had to borrow a friend's power mower. The previous strain involved carrying some luggage into the house after vacation.

These little indignities are getting to be more than just embarrassing. What to do?

I called my SEAL buddy to ask him for his advice on how to get back some functional strength in some simple workouts. His advice:

1. Install pullup bar in a well traveled area of the house. Do pullups (or in my case, attempt pullup) whenever you pass by.

A few notes on this: you can't just put the pullup bar in the basement. It has to be front and center, in a part of the house you visit often during the regular course of the day. If installing the pullup bar doesn't elicit gasps of horror from your spouse because of how the pullup bar's presence ruins the aesthetic of the room, you haven't put the bar in the right place. You want the pullup bar in the most obtrusive spot possible. In my case, it's in the pantry door frame. The lure of the pantry is irresistible: it contains cookies, chocolate, and the coffee grinder. I'm in there at dozen times a day, which means I'm trying pullups a dozen times a day.

I was amazed at how many people report nearly killing themselves doing pullups on improperly installed pullup bars. It's pretty simple: screw it into solid, load bearing wood framing, like a door frame.

2. Get a book by Pavel Tsatsouline and start doing kettlebell workouts with a 25 pound kettlebell.

A few notes on the kettlebell: No way in hell could I handle a 25 pound kettlebell. If I dropped it, I'd either crush my toes, the cat, shatter the floor, or sustain numerous strains at the very least. I picked up a lady's 10 pound bell to start with, just to make sure I could get the technique right before moving up to real man weight. Kettlebell workouts--even with light weights--are killer. Just you try 5 or 10 Turkish getups. You will be huffing and puffing in no time, and as Pavel says for many of his workouts, "Bring a puke bag."

The results, after just 10 days or so, are striking. I find myself with more energy. My arms are well toned--heck, I even have deltoids. Me--deltoids! When I used to lift weights at the gym years ago, I ended up with weird arms--big biceps, smaller triceps, and no shoulders. It's hard to do well-balanced weight lifting workouts what with all the goofy Nautilus equipment, but with a few simple kettlebell moves, you'll be amazed how your body changes.

I've also been amazed at how kettlebells develop real functional strength. Traditional weight lifting isolates your muscles and gives you the ability to lift weights in a particular way. For example, curling 50 pound dumbells makes you really good at...curling 50 pound dumbells. But day-to-day life doesn't present you with the opportunity to bench press something in order to accomplish anything of practical value.

Carrying a squirming two year old up steep stairs with one hand while dragging a 50 pound bag of concrete requires full body functional strength, not just huge biceps. The balance involved in these sorts of tasks is important, as is your body's ability to shift loads around. I find kettlebells are very good at building what I'll call "practical" strength. Ditto for the pullup bar.

Tuesday, May 26, 2009

Whither javaworld.com?

I have been losing faith in javaworld.com lately. Maybe I've just advanced to the level of programming where I'm highly skeptical of programming advice dispensed by people who work in development environments that are highly dissimilar to mine. It's not that the advice is bad; it's just that it can't be applied in any practical sense to my environment, or it yields no benefit in my environment.

I still read javaworld.com from time to time, but this little nugget in an article describing cloud computing really makes me wonder:

"Likewise, you should not be concerned with writing application code to implement scalability, reliability, and other cloud and distributed computing features that a cloud platform could provide..."

Uh, right. I'll just store everything in Object[], read and write data using unbuffered classes, call read(byte b) a billion times instead of read(byte[]) once, etc., etc., because the cloud will manage scalability for me. WTF?

Building scalability into the application code is incredibly important. Likewise, if my application dies all the time, the proper solution is not just to let the cloud restart it and cross your fingers. The proper solution is to code the app so it doesn't crash. The article's advice is especially ludicrous because when you use the pay-as-you-compute model, application crashes waste your money because you're paying to crash and restart, instead of paying for your app to do something useful. Furthermore, if you build scalability into your app, your app becomes cheaper to run on the cloud because you use fewer resources (electrons, clock cycles, network, storage) to run it.

Tuesday, May 12, 2009

Random Hilarious Error Messages

I come across silly error messages fairly often. I thought I'd capture them here for entertainment value.


This one comes at me from Oracle's Corporate Time Mac Client. How many windows does it take before you have too many? Five. Yup, five. I routinely have about 20 Firefox windows open without any sort of trouble. Five windows? Seriously?


I think this message came from SQL Navigator 5.5.4.847. I can't tell, though, because the dialog box doesn't give me the slightest clue. Wouldn't it be nice if my computer at least told me which application caused this?

Tuesday, May 5, 2009

Attempo Live Backup makes life miserable for developers.

Our company has had to switch from TSM to Attempo's Live Backup. It's wreaking havoc on developer productivity.

Builds, of which developers do dozens each day, delete and recreate thousands of files over the period of about 5 minutes. The backup system sees all of this disk activity and thinks "Oh boy, better get started caching changes and streaming them to the backup server!" Never mind the fact that those 150,000 class files are a compiler artifact and can be reproduced at will by running javac after checking out the appropriate source code from subversion--another backup system.

So what's a developer to do? Spend an hour configuring Attempo to ignore class files, war files, jar files, zip files, etc. The biggest bang for the buck I got was telling Attempo to ignore the many dozen "target" directories in my build tree, which is where most of the action takes place. It takes an hour to do this because you have to add the directories one by one. It would be really great if Attempo could use regexes for client side exclude directories and files.

Doing all of this configuration of Attempo greatly sped up my builds.

Friday, April 24, 2009

If Hibernate is so great for developers, why does it make my unit tests run 400x slower?

I am insanely frustrated by how sluggish hibernate startup time is during unit tests. I have explored ways to optimize this, but I believe Hibernate has made a fundamental mistake with boot-time loading. It would be far,far better to do these sorts of checks at compile time instead of at load time. It would be so much better to do it at compile time that I consider it a moronic blunder to do it at load time.

Here are some of the things I've tried to do to speed up Hibernate load times, both for unit tests and for deploying to our dev/test tomcats.

  • 1. Try -Dhibernate.use_lazy_proxy_factory=true

    Nope, that doesn't do a thing. ~19s startup for my piddly unit test that needs to grab one row, from one table, and parse it. Without hibernate, it would take about 3.6 microseconds.

  • 2. Make your own custom persistence.xml

    Nope, this doesn't work because we have a highly interconnected schema, with over 700 tables. Some folks will point out that my organization must be brain dead for having that many tables in the same schema, but our application is all about interogating data. Besides, the schema evolved over ten years, starting back when Java was in its infancy. So don't tell me that the solution to long startup overhead in a unit test is to refactor a mult-terrabyte schema.

    This interconnected-ness, even on a smaller schema, causes problems. Hibernate isn't a compiler; it's not doing static analysis to determine what Entity classes to load. It does a brute force, load-everything-at-boot-time, regardless of whether the JVM will actually ever need the classes in question. Our home grown RO tools did this once...at compile time. Ironically, developers complained about this. I thought it was great; I love static analysis. Hibernate doesn't do this once at compile time, it does it hundreds of times per day, each time a developer runs a unit test. By my calculation, the per developer hibernate tax, if you run about 50 unit tests a day during your personal development work, eats at least 25 minutes. It's not just 25 minutes, though, since each occurence is a distraction, and distractions are a waste-multiplier (like the military's "force multiplier", only for wasting time).

  • 3. Just write your own weird class loader.

    Seriously? If Hibernate is a tool whose goal is dumbing-down SQL and relational databases, do you really think that your average Hibernate user is going to be able to navigate a custom class loader?

  • 4. Re-architect your code so that all database access goes through a separately deployed, long-lived service layer like a web service or RMI.

    Everything that makes "option" 2 above impossible also makes this advice worthless. When you have a large (table-wise), highly interconnected schema, you can't just start shuffling logical subgroups of tables into separate schemas without spending a few years refactoring everything to go through those service layers.


  • TL;DR: Please, Hibernate, I beg you: move load-time checks into compile-time checks, or just optimize the bejeezus out of whatever is going on during load time so that it takes less than 2 seconds.

    Thursday, April 23, 2009

    Exception-Driven Development and Breadth-First Coding

    We've all heard about test-driven development: write your tests first, then write the code. This forces you to think about what you're going to do before you do it, which is always a good thing to do. I don't take it as dogma, but it's a helpful way to think about programming.

    Here's another one: exception-driven development. When you sit down to write a method, think about how it could explode and write some scaffold-like exception handling. If you do this right, then you should have lots of red junk show up in your IDE. The idea isn't to totally write all the error detection code first; just write a scaffold.

    For example, in my current project I'm talking TCP to a flatbed rack scanner that scans racks full of 2D-barcoded tubes--thanks, Ziath! It should be impossible for the same 2D-barcoded tube to appear in more than one slot (well) in the rack, so I want to be sure to detect this because it'll have horrible downstream consequences. So I wrote some stube code like so:


    if (haveSeenTubeBarcodeAlready(tubeBarcode)) {
    throw new ScanningException("Barcode " + tubeBarcode + " appears in multiple wells in rack " + rackBarcode);
    }


    Notice that it won't compile. That's sort of the point. It's half baked. I've found that always having code that compiles isn't necessarily the best way to work. I tend to write code by stubbing-out uncompilable chunks of 10-20 lines of code like this, work on implementing a few lines here, a few lines there, bouncing back and forth implementing a portion of this method, a little bit of this exception-handling code, a smidgen of this loop. It's a breadth-first approach to coding.

    The depth-first approach to coding, where you focus intently on implementing a larger chunk of code (like an entire 50 line method) often results in a lot of wasted time because you quickly get very detail-oriented and distracted. For example, you might spend three hours writing a nice, tight method, with lovely unit tests, only to realize that you actually needed to pass in two other parameters that you need to consult in some conditional logic.

    This approach doesn't work well for everything, which I think is why it's so counterintuitive. If you painted your living room by rolling a roller on one portion of a wall, then stroked a few brush strokes on another wall, then did some sanding on a different wall, only to then put some paint tape and plastic down on the floor and paint three inches of trim, it would be a disaster.

    If this doesn't make sense, you can just write //todo's all over the method about how you should deal with error handling. The basic idea is to think about error handling first, and then don't forget about it when you actually write the implementation.

    Tuesday, April 21, 2009

    Conditional breakpoints are my new best friends.

    Intellij's IDEA has a wonderful feature I discovered the other day: conditional breakpoints. Because the applications I write tend to operate on large amounts of data, the bugs I tend to see are the sort that show up once every thousand or so iterations through some loop. Often the defect is the result of not totally bomb-proofing the application to deal with dirty data. We tend to stream data around, so we don't get nice Lists--often we just get Iterators or Iterables, and for performance reasons, we often don't impose any reproducible order on these collections (so we tend not to use Comparators and my favorite interview question whose answer is LinkedHashMap).

    The result is that you can't just pause at position 3721 in the List. You have to wait for the chunk of data that has exactly the structure you're looking for. Enter conditional breakpoints: right-click the breakpoint, go to properties, click on "Condition". What's really great is that the expression box is itself a fully featured mini-IDE, so you get autocomplete goodness. Sweet!

    Sunday, April 19, 2009

    The No-Argh! Constructor and Dependency Injection

    I've done a few code reviews lately and been really puzzled by the proliferation of no-arg constructors, or as I call them, no-argh! constructors. The developers who make these objects tend to justify them in one of two ways:

    1. Total ignorance
    2. "The framework made me do it."

    Ignorance, as my septugenarian neighbor told me in the yard the other day, can be fixed with education. So here's the education I give people who just seem ignorant of the value of constructors:

    1. Constructors clearly communicate what your object needs to operate. Even without documentation, auto-complete in your IDE will tell you what the object needs in order to function.
    2. In the current environment of fear-of-multicore-and-latent-concurrency-bugs, making a nice constructor lets you mark fields as final and encourages object immutability, thus rendering your code threadsafe without your having to think at all about complicated synchronization policies.

    I usually then go through the exercise of newing-up one of their no-argh! constructors and asking "Now how do I know which fields to set?" Often, their classes aren't missing just constructors--they're missing documentation of all sorts, so it's totally unclear which fields I should set (which relates to my other post about thinking twice about autogenerating setters and getters for everything).

    Very often at this point the light bulb pops up and the developer gets it, at which point I congratulate myself for improving all future code the developer makes and justifying my outrageous salary.

    But what about the folks who use the framework as the scapegoat for their no-argh! constructors? Thou shalt use constructor injection. Any framework without constructor injection is at this point worthless. Even Spring now has constructor injection. I absolutely refused to use Spring until it had constructor injection. I considered it a fatal flaw with Spring, and I'm still highly suspicious of Spring in general because of this blunder, but I'll admit that Spring does have some very nice features.

    Friday, April 17, 2009

    Deploying JPA dependent code into Spring CMT and non-CMT applications makes a big mess.

    I use the Spring framework because my organization insists that I do. I try to not waste too much time complaining about it; the decision was made and I want to be a good corporate citizen. Yesterday I met this Exception:

    Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead.

    What got me here? I wrote a method that calls a "native" SQL query (BTW, I am offended that JPA decided to stick "native" in front of anything involving programmer-defined SQL. "Native" is C/C++, or any language that's close to the metal. SQL defines data relationships, and is probably farther from the metal in the database than Java is from a register). Prior to calling the query, I'd like to flush() the EntityManager so that a client doesn't accidentally leave some data in the EntityManager and then get confused by the fact that the method I'm writing doesn't pickup the data they thought they'd written to the database. Basically I want to save them from themselves, just by a simple call to flush(). I could just mention this in the javadocs, but calling flush() is so simple I figure I'd do it in my method.

    Works great in my sandbox; explodes for another developer with the exception above. What gives?

    In my sandbox, I'm newing things up on my own, explictly creating a new EntityManager and an instance of my class. In the other guy's environment, he's injecting the stuff through Spring. So I have to make my class Spring compliant and not Spring compliant at the same time because one client uses Spring and the other doesn't. So now I need to know the intricacies of the framework into which my code will be deployed, while making sure that it works in the absence of the framework. But the raison d'etre of the framework in question (Spring) is to save me from knowing all the details of transactions. Instead, I have to know all the details of Spring. It's not really 6 of one, half a dozen of the other--Spring is supposed to make my life easier, but it doesn't. It just forces me to let go of some details with which I am very familiar, comfortable, and adept at working with, and pickup a new bunch of new abstractions that so far I find pointlessly cumbersome. If I were a junior programmer, with no working knowledge of JDBC, I'd be utterly clueless about transactions, since they're all hidden by CMT. I find this a disturbing trend because programmers who work with databases and don't know anything about transactions are dangerous.

    At least the JPA docs offer a clue here, stating that getTransaction() can throw IllegalStateException if called on a JTA EntityManager. So it's at least clear that you have to worry about the framework in which your code is going to run.

    Clearly you have to test your code in each framework into which it will be deployed. If you were just using one framework (or no framework), you could rest assured via static analysis and documentation that things were going to work one way, all the time. Using a mixture of frameworks (or a mixture of frameworks and no frameworks, which we do) takes that away from you and requires that you write explicit tests per-framework, which often requires obtuse XML configuration (hello applicationContext.xml and copy-pasted applicationContext-test.xml).

    In short, you give up static analysis (something that serious CS people are really, really good at) and instead spend your time:
    1. Writing an increased volume of tedious unit tests and configuration files, often feeling like your unit tests are really testing your XML configuration. The XML configuration is so complex that it really does need its own tests, so you can't skimp on this and it becomes part of your code "overhead" (tax).
    2. Wrapping your head around declarative transactions

    All for the alleged benefit of CMT (which seems mostly just to avoid the minor nuisances of passing around Connections and being careful about calling close() in finally{} blocks).

    Tuesday, April 14, 2009

    Declarative transactions replace old complexity with newer, more awful complexity.

    JDBC is a bit hard to use in the raw. Dealing with cursor leaks and making sure that you've closed things in finally{} clauses is tedious. But it's well understood. I don't mind dealing with it. In fact, I think application programmers really need to understand transactions in order to code the application properly. That's not to say that I want to force JDBC down everyone's throat. I just want application programmers to appreciate that talking to a relational database does require some understanding of what a relational database does.

    Declarative transactions were supposed to make transaction management simple. They don't. They just make it confusing in new ways that involve annotations, XML, third party frameworks, and good god, AOP. All of this just to avoid a few finally{} clauses and a few calls to close{}? Take a look at this 15 page article from developerworks. 15 pages, just for some common transaction pitfalls. Something smells funny.

    TL;DR: Transaction management is still complicated. I liked it better when I could deal with it in plain Java instead of having to learn Spring, AOP, and annotations.

    Thursday, April 9, 2009

    The Human Genome Project is history. Now what?

    I've had the luxury of working at my current job for nearly 10 years. When I graduated from school with a CS degree in 1999, people were getting paid $60k to write HTML, with the only qualification that you have a pulse. At one interview at the time, I mentioned this "new" thing called XML (which actually was relatively new in 1999), and the CIO of one company said "Oh yeah, I need to look into that." I didn't end up working for that company.

    Anyhoo, I decided to work for this great place that was ramping up its work on the Human Genome Project. For the next few years, at cocktail parties I could actually attract a crowd by telling people I worked on the Human Genome Project. Ah, those were the days. When I catch up with friends nowadays, they invariably ask me what has become of the HGP. What the heck am I getting paid to work on, now that the HGP has been done for the better part of a decade?

    The technology for doing genome sequencing has evolved by leaps and bounds. We're not quite at GATTACA yet, but I no longer consider the rapid DNA sequencing in the movie pure science fiction. So where are we now with DNA sequencing? I'll answer by way of analogy.

    Imagine the year is 1979, 10 years after the Apollo landing. The space shuttle hasn't even lifted off for the first time. NASA's technology has found its way into all kinds of interesting practical uses, but people aren't zooming around the globe much faster. Spaceplanes still haven't shown up. Rapid sub-orbital intercontinental travel still isn't available today, 40 years later.

    If space travel had progressed with the speed of DNA sequencing and genome biology in general, by 1979 the US alone would have been launching spaceplanes about twice a day. Other countries would have built similar facilities. Suborbital flights would have been fairly routine, but not quite yet widespread. Still expensive, but in the $10k range. This would have profoundly changed the world in numerous ways. Just think of how it would change the shipping industry, and how this would ripple through other sectors of the economy to effect daily life. At this pace, by about 1990 (which in my comparison puts us at about 2020), people would be zooming around in spaceplane buses (I doubt folks will ever have their own space plane for a variety of technical and safety reasons). What's the genome biology equivalent here? Right now there are ways to sequence a single human genome in about a week for under $20k. The price continues to plummet, by the way. Illumina, Roche, and SOLiD are some of the more popular technologies. Pacific Biosciences and Complete Genomics are two other companies currently developing the next-next generation of sequencing systems.

    But so what? Why should we want to sequence a genome for $20k? Many of us don't. We want to sequence fractions of genomes from thousands of patients. The price per genome is an interesting point of comparison, but it misleads people into thinking that what we want to do is sequence all parts of a genome. Sometimes that's useful and sometimes it's an incredible waste. Imagine if UPS only sold shipping services in 747-sized batch. You couldn't send a small box without purchasing an entire 747 flight. Focusing on the cost-per-genome number misses the fact that there are thousands of researchers and clinicians out there who could greatly improve your health by just sequencing a tiny fraction of your genome, if only they could order a few kilobases--or even hundreds of bases--cheaply, quickly, and accurately. Many of us are focused on inventing these massively parallel, multiplexed sequencing systems.

    So why is this a good thing to throw money at?

    Because doing so will bring about incredible new ways to accurately diagnose (and eventually treat) the myriad forms of cancer. Because doing so will bring about tremendous advances in the way we treat and diagnose terrible infectious diseases and drug resistant strains of all sorts of nasty bugs. What's the timeline? It's hard to say, but I'm banking on two or three generations being able to look back at how we treat cancer and infectious disease now and being as repulsed as we are now by Civil War era "surgery".

    To use another analogy, if car mechanics worked the way our health providers worked, they would replace the engine every time you needed your oil changed because they couldn't tell that the problem was your oil. In the rare circumstance that the mechanic could deduce that the oil was the problem, he would replace it with olive oil because he couldn't tell what kind of oil your particular kind of car needs. This isn't a diss on health care providers. They're doing the best they can with what limited knowledge they can glean from current technology. Doing all of this sequencing--along with all the other painstaking population genetics and clinical work--will bring about incredible health benefits.

    But what in particular are they paying me for? To help design and build the software systems that keep track of many terrabytes of sequence data that we produce each day.

    Sunday, March 29, 2009

    Got Water?...In Your Basement?

    If you have water in your basement, the first thing to do is read The Original Basement Waterproofing Handbook. Do this before you shell out $10,000, or even $100 on any work. Basement water is one of those things that requires a bit of book larnin' before you go digging around (so to speak).

    This book was the best gift I got last year for Christmas. I kept interrupting my wife's Mario Kart-a-thon with exclamations like "Oh! Sheet Drain!" and "Look at this diagram--this is how our foundation was built!"

    In very heavy rains, especially so during the snow-melt season, our basement has some "minor puddling", as the previous owners told us prior to sale. It dries out pretty fast on its own. I shop-vac it up in about 20 minutes and it dries out in a few hours. Pretty minor. It's a nuisance because it renders a corner of the basement floor unuseable for long-term storage, so I went looking for a solution. Jack Master's wonderful book pointed out that yes, I could opt to tear out the tomato beds, destroy the lawn, put in a trench drain and coat the foundation for about $10,000. Or I could use my incredible powers of observation to notice that the roof downspout dumps water right onto the ground at the corner of the house where the basement gets wet.

    The solution: about $100 worth of flexible downspout pipe. The downspout now connects to this "pipe" and delivers the roof runoff to the back of my lot, downgrade, away from the house.

    Tuesday, March 24, 2009

    Hacking the Keurig

    The good news: my office has a Keurig coffee machine. The bad news: all the free coffee is from Van Hout, which I find undrinkable. The happy medium? I buy my own Green Mountain Coffee pods on the cheap just for me.

    This is my recession coffee plan. It doesn't come close to the tastiness of the $4.00 mocha from 1369, but I just can't afford those anymore.

    Monday, March 23, 2009

    Books: Combat Trauma and the Commando Life

    I keep pestering my commando buddy about what the commando life is like. He declines to comment and instead suggests various books to read. Here are a few I've found really interesting:

    Achilles in Vietnam reveals the astounding differences between society's attitudes towards warfighters and between warring nations in the days of Homer and in modern combat. In Homer's time, warriors respected each other and nations understood that returning warriors required and deserved support, sympathy, and respect. In our time, dehumanization of the enemy leads society to recoil from the warfighters, greatly exacerbating the warfighter's trauma, which can in turn inflict terrible violence on the domestic population.

    On Killing informs us that lethality of warfighters has greatly expanded in modern combat, even since WWII. During WWII, soldiers tended to aim high and deliberately miss their targets, but by Vietnam, soldiers hit their targets with far greater frequency. The book discusses how this change has come about and what price it exacts on the survivors. There are many hard-to-read first-hand accounts of combat trauma here, and simply reading about them is disturbing. Anyone who dreams of war as glamorous needs to read this book to understand how gruesome war really is. An interesting psychological observation here is that in an abnormal environment, abnormal behavior is to be expected. The combat zone is inherently abnormal, and the appropriate response to it is often to act abnormally. Survivors of these situations may suffer terribly while they come to grips with their actions.

    UDT /SEAL Operations in Vietnam blows away the Hollywood mythology of special operations commandos by examining SEAL/UDT operations throughout the Vietnam war. Small teams of highly trained soldiers, supplied with the right gear, good and timely intelligence, and the support of the traditional war machine proved extraordinarily successful against Communist forces. Note: it's out of print, but I got a copy used from Amazon and it was in great condition.

    Spec Ops is a fascinating analysis of a number of the more spectacular declassified special operations over the last 50 years from a handful of different countries. It compares each of the operations along a few different dimensions to highlight the common elements to successful special operations missions (rehearse, rehearse, rehearse, for starters).

    Blackhawk Down stands in contrast to these books by detailing how a seemingly straightforward kidnapping mission turned into a disaster over the course of about 24 hours in Somalia. The book includes many personal vignettes of the soldiers. Another good related book (also by Bowden) is Killing Pablo, which details the murkier (and I suspect more frequent) relationships between our special operations forces and "friendly" foreign governments.

    Wednesday, March 11, 2009

    Scripting the Southwest Checkin Page

    I get kind of antsy flying Southwest. I sit by the computer counting the seconds before I can check in. I start wondering "Can't I just write a script to do this and run it at just the right moment?"

    Yes I can. Someone tried it in Python, resulting in a 1,538 line nightmare and a lot of bug reports from users. It made me think about cost/benefit. How long does it take me to check in online like any other normal human? About 7 seconds. How long is typing 1,538 lines of anything? At 85 wpm, assuming 1 word per line, about 18 minutes. Just typing the script, if you had it in your head, takes 18 minutes. In reality, it probably took the developer about a week.

    It's an interesting coding project, but it's just not worth the effort. I like being the one doing the clicking. If something goes wrong, I need to know immediately during checkin. I don't want an extra piece of software in the way. Using the web site directly is reassuring in this case.

    Here's another option, though: use Selenium help you build a unit test. Execute the unit test to do the login. Here's what it looks like:

    public void testNew() throws Exception {
    selenium.open("http://www.southwest.com");
    selenium.click("//img[@alt='Check In Online']");
    selenium.type("txtConfirmNum", "YOUR CONFIRMATION NUMBER HERE");
    selenium.type("txtLname", "YOUR FIRST NAME HERE");
    selenium.type("lastName", "YOUR LAST NAME HERE");
    selenium.click("submitCheckIn");
    selenium.waitForPageToLoad("30000");
    selenium.click("link=(CHECK ALL)");
    selenium.click("submit");
    selenium.waitForPageToLoad("30000");
    }

    There--10 lines of code. Took about 30 seconds. I myself typed a whopping three lines of code for this (the three lines with "YOUR..." in them).

    Please don't use this code. This should go without saying. If the southwest.com developers feel like it, they can change the name, labels, or css IDs of anything on the page and then you're hosed. By the time you figure this out, you'll be stuck with "Group C" boarding passes.

    Tuesday, March 10, 2009

    Do your unit tests even get to assertFoo?

    Lots of my unit tests involve looking up data in a database. My tests often look something like this:


    public void crazyDatabaseTest() {
    if (...) {
    if (...) {

    }
    else {
    for (SomeThing doodad: things) {
    assertFuz(...)
    }
    }
    }
    }



    Now what happens if things is empty, or if by some chance my conditional logic breaks down and no asserts even get executed? To catch myself from this blunder, I often set a testAnything boolean like so:



    public void crazyDatabaseTest() {
    boolean testedAnything = false;
    if (...) {
    if (...) {

    }
    else {
    for (SomeThing doodad: things) {
    testedAnything = true;
    assertFuz(...)
    }
    }
    }

    if (!testedAnything) {
    fail("Nothing got tested!");
    }
    }

    Friday, March 6, 2009

    Girl Scout Cookies Good. Boy Scout Fudge...nasty.

    It's Girl Scout Cookie season. I love Girl Scout Cookies. On one of the days that the Girl Scouts weren't at Alewife station, a small troop of Boy Scouts showed up pushing...Boy Scout...ahem...Fudge.

    Girl Scout Cookies sound sweet, wholesome, and tasty. Boy Scout Fudge sounds totally gross. Boy Scouts, stick to popcorn please.

    WiFiTrak is an interesting tool to see WiFi networks on your iPhone/IPT.

    WiFiTrak does a great job showing you the WiFi networks in range of your IPT or iPhone. It does a terrible job connecting to them, however. I've bounced around Arlington and Cambridge and I can't connect to the vast majority of the "open" networks. How is a network "open" if I can't join it? I find it unlikely that every "open" network has MAC address filtering. If they did, I'd argue that the network shouldn't be considered "open".

    Nonetheless, it's fun to see all the networks.

    The app is pretty unresponsive when you attempt to connect to a network. The "back" button becomes totally unresponsive and I have to hit the home button and restart the app almost every time it fails to connect to a network. And check out this screenshot.

    I went from waiting for 30s for a timeout to 29, 28, 27,...,3, 2, 1, 0, -1, -2. I'm not sure what it means when I have -7 seconds to wait for a timeout.

    The key to cooking steak

    I have been trying to cook steak like they do in fancy shmancy restaurants for years. I wanted a steak that had none of that tough gray color. I wanted the outside to be nice and seared. I wanted the interior to be about 135° throughout. I wanted it to be reddish throughout. And I wanted it to be tender. I wanted it the cooking to truthfully reflect the quality of the meat. I have been failing in almost every category save the searing.

    Until I stumbed upon a tip in a recent Cook's Illustrated: take your steaks out of the fridge and pop them into a 285° oven for 8 minutes. Then start cooking them as you would normally. This little nugget turns my steaks into exactly what I want. Perfect.

    Thursday, March 5, 2009

    The 2nd track of U2's No Line on the Horizon rocks.

    Buy the 2nd track of U2's new album. Turn it up. I'm trying to rock it to it now without waking up the baby. It's hard not to crank it up...to eleven. It's a bit overly processed and cheesy for U2, but I still dig the track.

    The rest of the album is so-so. U2's slower songs just aren't very consistent. But the second track--holy cow. The production quality is amazing, but I'm reminded by this album that so much of what makes this album good is its production technology. The Edge's riffs are generally kept in balance...maybe too much in balance. I do miss the raw sound of older U2, but at this point, that's very much older U2, like before I was born, so before I get totally pretentious, I think I'll end this post.

    NYTimes iPod App 2.0: better, but offline mode still stinks.


    I started trying to use the NYTimes App on my IPT (iPod Touch--the non-phone iPhone) in the hopes that I could queue up the days' news just before going underground on my commute. Underground without WiFi. The "offline" part of the NYTimes app is what caught my interest. Previous versions of NYTimes haven't been worth the frustration, even when connected to WiFi. I'd try each new update for a few hours, grow tired of the crashing, the empty articles, and the general sluggish performance and duly uninstall it.

    Version 2.0 is a pretty decent app--finally--for use when you're connected to WiFi. If you want to use it while you're connected to the internet, it's finally usable. Enjoyable, even. It crashes less and is more responsive.

    But I'm still not going to use it because the offline features continue to be awful. Yup, I'm going to keep on scraping pages off nytimes.com using Instapaper, the single greatest IPT app out there. Well worth a few clams.

    My first gripe about NYTimes app 2.0 is that the synchronization now happens by section, whereas it used to (or at least appeared to--which reminds me, since there is zero documentation for this application and zero response from the emails I've sent the developer(s), I'm forced to conclude the app is maintained, shall I say, somewhat haphazardly) be done once for all sections. What makes this change unbearable for offline mode is that if I want to read the books section, I have to remember to start the app and then click on the books section before I go off the grid. So if during my underground commute I want to read each section, while I'm in my kitchen before I leave the house, I need to click on each section. Bleah. The NYTimes app makes me want to subscribe to the print version of the New York Times, but for all the wrong reasons. Digital distribution is supposed to make the reader's life easier, not miserable. Using NYTimes app 2.0 in offline mode is like trying to read the paper when someone keeps randomly yanking pages out of your hand.

    Argh! Phantom Article Screeshot

    My other gripe has been there since day one: the app is still plagued by phantom updates. The "phantom update" is a situation in which the app tells you that it's updated the section, but when you click on an article in the section, you get an error message telling you that you have to go online to read the article. This violates one of my rules of user interfaces: don't let your program lie to your users. When the NYTime app tells me that a section has been updated, it means that I can read any article in that section. It doesn't mean that I have a 50/50 chance of being able to read any article in that section.




    If anyone from the NYTimes app dev team is reading this, I will buy the NYTimes app for $1.00 if you fix these problems. I would have paid you $10 for the application when it came out last year, but having suffered through all the buggy updates which claimed to improve stability but didn't, my good will has plummeted by $9.00 (or more accurately, 90%).

    Wednesday, March 4, 2009

    Oracle's dbms_random is really handy for tests.

    I often want to throw random data at some of my unit tests. All I need to do is yank some data out of the database, randomize it, and set it as the driver for a test. Collections.shuffle() is a good way to do this when you can afford to pull a large number of rows into memory, but what if you have a massive data set and you want to get 5 rows, at random, out of the database? I add dbms_random.string('A',80) as random_number to my select statement, slap on an order by random_number at the bottom, and then add Query.setMaxResults(5) to my JPA query.

    Intellij IDEA 8 + MacPro + 64bit VM = HTML Preview Kablooie

    Oh man, am I bummed. My development environment is huge, and I've really been loving running Intellij IDEA 8.1 with -d64. It's a barn burner. But apparently Mozilla HTML Preview doesn't work in 64 bit mode. Argh! The "solution" is to go back to -d32. This is going to be painful. According to Activity Monitor, IDEA is using 5.54Gb of of virtual memory and 1.09Gb of main memory. That's enough memory to make me think that I really do want to keep using the 64 bit VM.

    I haven't been able to find a way to get IDEA to launch with a 32-bit VM. I've edited Info.plist to change the VM to 1.5. I've changed the VMOptions and added in -d32. I even changed the JVMArchs String to x86 instead of x86_64. And in the Java Preferences pane from the Finder, I've put 1.5 on top. Under all circumstances, according to Activity Monitor, the system still thinks IDEA is using a 64-bit VM (it has the "(64 bit") value in the "Kind" column), and HTML Preview won't load. Oh well.

    The other workaround is to just uninstall HTML Preview and use Alt-F2 to open up your HTML files the old fashioned way directly in a browser.

    http://www.jetbrains.net/jira/browse/IDEA-18145
    http://www.jetbrains.net/jira/browse/IDEA-18900

    Tuesday, March 3, 2009

    JPA Strengths and Weaknesses

    From the JPA Spec: "An entity is a lightweight persistent domain object." Right there, on page 17 of the spec, is evidence that hard-core, efficient querying is not something that JPA was designed to do. Not convinced? Page 22 offers another nail in the coffin: "Every entity must have a primary key." This is a problem because complex queries tend not to return Entities--they return fragments of Entities and the results of functions applied to their properties. The resulting cursor thus does not map to one or more Entities, which makes hacking the result set into a spec-violating fake-o Entity via @SqlResultMapping with a fake @Id a bad idea. The spec makes no allowance for cursors whose columns are just data, not Entities. Any result set mapping has to refer to at least one Entity. Violating this requirement in Hibernate gives you org.hibernate.cfg.NotYetImplementedException: Pure native scalar queries are not yet supported.

    A SQL query returns a result set. A row in a result set is often not persistent because it's the result of a join of some number of tables. Furthermore, these virtual rows are just that--virtual--and thus don't have a primary key. I'm not dissing JPA; I'm just saying that it's not the right tool to use when your queries require multiple table joins. Put another way, if you are stuck with a hyper-normalized database, where you can't build a meaningful piece of information without joining a half dozen tables, JPA is not going to help you with queries. If you have the luxury of not having to do joins to extract useful business objects, consider yourself lucky.

    JPA is great at what it's designed to do: make it easy to new-up some objects and cram them into the database, or update some existing records. I find it misleading, though, that JPA claims that Entities are POJOs because I am always fighting "detached" entities (org.hibernate.LazyInitializationException: could not initialize proxy - the owning Session was closed) resulting from lazily instantiated fields. Maybe we haven't configured Spring properly. Maybe something about the way we're using HiveMind in Tapesty 4.0 is violating the spirit of container-managed persistence. But I think it's insincere to claim JPA Entities are POJOs. They're closer to POJOs that previous EJBs, but they still have their encumbrances.

    JPA is a huge help to my tests, where most of the time performance isn't a concern, and traipsing over half dozen object relationships just ain't no thang. But when I care deeply about performance, I'm always going to revert to SQL. I know SQL very well, and I know database optimization very well. EJQL just doesn't have the power of SQL, and I doubt it ever will. The spec allows one to escape to "native" SQL, but whenever you do this to write a query, you end up returning a List<Object[]>. Ugh. Anytime you see that as your return signature, you know you're at the edge of the framework. It's a necessary evil for JPA, given the database vendor neutrality that the spec is trying to give us, but it's an enormous downer, and is one reason why I'm still looking for a good O/R tool that lets me write plain SQL but gives me a type-safe way to interrogate the cursor that comes out of a query.

    One of my colleagues actually wrote just such a tool. It's phenomenal. You write a simple stored procedure, and the tool inspects Oracle's metadata to autogenerate a type safe "bean" class that describes the columns in the cursor. Using event-driven architecture, you listen for rows, or you can get them all at once as a List. It's dead simple to use, and I'm surprised Hibernate and TopLink don't have something like it. I need to change it to be a bit more declarative, but basically here's how it works:

    List<ListFoo.RefCursor1> fooRows = new ListFoo("Param1",34.2).executeQuery();

    ListFoo.RefCursor1 is an autogenerated bean that echoes the columns in the ref cursor defined by the stored procedure.

    Nice and neat. Yeah, you gotta know SQL. But some of us actually think that knowing SQL and Java makes you a kick ass programmer.

    Selenium Tests and Firefox Crash Recovery

    When my Selenium tests start today, they first popup the Firefox dialog asking whether I'd like to start a new session or restore my old one. Yesterday Firefox didn't nag me like this.

    Solution: type about:config into Firefox's address and then set browser.sessionstore.resume_from_crash to false.

    I personally find the crash recovery to be a big pain in the butt anyway. I am always crashing Firefox. Crashing Firefox is pretty much a job requirement.

    Why is Selenium starting all my Firefox windows with an empty profile?

    One of the things I love about Firefox is how you can customize it. I will forever use Firefox because of the web developer plugin. I save countless hours by editing HTML and CSS in place with this plugin, bypassing the various pitfalls of different web UI frameworks. One reason why we started using Selenium is because of the Selenium IDE plugin.

    So imagine how startled I was when my automated UI tests started popping up screens on my desktop with the empty default profile. Writing a test is like writing other code--you make mistakes. And to fix those mistakes, you need the Selenium IDE. But the plugin wasn't showing up in the Firefox windows that my test cases were spawning.

    Simple solution here: when you start up the selenium server (I use selenium-server.jar), tell it where your Firefox profile is by using the firefoxProfileTemplate option. On my Mac, it's under /Users/[your user name]/Library/Application Support/Firefox/Profiles.

    This cuts both ways, however. If you have some non-standard plugins (such as NoScript), you may end up running your Selenium tests in an environment that's totally different from that of your typical user, which is a major no-no. To avoid this, you could configure a different, cleaner profile, marred only by the Selenium IDE plugin. Or you could be lazy like me and just leave it to the build and QA team to run the tests in a cleanroom.

    Burned By Tapestry 4's disable-caching again.

    Using my fancy new development desktop (MacPro 8 Core, 8 Gb), I figured I'd save some time by running my Selenium tests in parallel. Tapestry started bombing at random, dumping this nugget to the log file:

    Property visitState has already been accounted for by the element at Annotation @org.apache.tapestry.annotations.InjectState(value=visit) of public abstract

    This is what happens when you turn on Tapestry 4.0's disable caching flag (-Dorg.apache.tapestry.disable-caching=true). Ugh. I have to turn this flag on for development, and then turn it off for proper testing? Check out http://issues.apache.org/jira/browse/TAPESTRY-846 and its sibling http://issues.apache.org/jira/browse/TAPESTRY-848. Given the comments on these tickets, I don't think I'll spend the time attempting the patch.