Ken Sipe

Technology Director at Perficient

Speaking at the following events:
 

 ()
This is syndicated content from http://kensipe.blogspot.com/feeds/posts/default
 
As part of the Lambda Lounge, established by Alex Miller (Thanks!!), we have started a group to study the SICP. We just started and are going virtual... if you would like to participate, email or tweet me (@kensipe). In the process of studying SICP and LISP, I plan on focusing on Clojure. Clojure was previously on my machine as I was reading Stu's book Programming Clojure, however increased usage would require some maturing of my tools. This led to the discovery of Mark Reid's blog on Setting up Clojure for Mac OS X, however I assume based on updates and changes over the last year, it requires some updates.

Read Mark's blog first... getting clojure, the clojure-contrib and jline. Here are the differences:
  • Create an environment variable CLOJURE_HOME assigned to the directory for clojure
  • Create the /bin directory for the clj script included below
  • Add $CLOJURE_HOME/bin in the $PATH
  • Clojure is distributed with clojure.jar in the CLOJURE_HOME... so I maintain that. It is expected that contrib and jline are in the same directory.
  • Mark's clj script has been updated below... The previous script invokes classes which are depreciated.
Here is the updated script:

#!/bin/bash
# Runs Clojure using the classpath specified in the `.clojure` file of the
# current directory.
#
# Ken Sipe 2010-02-20
# Inspired by Mark Reid
# original: http://github.com/mreid/clojure-framework/blob/e1c80cc650f448713243be8272dba1fa3c1a7cea/clj
#
# This scripts expects $JAVA_HOME and $CLOJURE_HOME to be defined for the system it is running on
# The clojure.jar is standardly in the clojure_home dir (and not in the lib directory), so
# the expectation is that clojure-contrib.jar and the jline.jar are placed in the same dir

CLJ_DIR=$CLOJURE_HOME
CLOJURE=$CLJ_DIR/clojure.jar
CONTRIB=$CLJ_DIR/clojure-contrib.jar
JLINE=$CLJ_DIR/jline-0_9_5.jar
CP=$PWD:$CLOJURE:$JLINE:$CONTRIB

# Add extra jars as specified by `.clojure` file
if [ -f .clojure ]
then
CP=$CP:`cat .clojure`
fi

if [ -z "$1" ]; then
java -server -cp $CP \
jline.ConsoleRunner clojure.lang.Repl
else
java -server -cp $CP clojure.main -i $*
fi
Love the TextMate support and the history support in JLine!
 
I was able to attend speakerconf 2010 this year in Aruba. It was an amazing gathering of talented leaders in the software development space. Each day of the conference started with several 15 – 30 minute talks lasting roughly 3 hours. According to the retro at the end of the conference, most people enjoyed and preferred these short talks as a primer to longer conversations in the afternoon along the pool or beach. Most talks were met with the struggle to maintain time often due to the many follow on questions generated by the subject. As far as I know the subjects were selected by the speaker and they were generally not preannounced. It is impossible to capture the essence of the conference and I make no effort to cover the most valuable part of it… the evening free form discussions. Most of the information went so fast that it was difficult to mentally be engaged and take notes. Here is a glimpse of what was discussed.

Day 1
Stu Halloway
Stu introduced the ADD development… which is alcohol driven. Stu then jumped into clojure, introducing a private testing project he is working on called circumspect. Stu offered up some advice for working in clojure:
* be less clever
* idiomatic clojure
* make adjectives not nouns

The focus on circumspect was to provide an idiomatic testing option for functional testing in clojure. Stu mentioned that through his testing exploration that an test he created actually detected a bug in the clojure code.

Dennis Byrne
Dennis is the first to discuss a topic on hardware and general process constraints in a talk titled Memory Barriers. The challenge is that virtual machines and hardware interfaces reorder code in ways most developers are not aware of. Fences or memory barriers define for the VM what can and can’t be reordered. There was some discussion on the expensive nature of these barriers.

Matt Deiters
Matt introduces a challenging problem that he has been working on in a story about vertex. The problem stems from social network analysis… it doesn’t work in SQL. Their previous implementation in SQL might take an hour to find all the details need for just 2 degrees of separation in a social network. Matt introduces neo4j an open source nosql graph database and neo4jr-simple, which is a RESTful ruby API. I enjoyed the fact that Matt wasn’t a No SQL bigot… He espoused a “Not all SQL” approach, using alternatives where SQL databases fall down.

Fred George

Fred displayed a triangle with management, story and programmers (or workers) at the vertexes. He wanted to focus on the relationship of the story to the programmer. Fred focuses on a Lean process and raises the question if acceptance tests are really lean? This stirred some debate. In the end, It appeared to me that what was being argued in most cases was the definition of acceptance tests vs. other types of tests. I don’t know if I agreed with Fred’s position, which we discussed at length the rest of the conference. My favorite part of his talk was a list of words or phrases that are flags that your project isn’t Agile. They include: Review, I need it in writing, write me a ticket, plan, checklist, code freeze, gant chart.

Brian Goetz
Brian’s talk is on how we find ourselves in this mess with multi-cores. Brian gives a wonderful history of processors defining them in eras: CISC era, RISC era, and Multi-Core era. The big take-aways for me include:
* a move from explicit to implicit parallelism
* moore’s law favors bandwidth over latency
* cost of memory access

With the speed of today’s processors, it takes 200-300 clock cycles to get 1 memory fetch, where it used to be 1 cycle. As latency increases we need more caches. Memory access times by clock cycle are roughly:
* Register ~ 1 cycle
* L1 ~ 3
* L2 ~ 15
* RAM ~ 200 – 300

Brian states “Memory is the new disk”. Clearly memory access is the new problem to solve in computer hardware. Brian also states that there is no programming model for this new multi-core era. Based on the talk, it appears to be that the industry has a new chicken or the egg problem. Hardware is struggling to evolve, because there is no software development model. Software is struggling because there isn’t a new standard. Brian concludes with latency is the enemy and the industries need to provide development teams with new tools. There will be a growing need to understand the memory access model of a running program, which can only be determined at run-time at the chip level. It will be necessary to understand the Integer pipe and memory cache misses. At the end of Brian’s talk, Nygard mentioned Chuck Moore’s Multi-Core chip where each core runs a tiny forth program. It is something I’ve added to the list of things to look into.

Michael Feathers
Michael discusses challenges to object oriented and functional language hybrids. There is background that I didn’t record, but one of the great observations is that functional programming is a “tell, don’t ask” model and object oriented programming is a “ask, don’t tell” model. In the FP approach, programming logic and data seem to bubble up the call tree and in the OO approach the opposite is true, logic seems to be pushed down the tree. Michael then looks at what it looks like conceptually if FP is on top of OO or OO is on top of FP. Michael then introduces the group to an algorithm used to solve morphological watersheds :)

Day 2
Steve Vinoski
Steve introduced us to a product his startup company Verivue, Inc is producing for the big media companies for streaming video. The IO throughput numbers were unbelievable. The software at the core of the product… Erlang!

Obie Fernadez
Obie and Desi introduced us to the agile practices of Hashrocket, indicating that they have evolved past iterations and they question the value added by iteration activities. They develop in one-week windows and use Pivotal Tracker for tracking stories and progress. The term “Iteration” isn’t even in their vocabulary. Later in the week, I challenged Obie to see if the word “week” replaces the word “iteration” as a convenience and to see if they were doing all things an ”iteration” would entail. The short answer is No! Clearly pivotal tracker desires a good look. Another tool Obie mentioned later in the week was Balsamiq... a tool used for screen mocking. Yet another tool to checkout.

Phillip Hanrigou
Phillip starts with the phrase “any sufficiently advance technology should feel like magic”. He continues with the fact that 30% of Amazon’s sales is based on recommendations. This was an interesting talk that had me flashing back to Matt’s talk on neo4j looking for synergies.

Oren Eini
Oren suggests that TDD doesn’t scale… with a beautiful picture of a motorcycle with training wheels. The major point is that unit tests are bad and scenario based tests are good.

Brian Marick
The take away… “I don’t want to care about anything until I absolutely have to care about it”… Love it!

Dave Thomas
Dave introduces us to vector functional languages such as KDB and k. These specialized languages are the workhorses behind querying ½ terabyte of data in 1 second for the hedge fund industry. Dave claims this is the most interesting technology to come along in 25 years. If interested, he suggests getting started with j from jsoftware.com

Eric Yew
Eric gives us a deep dive into GPUs. He was on the NVidia team that built the technology. Eric suggests that GPUs are making the industry rethink previous assumptions that are not correct in the GPU world. How would we code differently if:
  1. There were no costs to threads
  2. And no costs to context switching
The approach suggests that memory lookup is too expensive (seems like we keep hearing this :). It is cheaper to recalculate a value on a 100 threads then to retrieve from memory.

This model supports scale and throughput and not latency. Latency once again is sacrificed.

The memory model is upside down as well from the traditional general processors. There are 3 memory access points:
  1. global - ~ 100 cycles to get to it
  2. shared
  3. local – stacks
It appears that the old paradigm of being abstracted from the hardware is over. The benefit is that the price of 2 Teraflops is near nothing… thank you computer gaming community!

Day 3
Neal Ford
Neal did a wonderful job presenting a presentation on presentation :) at a conference on conferences… it was a very meta way to start the morning. The big take away is http://presentationpatterns.com/ a site and forth-coming book on presentation styles, techniques and anti-patterns. The biggest benefit of the book is its focus on the practical. It will include templates for good practices.

Amanda Laucher
Amanda starts by defining the classifications of assholes and morons which got a lot of mileage the rest of the day. Amanda focused on educating us on terminologies in the category theory space with examples in F#. Here was what I was able to capture:

* Catamorphism - a fold
* Anamorphism – unfold
* Endomorphis – takes a type and returns the same type
* Homomorphism – takes a structure and returns the same structure or shape
* Monoid – A Functor
* Monad – Applied context
* A Monad is a monoid in the category endofunctors

Dave Hoover
Dave discussed the value of Resque and how it help in the development of Groupon.com

George Malamidis
George continues the conversation on memory access times and threads, introducing us to Node.js as a high scale solution for the web.

Michael Nygard
Thoughts on consistency – what a great talk! Michael introduces the concept of an observer and a super observer in a system through an object lesson. He then follows it up with the fact that with in a database there are often inconsistent states… the point is at the end of a transaction there cannot be any inconsistent states. For example, it is possible that an insert into the database would result in a violation of referential integrity… this is ok and necessary until a commit occurs.

Consistency is based on time. Frozen time means no change.

Another practical example:
2 ebay users making the same query at roughly the same time get different query results. This is viewed as being ok, because from the perspective of the observer it is consistent (they have no idea). From the perspective of a super observer there are some inconsistencies. This led to some discussion in mainly of the corporate failures stemming from the complexity created when trying to maintain consistency from the perspective of a super observer (which is a fallacy and trap).

Aslask Hellesoy
Another agile talk… this one focused on visualization of the return on development through CFD graphs. The suggestion was that velocity is a questionable valuation.

References included: CFD Details and a google spreadsheet
Robert Martin
I don’t think you can summarize Uncle Bob… you have to experience him :)


Summary
I made a couple of interesting observation / assessments through the conference. The first is there was a large percentage of the speakers / attendees that had a strong EE background. I truly enjoyed revisiting challenges at this level again.

Here are some of my takeaways from the conference:
  1. We are at the beginning of a new era in field of computing. One in which we do not have a standardize model or tools
  2. There is a need for programming solutions which reduce (or leak) the abstraction from the memory model that is generally preferable in the previous programming model. It will be necessary to indicate that values in memory are near, far or very far. It will be necessary to be able to measure and understand the cache miss rate.
  3. We appear to be at our limits in our pursuit to improve latency. Worse…. Latency is the component that is consistently sacrificed to increase throughput.
 
After reading the comment on a post a few weeks ago regarding Intellij 9 and Gradle, I had to laugh. Having worked on open source projects for 15+ years and presenting at countless conferences and user groups, I have stated roughly the same comment countless times... "It's open source, if you don't like it or you want a change, then do it!". Thanks to Peter for the suggestion and encouragement. It clearly seems that my actions were a surprise as Peter's first comment in a private email exchange was "Great you're doing this!"

Intellij 9 Development Experience
Getting the source and setting up for the first run was fairly painless after you find the download information. It was nice to see that the code repository was git. I ran into 2 issues in getting setup:
  1. Git clone failed with an unclear message initially. It turned out I was trying to clone using the git protocol behind a corporate firewall which was blocking it. As soon I was not behind the firewall, all went well. Be prepared... it is 924 MB.
  2. Setting up the run configuration wasn't complex, but it wasn't as advertised.

Here is what the Building and Running from the IDE from the checkout and build page should read:
  • Open the project as a "new" directory-based project
  • Configure a JSDK named "IDEA jdk", pointing to an installation of either JDK 1.6 (recommended) or JDK 1.5. This is on the project tab of the project structure.
  • Increase the compiler heap size... default is 128M, it works with 512M :)
  • Use Build | Make Project to build the code
  • To run the code, use the provided shared run configuration "IDEA".

Important notes:
  1. You will see that there are circular dependencies between 4 of the project modules.
  2. With the default compiler heap size, you will get a compilation error ( which is erroneous )
  3. There are a number of deprecation warnings during compilation... this might be a great place for someone to get involved and do some clean up.

After you have an Intellij running another instance of Intellij :) Here is a list of resources to be familiar with:
  1. Source Repository Layout
  2. View PSI structure (under the tool menu) - PSI is an IDEA AST abstraction, which was the toughest learning curve. Learn it! Then look at the following clases and their usage in the code base: PsiUtilBase, PsiUtil, PsiResolveHelper, PsiResolveHelperImpl

End Result
In the end, I spent 4 hours adding in additional gradle task support into Intellij 9, created a patch and submitted to Peter... a few days later I was informed that it was committed to repository. Enhanced Gradle support is on it's way!

I had mixed feels when IDEA announced it was going open source, but now that it is... Let's get in there and make this an even better editor!

Kudos to JetBrains and it's community!
 
I was off for the holidays which gave me some time to clean out the storage area. I ran across some notes from a conference I attended in 1998 and 3 core principles stood out that I thought I would share as we start this new year.

Core Principles (as I wrote them many years ago):
1. Smaller is better than larger
2. Understood is better than unknown
3. Progress is better than promises

Smaller is better than larger
There was nothing else written and I left nothing by way of context... regardless of what it meant then, it is clear that this is a great principle when it comes to code. Less code that does the same amount of work is better. The paradox is that it may take a little more time to develop a smaller code solution, but it pays off. The skill is in not being so abstract and so small as to not be readable or maintainable. It is this balance that makes a true software craftsman.

Understood is better than unknown
If there is one word that we would use to describe the issues of software development... it is the word "unknown". Estimating the known is easy. Estimating the unknown, is unknowable. The skill in software development is to separate the unknowns from the knows. Estimate the knowns and provide a SWAG estimate for the unknowns... these are estimates that you need to keep in check. It is best to delay the estimates and work of the unknowns until ( what Kevlin Henney brilliantly describes as) the last responsible moment. Of course this only works for the known unknowns... what will get you is the unknown unknowns:)

Progress is better than promises
This is why I have been a practitioner and trainer for XP and agile practices for many years. It is all about developing business value on a regular and iterative basis. Promises are meaningless... progress is all that matters.

Happy New Year!!
 
One of the hidden gems of the Intellij 9 release is it's support for Gradle. Some of the information on the web is out of date and some features are not intuitive. This post will detail some of the nuances and follow it up with a wish list for the next update :)

Gradle Support
The information from JetBrains is out of date. In order to setup gradle after the GA release, all you need to do is configure the gradle home under File -> Other Settings -> Template Setttings.


Gradle Intellij Nuances
After setting up IDEA to work with Gradle, you'll need a build file. I didn't discover a way to create a gradle build file from IDEA. Just create a new file and name it "build.gradle" in the root of the module directory structure.

When writing the build script, Intellij expects to work with gradle in a way that is depreciated. It will pick up on a createTask(..) method. If the cursor is on a task such as this, and you hit ctrl+shft+F10, it will auto-discover and run that task in the script. (cool stuff)

Hotkeys to know:
ctrl+shft+F10 - runs task the cursor is on
shft+F10 - runs the last script (or the last run)
alt+shft+F10 - pops up a run menu (with all previously run gradle run configs)

The figure below shows an alt+shft+F10 when the cursor is on a task called task2.


If you code gradle in a non-depreciated way... then IDEA does not auto-discover the tasks. Below is what one would expect in a gradle build file:

usePlugin 'java'

repositories {
mainCentral()
}

// from the gradle user guide
// http://www.gradle.org/0.8/docs/userguide/tutorial_using_tasks.html
task hello << {
println 'Hello world'
}
task intro(dependsOn: hello) << {
println "I'm Gradle"
}

Tasks would normally be created this way... It is still great that IDEA has gradle support in general regardless of this oversight. After a run configuration is configured with a targeted task, it worlds with no pain. Also you could add:
 defaultTasks "intro"
and gradle runs this as the fallback when IDEA doesn't provide the required task.

Intellij / Gradle features I'm looking forward to:
  1. IDEA understands proper task configuration
  2. IDEA makes it easy to create new projects with a structure maven and gradle expects.
  3. IDEA treats gradle as a peer with Maven and Ant. This includes:
    a. Gradle Window (similar to the ant build and maven projects panels)
    b. Before Launch Gradle tasks in the run configurations

 
Well the latest update for Java from Apple came through recently. Destroying all in its path...
If you followed my advice in the past on getting Java 1.5 working on a Snow Leopard, then the new Apple update destroys that with the follow error for a Java 5 java -version:
Error occurred during initialization of VM
Unable to load native library: libjava.jnilib
Abort trap

I didn't track down the exact issue... however it is easy to detect that the Java update converts the symbolic links for Java 1.5 to point back to Java 6... Apparently Apple is NOT sorry for their crazy choice of ignorantly doing this in the first place. The solution is to completely go through the process out lined at OneSwarm again and re-establish the symbolic links. This will require you to delete the symbolic links that point to "CurrentJDK" for 1.4, 1.4.2, 1.5, and 1.5.0.

After edits an ls -l in the /System/Library/Frameworks/JavaVM.framework/Versions should look like:

drwxr-xr-x 15 root wheel 510 Dec 5 22:18 .
drwxr-xr-x 12 root wheel 408 Dec 5 22:15 ..
lrwxr-xr-x 1 root wheel 5 Dec 5 21:35 1.3 -> 1.3.1
drwxr-xr-x 3 root wheel 102 Jul 20 18:35 1.3.1
lrwxr-xr-x 1 root wheel 5 Dec 5 22:18 1.4 -> 1.4.2
lrwxr-xr-x 1 root wheel 14 Dec 5 22:18 1.4.2 -> 1.4.2-leopard/
drwxr-xr-x@ 9 root wheel 306 Feb 12 2009 1.4.2-leopard
lrwxr-xr-x 1 root wheel 5 Dec 5 22:01 1.5 -> 1.5.0
lrwxr-xr-x 1 root wheel 14 Dec 5 22:00 1.5.0 -> 1.5.0-leopard/
drwxr-xr-x@ 10 root wheel 340 Dec 5 21:59 1.5.0-leopard
lrwxr-xr-x 1 root wheel 5 Dec 5 21:35 1.6 -> 1.6.0
drwxr-xr-x 8 root wheel 272 Nov 8 14:35 1.6.0
drwxr-xr-x 9 root wheel 306 Dec 5 21:35 A
lrwxr-xr-x 1 root wheel 1 Dec 5 21:35 Current -> A
lrwxr-xr-x 1 root wheel 3 Dec 5 21:35 CurrentJDK -> 1.6


Happy coding!
 
I've been very interested in providing some syntax highlighting in my blog posts, which I've been slow in general getting out... ( I have 6 or so queued up ). I finally found something I like and thought I would share the love. Special thanks to @ecounysis on twitter from whom I made this discovery. There are no excuses for my future posts... they will all have nice highlighting and be more easily copied (via the clipboard feature).

The place to start is: http://alexgorbatchev.com/wiki/SyntaxHighlighter:Hosting

From there you'll see the link to: http://blog.cartercole.com/2009/10/awesome-syntax-highlighting-made-easy.html

One important thing to note for those on blogger / blogspot. The initial update of the head tag didn't take for me. The solution for me was to add the highlighting entries at the tail end of head, just prior to the closing tag.

In addition to many of the bundled options... I've added F# and Objective-C from undermyhat.
 
I have long documented some of the issues and differences for a Java developer on the Mac OS X. Most of that has been on Leopard, OS X 10.5. For quick links the most significant articles are: Java is a 2nd Class Citizen on MAC OSX, Fixing Java Memory Tools on Mac OS X and Java 7 on Mac OS X.

Monday I was faced with a new situation, Snow Leopard OS X 10.6. There are several issues with Java on the new Mac OSX 10.6 "Snow Leopard". This post will outline the known issues and will provide fixes for several of them (or provide links where fixes are already explained).

Known Issues:
  1. Java 5 is not installed
  2. Java 5 symbolic links are set to Java 6
  3. jmap - some aspects of jmap do not function
  4. javascript runtime is still missing
As a consequence, applications that used Java 5 either don't work, or provide a message as shown here from my favorite stock trading application.



* I'm still looking for more issues, so if you can add to the list, please add comments, email or twitter.
** I have tested the latest BTrace and it works fine.


Java 5 on Snow Leopard
Whither you agree with the removal of Java 5 or not, it is unbelievable to me that the Java 5 symbolic links are pointing to the CurrentJDK which points to 1.6. For those new to this space the Java version live under /System/Library/Frameworks/JavaVM.framework/Versions . I would advise you to either:
  1. replace Java 5 using instructions provided at leopard http://wiki.oneswarm.org/index.php/OS_X_10.6_Snow_Leopard
  2. simply delete the java 5 symbolic links
As a side note, I use a script for swapping between JVM versions. I modified the original script to just include 3 important functions when sourced in. I'll add the script to the bottom of this post as a reference. The 3 functions are jvms, setJava, unsetJava.
  • jvms reads through the version directory and reports what it reads of the file system.
  • setJava allows you to type in the name of the version and switches JAVA_HOME and path to this version of the JVM.
  • unsetJava reverts the set to the original java version
JMap on Snow Leopard
My initial execution of jmap on snow leopard failed with the following error message:

kensipe$ jmap 3407
Attaching to process ID 3407, please wait...
sun.jvm.hotspot.debugger.NoSuchSymbolException: Could not find symbol "heapOopSize" in any of the known library names (-)
at sun.jvm.hotspot.HotSpotTypeDataBase.lookupInProcess(HotSpotTypeDataBase.java:399)
at sun.jvm.hotspot.HotSpotTypeDataBase.readVMIntConstants(HotSpotTypeDataBase.java:319)
at sun.jvm.hotspot.HotSpotTypeDataBase.(HotSpotTypeDataBase.java:88)
at sun.jvm.hotspot.MacOSXTypeDataBase.(MacOSXTypeDataBase.java:36)
at sun.jvm.hotspot.bugspot.BugSpotAgent.setupVM(BugSpotAgent.java:578)
at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:499)
at sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:337)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:163)
at sun.jvm.hotspot.tools.PMap.main(PMap.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.tools.jmap.JMap.runTool(JMap.java:179)
at sun.tools.jmap.JMap.main(JMap.java:110)
Debugger attached successfully.

The good news is that the debugger attached successfully :)

Information regarding this issue is scarce. It turns out to likely be an issue with the version of Java that is distributed with snow leopard. The JDK 1.6 rev 14 appears (based on non-confirmed web sources) to have this issue. The latest is JDK 1.6 rev 16 should have the fix, however using the apple distro, we are left waiting for Apple for the fix. Although jmap fails with no arguments it does succeed for certain commands such as jmap -histo 3407.
After replacing the JDK 5 with the leopard distro as advertised above and switching to this version with the script above, jmap works fine. However jdk 5 jmap only works against jdk 5 running applications. It fails against java 6.

javascript
As record in a posting from a year ago, called Fixing Java Memory Tools on Mac OS X, the javascript library is still not provided by Apple as part of the Java distro. You will need to following the instructions there to fix it. Unfortunately the Rhino download has disappeared from the Mozilla site, so if you didn't keep a copy, ping me and I will send you want I have. You will also need to duplicate this for each version of Java if you install the Java 5 Leopard package.
The test to see if this is setup correctly for you is: jrunscript -q
If that command only returns AppleScript, then you are missing the javascript engine which is need for a number of tools in the JDK.

After these changes, it appears that Snow Leopard is ready for some hard core Java development. Good Luck and Happy Coding!

Java Switching Script that I use:



# *************************************************************************
# Originally written by Shawn Erickson - shawn at freetimesw.com
#
#
# Last Update: 2008y 08m 8d
#
# The following functions are meant for use with bash shell which is currently the
# default on Mac OS X 10.4 (starting with 10.3 IIRC) unless otherwise configured.
# Good info on why not sym links: http://lists.apple.com/archives/Java-dev/2006/Jan/msg00290.html
#
# from: http://lists.apple.com/archives/Java-dev/2005/Aug/msg00506.html
# http://homepage.mac.com/shawnce/misc/java_functions_bashrc.txt
#
# *************************************************************************

### Prompt ###

export CURRENT_MODE_STRING="";

### Java Environment Functions ###

J_VERSIONS_DIRECTORY="/System/Library/Frameworks/JavaVM.framework/Versions"
J_COMMANDS_SUBPATH="Commands"
J_HOME_SUBPATH="Home"

function availableJVMs()
{
ls -1 $J_VERSIONS_DIRECTORY | grep ^[0-9].[0-9]
}

function jvms()
{
clear
echo "Available JVMs: "$jvms
ls -1 $J_VERSIONS_DIRECTORY | grep ^[0-9].[0-9] # look to remove redundant call
echo " "

echo "Current Java:"
java -version
}

function setJava()
{
local target_jvm=""
local jvms=$(availableJVMs)

# Validate that the user requested an available JVM present on the system

for jvm in $jvms ; do
if [ "$jvm" == "$@" ]; then
target_jvm=$@
fi
done

if [ "$target_jvm" == "" ]; then
echo "Unsupported Java version requested"
return;
fi

# If we get here the user asked for a valid JVM, so configure it

echo "Configuring Shell Environment for Java "$@
export JAVA_VERSION=$@

# First unset any current set java, back to default before doing configuration
_unsetJava

# Generate the paths needed for the JVM requested
local jcmd="${J_VERSIONS_DIRECTORY}/$@/${J_COMMANDS_SUBPATH}"
local jhome="${J_VERSIONS_DIRECTORY}/$@/${J_HOME_SUBPATH}"

# We save the original path so we can toggle back if unset
ORIGINAL_PATH="$PATH"
PATH="$jcmd:${PATH}"

# We save the original JAVA_HOME so we can toggle back if unset
ORIGINAL_JAVA_HOME="$JAVA_HOME"
JAVA_HOME="$jhome"

# Update command prompt mode tag to note JVM setting
CURRENT_MODE_STRING="J$@"

echo "Current Java:"
java -version
}

function _unsetJava()
{
if [ "$CURRENT_MODE_STRING" != "" ]; then
PATH="$ORIGINAL_PATH"
JAVA_HOME="$ORIGINAL_JAVA_HOME"
CURRENT_MODE_STRING=""
fi
}

function unsetJava()
{
echo "Configuring Shell Environment for default Java"
_unsetJava

echo "Current Java:"
java -version
}


 

The book 97 Things Every Project Manager Should Know put together by Barbee Davis is out, and worth picking up... why the magical 97? Apparently 101 was way over used... or perhaps Barbee couldn't get the last 4 in before it was time to publish :)

This book has some great insights from a number of sources, which is a huge value add to any project manager or senior developer. This is the top advice from experts in the field around team building, running a software project, and generally how to herd cats.

Most books in this space are limited to the experience of the author and his/her circle of influence. The beautiful part of this book, is the fact that Barbee has captured the most interesting and important tid bits stories of success and failures of software projects from a large sample of project managers and developers each providing a view from different industry verticals.

Well done Barbee!
 
It's official... I will be heading to Oslo, Norway in September for JavaZone. I will be presenting Debugging your production JVM, which is talk largely focused on BTrace. Also if you didn't realize it yet... BTrace has moved to the kenai site: http://kenai.com/projects/btrace/pages/Home
 

I will be speaking on groovy and grails at the up coming 2GX conference in New Orleans in October. If you are in the groovy space or you are just looking, you don't want to miss this... all the big names will be there. Most of the leading authors and committers in the groovy, grails and griffon space will be there!

How I got to be among them is beyond me... but it sure is groovy!

My topics include security, where the focus will include defense against hacking techniques such as injection flaws and XSS, followed by a walk through of the security plugin and how to secure urls and services. The second session is on Java memory management, the memory demands of a dynamic language, along with tools on debugging Grails applications in production.
 
I have received a number of requests from team members and session attendees for career path guidance. Here is what I threw together... I would enjoy others comments on what makes a difference for them!

My Guiding Principles

1. Become an expert at what is coming...
I've been lucky to live in an area of the country which is conservative when it comes to technology, which can at times be very dissatisfying. I travel to the coast (usually west) at least once per year for a technical conference. Through observation and networking, I learn the passions of the technical industry (not the vendor hype... beware the vendor hype). I make time to become well versed in these spaces. Within 1 year clients are interested and are looking for advice, within 2 they are looking for consultants. This is absolutely necessary to be a good architect anyway.
Suggestion: No one should work in our business without reading the mythical man month. Also read the pragmatic programmer or the quick guide if not the book: http://www.codinghorror.com/blog/files/Pragmatic%20Quick%20Reference.htm

2. Have the final say on matters that affect you...
Each of us is responsible for ourselves. If a company says no to training, or a conference or... that doesn't mean no to me... it means that this activity will not be subsidized. The next question to me is it worth the full obligation. There are plenty of training courses I've not only paid for, but took vacation time to be at. I currently have a nice large LCD screen, a wireless headset for my office phone and I don't use the corporate PC, I bought a MacBook Pro... all of which I paid for. I am generally unwilling to compromise working with the best tools. It shocks me in fact that Chefs bring their own cutlery, mechanics bring their own tools and software developers have better machines at home then they use at work, with an expectation that all their tools are provided for them by some company. In addition to the MacBook, I have invested significant dollars in software. Although I like open source, in many cases it is not the best. So I have TextMate, Intellij, MS Office, Keynote, etc. This wasn't as easy starting out, now frankly some vendors send me licenses now so that I'm showing their tool when I'm demoing. It is important to note that I do not accept licenses even for free if I can't stand behind the tool.
Suggestion: Invest in you! Be a master craftsman. Read: http://blog.objectmentor.com/articles/2009/04/01/master-craftsman-teams

3. Seek out Mentors and Network...
Find people who are great at what they do in an area of interest and seek them out, read their books, articles and blogs. Email them. Network with them. Obviously don't annoy them. Perhaps buy them lunch. Challenge them (for instance, "you said xyz, why would you say that?" or "why wouldn't you say yyy?"). Better yet, see what they are thinking about, what they are working on and what they might need help with. Help them! Pass them some work (if they have time), and ask their opinion.
Suggestion: Go to conferences and network.

4. Build your brand...
Create a brand for yourself. Be the Java Memory guy, the F# guy. Dive deep and market yourself. This means; a) find jobs or opportunities to work in this space, b) blog in this space, c) provide presentations in this space d) write articles or a book in this space. Your digital footprint is very important and it takes time to build out. As an example, if you search my name, the first several pages are me. This is significant. The goal here is to in a position to be hired for a gig without a resume. Those asking for you by name don't need a resume, they know what you represent. Reputation is everything! The great thing about a personal brand is it is more reasonable to change than a business brand... so 2 years as the czar of java, followed by...
Suggestion: Read Groundswell.

5. Know who you are and what makes you special...
This takes time for most people. You have to know what your talents are and what you are not good at. When you know what you lack, find teams / team members to compensate. For instance, I am a knowledge sponge and learn very quickly, but I get bored quickly. I am not as good working with a plain piece of paper, but I can perfect and analyze the dickens out of any architecture, system or code put in front of me. Take some personality tests and understand your personality. It's worth it! In the process of learning you.
Suggestion: Read Brain Rules.

6. Learn and consistently work on soft skills
Don't be the average developer... It is ok to be a geek, but be an alpha geek! Most developers will have one of two paths; PM or architect. Either way, the skills are completely different from what made them great as a developer. In the end, developing software is about enabling the business to a) make money or b) save money. Learn their language.
Suggestion: Read How to win friends and influence people and seven habits of highly effective people.

7. Be a Leader!
Leadership is earned... it isn't a title. For a technical person, It consists of having deep technical skills in at least 1 area combined with good morals. People follow people who know what the heck they are talking about and they trust. Knowledge without morals and people don't trust you. Morals without knowledge and you are a really nice guy. It has to be both.

8. Don't be Afraid:
- to say no (especially to things that don't align with you, know what you want)
- to have an opinion or stance on a subject (as soon as you do someone will shoot arrows at you)
- to change your opinion when the evidence dictates... but not based on pressure or opinion
- to make a decision (even in the absence of information)
- to be criticized
Suggestion: Read Who moved my Cheese.

9. Eat like a Bird and poop like an elephant
This is a wonderful phrase by Guy Kawasaki. Birds eat half their weight in food per day... Elephats poop... well they poop a lot. The idea is to read, listen, consume knowledge... then poop... I mean share that knowledge with anyone and everyone. I have a large library (which I read), and an account with Audible and Audio-Tech Book Summaries.
Suggestion: Read made to stick, and rules for revolutionaries.

10. Passion!
Have some... find it! There is nothing more convincing and more contagious than someone with passion. Change you, then change the world!

11. Have fun...
Life is too short
 
I just finished a JavaOne presentation on Debugging your Production JVM. The killer part and climax of the presentation was on BTrace. BTrace just rocks! As good as it is, the documentation and javadoc information is somewhat dated. This post will explain one of my favorite tool sets working together; that being jstat and Btrace.

JStat
Jstat is a tool that has been provided in the jdk bin directory since Java 5. It has a number features, which can be examined by using the flag -options:
jstat -options
The most significant in my opinion being jstat -gcutils which provides an output of jvm memory by compartment; survivor spaces, eden, old and perm space. Here are the steps if this is new to you:
1. start an demo application: java -jar Java2D.jar
2. get the pid: jps
3. lauch jstat: jstat -gcutil 1483
output:

S0 S1 E O P YGC YGCT FGC FGCT GCT
0.00 0.00 10.61 59.98 76.81 8 0.057 9 0.460 0.516
BTrace
Btrace is a tool that allows you to inject probes into a running java process to observe and debug an application. I won't go into "how" to use BTrace, as I've blogged on it in the past and the BTrace document does a good job of explaining it.

The area that is lacking is around the @Export annotation for BTrace scripts. There are no details in the documentation and the sample code comments says "// create a jvmstat counter using @Export"... yeah... right... Even the internal code comments and JavaDocs are just as vague. So if you are lost at this point, jstat use to be called jvmstat and was provided as part of the distribution of jvmstat. Even then there is no information on how to leverage the two... until now!!

Creating the Script
Take a look at the ThreadCounter.java file in the sample directory of BTrace. Any value you want to be exposed to jstat will need to be annotated with @Export. Then you will need to assign a value to this exported value with a static method from BtraceUtils. In the ThreadCounter example this is accomplished with perfLong("btrace.com.sun.btrace.samples.ThreadCounter.count"); The next step is to inject this code into a targeted JVM; such as:
btrace 1483 ../samples/ThreadCounter.java 

Accessing BTrace exports with jstat
The undocumented trick for jstat is that you have to specify -J-Djstat.showUnsupported=true and -name with the name of the exported variable defined by the perf statement. Here is the full command-line:
jstat -J-Djstat.showUnsupported=true -name btrace.com.sun.btrace.samples.ThreadCounter.count 1483




Thanks to Sundar at Sun for the enlightenment!
 
It is curious that my weakest subject in school was English, and that I struggle so much with the miss use of certain words. Words really do matter. As words are misused, it starts to become commonplace which through a domino effect results in the complete lose of their meaning.

The word most in jeopardy today in the domain of programming is Closure.

WikiPedia has a good definition which states: a closure is a first-class function with free variables. The critical phrase is: free variables. I realize how subtle this is... however without the free variables it is just another function... and IMO it doesn't have to be a function that is just the common mechanism. Adding to the list of good examples is the docs on closures in Javascript.

Examples that are wrong or vague are commonplace. First Example: Groovy, which defines it merely as a code block... or a second example, how about Zdeněk Troníček's blog support Closures in Java which defines it as an anonymous function. A Google search will provide a fairly equal number of good and bad examples.

Defining Closure
The point to be made here is that if you pull the "free variables" out of the definition of a closure than all you have an anonymous function and the two words would be synonymous making one redundant to the other. Put another way... Closures are typically anonymous methods, but not all anonymous methods are closures.

Understanding Variable Scope
When defining a variable, our typical scopes are global, part of a class, parameter to a method, or local to a method. This would be complex if we tried to discuss this for all languages, so I will focus on Java. In Java there is no global scope, this is typically handled as a static member of a class. In this case, it lives in memory in a space called perm space. If the variable is part of a class, then the instance is part of heap memory for an instance of that class. If it is a passed parameter or a local member variable, then the storage is part of the stack frame pushed on the stack. As a reminder, as a method returns the stack frame for that method is popped off the stack and the scope has ended for local variables, references, etc. What is interesting about the proposition of a closure, is we want to be able to create a method (method 1) with some "variables" in the scope of another method (method 2). After the return of method 2, which means everything is out of scope, we still want to be able to pass method 1 to other parts of the program with the state of local variables maintaining state. Perhaps you can understand the issue now. If the scope is lost, where and how is the state of the anonymous method 1 being maintained? This is what closure is, and it is up to the language, compiler or run-time to figure that out.

I prefer to consider closures as another level of scope, as oppose to just another function.

Functional Example
It is easier IMO to see this with a functional language, so here is an example:
Function powerFunctionFactory (int power) {
int pwrFunction(int base) {
return pow(base, power);
}

return pwrFunction;
}

Function sqr = powerFunctionFactory (2);
Function cube = powerFunctionFactory (3));

sqr (3);
cube(3);
Looking at this example when the function sqr is defined by invoking powerFunctionFactory(2), the scope of the value of 2 should be lost through normal scoping rules, however due to the feature of closure it is not and this technique is possible for a number of languages. Not only must it maintain state, but it must do it for each instance of closure. Notice that the cube function has a power state all it's own.

In closing I just ran across this great post by Neal Gafter, which is titled the definition of Closures. I didn't get to it in time to reference his material, which looks great. Perhaps there will be a follow up.

If we can't agree on the nature of closures containing variables... we will never understand monads :)
 
Speaking at JavaOne this year, on Debugging Your Production JVM. The session is on June 3, 2009 at 4:10 PM. This session will cover a number of great JVM tools for debugging and managing a run JVM, with a clear focus on BTrace. This is the 3rd time speaking at JavaOne (2001 and 2003 being the times of past). With the Oracle buyout, this could easily be the last for all of us. Welcome to Oracle OpenWorld! The question which remains...

What will happen to Duke?
 
With all these "Best Practices" floating around... I was wondering. Is anyone documenting the worst practices? One Google search later, I discover that not only is there a number of them, but the domain name is taken by someone just parking it... boo!! Here are a couple of my favorite:

Defining Worst Practices - in summary says you are violating a principle or natural law.

Scalability Worst Practices - brilliant... a fun read. Golden Hammer, Big Ball of Mud.

REST Worst Practices - Also a good read. Summary... Rest fails with highly de-normalized systems... oh and don't hard code stuff.

and my favorite worst practices for this post:
Presentation Worst Practices



Happy Coding!
 
Until today, I've been keeping up on the Spring 3 development off the svn repository with svn. There is a great post by Chris Beams from SpringSource that provides adequate detail and is a great resource to read as a precursor to this post. In general, I'm moving all my development over to Git, and it was time to get Spring 3 inline. It was not without pain. For those who just want a quick reference, I will provide a quick step by step on how to achieve the end goal of setting up a git client to the spring 3 svn repository. Following that I will provide details for those who want a deeper understanding.

Quick Step-by-Step Spring 3 with Git
  1. In the directory you would like the spring project execute : "git svn clone --stdlayout https://src.springsource.org/svn/spring-framework"
  2. change directory to the newly created spring-framework directory
  3. copy forked script from github.com/kensipe named git-svn-clone-externals to the current directory; make executable and run.
  4. run "git svn show-ignore > .gitignore"
  5. run "git gc"
  6. from the current directory you'll need to create the following directories
  7. mkdir org.springframework.instrument/src/main/resources/
    mkdir org.springframework.instrument/src/main/resources/META-INF
    mkdir org.springframework.instrument.classloading/src/main/resources
    mkdir org.springframework.instrument.classloading/src/main/resources/META-INF
    mkdir org.springframework.asm/src
    mkdir org.springframework.asm/src/main
    mkdir org.springframework.asm/src/main/java
    mkdir org.springframework.core/src/main/resources
    mkdir org.springframework.core/src/main/resources/META-INF
    mkdir org.springframework.expression/src/main/resources
    mkdir org.springframework.expression/src/main/resources/META-INF
    mkdir org.springframework.web/src/main/resources
    mkdir org.springframework.web/src/main/resources/META-INF
    mkdir org.springframework.orm/src/main/resources
    mkdir org.springframework.orm/src/main/resources/META-INF
    mkdir org.springframework.context.support/src/main/resources
    mkdir org.springframework.context.support/src/main/resources/META-INF
    mkdir org.springframework.web.portlet/src/main/resources
    mkdir org.springframework.web.portlet/src/main/resources/META-INF
    mkdir org.springframework.test/src/main/resources
    mkdir org.springframework.test/src/main/resources/META-INF
    mkdir org.springframework.integration-tests/src/main
    mkdir org.springframework.integration-tests/src/main/java
    mkdir org.springframework.integration-tests/src/main/resources
    mkdir org.springframework.integration-tests/src/main/resources/META-INF
    mkdir org.springframework.instrument/src/test/java
    mkdir org.springframework.instrument.classloading/src/test/java
    mkdir org.springframework.asm/src/test
    mkdir org.springframework.asm/src/test/java
    mkdir org.springframework.aspects/src/test/java

  8. export ANT_OPTS="-XX:MaxPermSize=256m -Xmx1024m"
  9. change directory to "build-spring-framework" directory
  10. run "ant"
  11. wait 20 minutes :)
The background and detail
The first line is standard git-svn to get the repository. If you are new to Git, this will take a few minutes long then a standard svn checkout. It is copying all the changes throughout history to your local .git repo. Fortunately spring 3 is still young from a history standpoint.

svn external - The first issue
The first thing to note is that part of the build process for spring 3 is a project which is added to this svn project as an external, which led me to this post on why svn externals are evil :) evil or not we have to deal with them. This led to a post by Alieniloquent which has some good explanations, followed by what I eventually settled on, a post by Andre Pang, which includes a script he created, which I forked off github. Kudos to Andre!! The script he created (which I can see using again the future), provides a one execution solution to what Alieniloquent blogged. It is beautifully simple. The script provides the following:
  1. pulls down all the svn externals (spring 3 only has one)
  2. creates a symlink (mocking what svn would have done)
  3. adds the symlink and the .git_externals directory to the excludes file
If you wanted to check your svn externals for other projects use the following command: "git svn propget svn:externals ." in the root of the project directory.

The next step is to get all the svn ignores into the .gitignores file.

version control directory management - the next issue
The explanation behind this issue is that svn allows for and manages empty directories... git does not. git only manages content. There are a number of empty directories which the spring build files expects to be present for the build to succeed. This is based on a standardize build system and a lack of error handling in the build system. If I get some time, I'll send a patch to the springsource guys to fix this. At this time you will have to create all the directories listed.

jvm out of memory issues - the last issue
ant will run out of memory without these changes.... first you will run out of perm space, followed by heap. You may be able to get by with less; I didn't spend much time here. The standard defaults will fail.

Also if you need to update your master branch with git... "git svn rebase" will do the job!

I need to say thanks to those bloggers already noted and to fellow NFJS speaker Matthew McCollough.
 
It appears that this subject has caught fire in recent years. The interest in St. Louis has resulting in the Lamdba Lounge started by Alex Miller. (Great name by the way). I have started reading (but not finished) Programming Clojure by Stuart Halloway and Programming Scala by Venkat Subramaniam. Additionally I just finshed writing a 10 page article on an introduction to functional programming, which includes 4-5 page on F#. The article is for NFJS who holds the exclusive rights at this time, but I plan to provide a link to it or post it here in its entirity after 90 days.

I should point out that I am no expert in this space, but I have been developing professional for 15+ years with a variety of languages (C, C++, Java, C#, a little Ruby, Perl, Groovy). In the last 6 months I've been "playing" with Scala, Clojure and F#.

The reason for this post is this: http://groovy.dzone.com/news/why-functional-programming. This article has been sent to me a couple of times and has been on twitter several. At this point I will make public comments I made in an email response regarding this subject. The first two subjects were points made via email, which make sense to maintain for this blog post.

The Argument to have UI and DB Support
I don't get the ui / db argument. This indicates an all or nothing mentality, which I would hope the industry could get over. A griffon front-end to a java hibernate db access and clojure rules engine make sense to me. Many of the functional languages provide the ability to produce a UI or interact with the DB, however that isn't really important to me as you will see below, but there are good solutions out there where this isn't really a concern.

Functional Notations and Code Beauty
I may be in the minority... but I could care less for "beauty" of code, I'm interested in less code that adds the same or more value. FP adds several notations that provide a conciseness to code, which is a huge value add. As I was writing up an article on F#... I became frustrated with my other programming languages... for instance: why is a switch in Java/Groovy so limiting. Groovy makes it better... but it is still limiting as you compare it to discriminated unions and pattern matching F#.

Getting Functional Programming
People don't get it yet (referring to the nah sayers)... they are in the battle pits and are lacking the big picture vision. This year quad core laptops are expected to be more common. That trend isn't going to stop. The last decade was all about die size and memory. This next decade is all about cores. The need for concurrency tools is a must. The language trend may be slightly ahead of its time.

I'll post more on this subject in the future... I just felt it necessary to provide some response to the value of functional programming. Frankly I've barely scratch the surface of the value.
 
This blog post is worthy of Retweeting:) http://infernus.org/2009/02/building-java-7-on-mac-os-x/

In 2 weeks when I hope I have time... I will be on this!
 
The last blog post showed a work around for getting BTrace to work on a Mac, however as explained it fails when trying to probe Grails. I've been working with the BTrace developers and now have a solution. It will likely be some time before the next release. So for those who don't want to build from source, I put a drop on drop.io of a build.

Side notes:
1) It appears that all the scripts and assumptions for Java do not take in account the Mac. Groovy, Grails and previously BTrace all have references to a tools.jar in a place which is not correct on a Mac.
2) drop.io is new to me... and it appears worth the look. I will likely use drop.io in the future. Mostly for presention code drops. I like github for source references, but sometimes it is just quick and easy to tar or zip up what you have and drop it.
3) There is an interesting situation with BTrace which can be frustrating if you are not aware of it. If you have a failed BTrace session... such as a connection refused on the BTrace side, then it is not possible to BTrace it again with a fixed version of BTrace. You have to restart the application to be probed.
 
Well it is time to fix another Java issue on the Mac... again... Sorry for the sarcasm. I love Java and Groovy and I love my MBP, but as already documented (on fixing Java Memory Tools and Java is a 2nd Class Citizen), these two don't get along as often as I would like. I thought for some crazy reason that I was the only one fighting this issue until all the comments from the fixing the tools article appeared. Glad to know, first that this blog is helpful to others, second that there are number of great people all over the world (offer drinks no less) and third that we are not alone in this fight.

BTrace and the issue
First... if you don't know what btrace is... you have to check it out. In the evolution of debugging tools for the jvm, it is the next big thing! Active in this space is A. Sundararajan, he has a couple of my favorite blogs, one on scripting btrace and another on jmx and btrace. Another great tutorial blog is by Igor Minar.

This blog isn't as much about btrace, as it is about fixing it on the mac. BTrace is new enough though, that it may require just a quick introduction. BTrace is a debugging tool, which injects "probes" (my term) into a running Java process. It is an open source option that has a high possibility of replacing Wily's Introscope. To be fair, Introscope provides a lot of extras which BTrace doesn't at this time... but Introscope is tens of thousands of dollars and BTrace is a free open source tool. I would be shaking in my boots if I was on the Wily team.

We will need to start a Java process in order to illustrate how it works (or in the case of the Mac... doesn't work). For the Java process look in the demo directory, there is a Java2D.jar. On the Mac it is located: /Developer/Examples/Java/JFC/Java2D. At the comand-line type: java -jar Java2D.jar. This will start up a Java process. To get it's pid at another command-line type: jps
If this is the only Java process running on your box, the pid with the jar notation is the pid you are interested in. If you want to be 100% certain type: jps -l . You'll get something like:
7897 sun.tools.jps.Jps
7838 Java2D.jar

Now in the btrace bin directory (I'm assuming you followed one of the referenced tutorials), for our example type: ./btrace 7838 ../samples/ThreadCounter.java . To which you will get:
Connection refused

In the terminal of the running Java process for the jar you would likely see:
btrace DEBUG: adding to boot classpath failed!
btrace DEBUG: java.util.zip.ZipException: error in opening zip file
java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.(ZipFile.java:114)
at java.util.jar.JarFile.(JarFile.java:133)
at java.util.jar.JarFile.(JarFile.java:97)
at com.sun.btrace.agent.Main.main(Main.java:108)
at com.sun.btrace.agent.Main.agentmain(Main.java:66)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:323)
at sun.instrument.InstrumentationImpl.loadClassAndCallAgentmain(InstrumentationImpl.java:348)

Yea... epic fail! That is BTrace failing on the mac. If you want more details, go to the btrace script and edit -Dcom.sun.btrace.debug=false to be true. This will indicate the following stacktrace:
btrace DEBUG: debugMode is true
btrace DEBUG: dumpClasses is true
btrace DEBUG: dumpDir is .
btrace DEBUG: probe descriptor path is .
btrace DEBUG: parsed command line arguments
btrace DEBUG: System ClassPath: /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/lib/tools.jar
btrace DEBUG: adding to boot classpath failed!
btrace DEBUG: java.util.zip.ZipException: error in opening zip file
java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.(ZipFile.java:114)
at java.util.jar.JarFile.(JarFile.java:133)
at java.util.jar.JarFile.(JarFile.java:97)
at com.sun.btrace.agent.Main.main(Main.java:108)
at com.sun.btrace.agent.Main.agentmain(Main.java:66)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:323)
at sun.instrument.InstrumentationImpl.loadClassAndCallAgentmain(InstrumentationImpl.java:348)

The line just above the fail point... hmmm.... it is looking for a tools.jar in the $JAVA_HOME/lib directory. As already noted, Apple likes to slice and dice the JDK... In trying to understand the issue, I uncovered an article released by Apple which says there is no tools.jar file... no tools.jar file!!! It is a classes.jar file and to make it fun it isn't in the $JAVA_HOME/lib directory... no, it isn't even under a subdirectory of $JAVA_HOME... I can't even write that without frothing at the mouth. I could start a rant here... but I'm assuming I'm writing to the choir so to speak.

Fixing the Issue
I didn't spend a significant amount of time on it... but I didn't get this working by changing the btrace scripts... something is in the code. I have another recommendation towards the end that the script change wouldn't work for as well. So we are left currently with the following work around.

Take the classes.jar and copy it to the lib directory as tools.jar (argghhh!!!) but it works. To be more specific... /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Classes/classes.jar to /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/lib/tools.jar. It is important to note that you have to be sudo to make this change and you will have to do this with each Java version you want this to work with. Let's hope that the next version of BTrace will be Mac friendly!

After you get that working... checkout visualvm with the BTrace plugin. It is great stuff and another reason that the script change alone is not a total solution.

Other Issues
OMG... I'm frick'n shaving the yak again... just as I'm getting ready to post this... I was about to trace a grails app. The changes described above, break Groovy and Grails. So pick one :) You can be Groovy or you can BTrace but you can't have both yet. I guess I'll begin digging through code and send a patch tonight...
 
I will be speaking at a number of shows with NFJS this year. Already scheduled is Milwaukee, St. Louis, Minneapolis, Boston, Seattle. I will likely be at Omaha, Atlanta, Denver, Columbus, and Raliegh. That's just the first several months :).

It is always around this time of the year I like to revisit my goals. Commitment comes easy around January of a new year. Around mid to end of Feb, is a good time to check to see which goals you are seriously committed to. One commitment, I've maintained for several years is a commitment to learning and a commitment to excellence. This is easier when surrounded with like-minded and committed individuals who help to hold you accountable. It isn't always easy... just worth it. That is one of the reasons I love NFJS. The caliber of the speakers are top notch and world class.

For some the economy will be a an easy excuse to not attend. Or perhaps an employer will not pay for it this year. What is clear to me is those with the skills and the networks will find it easiest to find the next opportunity. This is the time to pick up new skills. As a local venue, no show makes it easier or less expensive than NFJS.

See you there !
 
Finally... a good java editor with good git support. I've been waiting for a long time (in internet time) for git integration in the editor. Download Intellij 8.1 for Git support.

I have a number of project demos which are already in Git. I expected that Intellij would just pick up on this fact... and it did not. I then created a new project, did a Git init from Intellij and all was good.

To get IDEA to recognize a project already in Git, under the Version Control menu, select Enable Version Control Integration, then Git.

So far... the Git integration is fantastic!!! Love it! Well done JetBrains!
Happy Coding
 
Needing to gather some details to help answer a question regarding JVM memory and the the distinctions of Perm Space. I thought it might be helpful to post the links of what I thought were incredibly great posts from the past... It is now 2009 and the best posts on this matter appear to come from 2006. Hmm.... perhaps there has been a JDK release in some time :)

On the subject of; the reason the permanent space exists. Jon explains the details of instances of objects and their dependencies on instances of classes and the need for their order as it relates to garbage collection. He also discussions the klassKlass... interesting stuff.

On the subject of String intern, XML parsing and it's effect on Perm Space. haakon or what I assume to be Jenny details the effect of String.intern() and how an out of memory can occur. She does a great job of explaining their troubleshooting approach.

And my favorite book on the JVM spec is Inside the JVM. It turns out there are free chapters of this on the web now and chapter 5 is the chapter detailing the taxonomy of the JVM memory space.

The weakest coverage of any subject based on my searches, is the subject of root sets. Brian Goetz's article on Java Theory and Practice is rock solid.
 
Having made several comments over the last several months, and in particular over the last couple of days regarding Java on the Mac, several people have asked me to explain myself. So here goes.

When I first moved to the MBP as my primary development machine, I was very pleased. As a developer, I was amazed at how all my development tools were just there. Ruby, Rails, cvs, subversion, 2 Java JDKs, it just worked out of the box.

When I first discovered the difference
I have a standard demo I use when show other developers some of the nifty debugging tools like visualgc or visualvm. In order to show the tool off, I need to start another Java process. So I standardly go to the demo directory and start the Java2d.jar. Oh but wait... where is that on the mac. This lead to the discovery that the jdk is in several different directories /locations.

JAVA_HOME = /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
Demos: /Developer/Examples/Java/
Bin:/usr/bin/java
Headers/Lib: /Developer/SDKs/MacOSX10.5.sdk/System/Library/Frameworks/JavaVM.framework/Versions/xxx
Java Lib: /Library/Java/Extensions; /usr/lib/java
Endorsed Dirs:$JAVA_HOME/lib/endorsed
System jars: /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Classes

OK... I'll live with that. I quickly discovered a very useful scipt for switch between JDKs. I've modified the original if anyone interested let me know. But since we are on the subject of being a second class citizen, If you go out to the Sun site and look for the latest jdk, the options are windows, linux, and solaris.

Other Issues in Jakarta
Then the next issue came up... jhat was broken. I wrote a lengthy blog post on how to fix this. The issue... Apple didn't include the javascript libraries, which are necessary for these tools to work.

So now I'm taking a serious look at btrace, which is a great tool. Just trying to do the simple stuff, works fine on Windows... but no joy for the Mac. I'm still looking into it, but it appears that Instrumentation..appendToSystemClassLoaderSearch() fails on the Mac. Could be something else... still looking.

If you are doing standard Java development or Groovy and Grails... you may never notice the Mac difference. When you venture into the debugging and instrumentation realm of Java be prepared for some frustrations. As I get a solution for the btrace issue I will post it.

On the positive side, I enjoy the development experience on the MBP better than my experiences on Windows and linux. The memory management is better and the startup times on Java processes is fantastic.
 

Photos
Videos
Help
Visit the FAQ Page