Wednesday, December 26, 2007

Happy Holidays

Christmas Pcitures

Getting the pictures out early this year. A few more from today that I haven't imported yet. I'll probably post those in march :)

Sunday, December 23, 2007

Are You Working for the hollidays?

Except for the possible negative of driving more traffic over to Mrs. Skrocki further cementing her in the list of hot bloggers at Sun. Her Putting 1st things 1st post is apropos of the season and the prevalent view at Sun.

With the exception of lugging my laptop along incase something breaks (I am on call) I am avoiding thoughts of work as much as possible.

All my previous thoughts about poking about in CAC or changing/updating my blog theme are on hold, now back to watching last nights hockey game and working on China pictures, possibly going to see Sweeney Todd before dinner with @cdash's mother later tonight.

Wednesday, December 19, 2007

OLPC Quick Pics for ThinGuy

On a totally shocking note, Opera on the XO-1 works great with! I was guessing that it would choke on high volume picture sites. I was able to take these pictures, browse to my smugmug page and upload them. Java isn't working as a plugin so I had to use the old browser POST method.

OLPC and Me OLPC in the mirror

Having proven I was able to blog from the OLPC XO-1 I have moved back to using ecto for normal blogging. Trying to add images and links by hand is just not what I want to be doing at the moment.

Give 1 Get 1 is still going, it isn't too late to donate one for yourself.

OLPC Give One Get One. Arrived

I am sitting here in my office writing this entry on my OLPC (One Laptop Per Child) XO-1. It arrived last night.

my first impressions:
The packaging is sparse but sturdy, Two bits of paper with a note and the saftey warnings (no babies is first on the list). I am guessing that this isn't humor but I am entertained.

The keyboard has a nice feel for a membrane type keyboard, the keys have a slight depression which makes them fairly easy to feel. It is a little small for my hands (a good thing given the target market).I tried an external keyboard and mouse (they worked as I was expecting) but I have revirted to the built in.

I have had some issues with the stock browser in terms of speed and compatabiliy. It also seems that downoloads from the browser disapear (I am probably missing something simple). Installing Opera does the trick for me. It brings back tabs! Image rendering is more consistent and reliable and I can choose a download location. Vital sites such as this blog, the full twitter web interface and google reader all work and appear correct. Sites of importance such as the managed operations portal also work.

Wireless support is quite good, at least as sensitive as my full size devices, with the exception of WPA support (comming soon), in the end I backed down the security stance of of my APs and turned on MAC filtering.

Remaining issues: paste, Copy seems to work, I can drag from the frame and paste into applications. Why can't I paste text into the terminal?

I am sure that many of you are now asking the same question...but will it run solaris? I think the answer is probably not. Atleast not with a gui.

Still others are likely wondering, will it run the super beta punchin for linux. To that I am thinking no, but man that would be sweet!

Monday, December 17, 2007

Hockey, Tied 3-3

Our team remains undefeated, for the first time since EVER. Our record is now 2-0-4. We knew that we were going to be down a some players and we had a substitute goalie which made for some excitement all around.

I was a bit tired and sore from the race on Saturday morning and we ended up with a lighter than expected turnout. Aside from the stamina issues I think it worked fairly well as a recovery exercise. A prophylactic dose of IB about an hour before the game and some hard skating seems to have loosened me up nicely.

I can still feel the race a bit in my legs, yesterday felt like I was riding big hills on my bike. Today feels like I rode some big hills on my bike a while ago.

Saturday, December 15, 2007

Celtic Solstice Five Miler in Druid Hill Park, Race Report

The Celtic Solstice Five Miler was a good race. I was initially concerned about the projected cold weather, indicated 34F feels like 20F, late the night before the race the projected temperatures were 23F feels like 0F. Courtney and I got on the road at 6:50 (aiming for 6:30) I was concerned about getting to the race with enough time to pick up my bib, chip and warm up.

The drive took about 1:15 and Courtney dropped me off near the south west corner of Druid Lake, when I got out of the car it was DAMN cold. Even wearing two pairs of tights one thermal, insulated shell pants, cold weather top, a running top, a mesh backed vest, light running gloves, a mid weight shell jacket and some lobster claw gloves and a hat with a polar tech band; I was still cold. I ended up not using the gortex booties having located a pair of smartwool biking socks (perfect!).

I have never been to Druid Hill Park however I knew there was a shuttle bus to the start. I saw a pickup and headed over at a light jog to try and get warmed up a bit. Missed the first bus, got picked up and then we all wondered why we took the shuttle, it wasn't much of a ride. After we were dropped of I walk/jogged down the hill to the start to pick up my number plate and chip.

I got my stuff and decided after getting moving that really what I was wearing (under the shell and pants) was going to be enough. I ditched them with my bag at the bag check and headed to the scrum of people waiting for the race to get under way. On my way to the start I saw Emily and turned back with her to get her bib and chip.

At this point running a bit late the tables had been setup for food and there was a little delay on the bib and some trouble finding her chip. As we were setting out for the starting line the last of the scrum started moving and had cleared through the start when we got there. We were hurried on through at around 5min after the clock started with the back of the field about half way across the bridge and caught up just before the starting hill.

A quarter of the way up the hill Emily sent me on ahead knowing that we were on a different pace, almost immediately after that Courtney was standing on the side cheering us on.

The rest of the race: I liked the course, it was indeed rolling hills, I was keeping an eye on my HR, aiming to keep it generally below 175 (I averaged 171 for the duration). The course was well marked, as advertised there were a few potholes and puddles nothing that caused any issues that I saw, it did have its narrow moments and aside from a little squeezing and difficulty passing it also wasn't bad. Around (or possibly at) mile 1 someone was calling the time, I was at ~9.5 (my watch) and feeling good, I missed mile two. A quick sip of water at the turn around (2.65) was welcome and three came quickly after. The middle of three was a little bit of a struggle with a twinge in my side, between 3.2 - 3.6, by four I was feeling better and started accelerating to the finish. It certainly didn't hurt that the last mile is mostly down hill but the kick took me up into the red and final sprint over the bridge and across the line drove my heart rate to a gasping 191bpm.

Took a seat for a minute to take off my chip and went out to cheer Emily across the line. Emily came across the line just about seven minutes later and on my way back to meet her I ran into Courtney who had been sitting looking for us to cross the line. We went to the tent to get a drink, a bite of food and collect my bag. The results were posted quite quickly, but the page with Emily's times wasn't up by the time we left. Unfortunately it appears that putting my warm clothes back on I managed to drop my sunglasses (Rudy Project Kerosene w/prescription insert in a black Rudy case).

On the way home we stopped to pick up my sister Karina and take her to breakfast at my Aunt Jean's. Learned that my younger sister Karina has been running and caught a hard time for not telling her we were going to be running. We stopped in at Jean's to see my Mother, my Aunt Mary and Uncle Bob and chat for a bit. A few cups of coffee, a snack, some fruit and few hours later we hit the road for home. Now we are chilling on the couch watching hockey with Courtney and the cats.

Lessons Learned:

  • One of these days, try to get nearer the front, say maybe near the pace you are intending to run. (It might be interesting to try a run where I didn't start out weaving around passing people, not that this tends to go too long)
  • Trust your previous experience, I was worried about the cold but what I selected was just about perfect (once I got moving).
  • Really do get to the race earlier, this worked but, I didn't really need the rush right before the beginning.
  • I need more practice on hills. Watching HR for climbing is working but descent isn't comfortable/smooth.

The Race site, currently disabled probably due to bandwidth (nice).

The Washington Running Report take on the race, indicating that it has grown too big and the next year it will be capped at 2500 entries.

Preliminary results: 1902 Finishers

Pl Name HomeTown XAG Club GunTime
1. Berdan, David Baltimore, MD M26 25:28.20
1348. Ferry, Shawn Leesburg, VA M31 51:00.20
1672. Ferry, Emily Columbia, MD F24 57:44.90

ChipPl Name ChipTime

1. Berdan, David 25:28.00
1177. Ferry, Shawn 46:44.00
1611. Ferry, Emily 53:26.95

In the end the temperature was around 33F. My pace was ~09:21/mi

Edit: Fixed some typos and editing issues, should have gotten a proof reader for posting while tired.

Friday, December 14, 2007

Celtic Solstice Five Miler, tomorrow

In ~20 hours I will be racing the in Celtic Solstice Five Miler my wife will be there to cheer and my youngest sister may also be running.

Went up to the community gym last evening, I should do that more often. It isn't big or equipped with all the latest gear but it has more than enough for a good workout. Did about a mile warmup, got a good stretch, did another slow mile, stretched some more and walked home. As a dry run I tried the gortex biking booties as the ventilation in my shoes has been a little on the chilly side. The booties are OK, but they are C-'s 41/42 and I wear 45s, they didn't bother me and I noticed a decrease in chilly feet on my walk to and from the gym. says Baltimore will have a "Fitness Comfort Index of 2 (where 3-1: Uncomfortable)" Mostly Cloudy to Cloudy with predicted temperatures of 30F feels like 20F. It should be fun, except for the getting up early to drive up for the race.

Thursday, December 13, 2007

Chicken Primavera w/garlic bread in 10min or less

Taking my cue from ThinGuy, only no one has been asking me for my recipes :)

This meal was prepared in the office kitchen, using only minimal tools and supplies.

Required Ingredients:
Garlic Salt (Garlic Powder recommended) I Can't Believe It's Not Butter, Feb '07 Vintage

Italian Spice Blend Martin's potato bread, Hot Dog Rolls
Lean Cuisine Chicken Primavera
Red (cayenne) pepper

  1. Set toaster oven to Bake at ~450
    1. Place baking rack on the top rail
  2. Prepare Garlic Toast
    1. Remove a roll from the bag, split and place on clean surface
    2. Lightly coat top of roll with ICBINB
    3. Apply Garlic Salt to taste
    4. Apply Italian Spice Blend to Taste
    5. Place prepared toast in toaster oven
  3. Chicken Primavera
    1. Remove from box
    2. Place in microwave
    3. heat until hot (we used 8:30 for two)
  4. Brown/Toast the Toast
    1. At ~2:00 remaining on the microwave
    2. Switch the toaster oven to broil
    3. Observe closely to prevent burning

  5. Enjoy!

I add ground red (cayenne) pepper to my Chicken

The found "I Can't Believe It's Not Butter" expired Feb 13 '07. If you can't find this vintage feel free to use whatever edible grease you can find. Butter or butter flavor recommended.

The random spices smell like an Italian blend. They have been allowed to mellow and blend on the office kitchen table for at least a month.

All times are approximate, a pre-heated toaster oven would clearly require less time.

Tuesday, December 11, 2007

Cat Proof External Storage

What happens when you disconnect the drive on the fly?
(e.g. set up a tripping hazard for your cat)

You get this message
The device you removed was not properly put away.Data might have been lost or damaged. Before youunplug your device, you must first select its icon inthe Finder and choose Eject from the File menu.
What happens when you do the same thing to a ZFS pool? This is no different
than what you get on Solaris, except it is on my Mac Book Pro.
  pool: p1
 state: DEGRADED
status: One or more devices could not be opened.  Sufficient replicas exist for
    the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
 scrub: scrub completed with 0 errors on Sat Dec  8 01:44:00 2007

    p1           DEGRADED     0     0     0
      mirror     DEGRADED     0     0     0
        disk7s1  UNAVAIL      0   114     0  cannot open
        disk0s3  ONLINE       0     0     0

Fortunately for me, my system keeps merrily chugging along.

Unfortunately (sort of) and this is much more of an issue on my Mac than on
any other system I am adding and removing devices much more frequently.

When I last connected the external disk and imported the pool I had 6
more "disks" visible to the system (really a usb thumb drive) another FW
device and a few SW RAID devices. This caused my nice portable external
FW disk to appear as disk7.
  diskutil list             
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *149.1 Gi   disk0
   1:                        EFI                         200.0 Mi   disk0s1
   2:                  Apple_HFS noroute                 88.9 Gi    disk0s2
   3:                  Apple_HFS                         57.0 Gi    disk0s3
   4:                  Apple_HFS noroute_2_1_2           2.6 Gi     disk0s4
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:     FDisk_partition_scheme                        *55.9 Gi    disk1
   1:                  Apple_HFS                         55.9 Gi    disk1s1


The difference between the zpool status and diskutil list output
is the device configured for the pool is disk7s1 not the currently visible

I don't know how to recover from this situation live. If the device has not
renumbered you just zpool online p1 disk7s1 to reconnect on a different
device number I export and import the pool again.

To state that another way, I don't know how to effectively do:
zpool online p1
similar to
zpool replace p1
(the replace, even with -f, failed telling me that was busy)

I have just requested a pre-release tarball that is reported to fix some bugs
in the current beta. We will see how it goes.

Other Observations:

Once you load the RW kext, it appears that anyone can do anything to the ZFS pools
and filesystems. On my single user system this is OK. I am the only user and guest
accounts are disabled. e.g. Initiate a scrub as a non-admin guest account or create a
snpashot as nobody (sudo -u nobody zfs snapshot p1/Music@nobody)

You CANNOT download music from the iTunes store in the current beta. you get a
permission error. I can however add music from local files.


Edit: Cleaned up the block quotes a bit, still not really rendering as one might hope.

Sunday, December 9, 2007

Hockey: Tied 4-4

We ended up tied 4-4.

Went down by two in the first, picked up one in the second and three in
the third.

The tying goal was a screened slapshot from THEIR blue line.

We are doing much better as a team, but we were having issues
getting the puck out of our zone tonight.

My legs were feeling dead almost from the start, ah well. Need to get back on
the regular exercise bandwagon.

Wii for the Hollidays

As I alluded to earlier, I won "Best Wow Blog" for CEC 2007. The Judging criteria was

Best Wow Blog:
Blog that is about CEC and leverages as many different technologies as possible to communicate its message. Chosen by a panel of judges, approved by the CEC Executive sponsor Dan Berg, will review blogs for:

  • Wow factor: Does the blog incorporate technology in unique ways? Is it fun to read?
  • Is this a blog that will last throughout the year?

Sweet, I have incorporated technology in an interesting and or unique way and I am fun to read.

Alec Muffett won best podcast for his #8220Terabyte iPod#8221 presentation (which was quite interesting, you
should watch it) he is wondering if he can get a UK Wii.


Lou Springer won Best Red Shift Blog.  Go Lou!  I met Lou at CEC having previously conversed on

twitter and had dinner with him (and some other people see my dinner post listed below :) ) the last day of CEC.


iTunes, ZFS and Time Machine

I have been blogging about ZFS on OS X recently
RW ZFS and Leopard first thoughts
How to get experimental rw ZFS support AFTER upgrading to 10.5.1

As a way to start using ZFS on OS X on a daily basis I have migrated the contents of
my Music folder to a ZFS mirrored pool composed of an external disk and a matching
internal partition.

After a slightly rocky start including reinstalling my laptop so I could do the repartitioning
I wanted and a fairly high number of initial crashes things currently seems to be going well.

One of the things that I have noticed is that my iTunes Music folder and Library are polluted.
Now embedded in my library with the same name as a number of songs I have '.htm' files.
(./path/to/foo.mp3 also has a matching ./path/to/foo.htm in a large number of cases)
The files are part of some O'Reilly books I have on CD, as such I know "where" they came from, sort of.
(My library from my pre-experimentation CCC backup also has the same corruption)

A restore from my Time Machine backup is in order to get things back to a known good state. Or a recovery
from my slightly older CCC backup which is also not corrupted.

I am not particularly impressed with the Time Machine interface. It is novel and seems relatively functional
for finding the most recent backup by using the timeline on the right.

[Time Passes]

The Time Machine restore failed after ~8GB on a couldn't read/write a file error aborting the whole restore.
I do have to admit that the backups are working I can browse the backup structure. The use of hard links and
metadata is cool. Restoring via rsync does the trick, possibly faster as well since it doesn't copy unchanged files.

Even better I don't see a log, I am feeling somewhat less than impressed with the feedback from and utility
of Time Machine from a administrator's point of view. Am I asking for too much? A log of the restore with an
indication of the failure?

I have a feeling that the restore is really just a copy in Finder. An automator type action passing the
selection to Finder and executing a copy.

Again what finally worked for me was a combination of methods. I used the Time Machine interface to find the
backup I wanted and CMD-I to see the path to the directory (although would have been just as good).
Once I found the backup I wanted rsync worked quite well restoring 48GB of Music in 12,273 files.
(rsync --archive -P /Volumes/TimeMachine/.../Music/iTunes ~sferry/Music/iTunes)

Thursday, December 6, 2007

CEC: Award Winning 'Wow' Blog


I won an award for blogging. It turns out that I had the "Best Wow Blog" at CEC.

I am still not sure what exactly a wow blog is. I made a note to see if there was contest information
on SWAN but I haven't checked yet.

Hopefully this isn't a sympathy prize :)

Once I find out a bit more I'll let you know. I did win a prize, more about that later.

RW ZFS and Leopard, my first impressions

Not long ago I posted about installing the RW ZFS beta seed on Leopard after updating to 10.5.1.

As I said after the end of that post, It does work. On the one hand I haven't had this many system
panics on a mac ever. On the other hand I have repartitioned my laptop and now have ZFS pools
mirrored to an external USB stick and a firewire drive.
Since the simple tests below I have re-installed to get more partitioning flexibility. I couldn't do a live
repartition to get the space I wanted. Every attempt to repartition after the first was informing me that
there wasn't enough space.

After pulling the drive but before directing any traffic to the pool:

errors: No known data errors

pool: pool
state: ONLINE
scrub: resilver completed with 0 errors on Mon Dec 3 15:44:43 2007

pool ONLINE 0 0 0
mirror ONLINE 0 0 0
disk0s3 ONLINE 0 0 0
disk1s1 ONLINE 0 0 0

errors: No known data errors

After initiating a scrub on the pool:

pool: pool
status: One or more devices could not be opened. Sufficient replicas exist for
the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
scrub: scrub in progress, 11.12% done, 0h1m to go

pool DEGRADED 0 0 0
mirror DEGRADED 0 0 0
disk0s3 ONLINE 0 0 0
disk1s1 UNAVAIL 0 0 0 cannot open

errors: No known data errors

zpool online pool

Disconnect while writing files to the volume via rsync:
pool: pool
status: One or more devices could not be opened. Sufficient replicas exist for
the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
scrub: resilver completed with 0 errors on Mon Dec 3 15:56:10 2007

pool DEGRADED 0 0 0
mirror DEGRADED 0 0 0
disk0s3 ONLINE 0 0 0
disk1s1 UNAVAIL 0 595 0 cannot open

errors: No known data errors

When the disk went offline the IO slowed (somewhat dramatically), when it came back online it also slowed (again dramatically for ~1-2s). The resilver started and writing data to the pool while resilvering slowed everything down a bit but nothing horrible.

More to come. On a day to day basis I am running iTunes and my music library on ZFS.

My first hint: I recommend turning off spotlight indexing on the ZFS volume it seems SIGNIFICANTLY more stable.

Tuesday, November 27, 2007

How to get experimental rw ZFS support AFTER upgrading to 10.5.1

The alternate title of this entry is "Force install of ZFS Beta Seed v1.1 on Leopard"


This is performed at your own risk. The steps described remove any logical restriction for installation of the package.


There is more than one way to implement this particular hack. This method uses the package installer (a cleaner more friendly hack than manually copying files). Other recommendations included installing 10.5 on a different partition and subsequently installing the patch and copying the package files.

A simple alternative left to the reader would be extract the files from the package Payload an manually copy the files into place (cat /tmp/ZFSseed1/ZFSBetaSeed1.pkg/Payload | pax -z -r -v).


On to the actual Implementation:

Download the dmg from


Mount the DMG:

open ~/Desktop/Inbox/leopard_9a559_zfsbetaseed1_0613523123.dmg

Expand the package:

pkgutil --expand /Volumes/ZFS\ 1/ZFSBetaSeed1.pkg /tmp/ZFSseed1

Edit the Distribution file and comment out the line that actually checks the requirements (and "causes" the failure):

vi /tmp/ZFSseed1/Distribution

Flatten the edited expanded package directory back into package format:

pkgutil --flatten /tmp/ZFSseed1 /tmp/ZFSrw.pkg

Open the package installer:

open /tmp/ZFSrw.pkg

Install the package and reboot.

Again this is performed at your own risk. The steps described remove any logical restriction for installation of the package and may cause you system to explode or you cat to catch fire.


ZFS Beta Seed v1.1 will not install on Leopard (10.5.1)

Edit: I guess I should mention that it does actually appear to work :) Next I'm going to try and switch access between Leopard and Solaris under Parallels

Edit1:  Changed comment marks from // to ; // works if you are commenting in the embedded script part not the XML part. Thanks to Colin Seymor for catching that.

Monday, November 26, 2007

SGE quick and dirty how to find jobs on 'bad' slots

I occasionally have a need to find queues in Sun Grid Engine that are in one of the possibly problematic states which have an occupied slot. It is just infrequent enough that I don't remember exactly how I did it the last time.

qstat -f | awk '$6~/[cdsuE]/ && $3!~/^[0]/'
queuename qtype used/tot. load_avg arch states BIP 1/1 -NA- sol-amd64 adu BIP 1/1 -NA- sol-amd64 adu

An alternate is "qstat -f | awk '$6~/[cdsuE]/ && $3~/^[1-9]/'" which also avoids printing the header line. In the example above 'state' in $6 matches 's' and 'used' does not begin with '0'.

The possibly more elegant 'qstat -f -qs cdsuE' still requires a second comparison in awk of '$0!~/--/' to filter out the queue separator lines. (qstat -f -qs acduE | awk '$0!~/--/ && $3!~/^[0]/')

Finally because I can never remember what exactly all the queue states are and the qstat man page doesn't have the nice table:

aoACD #8211 Number of queue instances that are in at least one of the following states:
a #8211 Load threshold alarm
o #8211 Orphaned
A #8211 Suspend threshold alarm
C #8211 Suspended by calendar
D #8211 Disabled by calendar


cdsuE #8211 Number of queue instances that are in at least one of the following states:
c #8211 Configuration ambiguous
d #8211 Disabled
s #8211 Suspended
u #8211 Unknown
E #8211 Error


Job State/Status:

d(eletion),  E(rror), h(old), r(unning), R(estarted), s(uspended), S(uspended), t(ransfering), T(hreshold) or w(aiting).

References: SGE (N1GE 6.0) -- Monitoring and Controlling Queues

Edit: Added Job Status, literally couldn't find that in any of the online docs (notwithstanding ~40% through the qstat(1) man page, targeted google searches do a poor job finding the link)

Wednesday, November 21, 2007

OS X Leopard, Tiger X11 and SGD (How I downgraded to Tiger's X11 and got SGD working again)

We use SGD to provide access to few applications. After installing Leopard I could focus and click with a mouse but all keyboard input was ignored for SGD applications.

Searching online I found some indications that X11 in Leopard has some application interaction issues. The solution presented in a number of different forums for various applications was to downgrade to Tigers I tried methods from a couple of posts and didn't have success. Instead I mixed and matched the steps from a couple of suggestions and found a solution that worked for me.

I have not tried to recover from this change, You can PROBABLY re-install X from the leopard DVD.

When this is complete you will probably have two X icons in the Dock when is running.

The steps can be summarized as:

  1. Download X11 Update 2006 1.1.3: ""
  2. Destroy your current X11 installation
  3. Install X11 update 2006
  4. Change the path to your window manager in xinitrc
  5. reboot

wget ''
open X11Update2006.dmg
sudo launchctl unload -w /System/Library/LaunchAgents/org.x.X11.plist
sudo rm -R /usr/X11R6
sudo ditto -Vx --noqtn /Volumes/X11\ Update\ 2006/X11Update2006.pkg/Contents/Archive.pax.gz /
sudo perl -i -p -e 's:exec quartz-wm:exec /usr/X11R6/bin/quartz-wm:g'

The instructions I found online indicate that a log out/log in should do it. I found that it didn't seem to start working until after I rebooted.

The instructions I based the above steps on:
Bring Back Tiger's X11 to Leopard in 3 Steps
easier instructions to install Tiger's

Friday, November 16, 2007

Two Storage Commands I Don't Know How I Lived Without

I am working on an issue which involves a 3510 and two dual connected hosts (Home grown Active/Active configuration). The customer's equipment has just been moved within the cage. When the systems were rebooted one of them reported multipath failures and both SCSI errors.

While I was investigating the problems I used cfgadm, luxadm and sccli the first two are common to Solaris the last is an additional package for management of arrays including the 3510. These commands are not new to me; while I was searching for alternative solutions to my problems I found fcinfo and mpathadm two fairly new (and definitely new to me) commands.

Using luxadm to display the state of the ports:
(One of the ports was not connected but I wasn't thinking about blogging it so I missed my chance to capture the output.)

luxadm -e port
/devices/pci@1d,0/pci1022,7450@1/pci1077,100@1/fp@0,0:devctl CONNECTED
/devices/pci@1d,0/pci1022,7450@2/pci1077,100@1/fp@0,0:devctl CONNECTED

Using luxadm to show link errors:
luxadm -e rdls /dev/es/ses0 

Link Error Status information for loop:/dev/es/ses0
al_pa lnk fail sync loss signal loss sequence err invalid word CRC
9e 0 1 1 0 2794 0
9f 0 0 0 0 243 0
1 0 0 0 0 0 0

Link Error Status information for loop:/dev/es/ses0
al_pa lnk fail sync loss signal loss sequence err invalid word CRC
a3 0 2 2 0 65535 0
a5 0 0 0 0 28481 0
1 0 0 0 0 0 0

Using cfgadm to see the configuration of the devices:
(In the original investigation c2 was displaying type fc and Occupant unconfigured)
cfgadm -al
Ap_Id Type Receptacle Occupant Condition
c0 scsi-bus connected configured unknown
c0::dsk/c0t0d0 disk connected configured unknown
c0::dsk/c0t2d0 disk connected configured unknown
c0::dsk/c0t3d0 disk connected configured unknown
c0::es/ses1 processor connected configured unknown
c1 fc-private connected configured unknown
c1::256000c0ffc86cfb disk connected configured unknown
c1::256000c0ffd86cfb ESI connected configured unknown
c2 fc-private connected configured unknown
c2::226000c0ffa86cfb ESI connected configured unknown
c2::226000c0ffb86cfb ESI connected configured unknown
I tried using 'cfgadm -c configure c2' and 'cfgadm -f -c configure c2' and finally 'cfgadm -o force_update -c configure c2' none of which succeeded in letting me recover the path. I just now found a bug for path shows NOT CONNECTED. It appears that I might have been able to recover using 'luxadm -e forcelip'. Since I needed to clear the 3510 error counters it was decided to take the systems down and power cycle the 3510.

Now on to the hook for this post (fcinfo and mpathadm). While looking at various documentation I found fcinfo and mpathadm!
fcinfo was added in S10u1. Using fcinfo I saw the some of the same information that I got from 'luxadm -e rdls' and more.
Using fcinfo to see local hba-port info:
fcinfo hba-port -l
HBA Port WWN: 210000e08b1cdb34
OS Device Name: /dev/cfg/c1
Manufacturer: QLogic Corp.
Model: QLA2340
Firmware Version: 3.3.117
FCode/BIOS Version: N/A
Type: L-port
State: online
Supported Speeds: 1Gb 2Gb
Current Speed: 2Gb
Node WWN: 200000e08b1cdb34
Link Error Statistics:
Link Failure Count: 0
Loss of Sync Count: 0
Loss of Signal Count: 0
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0
HBA Port WWN: 210000e08b1124bf
OS Device Name: /dev/cfg/c2
Manufacturer: QLogic Corp.
Model: QLA2340
Firmware Version: 3.3.117
FCode/BIOS Version: N/A
Type: L-port
State: online
Supported Speeds: 1Gb 2Gb
Current Speed: 2Gb
Node WWN: 200000e08b1124bf
Link Error Statistics:
Link Failure Count: 0
Loss of Sync Count: 0
Loss of Signal Count: 0
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0

Nothing extremely interesting on the hba-port side, however fcinfo also shows remote-port information.
Using fcinfo to see remote-port info: (the -p option shows information visible from the hba-port WWNs seen above)
fcinfo remote-port -l -p 210000e08b1124bf
Remote Port WWN: 226000c0ffb86cfb
Active FC4 Types:
SCSI Target: yes
Node WWN: 206000c0ff086cfb
Link Error Statistics:
Link Failure Count: 0
Loss of Sync Count: 2
Loss of Signal Count: 2
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 65535
Invalid CRC Count: 0
Remote Port WWN: 226000c0ffa86cfb
Active FC4 Types:
SCSI Target: yes
Node WWN: 206000c0ff086cfb
Link Error Statistics:
Link Failure Count: 0
Loss of Sync Count: 0
Loss of Signal Count: 0
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 28481
Invalid CRC Count: 0

fcinfo remote-port -l -p 210000e08b1cdb34
Remote Port WWN: 256000c0ffd86cfb
Active FC4 Types:
SCSI Target: yes
Node WWN: 206000c0ff086cfb
Link Error Statistics:
Link Failure Count: 0
Loss of Sync Count: 1
Loss of Signal Count: 1
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 2794
Invalid CRC Count: 0
Remote Port WWN: 256000c0ffc86cfb
Active FC4 Types:
SCSI Target: yes
Node WWN: 206000c0ff086cfb
Link Error Statistics:
Link Failure Count: 0
Loss of Sync Count: 0
Loss of Signal Count: 0
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 243
Invalid CRC Count: 0

Fcinfo and luxadm are clearly showing me that there are problems reported for the remote port in the 'Invalid Tx Word Count'.
The primary recommendation is to reseat the cables SPFs and blow out the ports. We are moving along with the process now, having replaced one of the cables, reseated everything and blown out the ports.

On to mpathadm, it was added in S10u3 and lets you discover and manage multipathing (shocking given its name).
I am using mpathadm to display information about the current configuration. Prior to the reboot the system with only one link in CONNECTED state showed only one path to all devices.
Output from 'mpathadm list lu':
mpathadm list lu
Total Path Count: 3
Operational Path Count: 3
Total Path Count: 1
Operational Path Count: 1
Total Path Count: 1
Operational Path Count: 1
Total Path Count: 1
Operational Path Count: 1
Total Path Count: 1
Operational Path Count: 1
Total Path Count: 2
Operational Path Count: 2
Total Path Count: 2
Operational Path Count: 2

Specific detail from 'mpathadm show lu' on path:
mpathadm show lu /dev/rdsk/c3t600C0FF000000000086CFB3597712416d0s2
Logical Unit: /dev/rdsk/c3t600C0FF000000000086CFB3597712416d0s2
Vendor: SUN
Product: StorEdge 3510
Revision: 415G
Name Type: unknown type
Name: 600c0ff000000000086cfb3597712416
Asymmetric: no
Current Load Balance: round-robin
Logical Unit Group ID: NA
Auto Failback: on
Auto Probing: NA

Initiator Port Name: 210000e08b1cdb34
Target Port Name: 256000c0ffc86cfb
Override Path: NA
Path State: OK
Disabled: no

Initiator Port Name: 210000e08b1124bf
Target Port Name: 226000c0ffb86cfb
Override Path: NA
Path State: OK
Disabled: no

Target Ports:
Name: 256000c0ffc86cfb
Relative ID: 0

Name: 226000c0ffb86cfb
Relative ID: 0
This is in no way a full exploration of the capabilities of fcinfo and mpathadm.
I hope that the next time you are (or I am) looking at FC or multipath issues these commands will be helpful.
Please see the links to the manual pages below for more specific information and examples from the fcinfo and mpathadm commands.

fcinfo #8211 Fibre Channel HBA Port Command Line Interface
mpathadm #8211 multipath discovery and administration

EDIT: Fixed some strange formatting issues
EDIT1: A bit more touch up

Monday, November 12, 2007

Hockey: We won

We are now officially the "Rink Rats" we couldn't get the "Farging Ice Holes", I was also partial to pylons.

We won 2-1, nearly shocking. Next game is a nice early 11:00pm.

My foot didn't bother me and thankfully the Refs were a little late so I had time to stretch out on the ice. A little short on stamina but I haven't been on the ice for at least a month and I really slacked off on the running and other generally health things in Vegas and China.

One Laptop Per Child: G1G1 (Give One Get One) Ordered

One Laptop Per Child

Donated to One Laptop Per Child at Laptop Giving so I will be getting an XO Laptop which I think should stand up nicely to having friends with kids over. Although I will surely have to take it to the office or out on the town. I wonder how functional I could be using it for work.


The mission of the One Laptop per Child (OLPC) movement is to ensure that all school-aged children in the developing world are able to engage effectively with their own personal laptop, networked to the world, so that they, their families and their communities can openly learn and learn about learning.

The OLPC Association focuses on designing, manufacturing, and distributing laptops to children in lesser developed countries, initially concentrating on those governments that have made commitments for the funding and program support required to ensure that all of their children own and can effectively use a laptop.

Initial focus is on the launch of the One Laptop per Child program. In the future, the OLPC Foundation will focus on the grassroots, #8220bottom-up#8221 aspects of the OLPC mission.

Give One Get One: 15 days left to Order/Donate one yourself.


If you live in the USA or Canada, and during a brief period of time, you will be able to pay USD 399 for two XO laptops. The first laptop is yours to keep (the get 1 part) and the second one is donated to the program to be distributed in one of OLPC's partner countries (the give 1 part).

The G1G1 program also includes 1 year of T-Mobile Hot Spot access. If you don't give away the 1 you get you can set up shop at any T-Mobile HotSpot with your distinctive laptop and be a living advertisement for the OLPC project.

More information at the OLPC Wiki

Back on the health wagon

Went for a run Saturday morning in part because I need to start getting more exercise again and in part as a dry run for the Leesburg, VA 5th annual Freeze Your Gizzard race.

6mi in 1:09:00, over did it a little but I was running to the office and back to pick up my laptop power supply that I had left on Friday. I should have slowed down on the return leg instead of speeding up. A little sore and although I stretched the crap out of my soleus my Right 5th Metatarsal is aching.

Some heat to keep it loose and some ibuprofen and I am good for my nice early 10PM hockey game tonight. First game of the Winter season; glad I had that early meeting this morning.

Monday, November 5, 2007

First try with Sun Service Tags and SXDE 09/07

I just installed SXDE 09/07 and decided to give Service Tags another shot. The installation of the Service Tags packages doesn't take a special effort.

Unfortunately I can't get the product registration agent to find anything. I checked with one of my co-workers to make sure that he had success before spending any time on the issue.

He confirmed that he was getting discovery for OS installs on systems without servicetag supported products.

Service Tag Discovery: No Products Found

So clearly I have just installed the packages. This is obnoxious.

So under preferences I enabled FINEST logging and tried again.

FINE: Checking ip addresses:
Nov 5, 2007 11:57:17 AM com.sun.scn.client.ui.RegClient getSystems
FINE: Getting ip addresses:
Nov 5, 2007 11:57:17 AM com.sun.scn.client.ui.RegClient getSystems
FINE: Checking:
Nov 5, 2007 11:57:17 AM com.sun.scn.client.ui.RegClient checkIPAddress
FINE: Checking if valid ip address:
Nov 5, 2007 11:57:17 AM com.sun.scn.client.comm.TCPProbe run
FINER: sending message to:
Nov 5, 2007 11:57:17 AM com.sun.scn.client.comm.Communicator$1 run
FINER: communicating with: /
Nov 5, 2007 11:57:17 AM com.sun.scn.client.comm.Communicator getFromAgent
FINE: Getting agent:

That look like communication to me, but wait a URI...trying in a browser returns: in.stlisten: fatal: open failed: No such file or directory

When I look on my system I see I know an easy first try to "fix" that.
ln -s /usr/sfw/lib/ /usr/sfw/lib/
It seems to me that we could do with some sort of error detection or a host level smf service failure.
e.g. We got a response from the polled host but it wasn't anything that we were expecting.

Trying again in a browser returns a result that looks a lot like I would expect given my understanding of Service Tags.

    Parallels Software International Inc.
    Parallels-18 F2 11 FF 3E 85 43 C5 B4 CC D7 85 04 84 A9 AD

Now everything is working as expected. That is what I am expecting to see!

Taking a look at the Sun Connection Inventory Channel I can see the host I just registered!
Sun Connection: Viewing Registered Hosts

Now everyone should go and install the Service Tags agent and register their devices (It's tied to an extra bonus for us next year :) ).

References: Sun Connection on BigAdmin

Thursday, November 1, 2007

Indiana IPS (Image Packaging System)

Boldly forward with minimal reading of the docs.

I can't obviously find where I would be downloading additional packages from. I feel like I am running in circles. It would seem that nearly the first thing I should be able to find would be download the rest from HERE. The single CD installer rocks, but we are up to 1 DVD for the normal full install.

I want the Firefox default home to prominently show me:
"Now that you have completed the Slim install get the rest of the packages. Use pkg list/status/something someargs"

The Preview includes the Image Packaging System. With IPS, you can select versioned builds of components to manage or create your own custom OpenSolaris distribution.

IPS packages that are not included in the Slim Install installation image, such as developer tools, can be downloaded after the installation. This prototype uses new IPS commands to access packages from the network repositories. Both IPS packages and SVR4 packages are supported.

The OpenSolaris Project: Image Packaging System project page contains man pages for the new IPS commands and a link to the IPS download site.

OK, pkg(5) has some indications of a repository (or authority) pkg:// No examples, is pkg:// actually a valid and running authority? OK says Looking at the list of available packages I now see that they are basically ALL installed. (maybe pkg(5) should reference

I can also see that when I pkg uninstall SUNWbind and pkg install SUNWbind the counters on the site increment. When I drop ni0 the install fails and with it enabled and snoop running there is http traffic downloading the package. Clearly I am hitting the remote authority. Somewhere I think I should be able to see what the default authority is or where to find it (without digging in /var/pkg and guessing that cfg_cache should be the source, or looking at snoop to see where my traffic is going)

pkg install/uninstall is fast and also easy (somewhat dependent on network bandwidth I would guess). It wouldn't suffer from the some sort of optional feedback that work is progressing.

Errors from pkg are quite ugly and straight out of python.

At this point I call the install a success simple, easy, fast. The post install experience is still missing something. The packages that are not included on the live CD, how did they get installed on my instance when I had no net during the install.

So we are down to
  1. Is the only authority?
  2. are all the packages that are available installed by default?
  3. If pkgadd can't add packages (or is this a bug) how do I add standard SYSV packages now

In the end this was fun and interesting, I have a much better understanding of where we are going but it doesn't look like Indiana is going to be my everyday system for a while yet,

OpenSolaris/Indiana first thoughts

(See Update 1 for how to get functional networking in Indiana under Parallels)

As ThinGuy just mentioned on twitter, no registration or login to download the image. Woot!

I am running the installer iso on top of Parallels on Mac OS X 10.5 (leopard)

  • Boot speed is fabulous
  • Installer is straight forward
    • so few steps it seems like I must be forgetting something
  • Base install is FAST

On the down side:

  • I can't seem to get an external network interface to plumb, but Parallels seems a bit flaky on Leopard
  • single disk contention is the long pole install is not as fast as it could be, should have burned a CD

So now as long as I can get a network interface working one way or another I am set. Fingers crossed. I have already nuked my other local install so I had space to play. I would rather not have to recover it.

Update 1:

Networking: It helps if you remember to install the Parallels provided network interface driver.

See the comment about so few steps I must be forgetting something (like installing the driver)

Parallels: Installing the Beta got rid of some VM related network error messages and appears to have fixed shared networking (my default for my simple test instance).

I guess I could have provided a link:

Issues with the base slim install No /usr/ccs/bin/make (or any make as far as I can tell in the default install). To install the required driver the following manual steps are required.

  • untar the ni*.tgz into /tmp
  • in the /tmp/ni*/i386 directory
    • cp ni dp8390 to /kernel/drv
    • cd ..
      (or you can change /etc/path_to_inst yourself)
    • modload /kernel/drv/ni
  • Assuming you are using dhcp wait a moment and get the popup telling you that you have an address.

Appendix C: is way out of date although it would have reminded me to install the network driver.

Monday, October 29, 2007

CEC non-technical video

I had plans for a video at CEC, in the end with so much going on and having brought a big still camera I ended up not doing an video.

I'll leave you with this small sample of one morning at CEC (audio required):

Wednesday, October 17, 2007

China: Shanghai Day 1

By the time we got to the hotel we had been up for ~25 hours. The flight was fine, a little bit worried at first when I went to check the status and saw canceled at 04:00 (it had been pushed back 7 minutes). Once we landed our guide was right past customs which was fortunate given the high volume of passengers and people waiting for them.

After about an hour in a van going to the hotel we checked in (now in a suite, not what we were expecting, we seem to have gotten a bump to the executive levels). @cdash took a bath to clear her sinuses and the rest of us went for a walk in the park across the street so we wouldn't fall asleep before we meant to. We took some pictures in the park, got turned around and made it back to the hotel for dinner.

View as we left the park at dusk

On the way out the door for our tours of

  • Yu Yuan Garden
  • The Bund
  • Shanghai Museum
  • Shanghai Urban Exhibition Center

Friday, October 12, 2007

CEC: Dinner w/ThinGuy Bhlackey, louspringer and some non-twitter heathens at Firefly*

Firefly* on Paradise Tapas Kitchen & Bar

ThinGuy's wife, Bhlackey, artp

louspringer, the guys who's names I can't remember

louspringer and thinguy

another name I can't remember and thinguy

(Update: Fixed louspringers name on the second pic. no idea how I missed that I typed lousmith the first time, sorry Lou)

The gallery

CEC: Wednesday GCS All Hands

CEC Red (or more orange)

CEC: Party (and beyond)

The party was interesting, the space was nearly big enough but the ability to move was constrained by the shape of the spaces.

A party in three parts:


The pool: Big hit, weather was really nice. Women dancing on platforms on the pool, RC boats running around the pool, piles of CEC attendees standing around the pool, live band. Long lines.

people watching the poolThe Pool

Air HockeyA spherical projector

The Band:

The bandThe BandThe BandThe Band

Inside, downstairs: Eh, dark, loud. Everyone was packing the pool side so lines inside (particularly upstairs) were non-existent when I was there. Easy access to taquitos wasn't especially engaging. We also had a briefly interesting but overall uninspiring Kill Bill interpretive dance also some disappointing fire play (I am spoiled).

Kill Bill as Interpretive Dance

Fire PlayIt is a tech conf after all

Little Buddha: Apparently there was sushi, I got a couple of pieces from the on the run guy, he would barely slow down to let you get anything. Couldn't find any of the carry out type boxes with food in them, no idea what they were. Dessert was tasty but I am not sure how the baklava relates to the sushi theme. Really tasty whipped peanut butter tart things (if you like peanut butter). I took one picture it wasn't so good.

The final I am sometimes a negative person verdict: music was to loud (always is) forgot my earplugs, party wasn't bad but it was somewhat hard to navigate. Got some good pictures of the dancing (sure to be a hit based on the long term ogling around the pool).

Ogling, take a picture it lasts longer :)A view from the upper deck

DancingdancingDancing for the cameraJust to make sure you are paying attention

Dancing for the camera

After the party we went back to the hotel had a drink and cigar at Napoleon's then on to an "outdoor" cafe/stage at paris.


I forgot the name of the group

After the party

Gallery of all the pictures.