19.12.12. Restoring Lost OS X Notes Folders

So I'm a loose fan of OS X's Notes app. It's simple - no having to think of filenames or storage, simple folders if needed, and it syncs everywhere. No dealing with Markdown. It's pretty damn close to a piece of notepaper.


Unfortunately, it has some weaknesses. And the most unfortunate weakness is that its data can be a bit fragile. The data is stored in IMAP on your mail server, and to keep things simple, I just had two folders in my iCloud mail account. All was going OK. Until the app had a mini-freakout earlier (it wasn't showing a cursor in the note area). I quit and reopened the app, and I was faced with a collection of local folders as well as iCloud folders. The local folders had 'recovered data', but I only had one new note in there. So I tried to empty and delete the 'recovered data' local folder.

Instead, I ended up accidentally deleting my Work Folder, and the ten or so notes I had in there. There was a warning dialog that popped up, but I thought it was in reference to the local folder and clicked through it.

So the notes were gone. I had to try to find them, but they were gone. The computer I was on didn't have Time Machine, so I had to hope that there was something I could go back in time and find at home.

Here's the location - {Home}/Library/Mail/{iCloud folder}/. I hit the time machine button once I was in there and was able to go back to yesterday and found a Work.mbox. I restored this .mbox folder, but that alone didn't cause the notes to come back. There are a bunch of folders and subfolders until one comes to .emlx messages. The easiest way to list all of them is to search for .emlx files. They don't open in Notes.app but they will open in Mail.app, with all of their formatting preserved. I imagine if you could open them in OS X 10.7 Mail.app, which still had the integrated Notes folders, you could move the messages into a new subfolder. But on 10.8, I had to manually copy the message text into Notes. It was only about 10 notes, so it wasn't bad. It's better than losing some of this data.

This did lead me to start looking at other options to handle short term reference that was as simple and convenient as Notes, but maybe with an easier-to-restore backup option, but I haven't found it. I am evaluating Evernote, but it feels more complex than what I use Notes for.

For me, Notes.app is a good place for temporary reference or some other minor/personal notes and lists. It's not a permanent archive, but it's a good place to collect thoughts before going into meetings or to hold onto some reference material from emails. Again, I don't need to think of filenames or tags or Markdown or anything fancy with it. I just wish it behaved a little better on OS X. I wish it was an iCloud Documents app that hid the file management part completely, but could take advantage of OS X document management (versions, time machine, etc).

Evernote may solve my problem of where to put notes after they're no longer needed in Notes.app, but feels a bit heavy for the use case outlined above.

Hopefully this is the only time I run into this problem.

18.11.12. My Email Inbox Workflow with OmniFocus

I had one of those mornings where I was dealing with personal morning stuff - exchanging some items with a neighbor, worrying about some odd behaviors from the dog, and the usual flurry of getting dressed and trying to get the dog out for a walk. As this is going on, a rush of emails starts coming in of actions to do for work, on a project that I had done little work on and wasn't immediately prepared to handle. In particular, I was seeing these emails come in on my iPhone where I could do little about it. My first instinct, and action, was to start stressing out a little bit: "Why this project? Why now? What do all of these things mean? Oh crap, is this new email related to another project?" Meanwhile, more emails were coming in and getting grouped together in my inbox in their own stream-of-receipt way.

But then I was able to let go of the stress and let the flood continue. Why?

1. It's my job to deal with this stuff. There was no bad news or anything here.
2. I didn't need to deal with it immediately. Just because there was an email on my phone didn't mean that I had to start working. It can be easy to forget this.
3. I didn't even need to read every email immediately. Once I recognized the pattern, I could just let them pile up like in the old days before constant communication, while still keeping an eye out for anything urgent or that could be answered quickly while waiting for the dog to finish sniffing whatever has her attention while we walk.
4. I strive to maintain "Inbox Zero", so I didn't need to worry about preserving an unread count as "emails that need processing". Basically, if it's in the inbox, it's awaiting processing. So after reading a couple of these emails and recognizing what was going on, I knew that I could wait until I got to the office to actually deal with them.
5. I didn't have worry about process or organization. I have OmniFocus, and I knew that it would be easy to turn these static out-of-order emails into actual items that I could process. OmniFocus makes it easy to copy all or part of an email message to its own inbox, but from there I can edit the text, re-order things, start picking apart subtasks, and so on.
6. Once those items are captured into OmniFocus, I can move those emails out of the inbox and into a project folder, possibly never to be seen again. I can open the original message from OmniFocus if needed. This will bring my Inbox count back down to zero (or near zero).
7. Now I can actually process all of this information that came flooding in and get things done.

Also, spending time with meditation has allowed me to recognize the silliness of that sudden stress-tick that comes when emails like this start coming in while I'm rushing to finish other things. It made it easier to recognize, as Merlin Mann once said in an episode of "Back to Work" (can't remember which episode), that it's just a new email. I'm not alone in the woods, trying not to be eaten by bears. Instead, I think of the list of things above: "wait, why am I worried about this? I have a process for dealing with this, it'll be OK."

At PyCon 2012, Paul Graham spoke on how he wanted a replacement for email that was really "send me to-dos." I guess for him, most real emails are actionable items and he wants them presented as such. Combining OmniFocus' mail clipping capabilities provides something similar for me, but with me taking an active step in transforming some inbound request into actionable data. I think this works better since not every email is actionable, or its action may be a quick reply. For me, the important thing is to figure out what needs to be done and get all of those messages out of my Inbox and filed away.

If the email is just reference info, or is not related to anything I'm currently doing but may come up again in a few days, I scan it and file it away. If it warrants a quick reply, that is done before filing it away. Longer term reference info is often copied into something like Apple's simple Notes app before filing the original email away.

If the email pertains to something actionable, it tends to go into OmniFocus for further processing. Most of the time this is related to work stuff, but it can also be reminders of something to do at home such as a reminder to download an update to some audio software. OmniFocus' quick entry window makes it easy to assign projects and/or contexts. If it requires deeper thinking, it just goes into the OmniFocus inbox. Then the original email may, of course, get filed away into an appropriate folder.

If the email relates to something like a shipping notification that I want to track, I copy the tracking number into an iPhone app dedicated to package tracking. For a long time, these tended to be the kinds of emails that would stick around in the inbox until the item arrived, but during some shopping sprees or slow-delivery items, I found that the emails were getting distracting. A dedicated app also makes it easier to visually track the status of multiple shipments. Once the shipping info is in the tracking app, the original email is deleted. No need to keep those around for reference.

Receipts get archived immediately to a folder named "receipts". Other emails that are not related to a project or receipt that are not spam are now moved to the "Archive" folder. I'm glad this is a quick action in OS X's mail (I used to keep a generic "reference" folder for this, and turned that into the "archive" folder).

And with all this, I tend to keep my email inbox count to zero or near zero. This makes it easier to process big volumes when they come.

See also: The OmniFocus and GTD White Paper (PDF). My personal system is not full GTD, but the basics are there and the core inbox-processing workflow is well diagrammed and discussed in this paper.

 

1.11.12. A few more thoughts on iA Writer

A day into the iA Writer experiment on my iPad and iMacs, and these are some quick thoughts:

Good: iCloud Support. I like Dropbox, but there are situations where iCloud documents just make more sense to my brain. I haven't had any strange behaviors yet, whereas some Dropbox based apps that I use on both my Mac and iPad do exhibit annoying behaviors. I like that I can have the same document open on multiple devices and that the document itself gets updated in place. I'm not trying to write on multiple devices at the same time, but I don't like getting funny "(saved on …)" files when I forgot to close a window or document on a device. I like OS X Mountain Lion's iCloud document management window within the app, and it's very easy to drag files in and out to file things away in the larger file system (including Dropbox via its OS X Finder Integration).

On iOS, iA Writer also supports Dropbox, but I have not tested it yet.

Good: Simple Interface. This is the first "distraction free" simple writing app that I've enjoyed since the original WriteRoom 1.0 (I didn't like future versions as much). It's very easy just to write, whether in windowed mode or full screen. Writing this in full screen on my iMac right now doesn't feel much different than writing on my iPad with a bluetooth keyboard last night.

The default (and only) font is quite nice on all platforms.

Mediocre: Markdown Links. While inline-style Markdown links are OK, I like the footnote style when I'm just writing things down and want to pile up a list of links to collect later at the bottom of the document. I'm interested in iA Writer as a blogging tool and like that I can compose entries so easily on my Macs and iOS devices, but this little missing feature is annoying. I'm not a heavy Markdown user and don't care about "MultiMarkdown" or other such enhancements, but footer-style links would be nice.

The developers say that such support is coming.

I'm also undecided about my blogging host/platform, so the ability to write in simple Markdown and "Copy as HTML" is a nice feature in iA Writer for posting via other tools that may not have (easy) Markdown support. This need of mine may change.

31.10.12. Testing iA Writer and Keyboard

This is something that may or may not get me back into blogging - being able to use my iPad but type at near-full speeds. I've added an Apple Wireless Keyboard to my setup. It's an experiment. A friend challenged me to participate in NaNoWriMo. I have doubts about that, but thought that it might be worth a try. I have characters and places and bits of story that are already lying around and it would be good to challenge myself to do something with it. But my main machine at home is an aging (mid 2007) iMac that is really my music studio machine. Well, it's really a media server that services the iPad and iPhone these days. Occasionally it's used for music, but that's like once per month. The point is, it's not the most comfortable writing environment due to the way the desk and iMac are set up.

Part of me was thinking this would be a good opportunity to buy a MacBook Air, I really can't afford one and don't have a strong need for one (they're not strong enough for music, and I don't move around enough to warrant buying another laptop. Besides, I still have an old white plastic MacBook that is serving its remaining years as a bedside Netflix and DVD viewer (and the DVD portion is important). I haven't found need to use for much else beyond that in the past couple of years, so it's getting a bit crusty.

Still, I was thinking about what would help me try to write from a more comfortable place? Well, there's the iPad, of course! Using its onscreen keyboard for longer pieces, such as the one I'm writing now, is inadequate. So I thought it was time to finally try adding a bluetooth keyboard, and most reports I've read indicate that the Apple aluminum wireless keyboard is generally unbeaten. I don't plan on taking this with me to very many places. I just want to stay on this side of the loft with a smaller, simpler, more focused machine. So far, it seems to be working.

For the start of this experiment, I'm typing this all out in iA Writer, a very clean and simple writing app that is a blank white screen and a nice big font. More importantly, to me, is that it supports iCloud's "documents in the cloud" feature. While I like DropBox and have had some success using text editors in DropBox, iCloud's setup when there's a nice corresponding Mac app seems even simpler. I've been wanting to give iCloud documents a serious try, as there are more and more places where I find myself wanting it.

So, a new experiment begins, starting to use the iPad even more seriously as a productive machine.

6.9.11. HTTP Basic Auth problems that affect Zope (2 and 3) in Safari 5.1, Mac OS X Lion

One very frustrating experience that I encountered after upgrading to OS X Lion (10.7) was that Safari 5.1, as included in Lion, would constantly pop up basic-authentication dialog boxes on our CMS sites, which are based on the Zope Toolkit (kind of between Zope 3.4 and Bluebream). Just about every page in our admin UI would do this. To get around this, I switched to Firefox for interacting with our admin screens, but still used Safari as my primary browser (better OS X citizen, takes advantage of Lion features which I enjoy, bookmark syncing, etc). This lead to problems as Firefox could get pushed way back in the usage stack and would be paged out, and it did NOT like to wake up after long periods of inactivity.

A couple of days ago I decided to take another look at the WebKit project as I was certain that I was not the only person having this issue. And I found a bug that had been recently closed relating to this issue. It's WebKit bug 66354 and it's resolution is in the WebKit nightly builds since at least September 2, 2011.

Apparently this only affects OS X Lion as it has to do with low level CFNetwork changes in Lion. The bug occurs not with basic auth, exactly, as I was able to use other systems behind basic auth. The bug occurs when there are redirects with Basic Auth, which our CMS uses a fair bit in its admins for basic navigation links.

As of OS X 10.7.1 and its Safari (Version 5.1 (7534.48.3)) this is broken. If you use OS X Lion and Safari and encounter HTTP Basic Auth problems, I'd recommend switching to the nightly builds.

 

19.8.11. PyFlakes and BBEdit, kind of together (at last?!)

I've recently returned to BBEdit as my primary development editor after spending the past few years with the increasingly neglected TextMate. One thing that I missed from TextMate was some nice 'PyFlakes' integration. Since adding PyFlakes to my daily workflow in TextMate, either through the 'PyCheckMate' option or via the 'PyFlakes On Save' option (which popped up in a tooltip), my module imports have been cleaner and refactorings have gone more smoothly as I can catch forgotten names and imports in moved code.

The first thing I did when I moved to BBEdit was to rig up a quick script that ran PyFlakes and bound it to command-shift-V (V for Verify, or validate), based on the PyFlakes-On-Save TextMate command (which I found in a Zope bundle, I think). My script just dumped the results into a new BBEdit 'Unix Script Output' text document window. It worked, but was not particularly helpful. BBEdit has a 'check syntax' command built in that will catch SyntaxErrors in a python file, and its results go into the 'search results' window which allows you to step through the errors nicely in a dedicated window. I remember using something like this with Emacs and things that modified Make-mode to jump to error results in unit tests, making it easy to step through failures with a single key-chord.

Well today, I finally found how to get my results into that window. The script is now up on GitHub in Gist 1157742.

 

24.3.11. Forget distraction free, I want file-management free!

A few years back I downloaded an interesting piece of software for the Mac OS - Writeroom version 1.0. Writeroom sells itself as a "distraction free writing environment." It's main feature was that it was plain text only and could run in a full screen mode with default colors of being green text on black background. There was some fanfare around it and some skepticism. Personally, I liked the idea - get rid of all of the other windows, instant messages, dancing gifs, email notifications, etc, and just write.

This is something which I like about the iPad - only the current application has my attention. I hope that this doesn't change. I do not like what I see in Android 3.0, nor what little I've seen of the Blackberry Playbook. Their interfaces look too busy. I deal with busy interfaces all day. For me, the iPad is a relief from the world of current desktop computing.

My favorite key combination on Mac OS X is option-command-h, aka Hide Others. When the screen starts filling up with windows and more windows and more windows, Hide Others temporarily brings some order to the madness.

Back to Writeroom. My favorite feature of 1.0 was actually not this 'distraction free writing environment' nonsense. That was a useful feature, to be sure. But the feature I liked most was that it was a file management free writing environment. Writeroom 1.0 managed its own files. There was no saving, no filename choosing (the document name was taken from the first line of the document). This made it a perfect place for just jotting things down, working small ideas out, and maybe taking them into bigger things. That file management free writing environment was bliss, at least for my uses. Every time you opened the app, there were all your documents. You could close them to hide them, but they were easily accessible from a menu and a keyboard shortcut.

There were some other apps that offered similar features - the Stickies application that comes with Mac OS X (and dates back to Mac OS System 7.5) is another nice tool for temporarily holding small pieces of information without having to worry about file management; Yojimbo is a collection bucket of notes, PDFs, etc, and can be another place for notes to quickly go; and Apple Mail in Mac OS X 10.5 added Notes which could be stored in IMAP (and can finally sync to the Notes app in iOS 4) and offer a nice quick jot-down place that also takes its title from the first line.

Writeroom 2.0 killed this feature. I avoided writeroom 2.0, but when version 2.5 was on the Mac App Store at a temporary discount, I bought it. And I hate it. Now I have to think about filenames. Now I have to think about where things are being stored. Writeroom 1.0 allowed you to easily export a document out, but that kind of decision was usually made AFTER you were finished with the document. There are times when I don't want to think about that stuff when starting out. This is not true for all cases - much of my day work is spent in TextMate writing Python code, reStructuredText documents, and managing configuration files. Of course I need full file management in that situation. But when it comes to just writing down some notes, capturing some ideas, and maybe thinking through a proposal, I miss that Writeroom 1.0 functionality.

And again, this is something that the iPad (and iPhone) gets right, for most instances. Whether its OmniGraffle, Pages, Garageband, or the built in Notes app, starting a new document is damn easy. Saving is automatic. You just start using it.

Fortunately, the next version of Mac OS X, "Lion", looks like it offers many long overdue file/document management features for developers so that they can offer functionality closer to Writeroom 1.0. Again - I'll always need TextMate and some apps to do manual file management, but those uses are specialized and becoming more rare. Mac OS X Lion adds support now for auto-save, reversion, locking, and historical change tracking. In some ways, this feels like going back to some of the ideas explored in the LISA and other pre-Mac user interfaces that weren't shackled to the hardware limitations of the original Mac OS (file management in the Lisa OS was quite a bit different - everything started from the finder, including new documents). In other ways, it feels like we're catching up to what the Newton OS, Palm OS, and iOS devices have long done in their aims of imitating paper notebooks and planners - everything is just saved, automatically.

It's not perfect for every situation, but it's something I've long been wanting in more places than just calendar and contact apps.

13.2.11. To Laptop or Not To Laptop

PyCon 2011 is coming soon, and as it does, I find myself facing the question "do I bring my Macbook this year or not?"

My Macbook has spent most of the past couple of years serving as a vehicle for viewing DVDs, Hulu, or Netflix streaming from bed. It does get used occasionally as a home / travel development machine, but that use has declined significantly over the past year. I'm fortunate to be at the place in my life where I can leave work at work; but I also have been rather uninspired to do personal development work. Hence, the laptop is basically a DVD viewer.

And for DVDs, it's quite nice to watch TV series DVDs from bed. I only watch about 20-30 minutes a night before tiring out. It's a fun way to revisit personal favorites like HBO's The Wire and Deadwood. I'm currently watching Twin Peaks, seeing season 2 for the first time.

But for DVDs, it's also starting to become a liability, as I own more and more blu-ray. As soon as I decide to get a blu-ray player for upstairs (either a portable one or getting a second television), the laptop's uses will be near zero.

One potential use is for travel, at least for technical conferences. I brought the Macbook along to PyCon last year. I did the whole trip out of my day-to-day backpack, which accommodates the Macbook just fine. While at the airport and in flight, the extra weight and space did stand out, but that was only a fraction of the trip. I'm trying to remember now just how much I used it once there. I took far less notes on the device than I expected, and have done little with the notes I did take. They still sit in a folder on the Macbook's desktop.

I know I did some coding while there, but not much. I don't believe I made any major commits or pushes while at the conference. I wasn't involved in tutorials or sprints last year and won't be this year either, so there's little need to code while at the conference.

Finally, my co-worker had brought his laptop and while he did use it to maintain some situations back at the office, he told me he never used it on the conference floor. I think the most use we got out of our laptops was for watching movies on the plane. And hey, I've got an iPad for that.

So as this next tech conference comes up, I'm seriously considering leaving the laptop behind and just using good old pen and paper. Maybe my personal era of the laptop is officially over. Weird.

 

1.10.08. Giddy-up 401, File Uploads, and Safari

I've recently been doing some work to support ZODB 3.8 BlobFiles in our Zope 3 based sites and applications. Doing this brought me back around to seeing some behavior I've seen in the past and probably learned to ignore: uploading a large file from Safari using a basic HTML form (with proper encoding type, POST, etc) seems to take inexplicably long. Even worse - once behind Apache, you might not get an expected response, if any. You might get a 'timed out' response, unsure if the app server has everything and will finish the request/response cycle on its own.

It turns out that Safari does not eagerly send along authentication information along with each request when logged in with Basic Auth. When it does, it seems to have a very short time window.

So say you're logged in to your application with basic auth (for better or worse). The normal pattern is that when encountering an unauthenticated situation, Zope will challenge with a 401 status code and the WWW-Authenticate header (or something like that - I'm away from all specs right now). If you're not logged in, then you get the basic auth dialog box and send along the credentials. If you are "logged in", then the browser doesn't ask for credentials again, but just sends them along.

The downside is that this causes a request to be repeated. And if you just tried uploading a 5 MB file, then that whole file has to be re-submitted to make the full request.

It's the right thing to do with the basic (ha!) information at hand - trying to post all of that data in a single request. But Safarishould recognize that it's submitting to the same server (if not the same URL!) and should automatically include the auth headers. Safari seems to do this, but only on requests in very short windows.

Firefox, on the other hand, seems to have a much longer window in which it will send the credentials along automatically on the first request, instead of waiting for the challenge.

I don't know how other browsers do it. I'm not sure what the spec says, if anything. Glancing at O'Reilly's "HTTP - The Definitive Guide" didn't seem to give any indication of whether it's better for the client to assume that it should send the authentication tokens along with each request back to the same server, or if it's better for the client to hold off on constantly sending that info along unless challenged.

Most of the time this doesn't really seem to matter - it's not something end users typically see as it goes by fast enough to rarely be noticed. Of course there are other ways of getting credentials (cookies, sessions, subdomain/remote ip mapping, etc) which we often use on the main public-facing side of our sites. But for content management and admin, Basic Auth is such an easy fallback, especially in a framework like Zope (1, 2, or 3) which has long had a strong security story and would automatically handle this crap for you way back in the days of Bobo (1996, if not earlier).

It's just an annoyance. Glad I nailed it down to this: uploading large files with Safari (I think IE is, or was, similar) to basic-auth protected sites often can time out because the browser posts once, gets the 401-Unauthorized challenge, and does the post again - this time with the auth info.

Solutions:

• don't use basic auth for sites that expect moderate to heavy uploading via forms.
• recommend and/or use browsers that send the auth token along more often in the first request.
• provide better interfaces for uploading files; providing better communication with the uploader about status, and perhaps having a better interface into the destination web app. Fortunately there are appear to be some free and open solutions out there already.

Wheee!

16.7.08. OmniFocus Revisited

I've been using OmniFocus for a while now on my Mac, and made occassional use of it's predecessor, Kinkless GTD. As an implementation of the "Getting Things Done" method, or just as a personal "what am I doing?" manager, OmniFocus 1.0 was pretty good. I, personally, liked it better than the alternatives I tried, but some of that may be due to familiarity with the OmniOutliner heritage. And OmniOutliner remains one of the greatest Mac OS X applications ever written. I believe it's one of the greatest outliners ever written. Sure, it doesn't have Mind Mapping support and it can't run a slideshow (although its data can be exported to OmniGraffle or Keynote). OmniOutliner is a joy because it's fast, beautiful, ridiculously easy to use, while still having a fair amount of power and flexibility. Kinkless GTD was actually a set of AppleScripts that made OmniOutliner act like a specialized GTD application (which OmniFocus now does natively).

But when it came to OmniFocus, I only used it at work, and even there I only used it intermittently. But with the 1.1 "Sneaky-Peek" version of the desktop client (still under development), OmniFocus supports synchronization via MobileMe or any WebDAV server. This not only enables me to share this information between the office, home, and the rarely used laptop, but it enables sharing with the awesome new OmniFocus for iPhone.

Second only to the iTunes Remote, OmniFocus for iPhone is the coolest mobile application I've seen so far on this young platform. OmniFocus for iPhone (or iPod Touch) can synchronize over Wi-Fi, Edge, or 3G. That alone makes it very useful. Cooler still is that it's location aware. "GTD" has a strong concept of contexts. A project may require picking up supplies at an office supply store, assembling at home, and mailing out items at the post office. With OmniFocus, you can add location information to any context. This location information may be based on where you currently stand (using Location Services), on a manually entered address, on an address from a Contact record, or on a business search. Then when looking at the "By Location" screen in OmniFocus, available actions get grouped by their location in relation to where you currently stand. "Grocery Store: 1 Mile; Post Office: 3 Miles..." Very Cool.

This is the first electronic implementation of GTD that actually appears usable. Even if you don't follow GTD religiously (I certainly don't), the projects/contexts combination is an effective way of organizing actions. The location-aware contexts in OmniFocus for iPhone help answer the question "what can I do based on where I am?" When used well, it should make it harder to forget those little items that you needed to do or pick up for some small project.

12.7.08. iPhone 2.0 and the iTunes Remote

Like many others, I was bit hard by the server failure that crippled the big iPhone 2.0 update process this morning. It robbed me of said morning, and I opted to take the day off work. It's been a bit of a tough week, and last night I did my first show in months, and had company over until past 3AM afterwards. Since things are about to get crazy at work, I decided that I needed a day to catch back up on sleep while waiting for my phone to work again.

A few hours later, the phone worked.

One of the most surprising and enjoyable elements is the iTunes Remote. Full and comprehensive access to my fairly large iTunes library on the iMac: all playlists, etc, with the ability to control volume, jump around in songs, see artwork - just like the 'iPod' iPhone application! Sometimes, particularly in the morning, I might turn on Front Row and carry the nice little Apple Remote around with me to have some control over songs while getting dressed in the upstairs loft, but using the remote from up there often required reaching far over the edges and trying to point the remote in the direction of the iMac, just to be able to skip forward a track or two.

But now, I can control it all from the iPhone, without needing a line of sight! In fact, this might be what pushes me to pick up an Airport Express or two (one for speakers in said upstairs loft, maybe one to bring music out on the patio or the bathroom). Full access and control of 29 and a half days of available music playable out of real speakers and controlled by an untethered iPod size device - beautiful.

10.6.08. Looking to a Snow Leopard Winter.. er... Summer.

I’m a bit excited about Mac OS X “Snow Leopard”. Few user-visible changes, with a focus on fine-tuning and giving developers better access to capabilities of modern hardware. It appears that Apple’s experience in making a lightweight Mac OS X “Core OS” for the iPhone will also drive this release.

One of my favorite operating system releases was OS/2 “Warp” (OS/2 3.0). OS/2 2.0 was a fascinating creature - completely divorced from Microsoft, OS/2 2.0 delivered an aggressively object-oriented runtime built on SOM (a desktop implementation of some of CORBA 1.x, I believe). It was radically different from Window 3.x. It’s hardware requirements were a bit high for the times, but it was a solid OS.

What impressed me about OS/2 3.0 “Warp” was that it’s system requirements were in some cases significantly LESS than OS/2 2.0, while performing better. I don’t know of any majoro user-visible adjustments (this was before operating system releases became the giant dog’n’pony shows that have been expected since both Windows 95 and Mac OS X).

I think that even though desktop and laptop hardware continue to get better, the rapid growth rates seen between 1995 and 2005 are slowing down. Now the pressure is on connectivity, portability, and storage storage storage for all of those mp3s and movies and photos. I think both Windows XP and Vista, along with Mac OS X 10.4 and particularly 10.5 have been a bit cavalier about their usage / expectation of resource availability without doing a good job of cleaning up afterwards. Removing a ‘TemporaryFiles’ folder used by Apple’s “Soundtrack Pro” program gave me back 25 GB of disk space. 25GB! I expect that when doing lossless audio work, I’m going to leave a lot of turds behind. But not that many. That’s an accumulation over only a few months. Now some of that may have been due to crashes brought on by the instability in Mac OS X 10.5.2’s audio subsystem (particularly in relation to some USB audio devices). But still - 25 gig! Over the course of just a couple of months!

I think that Apple is at a good place to do this. Good housekeeping is required - otherwise you end up with situations like Mac OS ‘Classic’ or even Windows Vista, where there is so much old baggage, bad hacks, outdated mentalities, etc, all in play; it makes it difficult to move the platform forward. Some companies and developers have always been mindful of this, electing to keep their products lean and fast, always (see Ableton Live - hands down, the most impressive audio application out there). Other companies don’t support that philosophy for whatever reason - backwards compatibility, rush to market, a combination of the two, etc.

This far into the Mac OS X life cycle, there’s not many new dog’n’pony features to add. The API’s have stabalized, the developer tools offer more than they ever have (Interface Builder 3 is a terrific update), the Finder and Spotlight are actually fast and usable; applications and utilities from both inside and outside of Apple are going to really shine on Mac OS X 10.5 with all that it offers to developers. A new age of PDA’s are upon us, whether it’s a device like an iPhone, an ultra-mobile Asus Eee-PC style portable, or even the Macbook Air: secondary and tertiary devices are really taking off.

I think that an underlying aspect part of the ‘Snow Leopard’ plan is to allow such devices, made by Apple (naturally), to proliferate. When it was announced that the iPhone was built on Mac OS X, I was surprised - Mac OS X has been a pretty wasteful OS - or at least, one that would consume more resources than realized (often for caching, interestingly enough). A standard install is full of crap that may be useful, but often takes up space. How many gigabytes of printer drivers now? To take the fine tuning and resource management ideas from the iPhone variation of OS X into the main system is what I think will allow for Apple to finally make the Eee PC style portable that everyone wanted the Macbook Air to be.

I’m putting my money on some kind of small device, priced around $600-$800, coming out at or around the same time as Snow Leopard. Combined with Mobile Me and Snow Leopard Server’s increasingly Exchange-like feature set (but better priced and more understandable for small organizations), the ubiquitous-data-access capability is there.

Today’s full-featured laptops (MacBooks, Inspirons, whatever) are their own entities; my aging iBook gets used rarely as I just don’t have as much data or software set up on it, and it’s sometimes too big of a pain to keep in sync.

The XO and Eee-PCs (or whatever they’re called) are also separate from the rest of one’s life; useful as a fun or educational toy, or as a geek’s favorite gadget to see what they can get running on such a little device. Most of the other developments I’ve seen in this area have centered around “how cheap and how small can we make a laptop/portable that will run (Linux/Windows XP)”. But outside of education, if this is the only focus being given, then these companies are going to be making nothing more than the next round of casual gadgets that get tossed or buried after a few months - especially if a key factor of what made Palm devices so popular (for a while) is completely neglected.

The Macbook Air is deliberately designed as a complementary computer, using the master’s optical drive even. While sexy, I think the Macbook Air misses the mark on a few items. But I think it’s an indication of things to come - laptops deliberately designed to complement your main machine. Smaller devices, from the Palm to the iPhone, have done this. And they’ll also be designed to work with your (or your company’s) data, which the Blackberry has done (and the iPhone will do when its new ‘enterprise’ support rolls out). Getting this onto other devices, without being constrained to an enterprisey system like Notes or Outlook, is where things really appear to be headed. It’s certainly something that I’d like to have. And the more I look at Snow Leopard, the more I believe that Apple is sneaking ahead of the crowd into delivering this into the hands of consumers. They’re skating to where the puck is going to be.

Granted, Windows “Live Mesh” looks to be heading in the same direction. But after Vista, Microsoft needs to reign in the Windows kernel and distribution. Windows Server 2008 and some of what has been leaked (or speculated) about “Windows 7” seem to indicate that Microsoft is aware of this. And how could they not be? But I think that even with their vast resources, Microsoft has a long ways to go to catch up - even though it appears that they’ve been playing in this area (tablet computing, ultra-mobile pc’s) for a while. A deep cleansing of the Windows core is desparately needed. And then a deep re-implementation of the UI may be needed.

Apple had a terrific luxury (and great idea) with the iPhone. While sharing the same kernel and many same APIs as the desktop (and server) Mac OS X, it has an entirely new UI that is dedicated to its intended use. Windows CE, on the other hand, tried to bring the Windows 95 look and feel to tiny devices and now I’m really not so sure it was a good idea. It allowed Microsoft to punt on some usability and design issues by falling back on the way things work on the desktop. I still see this, even in some of the newest and fanciest “iPhone killers”: some of these have a very fancy launcher app; some even have a very fancy phone and contact app that spins around in 3D and responds to gestures. But then, suddenly, you’re in the tiny-font, tiny-scrollbar, pixelated, stylus-driven world of the interior. It’s like going into a grand building like The Plaza (back when it was a hotel, at least), and finding the inside full of grey linoleum floors, flickering flourescent lights, and cinderblock walls reminiscent of an old hospital or elementary school. Quite the let-down (a lot of courthouses are like this, actually).

I also think Apple was smart to NOT have an SDK at the launch of the iPhone. I bet they would have liked one, but I think the iPhone had to launch when it did, and perhaps not-quite-everything was ready yet. If one looks back at the classic Macintosh and Palm devices and operating systems, you see systems that pulled of very clever hacks to fit within the price and size constraints of the time. The Lisa was much more than a $10,000 Macintosh - it had many features from power management to an OpenDoc style multi-tasking document based UI. But to offer those features, it was priced well out of reach. The Macintosh squeezed as much as it could down into a 128K Ram machine, and the compromises they had to make in order for that to work would end up haunting the company until its near-death. The Palm, too, took the ideas of the Newton and other tablet devices and stripped them down into a size and price point approachable by the masses. And like Apple, the design decisions that were made to make that work have crippled the Palm OS so much that even Palm sells half of its devices with Windows CE (or whatever CE is called these days). Those compromises are bad enough to deal with on your own - but when having to support third party developers and then provide some degree of backwards compatibility, it can just kill you.

By taking the time to put the SDK into beta, to polish up the OS and its APIs, I think Apple will avoid a repeat of that story. Instead of having to support every little exposed compromise that may have been made to get the iPhones out the door last June, Apple could tidy them up. By using a beta period for the SDK and next major release of the software, Apple can respond to feedback and make changes and adjustments before they become permanent.

1.11.07. AppleScripting Across the Universe

After a long day at work, I wrote a long message in Basecamp about what I had accomplished, how to access it, etc. But I forgot to submit the message! Crud. I wanted to send it out before morning and didn’t want to go into the office. I couldn’t get any screen sharing connection to go between the machines. I just had a handful of SSH leaps.

AppleScript to the rescue!

This is probably the most AppleScript that I’ve ever written. Fortunately, Safari supports the command do JavaScript ... in tab. After some floundering around with a similar setup on my local machine, I finally figured out AppleScript’s interesting reference notation and was able to ferret out the window and tab containing the unsent message, add some text to the message’s textarea element, submit the form, and return the extended value.

tell application "Safari"
    set message_tab to current tab of window named "Web site > New message"

    set extended to ".... Fun fact - i wrote this before i left the office and forgot to submit it. as a result, i now know how to submit forms like this via AppleScript."
    set post_body_value to "$('post_body').value"
    set extend_value to post_body_value & " += '" & extended & "';"

    do JavaScript extend_value in message_tab
    set body_value to do JavaScript post_body_value in message_tab

    do JavaScript "document.forms[0].submit();" in message_tab

    return body_value
end tell

I pasted the above code into VIM and ran it with the command line osascript command. Worked like a champ.

And because sleep is for the weak, I decided to track down how to do the equivalent in Python. Mac OS X 10.5 provides a “Scripting Bridge” for Python and Ruby (and potentially others), which causes many frameworks and other objects to be dynamically exposed. Without the need (for better or worse) of yet-another-virtual-machine. Anyways, I cobbled the following together:

from Foundation import *
from ScriptingBridge import *
safari = SBApplication.applicationWithBundleIdentifier_('com.apple.Safari')

def find_window_named(name):
    for win in safari.windows():
        if win.name() == name:
            return win

window = find_window_named("Web site > New message")
message_tab = window.currentTab()

print safari.doJavaScript_in_("$('post_body').value", message_tab)
safari.doJavaScript_in_("document.forms[0].submit()", message_tab)

There may be a better way to do the find_window_named method, but I didn’t have the time to track it down. As it was, I was able to do do the above by playing around with everybody’s favorite Python tool, dir(), which verified my suspicion that many of the commands exposed to AppleScript were also available via the Scripting Bridge. This is evidenced by the currentTab() method of a Safari window, analogous to the current tab of window ... AppleScript. And I imagine most of these are just Objective C methods. And since AppleScript editor’s Dictionary browser told me about the do Javascript [v] in tab [t] command, it stood to reason that it would exist on the Safari object. It was there when I did pprint(dir(safari)), and I knew that I’d need to pass in a Tab object.

In any case, it’s awesome that Apple has embraced Python and Ruby and has tied them in to the Cocoa runtime. Historical note: the first Python - Objective C bindings that I know of where commissioned by a NeXT Developer who wanted to use Python and Bobo (zope.publisher) to do web work with NeXT’s Enterprise Objects Framework, without the weight and cost of WebObjects. I think that means that Python was bridged into the Objective C runtime and NeXTStep frameworks before Jython ever got going. I believe that work was done by the developer who later released Objective Everything which bridged into Perl and TCL as well as Python.

Of course, traditional MacPython from the classic Mac OS was also natively tied in to the AppleScript of that era; AppleScript has always supported other dialects (FrontierScript was a common one).

But it’s nice now to see support coming out of both Apple and Microsoft (and Sun too, I guess) for these languages. The above scripting of Safari was surprisingly easy. As was an earlier experiment to fish around my calendar store for incomplete To-Do items. Quite nice.

But what’s especially nice is that I was able to SSH into my office Mac and tell Safari to submit that form that I had neglected earlier.