Here's what the top 25 looks like for this week. This is from an iTunes smart playlist that shows the top 25 most played songs that have been added to my library in the last month. iTunes smart playlists are an absolutely great feature that I hope shows up in other places in Mac OS X, like oh maybe in the Finder as a smart list of files.

Artist	Song	Album	Play Count
Metallica	Frantic	St. Anger	12
Massive Attack	Antistar	100th Window	11
Massive Attack	Butterfly Caught	100th Window	11
Massive Attack	Everywhen	100th Window	11
Massive Attack	Future Proof	100th Window	11
Massive Attack	Name Taken	100th Window	11
Massive Attack	Hymn Of The Big Wheel (Origin	Hymn Of The Big Wheel	11
Metallica	Dirty Window	St. Anger	11
Metallica	My World	St. Anger	11
Annie Lennox	A Thousand Beautiful Things	Bare	10
Annie Lennox	Bitter Pill	Bare	10
Annie Lennox	Erased	Bare	10
Evanescence	Bring Me To Life (Feat. Paul	Fallen	10
Massive Attack	Prayer For England	100th Window	10
Massive Attack	Small Time Shot Away	100th Window	10
Massive Attack	Special Cases	100th Window	10
Massive Attack	What Your Soul Sings	100th Window	10
Massive Attack	Any Love (Larry Heard Mix)	Hymn Of The Big Wheel	10
Massive Attack	Home Of The Whale	Hymn Of The Big Wheel	10
Massive Attack	Hymn Of The Big Wheel (Nellee	Hymn Of The Big Wheel	10
Annie Lennox	Honestly	Bare	9
Annie Lennox	Loneliness	Bare	9
Annie Lennox	Oh God (Prayer)	Bare	9
Evanescence	Everybody`s Fool	Fallen	9
Evanescence	Going Under	Fallen	9

What's funny is that I have a tremendous breadth of musical interest, but you sure wouldn't know it from this list. I bought a number of more popular albums a couple weeks ago which skews the results away from the more eclectic mix I usually get from eMusic.

Currently Playing "Marquis Cha-Cha" by "The Fall" from the album "Palace Of Swords Reversed", a little more eclectic bit from eMusic.

Mac OS XMusicProgrammingPythonWeb Services2003-06-29T17:01:15-07:0010 The improvements include: Improved performance and stability, Better tutorials and examples, Initial support for MacOS X 10.1, Support for the WebKit framework, Write plugin bundles in Python (requires Python 2.3) [Studio Log]

I've been playing around with the 0.9 release of PyObjC. It's a very promising project badly in need of better documentation. I've really been getting into Python lately and have been using it heavily with my Cocoa projects. I'd love to see Python included by Apple as a full peer with Objective C and Java for Cocoa applications. It would make Cocoa development even easier then it already it. Applescript Studio is nice, but Applescript is still not a language that I feel comfortable with and the way it is integrated with Cocoa means there are quite a few things you can't do with Applescript. I've found it easier to just stick with Objective C. The way PyObjC is being integrated it is functionally equivalent to Objective C and should bring all the power along with the ease of a scripting language. Mac OS XProgrammingPython2003-07-09T12:18:13-07:0000

... and if the release of Python 2.3 w/first class OS X support were not enough ....

Excellent. And, of course, PyObjC will continue to provide first class support for integrating Python and Objective-C, including full blown Cocoa application development using Python in place of Objective-C.

... and on my son's third birthday and everything. [bbum's rants, code & references] Mac OS XPython2003-07-29T23:43:26-07:0000 The libxml Python API is very lightly documented, so this is an attempt to fill in some of the holes that exist.

Creating a new document

To create a new document using the libxml2 API you use a document constructor function that returns an empty document instance. This method takes one argument that repesents the XML version of the document being created.

 import libxml2
 doc = libxml2.newDoc("1.0")

Creating elements

Once you have a document instance you then need to add elements to it. First off you need to create the root element.

root = doc.newChild(None, "root-element", None)

The root node is created using the xmlDoc.newChild() method. This method takes three parameters.

• namespace - The namespace that the element should belong to or None if no namespace.
• node name - The name of the node with no namespace prefix.
• element content - The content for the element or None if the element is empty.

  In this particular case we're creating an empty element named root-element. If we were to print this out at this point it would look something like this.

<?xml version="1.0"?>
<root-element/><p />

If we wanted to put the node into a namespace we would write this instead.

root = doc.newChild(None, "root-element", None)
namespace = root.newNs("http://example.com/sample", "sample")
root.setNs(namespace)

The resulting document then becomes.

<?xml version="1.0"?>
<sample:root-element xmlns:sample="http://example.com/sample"/><p />

Now that we've created the root we can continue adding elements to the document. We can add a element child-node in the http://example.com/sample namespace by adding.

child = root.newChild(namespace, "child-node", None)

And our document now looks like

<?xml version="1.0"?>
<sample:root-element xmlns:sample="http://example.com/sample">
    <sample:child-node/>
</sample:root-element>

If we had wanted to included some text within the added child it's as simple as just changing the third parameter to newChild.

child = root.newChild(namespace, "child-node", "Some sample text")

Which generates the document

<?xml version="1.0"?>
<sample:root-element xmlns:sample="http://example.com/sample">
    <sample:child-node>Some sample text</sample:child-node>
</sample:root-element><p />

Adding an attribute to an element is also very easy.

child = root.newChild(namespace, "child-node", "Some sample text")
child.setProp("an-attribute", "with a value")

Which of course generates a document that looks like this.

<?xml version="1.0"?>
<sample:root-element xmlns:sample="http://example.com/sample">
    <sample:child-node an-attribute="with a value">Some sample text</sample:child-node>
</sample:root-element><p />

If you wanted the attribute to be part of a namespace, you use setNsProp instead of setProp.

child = root.newChild(namespace, "child-node", "Some sample text")
child.setNsProp(namespace, "an-attribute", "with a value")

And the result

<?xml version="1.0"?>
<sample:root-element xmlns:sample="http://example.com/sample">
    <sample:child-node sample:an-attribute="with a value">Some sample text</sample:child-node>
</sample:root-element><p />

Beside simple elements and attributes libxml defines methods to create all the other common XML types. Here's a summary of the methods that are available.

• xmlDoc.newDocComment(comment) - Creates a comment node.
• xmlDoc.newCDataBlock(content, length) - Create a CDATA section.
• xmlDoc.newDocText(content) - Creates a new text node.

These methods are all node construction methods that are called to create the instance of the required type. Once you have the instance you then need to add it into the document tree where ever you want it. There's also a function available to create processing instructions. This function differs in that it called on the libxml2 module, rather then an xmlDoc instance.

• libxml2.newPI (name, content) - Creates a processing instruction

Since these functions require you to create the node and then add it to the document in two steps, libxml provides a number of methods to control where the node is placed in the document tree. These methods are available on any instance of an xmlNode.

• xmlNode.addChild(node) - Appends the new node to the list of children for the node.
• xmlNode.addChildList(nodeList) - Appends a list of new nodes to the children for the node.
• xmlNode.addNextSibling(node) - Adds the new node as a sibling after the selected node.
• xmlNode.addPrevSibling(node) - Adds the new node as a sibling before the selected node.
• xmlNode.addSibling(node) - Adds the new node as a sibling after the selected node. (similar to addNextSibling)
• xmlNode.addContent(content) - Appends additional text content to an element.

  Here's an example that puts everything together.

#!/usr/local/bin/python 
import libxml2
doc = libxml2.newDoc("1.0")

root = doc.newChild(None, "root-element", None)
namespace = root.newNs("http://example.com/sample", "sample")
root.setNs(namespace)

child = root.newChild(namespace, "child-node", "Some sample text")

child.setNsProp(namespace, "an-attribute", "with a value")

comment = doc.newDocComment("Just commenting")
child.addPrevSibling(comment)

pi = libxml2.newPI("a-sample-pi", "with some useless content")
root.addPrevSibling(pi)

text = doc.newDocText(" This will be added to the existing text.")
child.addChild(text)

child.addContent(" This will also be added to the text")

print doc.serialize(None,  1)

And a final result.

<?xml version="1.0"?>
<?a-sample-pi with some useless content?>
<sample:root-element xmlns:sample="http://example.com/sample">
<!--Just commenting-->
  <sample:child-node sample:an-attribute="with a value">Some sample text This will be added to the existing text. This will also be added to the text</sample:child-node>
</sample:root-element>

ProgrammingPythonXMLXML Databases2003-08-09T19:42:38-07:0010 I just wanted to post some notes about installing Sleepycat Berkeley DB XML on Mac OS X 10.2 with Perl and Python support. The builds are relatively straight forward and Sleepycat has posted a simple script to help build Berkeley DB XML it self. However, it isn't clear what is necessary to get Perl and Python working.

The most important thing, before you start compiling anything, make sure you have the latest GCC 3.3 from Apple. This is distributed as a patch to the December 2002 developer tools. This is critical, without it Python and Perl support will not work.

Next, unfortunately, you'll have to build a new Perl and Python. The Mac OS X 10.2 Python should be the right version, but I couldn't get it to work. Building a fresh Python 2.3 does work. For Perl, Mac OS X includes Perl 5.6 and Berkeley DB XML requires 5.6.1 so you have to build a new one. I used Perl 5.8.0 and it seems to work fine. So you have to build a new Python, a new Perl and the Berkeley DB XML distribution. These should all build using the standard instructions and for DB XML you can use their script.

Once you have all that built, you can then build the DB XML Perl and Python libraries.

For Python you first need to build and install bsddb3, once that's done you can build the python support for DB XML in the usual Python fashion. Make sure the python you're using is the one you built previously. Unless you specified otherwise, it's installed in /usr/local/bin/python.

cd dbxml-1.1.0/src/python
/usr/local/bin/python setup.py build
sudo /usr/local/bin/python setup.py install

There's an example Python program in dbxml-1.1.0/examples/python/examples.py that you can run to test the build.

For Perl you just build it in the usual Perl manner. Again, make sure you use the perl you compiled.

cd dbxml-1.1.0/src/perl
/usr/local/bin/perl Makefile.PL
make
sudo make install

There are some examples for the Perl API in dbxml-1.1.0/src/perl/examples.

Mac OS XProgrammingPythonXMLXML Databases2003-08-09T20:33:00-07:0010 I'm thinking about moving this site to Blosxom from MovableType. I'm getting annoyed with a lot of things about MovableType, in particular how slow it is when publishing. Each time I add a post it gets a little slower and it's starting to really bug me. I'm also annoyed by how cumbersome it is to edit templates, especially since I have more then one blog and they all use the same templates. I also like the way NetNewsWire handles Blosxom blogs, it shows you the hierarchy and all posts you've made at all times. With MovableType you only get your recent posts and if you restart NetNewsWire you'll only get the last 10 or so posts you've made. This makes it really cumbersome to go back and edit your old posts. There are many other little things as well that are bugging me as I use this system more.

I really like the simple, simple file based mechanism that Blosxom uses. I've always considered MovableType's use of a database as massive overkill (even if it is just MySQL). I also like the idea of being able to build the whole thing locally and then just shove it up on the server with nothing major needing to be installed on the server.

After reading through the documentation it looks like the one problem area with Blosxom may be categories. I tend to add posts into several different categories, but it looks like Blosxom may only support one category for each post. I'll explore this a little more to find out for sure, but it can't be that hard to make it use symlinks or something to do it.

I found this site which has some pretty good information on making the move.

There's also a version written in Python that looks interesting. Hmm, that could actually be fun. Blosxom is written in Perl and even though I'm perfectly comfortable writing Perl code, it's not something I really enjoy anymore. Python, however I do like. Very, very tempting.

If I do make this move it will be the fourth time I've changed the software I use for this blog. Ugh! Sadly, it probably won't be the last either. Python2003-08-10T02:04:33-07:000 A good list of common problems in Python. I like Python, but I'm anything but an expert so I learned a thing or two from this. Python2003-08-14T10:00:10-07:000 I'm still thinking about converting this site to running on a Blosxom derivative, in particular on Pyblosxom. I spent a fair amount of time over the last week getting everything working in a test setup and I've worked everything out, except I'm worried about the performance of it. Even running on my dual 1.25Ghz Powermac there's a noticeable delay when viewing a page. This concerns me as the server this site runs on is only a dual 266Mhz Pentium II. I haven't tested it on this machine yet. This really shouldn't be a major problem as this site doesn't get all that much traffic and the network is kind of slow anyway, but CGI scripts always bug me. This is the one good thing about MovableType, it's slow to post, but that's because it creates static pages for everything.

The Perl version of Blosxom can also be used to generate static pages, the Python version can't.

So far I've written four plugins for Pybosxom to make it as compatible with the current site as possible. I had to add RSS 1.0 support, Textile formatting support and a post body summarize function. Along with these I also created a new plugin that tracks referrers and hit counts on a per post basis. That one was more my experimenting with Berkeley DB XML then anything, but it's very useful.

Anyway, now I'm stuck trying to decide whether to go with Pyblosxom, go with the original Perl Blosxom or to punt on the whole thing and just stick with MovableType. Mac OS XPythonWeb ServicesXML Databases2003-08-18T22:11:43-07:000 In his comparison of Java and Python productivity, Steve Ferg notes that: A programmer can be significantly more productive in... Via: Nu Cardboard

This is kind of funny, but I do agree with the assessment that Python is vastly more productive then Java. I spent several years working with Java and in retrospect it was astonishingly unproductive. This included a project where I converted a team development effort from Perl to Java for all the reasons that are commonly stated for making such moves. In retrospect that was a horrible decision. I did it under the belief that we would have a more scalable process, more stable code and ultimately faster development of new applications because of this. It never really worked out that way and this was with Perl. I really regret making that move even though Perl is a far worse language for team projects then Python is. Over the last few months I've been working more and more with Python and I'm pretty convinced that the difference in raw productivity makes up for any loss from static typing and a compilation phase. I believe the same applies to Perl, although not quite as much.

What's important to understand about running scripting languages in large projects is you have to have good tests. When we were using Perl we had lots of unit tests, however when we switched to Java the code itself took too much precedence and the creation of unit tests suffered. Sure it's nice to have a testing policy, but schedule pressure can have nasty ways of interfering, especially when the overhead of the language is slowing things down. In the end our system became even harder to change then when we started out. The problem of course is that just because a language is statically typed and compiled it does not remove the need to write tests. So you're now taking on the burden of a less productive language while not shedding the burden of writing comprehensive tests. In addition, because of the static typing, compilation phase and OO access protection features of languages like Java your tests have also become much harder to write.

Because of this, I'm really starting to believe that efforts to improve software quality by tightening up the language features through stronger typing and rigid language features are really the wrong approach. I tend to believe a more profitable future will be attained by making languages easier to use and building in mechanisms that improve the testability of code. My feeling is that it isn't really important that you pass the right type to the right method, what's important is that the code does what it is supposed to do. Passing the right type to the right method may be a precondition for this, however it is not sufficient to guarantee it, and that to me just makes it overhead. Tests are still required. If dealing with all the static typing and compilation phases makes test development suffer, then I tend to believe those features are counterproductive and just get in the way. ProgrammingPython2003-08-19T10:46:59-07:0000 Daniel Veillard has released new versions of libxml2 and libxslt, classifying the libxml2 release as a "major bugfix release", and writing that libxml2 2.5.9 and 2.5.10 "include a lot of bugfixes spanning the whole library; upgrading is strongly recommended." The libxslt 1.0.32 release is also significant in that it is the first to include Python bindings for extension elements. [xmlhack]

I've been using libxml2 in Python a lot lately. It's the first XML parser I actually like, mainly because of the very convenient XPath API. PythonXML2003-08-21T11:53:40-07:000 This is just a note about writing extension functions for libxslt. The documentation on this is slim at best and the only example that comes with libxslt is minimal.

I'm working on a project where I wanted to run a piece of text through a textile processor and then insert the result into the result tree of the XSL document. I thought this would be pretty simple, but it turned out to be a bit more work then I expected. Here's the code that finally works.

Update: the original version of this code inserted the output directly into the tree. That's probably not what you want to really do, so I updated it to return the content.

def textile_process(ctx, content):
    """
    An XSL-T extension function to process textile formatting included in a post.
    """
    try:
        node = libxml2.xmlNode(_obj=content[0])
        parserContext = libxslt.xpathParserContext(_obj=ctx)
        xpathContext = parserContext.context()
        
        resultContext = xpathContext.transformContext()
        source = "<div>" + textile.textile(node.content) + "</div>"
        
        doc = libxml2.parseDoc(source)
        
        root = doc.getRootElement()
        root.unlinkNode()
        
 	# If you do this you insert the result directly into the output tree
        # resultContext.insertNode().addChild(root)	return [root]
    except Exception, err:
        sys.stderr.write("Context error " + str(err))
        
    return ""

The tricky parts about this code are converting all parameters into the right types and getting to the pieces of the documents that you need. The problem is that the parameters come in as raw PyCObject instances so you have to convert to the more specific object types manually.

For the way I was using this function the content parameter contains a list with one item and that item is an xmlNode. So the code.

node = libxml2.xmlNode(_obj=content[0])

converts the PyCObject into an xmlNode instance that you can then use as you usually would. You have to do similar things with the xpathParserContext that comes in as the first parameter. From the xpathParserContext you then have to get an xpathContext and from that you get the context for the result document. The xpathContext variable provides a reference into the source document and the resultContext variable provides access to the document being created.

Update: this paragraph applies to the commented out version of the original code. One confusing thing about the resultContext is the use of the insertNode() method. At first I thought that was inserting a new node into the result document, what it really is doing is requesting the node under which the result of the function will be inserted. It would probably have been clearer if the method was named getInsertionNode() or something like that.

Now that this is working it's pretty slick, but it sure took a while to figure out exactly what needed to happen. PythonXML2003-08-24T04:23:50-07:00 I mentioned in my rant about window proliferation on Mac OS X that I was looking at using JEdit as a solution. It turns out JEdit is almost a perfectly workable solution, through various plugins I was able to get it to do everything that I wanted and it felt close enough to a native app to not drive me crazy. Unfortunately, all is not well in this world.

The JDK on Mac OS X has some serious problems when running JEdit. With the 1.4 JDK the VM crashes far too often. This shouldn't be JEdit's fault as it's the VM that's crashing. Unless jEdit is running some native code a Java app should never be able to crash the VM. This wouldn't have been a huge deal as JEdit is really good about saving its current state, just restarting the app is a minor annoyance. However, the real problem is that sometimes when the VM crashes it takes the whole WindowServer process down with it. This results in instant termination of all your running apps and getting logged out of the system. To say that's a bit of a problem is an understatement.

When you install jEdit it recommends that you use the 1.3 VM. Unfortunately doing this also results in the loss of a number of functions (in particular mouse scroll wheel support) and substantially changes the display of the app (i.e. tab layouts). Also if you turn on the hardware acceleration it has problems with garbled text and incorrect positioning in text areas. Running without mouse scroll wheel support is extremely annoying.

I'm quite disappointed with the current situation of Java on Mac OS X for running GUI apps. I've had no problems with it running server apps, it seems most of the current problems can be traced to the things they're doing with Swing, i.e hardware acceleration and the switch to the Cocoa toolkit for GUIs in 1.4. This stuff will be great once it's fully stable, but I've heard nothing about progress in this area for some time.

I'm still trying to make do with jEdit running on the 1.3 VM with no hardware acceleration. However, I would definitely prefer to be able to take advantage of the 1.4 features, that scroll wheel thing can really spoil you. Mac OS XProgrammingPython2003-08-27T23:08:09-07:0020

Today, Just van Rossum released Drawbot. Drawbot is significant in a couple ways. Firstly because it's the first real application I've seen that was written in Python using PyObjC. This means it's a Cocoa application, but wasn't written in one of the usual Cocoa languages (i.e. Objective C, Java, Applescript). Secondly because it's an interesting tool that allows you to play with the Cocoa 2D drawing API from within a scripting environment. It provides its own simple graphics API, but it's also possible to use the Cocoa APIs directly. This makes it a good way to explore and play with the Cocoa drawing APIs from within Python. Pretty cool.

I was recently asked in an email my thoughts about Webware's performance. Figured I'd share them here as well.

So far I've been entirely satisfied with the performance and stability of Webware. Running on my dual 1.25Ghz G4 desktop it serves data out of Syncato's cache at roughly 45% of static HTML speed. That's really not too bad given it's a dynamic request and is traversing a socket from Apache. It serves roughly 110 requests per second which is a 9.5 million requests per day pace. My production server is considerably slower since it's about 6 years old, but Webware is anything but the bottleneck in the system.

The one potential problem with Webware is really a problem with Python in general and that's the global interpreter lock in the Python interpreter. That will serialize certain operations that could be more efficient if run in parallel, especially when you have multiple CPUs. This would maybe be a problem if you were to actually try to push 9.5 million requests on a single machine. Fortunately 99.9% of sites don't get anywhere near that and if you do you should definitely have more then one machine anyway. My modest little site doesn't even begin to tap into that.

I started working with libxml2 a few months ago and I've really grown to love it. After spending several years working with Java APIs like SAX and DOM, I'm absolutely thrilled with the simplicity of the libxml Python API. Yeah it's not standard, but who cares. I could never remember how to write SAX or DOM code anyway. No matter how much I used it I always had to open a book, with libxml it's just so much more natural. The libxml API still has all kinds of problems, but it's by far the most productive XML API I've ever used.

My favorite feature by far, absolutely no question, the brain dead simple XPath API.

A few lines of code, whole worlds of power. To me XPath is the swiss army knife of XML tools. Of course you'd never guess that if you were to look at the source code for Syncato.

PyObjC 1.0 released
PyObjC allows developers to write Cocoa apps in Python. [ranchero.com]

I keep hoping to have more time to play with this. Objective C is already pretty easy, Python is even easier.

If nothing else comes out of my work on Syncato it does seem to be raising the visibility of XPath as a general XML processing tool. First it was Sam Ruby attaining enlightenment, now Simon Willison has had his own "enlightening experience" after reading my post about libxml love.

Syncato takes XPath to the extreme and uses it for all XML manipulation, not just the queries through the URL. The next version of Syncato will also add a Python XPath document creation and modification API called PathUpdate. This allows you to create and modify documents by just providing paths to the nodes you want to create or modify. You can specify a deep path and it will create the whole hierarchy. Here's a sample from my unit tests that gives a flavor of how it works, it's pretty cool and very powerful.

\ntext\n')

# add another child and then update both to the same value
doc.addNode("/root/child")
doc.updateNode("/root/child", "newText")

self.assertEqual(doc.serialize(), '\nnewTextnewText\n')

# Now just update the second child.
doc.updateNode("/root/child[2]", "text")

self.assertEqual(doc.serialize(), '\nnewTexttext\n')

# Now update an attribute
doc.addNode("/root/child[1]/@attr", "text")
doc.updateNode("/root/child[1]/@attr", "newText")
self.assertEqual(doc.serialize(), '\nnewTexttext\n')
 
]]>

I've been meaning to point to this for a while, but kept forgetting. Don Park has been talking with Sleepycat about commercial use licenses for Berkeley DB XML. The way they define distribution is a little weird so it's worth reading what Don has to say about it. They're also being somewhat generous with licensing right now while the product is new.

Does anyone else ever get this or is it just me? What I'm talking about is a deep mental block that destroys productivity. Your mind is full of ideas, but when you sit down to try to realize them you get nowhere and your mind just starts to wander.

I have this problem in a big way, I sometimes go for days or even weeks at a time where I can do absolutely nothing productive with the computer. Over the last few months I've been trying to battle this and to figure out how to control it so that I can be happier with myself. So far I've had very limited success and it's a very frustrating experience.

First you sit down and try to do something but for some reason it just doesn't feel right, so you go and do something else and that doesn't feel right, and on and on. What's the worst is when you start to get mad at yourself for not being productive. At that point you're just lost, I often tend to completely crash then and end up sleeping 18 hours straight or something. That's a huge waste of time and you wake up feeling like utter crap which makes it even harder to get your mind back in gear. It's like a deep downward spiral that is seemingly impossible to pull yourself out of.

I call it programmers block because I recently read a couple books on writers block. I figured it's a similar activity and the symptoms may also be similar. It turns out they are exactly the same and reading those books has been the one thing that has helped with the situation.

I think my biggest problem is that I have big ideas. I constantly get these grand visions for some great new technology or piece of software or service or what ever and my adrenaline really gets pumping. I sit down, sketch it out and then start thinking about how great it's going to be when it's done. Boom, crash, there's the mistake. It's that vision of greatness that's a killer. Why? Because your mind gets about 100 steps ahead of reality and then when you sit down to try to turn it into a working system the mountain you've built seems too large to climb. Basically it's day-dreaming and dreams are often far removed from reality. This is something the books on writers block made very clear, the number one enemy of writers around the world is self doubt. That mountain of an idea that you have, even if you can envision how great it will be when it works, looks 100 times larger when you actually have to put all the pieces together to make it work. You look at that and reality sets in, and at least for me, I have a bad habit of crashing into a complete state of no productivity. I get frustrated with my inability to make progress then start to doubt my ability to do anything at all, and the situation just becomes ugly.

So how do you combat this? I really don't know for sure, but I do know it helps to think small. It especially helps to avoid the dangerous day-dreaming of greatness. I'm trying to consciously watch for this and kick myself every time I find myself thinking about how great something will be without actually working on it. My success has still been limited however. For instance I just came off a four day crash over the weekend. I did manage to dig myself out of it by forcing myself to just tackle a tiny little piece of a project, then the next little piece and so on. With this I was finally able to put together some good productive hours. Unfortunately it's hard, I tried and failed to do the same thing several times during the weekend. It doesn't help the situation any that I"m still battling pain in my wrists which tends to make me want to stay away from the computer too.

I know that other people have this problem. How do you beat it?

As I suspected, programmers block is a common problem and many other people have written about it. Buzz Andersen posted a link to his very similar experiences. From there I found some thoughts by Russell Beattie and Joel Spolsky. Russell looks at it as procrastination and points to another post that talks a little about that.

I think calling it procrastination is a common thing, but that's definitely not what it is. It feels that way, and I've certainly spent a lot of time kicking myself for procrastinating. The problem is deeper though and writing it off as procrastination makes the situation worse. Procrastination is something that you feel you should be able to control, "if you'd just stop being lazy". However programmers block is not about laziness, for me it's a fear. A fear of failing to live up to the expectations that you impose on your self.

Andrzej Telezynski said it pretty well in the comments to my previous post, " "Perfectionism" is the evil. It is one of the dreams that never can be true, just a poison for the mind". I think perfectionism is what drives the creative process, but it's definitely both friend and foe.

This is a wish list item, call it a dream maybe, but I really want to see XML support baked directly into a programming language. Python in particular would be great. Here's what I want.

• Seamless XML support. Never having to explicitly parse an XML document.
• XPath as a native language construct.
• Dynamic conversion between text and parsed representations of the XML.
• XPath manipulation for XML modifications.

Here's an example of the flavor that I'm looking for. This is pseudo Python though what I'm thinking may be considered un-pythonic. That's not my concern though, it's just a concept.

just an example"
doc2 = "just another example"

# get the title as a string
title = doc/item/title/text()
print title

# get the title as a node
title = doc/item/title
print title

# a collection of documents
docs = [doc, doc2]

# find all the titles in the collection as strings
titles = docs/item/title/text()
print titles

# find the node for the title that equals 'example'
titles = docs/item[title = 'example']/title
print titles

# find all items that contain the word another in their description
item = docs/item[contains(description, 'another')]
print item

# Use dynamic conversion to treat the XML as a string and 
# rename the description element to body
doc = re.sub("^description", "body", doc)
print doc

# append some new content to the body
doc/item/title/body += " with new content"
print doc

# Delete the title element
doc/item -= title
print doc

# append the new title
doc/item += ""
print doc

# delete the title again and then insert a new one before body
doc/item -= title
doc/item/body <= ""
print doc

# delete the title again and then insert a new one after body
doc/item -= title
doc/item/body => ""
print doc

]]>

If you could run this program here's what you would get as output.

example
["example", "example2"]
[]
[just an example with new content]
just an example
just an example with new content
just an example
just an example
just an example
just an example

]]>

The XPath modification syntax is an experimental one I came up with sometime last year but never implemented. Here's how it translates.

• += is append
• -= is delete
• is insert before
• >= is insert after
• = is replace
• ~= is rename node

BTW, I am aware of one language implementation that does some of what I want. The XQuery implementation that's part of Sherlock on Mac OS X has the dynamic conversion between text and an in memory representation of the XML. It doesn't have a modification syntax though.

Syncato 0.6 is now available for download. This release is a substantial improvement over the 0.5 release, but is still mainly aimed at people who are pretty technical.

Syncato is the weblog system that runs Inspirational Technology. Its major distinguishing characteristic is that it is based heavily around the concept of manipulating XML fragments via XPath and XSL-T. This principal is used to implement the system as well as being exposed to the world via XPath in the URL. Using XML in this way results in a very powerful system for the storage of all kinds of data.

The data storage architecture is an open container for any type of XML fragment that you want to throw into it. The system then provides the ability to pick from that container whatever you want to include within your posts. This mechanism also encourages you to include richer markup within you posts. Syncato basically encourages you to either make up tags that mean something specific to you or to use standard markup in a way that provides some additional meaning.

For example, in my posts I sometimes recommend products and then include a link to Amazon for the product. To make this simpler I created a new "asin" tag that includes the Amazon product identifier. When the site is rendered the tag gets converted into the appropriate markup for Amazon product links. However, beyond that it also enables you to do interesting things like get a page containing all posts that have recommended products on Amazon. This kind of thing is all accomplished via XPath in the URL.

This is just a simple example, but the possibilities are completely open depending on how you markup your posts.

The single biggest change in the 0.6 release is the inclusion of a file based database in addition to the Berkeley DB XML database. This provides a much simpler way to get an initial system up and running. The dependencies are reduced to just Python, libxml, libxslt, Webware and a webserver. It's still harder to install then I'd like, but it's getting better. The file database works well for small data sets up to a few hundred entries. From there you can then update to Berkeley DB XML to take advantage of its indexing capabilities. Since everything is XML it's pretty easy to convert from one database type to another.

The other notable change in this release is the inclusion of the Blogger and Metaweblog APIs. Darryl VanDorp stepped up to the plate and got them working just in time to be included.

Here's a list of what else changed this time around.

• Added comment support.
• Moved weblog pings into a separate thread.
• Made pings generic so that any site that supports the weblogs.com interface can pe pinged.
• Added technorati pings to the default configuration.
• Added a very rudimentary HTML administration interface. (Just add and edit entries)
• Added the PathUpdate API to enable document contruction and modification using XPath to select nodes.
• Added category RSS feeds
• Added a file based database to make the software easier to get up and running.
• The file based database is the default out of the box.
• Began rearranging the XSL files to start separating user styles from system styles.
• Initial Blogger and Metaweblog API implementations. (thanks Darryl VanDorp)
• Changed the command line scripts to use the web interface instead of accessing the database directly.
• Switched categories to key off of title instead of id.
• Substantial improvements in the stability of the Berkeley DB XML backend.
• Changed the directory structure for servlets. There are now three types based on access restrictions.

I've begun updating the list of features that will be coming in the next release of Syncato. I'll be adding to the list as I get things working. I'm not listing anything until I actually have it working. This post will update automatically as I add features on the main page.

I think the next release will pretty much complete the list of basic weblog features that Syncato should have, i.e. the stuff that most systems already do. Syncato already does a whole bunch of stuff that no other system does. Going forward I'll be focusing on usability, in particular making the system easier to setup.

In Syncato 0.6 I added a Python API for constructing and modifying XML fragments using XPath specifiers. So far this has proven to be a really powerful way of working with XML fragments the way Syncato does.

The API is pretty simple.

You can use it to create a new document. Here's an example to create a simple weblog post.

embedded", 1)

]]>
    

Printing this out then gives you a document that looks like this.

Some body content with embedded


]]>
    

Modifying is also easy

") 

doc.updateNode("/item/title", "Another example")

]]>
    

Of course you could also use a more complex query to select the node. If the query selects multiple nodes then all will be updated. Pretend this example has lots of title elements, but we only want to change those containing the word "example".

") 

doc.updateNode("/item/title[contains(., 'example')]", "Another example")

]]>
    

Right now the code is available as part of Syncato, but isn't tied to Syncato at all. It just depends on libxml2. Once it matures a little more I'll probably make the API available as a standalone download since it's quite powerful. It basically gives you an XML update language that uses Python syntax.

Syncato 0.7 is now available for download. Syncato is the Python and XML based weblog software that runs this site. In addition to new features, performance increases and bug fixes, this release also brings a change in the license of the software. Previous releases were released under an Apache style license. Syncato 0.7 and all future releases are being made under a license derived from the Sleepycat license. This is still an Open Source license and the terms are very simple. The main difference is that it does restrict commercial redistribution of the code unless full source of any derivative work is also made available.

Here's the list of changes in this release.

• Added trackback support
• Added trackback autodiscovery on post pages
• Added remote-xpath-query and remote-content include tags that allow you to include remote XML content into your posts.
• Greatly improved the HTML admin interface.
• Added comment and trackback spam URL blacklist support
• Refactored the stylesheet locations to separate system and user stylesheets. All stylesheets in dist/stylesheets are intended to be updated any time a new version is released. User customizations can be made by overriding any of the templates from the stylesheets in that directory. User styles are stored in the top level syncato stylesheet directory.
• Reworked the URL scheme to add the ability to use post titles to provide more descriptive URLs.
• Changed output from XSL process to generate XML instead of outputing HTML.
• Added a new URL parameter c to enable content queries. These are XPath queries run on rendered content. This allows you to extract fragments from the HTML result.
• Added configuration options to turn url xpath queries and content queries off.
• Changed summarize function of WeblogXslExtension to no longer require sgmllib.
• Fixed summarize so that it expands embedded tags prior to doing the summary.
• Lots of cleanup to simplify the XSL files.
• Added additional indexes to the DB XML backend. This tuning provides a substantial improvement in performance when running the latest version of DB XML.
• Renamed the root element on category definitions from category to category-def. This was done to take better advantage of the indexing facilities of DB XML. There's a script update.py that will update an existing site

Ryan Wilcox has wrtten an interesting piece on building Mac OS X Cocoa applications in Python with PyObjC. Digging more into PyObjC is still very much on my todo list.

A couple days ago I came across a new scripting language for the Java platform called Groovy. This language is pretty slick and is actually approximating what i was talking about in my post about baking XML into the language. It doesn't use XPath, but does have a language GPath that is based on the base syntax of the language and can be used to query any in memory data structure. The language does have some built in XML support and as part of the syntax enables a Groovy program to be treated as an alternative markup language. Very lisp like, but also very cool to see in a Java like language.

Since Groovy targets the Java platform it can use any java classes and provides nice builtins for lists, tuples and closures. Overall a very promising looking language. In a lot of ways it does the same thing as Jython, but the features like GPath and GroovyMarkup are pretty compelling.

While playing around with groovy it becomes painfully obvious how Java classes are often way too hard to use in a scripting context. Here's an example that uses Groovy to parse an XML file using Saxon and then run an XPath query against it.

Not too bad I guess and a hell of a lot better then straight Java, however, here's how you do the same thing in Python with libxml.

The Python is vastly more direct and to the point, even in this simple example the Java API is saddled with way too much abstraction. This drives me insane with Java, you can of course wrap whatever you want to gloss over the abstractions, but I really don't get why API designers for Java so adamantly refuse to make the common cases easy. I was there once myself, I had a wee bit to do with creating the overly abstracted mess that is the XML:DB API. I regret that design every time I look at it. it could have been soooo much easier to use. Why isn't it? Ugg, who knows, maybe that wouldn't have been enough Java like.

Anyway, to reinforce the point that the problem is with the API designs, here' s the Groovy example re-implemented in Jython.

You might complain that Saxon doesn't have the best API, but all the Java XPath tools are bad, most of them much worse. DOM4j is probably a little better then Saxon, but not substantially.

Looking at these APIs it's also obvious why so many programmers like to moan about how hard XML is. Who can blame them when it takes four classes from four different packages just to parse an XML file. We won't even talk about the W3C DOM. We also won't talk about the fact that the Groovy example hides the fact that those APIs throw checked exceptions.

Oh yeah, how about serializing a document back to text? With libxml it's just doc.serialize(). How do you do it with your Java API? Honestly, I really don't even know with Saxon, but I know it takes a hell of a lot more then one method call. It sure did when working with Xerces and I could never remember how to do it without looking it up.

I used to love Java, I also used to think Java was THE language for working with XML. I don't believe that anymore. Sure it's great to have an API that can have any individual component replaced, but that makes the API also have a massive barrier to actually writing code with it. Most Java APIs make the hard stuff possible, but they've really lost sight of the idea that the easy stuff should be easy.

Since Syncato is an XML microcontent management system the playlists I've been posting are of course XML data like everything else in Syncato. To create the lists I use iTunes to create a playlist, fill it with the songs and then extract the playlist using a combination of Python and Applescript. This could of course be accomplished with just Applescript, but I'm much more comfortable with Python then I am with Applescript. This script is actually mostly Applescript since it was originally part of a larger Python program.

"
	repeat with i from 1 to (count of items in allsongs)
		set songs to songs & ""
		set songs to songs & "" & (item i in artists) & ""
		set songs to songs & "" & (item i in albums) & ""	
        set songs to songs & "" & (item i in trackTime) & ""
		set songs to songs & ""
	end repeat
	set songs to songs & ""
end tell
"""
# Read the contents of the playlist from iTunes
content = os.popen("osascript -e '" + getPlaylist + "'").read()

print content
]]>
  

I haven't really had much time to experiment and develop the microcontent aspect of Syncato, I've been rather caught up with other things. There are a lot of interesting things that can be done with it, but it will take sometime to build the necessary concepts and tools. Posting playlists is fun, but isn't the most compelling thing that can be done. We'll see what happens over the coming months as I get more of the pieces in place.

From Jarno VirtanenI found a link to an article titled Understanding Object Oriented Programming. Wow, all I can say is go read it and then you'll know exactly how not to build a maintainable system. I'll refer you back to my post on the pain of over abstraction. In that post I was criticizing the design of many Java APIs, especially XML APIs, however this is really a general problem with OO development. Fancy object oriented architectures have become the goal and this article maybe makes that point more clearly then anything I could ever say. It's representative of the thinking from a few years ago (written in 2000), and shows us just how much damage we now have to undo. It basically says that the simple solution that just works is wrong and will be unmaintainable. Maybe that's true, maybe it's not, nowhere does the article consider the question of whether or not that code actually needs to be that generic. It simply says that the simple solution is bad and that the seven class monster they came up with is the right solution. Talk about doing a disservice to students trying to learn how to build solid computer systems.

OO techniques are a tool, they are not an end goal. Starting to design a system to achieve OO purity is the fastest way to build a system that will be ten times more complex then it likely needs to be. This article was actually written in 2000 and thankfully Agile development methods that understand this distinction have been gaining substantial mind-share since then. Hopefully Pace University and King's College London are no longer teaching their students that this is the "right" way.

BTW, Simon Willison has posted an alternative solution and says "smart data-driven programming beats over-engineered class-based programming any day of the week". All I can say is, I agree completely. Programmers like to write code, and personally I think write too much of it, way too much of it. Data-driven programming is a method that's been lost in all the OO hype, but it works, and I believe it's going to play an increasingly important role going forward. This will especially be driven by XML ubiquity. Syncato is my experiment in exploring XML based data driven programming.

In that spirit, here's my solution to the problem using XML, Python and Simon's mythical System.getProperty. Using XML here is of course overkill too, but it solves their original problem in a much more maintainable manner, requiring zero code changes to add a new system.

An XML description.

    
        Windows NT
        Windows 95
        Windows CE
        This is a Windows box and therefore bad.
    
    
        SunOS
        Linux
        This is a UNIX box and therefore good.
    

]]>
  

And a simple program to glue the pieces together.

 0):
    print message[0].content
else:
    print 'This is not a box.'
]]>
  

This particular example was contrived in the original paper and it's contrived here too, however when you teach OO developers to think code first, you end up with lots and lots of code. Want to know the secret to building maintainable systems? Quit writing so much source code. Ask your self if that fancy OO technique you're about to apply is really necessary or if a simpler data-driven solution can solve the same thing. I'll argue every time that if a simple XML file can eliminate a dozen classes, you need a damn good reason not to use it. That's not saying that there aren't times when it's inappropriate. Like everything else, this is just a tool too, but I believe it's an under-used and extremely powerful tool.

Once upon a time we understood this, we used to drive systems off the data model, now it seems everything has to be objects. This presents a problem since relational data models have presented a persistent mapping problem into object oriented domains. And many of the techniques developed to solve the problem are some of the worst offenders of the too much code phenomenon. EJB entity beans are a pretty good example of this. Forcing an OO layer between the program and its persistence introduces so much overhead that it's often just easier to write a dozen classes to solve the problem. XML now provides you the ability to start small and scale up as needed. The model is always the same no matter where the data comes from.

XML is about options. For instance in this silly example I could have certainly used a simple tab delimited file or some type of properties file. However, there are benefits to this being XML. Obviously, XPath is a big one, but so is the ability to change the underlying data storage technique. To your application it's just XML, but underneath it could be coming from the file system, over the network via HTTP, from a native XML database or it could be an assembled document from an RDBMS, an LDAP server or just about anywhere. To your application, outside of changing the location of where to read the data, it makes no difference. This provides the ability to evolve the system over time well beyond what something like a properties file would enable. To boot, using XPath, it probably even takes less code to process. Could you do all this with a properties file? Sure, but you'd have to build it all your self because everybody else is using XML.

The key here? Don't insist on putting a fancy OO layer on top of the XML. In some cases that will help, but in most cases it's just overhead. XML has powerful tools, learn how to leverage them. XPath in particular provides a tremendous amount of power for very little effort. It's not suitable in every situation, but is a very good way to bootstrap a simple solution.

Yesterday I finally sat down and got PyObjC working on Panther. I had been having some problems with it earlier until I realized I was installing the Jaguar version on Panther. Doh! If you want to install it, don't download the installer, it doesn't work on Panther. That isn't real clear on the download page. Anyway, I put together my first program with it. So here's my PyObjC version of a first program.

Yep, that's a very, very rudimentary web browser. That's how simple Cocoa makes this stuff. Doing that was just as easy as throwing up a simple text area to say "hello world". Of course it helps that I already know Cocoa, but ... anyway, here's the Python code for creating a very, very simple web browser using WebKit and PyObjC.

Following up on my previous post about writing Cocoa applications to handle custom URL schemes, I wanted to add on how to do it with a PyObjC Python application. I'm definitely still getting up to speed on PyObjC so there may be an easier way to do this, but I was able to get it to work. All the scripting dictionary and Info.plist settings are the same, you just need to add a Python implementation of URLCommandHandler. The tricky part is getting the Python URLCommandHandler object to be made available to the Objective C runtime so that it can be linked up with the incoming apple event.

The way I did that was to create a Nib file that contained a subclass of NSScriptCommand named URLCommandHandler. NSScriptCommand is not part of the normal list of classes you can subclass in Interface Builder so you need to tell IB to read the header file for it, it's part of Foundation.framework. Once you have that defined you can use the Nib in the same way as usual for PyObjC and the Objective C runtime will be able to see your Python class. You just need to create the subclass, there aren't any other connections involved. Because of this, I'm guessing there's probably a way to do this without using IB to make the connection. I just haven't figured out how yet.

The Python class is really simple.

It's actually very easy once you know how to do it. Doing this should also enable you to add general AppleScript support to Cocoa applications written in Python.

Over on his O'Reilly Network blog Uche Ogbuji took issue with my use of XML in my response to the extreme use of OO. The problem that I face is that I'm not a Python programmer, even though I program in Python, just as I was never a Java programmer even though I programmed in Java, nor was I a Perl programmer when I programmed in Perl. Languages are just tools. Data, however, is different. It has a permanence that exceeds the bounds of a particular programming language and to me placing data within the syntax of a particular programming language is not necessarily a good thing. Like Uche, I believe that dynamic languages are the way forward, however I also believe that we need to weaken the coupling between a particular language and the actual solution to a problem to whatever degree we can.

As for the specifics of using XML in that case it was just an attempt to show it as an option. Simon Willison had already demonstrated a solution exactly like Uche is advocating, so I simply wanted to add on another perspective that pulled the data out of the language. XML was the easiest way for me to do that. When I mentioned XML was overkill what I was thinking was that it was overkill compared to using something like a Java style property list file, not compared to expressing the data in a programming language. Whether the use of XML in this case was or was not a good idea is an unanswerable question without considering much more context (that doesn't exist) so we won't worry about it anymore.

Oh yeah, I've also seen a few people say that I'm an OO hater or some such nonsense. Like everything, OO is a tool, sometimes it's good, sometimes it's not. The same applies to data oriented programming and XML. Unfortunately since I'm experimenting heavily with pushing the use of XML and data oriented programming to extremes, my posts often come off pushing the techniques more then I would when building a real world production application. That's the bias of my writing, simply because that's what I'm interested in and that's what I want people to be more aware of. I do believe we've gone too far with OO and that it is getting over used and is resulting in systems that are far too complex, but that's a long way from hating OO.

A while back I wrote up a quick post on writinglibxslt extension functions in Python. Writing extension functions allows you to easily call python code from within an XSL-T stylesheet. I wanted to add a little more detail to that post and show how you can build extension modules instead of just single functions. As far as I know there isn't any documentation as part of the libxslt distribution.

Here's an example program.

This contains 27 characters
"""
        
styledoc = libxml2.parseFile("style.xsl")
style = libxslt.parseStylesheetDoc(styledoc)
doc = libxml2.parseDoc(data)
result = style.applyStylesheet(doc, None)
            
print style.saveResultToString(result)
                           
doc.freeDoc()
style.freeStylesheet()

]]>
  

This example registers two functions under the namespace "http://www.xmldatabases.org/extension". To make them part of a module you have to call the libxslt.registerExtModuleFunction to register them with the libxslt environment. You can register as many functions as you want this way. The first parameter is the name you use to call the function from within libxslt, the second is the XML namespace URI that identifies the module and the third is a Python callable that implements the function. All functions with the same namespace URI are considered part of the same module.

The functions always take at least one argument, the xpath parser context. However, you can add additional arguments to pass other data from the style sheet to your function. These arguments will come in as various different types based on how they're passed from XSL-T. For instance they could be a libxml2 node, a list, or a string. In this example I convert the arguments to a string within the XSL-T before passing the data to the function. If I didn't do that I'd have to convert the argument into a node and then get the content of the node as a string.

The same consideration must be given to what you return from the function. In this case, both functions convert their results to strings, but they could also return libxml2 nodes or lists of nodes that will be inserted into the result document.

Here's the associated stylesheet that uses the functions.

        
    
        
        
    
    


]]>
  

There are two things you have to do before you can use the functions.

1. Declare the namespace URI that you used to define the functions within the libxslt environment. You can bind this namespace URI to any prefix you want.
2. Declare the prefix that you chose as an extension element prefix by adding it to the extension-element-prefixes attribute.

Once you do both of those things you can call the functions like any other XSL-T functions. Just make sure you pass types that your functions know how to handle. It's obviously a good idea for your functions to explicitly convert the different types if they can.

There's a new release of pybsddb available. 4.2.4 is a necessary release if you want to use Berkeley DB 4.2 with Python. Berkeley DB 4.2 is recommended for use with Berkeley DB XML 1.2 and without this pybsddb release Python crashes with DBXML 1.2 and DB 4.2.

Because I haven't had Python working with my Berkeley DB XML 1.2 install until now, I haven't yet verified that Syncato will work with it.

I can now confirm that Syncato does NOT work with DB XML 1.2.0. Actually, it looks like the Python bindings for deleteDocument and queryWithXPath in DB XML 1.2.0 have problems with transactions. So if you want to use transactions wait for DB XML 1.2.1 which will have a fix. The Python bindings seem to work fine otherwise. Waiting for 1.2.1 will also be necessary for Syncato to run.

Update: I've confirmed that Syncato will work without changes when DB XML 1.2.1 is released.

The last couple days I've been playing around with Mono Beta 2 on Mac OS X. It's kind of interesting to see things like ASP.net running directly on a Mac, pretty cool too. While poking around I also came across some promising looking work for implementing Python on top of the CLR called IronPython. It's currently an "unreleased research prototype", but sounds like something that would be interesting to see in action. Performance on some basic benchmarks actually looks pretty surprising. Hopefully it moves beyond research prototype some day.

Lot's of interesting programming languages have been popping up lately. Groovy has been one that initially caught my interest and now there's a new language called Boo that looks really interesting. Boo is a language that targets the CLR, but has syntax based on Python. It's not out to make a Python clone, just to have a similar syntax that can leverage the class library and other facilities available through the CLR. It's a really interesting language and runs just fine on top of Mono on Mac OS X.

Another language that looks interesting is from Microsoft Research and is now called Comega. This is an evolution of the language that used to be referred to as X#. This language also targets the CLR and has integrated direct support for XML within the language syntax. I'm dying to have a usable language with tight XML integration so it's good to see progress in this area. Unfortunately, Comega is based around the concept of strongly typed XML which is something that I think defeats the whole point of using XML. Also in typical Microsoft fashion the compiler is available in a format that can only be installed on Windows. So far I haven't been able to try it out to see if it works on Mono. Well, actually I did try to install it on a Windows machine but the installer just silently fails so even that was unsuccessful. I suspect it's looking for the .Net framework or something else that isn't installed on that machine, but there's no information at all on what it requires. Anyway, the compiler download isn't actually linked from the Comega site as far as I can tell so you have to go to the download section on the Microsoft Research site if you want to get it. If it turns out it will run on Mono I'd love to hear about it.

In my "secret" efforts to finally make some real progress on Syncato I just added a pretty cool feature to the Syncato XMLFragment API that allows you to treat an XML document sort of like a Python dictionary. Basically it allows you to use an XPath in place of the usual string key. This includes both reading and modifying the document. These definitely won't be interchangeable with Python dictionaries so call it an experiment if you must. My interest here isn't in being pythonic, it's in having maximum power and simplicity for using XML as a core application data structure. Given how the Syncato internals are written this provides a very valuable tool.

It looks something like this.

    
    Just a simple post to get started.
    XML
    Java
"""

        fragment = XMLFragment(doc)
        
        print "title " + fragment["/item/title"]
        
        fragment["/item/title"] = "New title"
        print "title " + fragment["/item/title"]
               
        del fragment["/item/title"]
        print "title " + fragment["/item/title"]

        print "Category " + fragment["/item/category[1]"]
        print "Category " + fragment["/item/category[2]"]

        print "title " + fragment["/item/title[contains(., 'title')]"] 
]]>

Running this you get output that looks something like this.

The API also supports setting a base for a query so that if you always have the same beginning path you can make it even more convenient to work with the XML using the dictionary mechanisms.

    
    Just a simple post to get started.
    XML
    Java
"""

        fragment = XMLFragment(doc)
        fragment.setBase("/item/")
                
        print "title " + fragment["title"]
        
        fragment["title"] = "New title"
        print "title " + fragment["title"]
               
        del fragment["title"]
        print "title " + fragment["title"]

        print "Category " + fragment["category[1]"]
        print "Category " + fragment["category[2]"]

        print "title " + fragment["title[contains(., 'title')]"] 
]]>

Ian Bicking has announced a new project called WSGIKit that implements WSGI and attempts to maintain Webware compatibility. This looks really interesting. Just today I was thinking about moving Syncato to a WSGI implementation so this may be the way to get started.

It's been kind of quiet on the Syncato front for ... well for quite a while. I've recently seen a few rumblings around the net about the project being abandoned and I'm happy to say that's not the case. I ran into two problems with the development of Syncato that stalled progress. First I got the system to the point where it worked well enough for what I needed. It's been running this site for about a year now and I've been happy with it. This kind of took some of my drive away while I went and worked on other things. However, that wasn't the real problem. The real problem was too grand a vision for what I really wanted the system to do. I've actually been quietly chipping away at the system during this period, I was just making too many major changes to release anything. I'm still caught in that cycle as the next release is still in pieces on my hard drive. The good news is that I've made substantial progress over the past week and the basic system is once again running with a much more advanced set of features.

The single largest change with the next release is that the Syncato system and the weblog application will be completely separated. This has been my major stumbling block for progress as I wanted Syncato to be a generic platform for developing XML data driven applications. This required a number of changes including adding support for multiple databases, an internal roll based security system, a reworked XSL templating system, completely reworked configuration system and an entirely new concept for running multiple applications under the same Syncato instance. I'm also converting the existing REST API to be more compatible with the ATOM API. The good news is that most of this work is done and I was finally able to run my first rudimentary application on the system last night.

The bad news is that the weblog application still has to be converted, of course it's also exciting to finally be moving on to that stage. While converting the weblog application I'm also converting it to use the ATOM data model and API. This means that the system will not be backwards compatible with the old data. Of course, it's all XML so XSL-T will be able to make the conversions.

There's still a lot of work to do before I'm ready to make another release, but I think the new system is really promising and could be quite cool once the kinks are worked out. Just don't ask me when the release will come. :-)

As I continue to examine what it is that has put me into a year long blogging lull, I want to take a closer look at the things I used to write about. First up is XML.

I've been a big proponent for the use of XML and Syncato was an attempt to show the possibilities for what can be done when you build a system with XML data running from top to bottom. My experiments in that area have been quite successful, building and using Syncato for the last two years have definitely convinced me of the value in this line of exploration. However, that's really an exploration in microcontent systems, semi-structured data and building a unified data model more than it is at pushing XML technology. It just happens that XML technology provided the most convenient toolkit for accomplishing what I envisioned.

My XML technology toolkit primarily consists of three things, XML 1.0 (without DTDs), XPath 1.0 and XSL-T 1.0. Occasionally I'll use namespaces, but I really try to avoid it. Most of the other XML specs, especially XML Linking, XML Schema and XQuery are simply too complicated for their own good. And worse, with the introduction of XML Schema the W3C started a generation of new specs based around the concept of strongly typed XML. That's a concept that in my mind completely destroys the value of using XML. It would be OK if you could just take XML Schema or leave it, but it's now getting baked into the core of other technologies like XQuery, and all the WS-* crap which makes all that stuff an absolute pain in the ass to deal with. It's also causing developers to adopt the strongly typed XML concept and create system of despicable fragility, when they should be leveraging the flexibility of XML to build systems of surprising resilience.

So it seems that for me, the XML technology stack is done. Even though some of the new features added in XPath 2.0 and XSL-T 2.0 would be extremely useful to have, in fact REALLY useful. The complexity introduced by the specs relationship with W3C XML Schema is going to keep support out of just about every toolset I work with. Right now I'm working on projects in PHP, Perl, Python and Ruby, none of which has any support for XPath 2.0 or XSL-T 2.0 and it seems likely won't gain support for it anytime in the near future. So thanks to the W3C and the abomination that is XML Schema, XML is now in practical usage, done, and in many ways that's too bad.

Another thing I used to write about a fair bit was Python. Python is a great language and it's the language that I used to write Syncato. However, I never really cared all that much about the Python language itself. For example, I've never had any interest in embracing the Pythonic way of doing things and I certainly never considered myself a Python programmer. For Syncato my goal was to actually shove the programming language into the background as much as possible and focus on seeing how much could be accomplished via data transformations using XSL-T. I had to write a fair amount of Python code to bootstrap the system, but the actual applications, like this blog, are about 80% XSL-T. I guess this is a testament to Python, as it allowed me to do what I needed without getting in the way and it had the XML tools available to accomplish what I envisioned.

I still like Python, but since I can't get excited about the language itself, I guess I just lost the motivation to write about the areas of Python that I did actually use.

It seems every developer you talk to these days has a side project of one kind or another. You know those little ideas for web sites that might make a few thousand a month in revenue. Some are intended to become companies, but most just generate some passive income for the developer. The problem is that most people don't usually have the skills to pull off these side projects on their own. I know I don't, but I do have a large number of little revenue generating side projects that I want to see constructed. For the projects that are big enough you can go out and raise angel funding or maybe even venture capital, but for those that aren't big enough, getting the ideas off the ground is a little tougher.

So we're doing something about that and I can now announce the existence of my (and co-founder Sean Tierney's) new company Grid7, LLC. Grid7 is a consulting and product development company and we recently introduced a new concept, the side project incubator, Grid7 Labs.

Grid7 Labs provides a marketplace where developers, designers, administrators and anyone else with skills to lend, can work together to help each other build out their side projects. It's a developer cooperative where everyone who participates shares in the revenue from the creations that become part of the Grid.

Grid7 itself provides the project management tools, hosting and other infrastructure services to further simplify the development of these projects.

The Grid also operates with a "community source model". Which means that within the grid all source code for the projects is open and free for use, but the source code is not made available outside the grid. The Grid will almost certainly produce some true Open Source code, but most of the projects won't make sense to be released that way.

The labs private pilot program, running with a set of selected developers, is now in progress and over the next few months we'll be releasing more information and opening the doors to more people. However, right now we do have one more spot for a Ruby or Python developer.

If you have a side project that can generate reasonable revenue, but can't be pulled off on your own, Grid7 Labs may have the answer. Feel free to signup so that we can notify you when we're ready to bring more people on board.