Yahoo Pipes UI

Yahoo pipes screenshot

A few years ago I bought my first Apple PowerMac G4 in order to try and get into producing my own music. At the time I was doing a bit of part time DJing and fancied getting into creating bootlegs and remixes. Sadly, a lack of talent let me down. However, one app I did find easy and intuitive to use was Propellerheads Reason. Reason allowed you to link up soft synths, mixers, drum machines and more into a virtual rack. When you spun it around, it displayed the back of the rack panel, complete with patch cables for joining everything together. You could direct an input such as a drum machine into a modulator such as the subtractor and produce all sorts of weird and wonderful sounds.

Propellerheads Reason rack

As you moved cables around and tweaked knobs on the front of the rack, so the output generated changed in real-time. All very impressive and fun.

The Pipes API is very similar and employs a lot of the same concepts. It has the same patch cable type approach to wiring the modules together, and the debugger shows you the results of your actions pretty much immediately. All in all it is one of the most impressive web based UIs I’ve ever seen.

Looking again at Tim O’Reilly’s post about pipes, I notice that Jamie Pitts makes the same connection to Reason in a comment. On his blog, he also references the similarity to Apple Quartz Composer, which I admit I’d not come across before.

My first pipe

Whilst the US are asleep and Yahoo Pipes is up, I’ve been having a quick play.

My initial idea was to utilise the machine tags feature of flickr to create a pipe which would work with photos tagged by members of the Hursley photo club. Members who post to flickr have used machine tags to tag photos taken on our recent trip to Winchester. The following tags are in use:

So the idea was to have a number of inputs which would allow you to then utilise the machine tags to find pictures that matched. To begin with I started with location and member. Not overly exciting, but a start.

Getting the input is easy, just use a couple of text inputs, then a string concatenation to turn them into the machine tags, which in turn was wired to another string concatenation to create the search string e.g. “hurlabphoto:member=aspender hurlabphoto:location=winchester”. I then fed this into the flickr source module. However whilst it appeared to handle a single machine tag ok, it didn’t return results if two were present. Instead, I used the URLBuilder and Fetch modules to create a valid flickr feed request (using the API reference documentation to get it right) which worked.

What this gave me was a feed containing the photos which matched both the location and member. Unfortunately this is where things started to get disappointing. Firstly, it would be nice to be able to use wildcards with the machine tags, but they are only supported when using the full blown flickr APIs. So you have to input both a member and a location. Additionally there are not any facilities for validating/checking inputs so you cannot even check for null/empty and not include that input as a result. What’s more, it would be nice to be able to provide a list/drop down style of input. For example it would be awesome to be able to do a wildcard search on a machine tag such as hurlabphoto:member and then take all the values to populate a drop down to present to the user.

One way I tried to get around this problem was to initially retrieve all photos tagged with hurlabphoto then attempt to successively filter on the tags returned for each photo using the inputs. In the results returned by flickr, the tags are contained as such:


<category term="hurlabphoto" scheme="http://www.flickr.com/photos/tags/" />
<category term="hurlabphoto:location=winchester" scheme="http://www.flickr.com/photos/tags/" />
<category term="hurlabphoto:date=20070127" scheme="http://www.flickr.com/photos/tags/" />
<category term="hurlabphoto:member=aspender" scheme="http://www.flickr.com/photos/tags/" />
<category term="hurlabphoto:subject=candid" scheme="http://www.flickr.com/photos/tags/" />

The filter node recognises the term attribute, but unfortunately doesn’t seem to successfully return results when you filter against it with a machine tag such as hurlabphoto:member=aspender.

So, the net result is that I’ve created a pipe which simply removes the need to insert the full machine tag and which also sorts the results. Not very useful. Most of this is down to the current limitations of machine tag searching by anything other than the full API, something which will hopefully be relaxed over time. When/if it does, then the combination of flickr and pipes will be very powerful indeed.

So, finally, if you want to have a play with the pipe and maybe play around with taking a clone of it, you will find it here. FYI, valid values for the inputs are:

location: winchester
member: aspender, currie, calanais, Fragments of eternity, Liam O’Neill

David LaChapelle – Heaven to Hell

Warning. Following the link below will take you to a site with some images you might not want to view at work!

David LaChapelle is a photographer, music video, advert and film director whose highly stylised and often controversial work has been at the forefront of modern high-concept art and he is responsible for images and videos which you’d probably find instantly recognisable. His book Heaven to Hell showcases his most recent photographic work and heavily features trademark conceptual shots of a wide range of celebrities in often uncompromising positions. It is vivid, over-saturated, sometimes disgusting but endlessly fascinating. He has an obvious obsession with pornography and fetish which creeps into some of his most well known work such as the Christina Aguilera’s Dirrty video and is certainly highly present in the book. That said, if you can get past that and look at his composition and also the sense of irony that many of his shots portray then you realise his obvious talent.

Looking at his web site, I was also very surprised by the amount of recognisable music videos he has directed, including ones for artists such as Norah Jones and Joss Stone which you wouldn’t have thought at first. However, I personally think he doesn’t excel in this medium to the level of Spike Jonze, Michel Gondry and Chris Cunningham to name but three.

Whilst not a style that I could ever imagine trying to imitate, and I imagine you either love it or hate it, the book is certainly very interesting from a photographic point of view and it is easy to see why he is lauded as one of the most important living photographers. Just don’t leave it around on the coffee table when the parents come to visit!

Yahoo Pipes

As previously mentioned, I have a grand plan to do some funky web2.0 style stuff on this web site. However, with the new Yahoo Pipes service, maybe I don’t need the space after all. The premise behind Pipes is to bring the power of Unix pipes and filters to the world of feeds, allowing the mashup of multiple feeds into one. The service supports some pretty powerful bit of functionality (or modules as they call them) to perform analysis and processing of the feeds. There’s some good posts over on the O’Reilly Radar about how Pipes work, and one from a rather excited Tim O’Reilly on why he thinks Pipes is a “milestone in internet history”.

Migrating from wordpress.com to .org – diversion strategy

Firstly, I’m going to talk about things in this post for which I am no means an expert and very much a beginner, so please don’t take any of this as gospel.

One of the biggest problems with the move from my wordpress.com hosted blog to hosting on my own site is managing to redirect people to the new blog, and generating new traffic to the hosted site.

Redirecting traffic

The old blog was obviously picked up by Google, other search engines, blog directories like Technorati, and other people’s blogrolls. Thus, any redirection strategy would have to:

  • Attract new visitors to the new blog location
  • Get people searching for popular post topics to go to the new site
  • Try and get people to comment on the new site rather than the old

These aims can be achieved by:

  • Making it obvious that the old blog is dead
  • Getting the new blog to appear in Search Engine Results Pages (SERPS)
  • Getting the old blog to disappear from SERPS!
  • Getting blog directories and blogrolls updated

The most obvious thing to do first was to add a post to the old blog which pointed people to the new one, making it clear that they needed to update bookmarks and feeds. I also pared down the template to remove categories, archives, calendar etc. In fact, all that is left is the blogroll. I also:

  • Added a text widget to the sidebar with a link to the new blog which would appear on every page of the blog
  • Changed the title of the blog to make it a bit more obvious what was going on

I considered disabling comments on each post and perhaps even adding a redirect comment to each post, but this appeared too much work to be worthwhile. So far I’ve only had one new comment on any of the old posts (but also no new comments on their equivalents on the new blog)

I’ll cover adding the new blog to search engines later on. However, getting the old one removed is a difficult thing for a wordpress.com hosted blog as you have little control over it. On a normal web site you could add an .htaccess file to the root directory with a 301 redirect. Not so when you don’t have sufficient access to do so! There is an option in the dashboard under Options -> Privacy to restrict access to the site to search engines and remove it from WordPress’ own listings. I suspect this does a similar job to a robots.txt. A robots.txt is the traditional way to restrict search engine spiders from indexing your site. Google also have a tool for removing specific URLs, but this is not meant for removing an entire blog for instance.

I then updated any links I have to my blog on other site profiles, created a new Technorati blog profile and fired off emails to a couple of specific IBMer blog rolls.

The final step was of course to get people’s blogrolls updated. One or two picked up on the change by themselves, but a quick email around the rest sorted out the majority of the rest. What you can’t help are links from comments you’ve made on other people’s blogs.

Up until today I’ve not seen traffic to the old site decrease at all. However it looks like my most popular post on the BT Homehub which normally attracted 60 or so views/day has disappeared from Google, and as such it has not received a single hit so far today. Thus it looks like things are beginning to disappear.

Eventually once the new blog is indexed and appearing in SERPS and I’m happy traffic to the old one has reduced to a trickle, it will be time to hit the delete button.

Attracting traffic

This is a whole different kettle of fish in the self-hosting world compared to wordpress.com. When I created the original blog it appeared on Google and co. in a couple of days. Not so now! As hinted at above, I am a complete beginner in the world of Search Engine Optimization (SEO) so use these comments as hints about what to go and research more thoroughly!

To begin with I set up an index.html (more on that in a separate post) linking to the blog, then went to Google and added the URL. This then led me to find the Google webmaster tools. This led me to research the whole area of robots.txt, sitemaps and the pro’s and cons of www versus non-www, and all sorts of issues around how search engines look out for duplicate content and the effect it can have on your results. The last item is especially relevant to blogs as the same content often appears on different distinct URLs (archives, categories, posts)

Along the way, I found and installed a couple of useful WordPress plugins:

  • DupPrevent – Allows you to configure areas of your blog which won’t be visible to spiders to avoid a duplicate entry penalty.
  • Google sitemaps plugin – Generates a sitemap from your blog content. Highly customizable and can include non-blog URLs.

The most disappointing aspect of my research however is the fact that it can take four weeks or so for new sites to appear in search engine results.

Southampton Apple store to open on 10th February

Apple Store Southampton (Matty Turner)
Image from flickr user mattyturner licensed under Creative Commons Attribution 2.0

For months now, a simple black facade with an Apple logo has hidden one of the shops in Southampton’s West Quay shopping mall. Each time I passed I wondered what was going on inside, and when the store would open. Yesterday Apple emailed out to say that the opening will be at 9am on Saturday 10th February. The first 1000 people through the door get a free t-shirt. I intend to be there with DSLR in tow to capture the events, and maybe pick up a couple of bits for the iMac!

More details on the store over at Apple.

Apple denounces DRM

I always though this would happen. Steve Jobs has posted what amounts to an open letter to the music industry to stop the use of Digital Rights Management for online music downloads. The thoughts contained within it are clear for all to see. Apple Inc. would switch the iTunes Music Store over to a non-DRM format “in a heartbeat” if the big-four music companies would allow it. Jobs elucidates the options they have today: to stay as they are and watch a market fragment into proprietary formats, license FairPlay and watch it get compromised quicker than the blink of an eyelid, or convince the music industry that DRM has never, and will never work.

Interestingly, Jobs effectively issues a call to arms to the citizens of Europe to put pressure on the big four, both because we have been most vocal in criticizing DRM and as the music industry is effectively centered here. Where do I sign up?

Hursley photo club Winchester challenge

Last Saturday a good number of Hursley folk met up in Winchester for a two hour photography challenge. The brief was simply to take photos, meet up and have a bit of fun. I, along with most people seemed to gravitate towards the cathedral, though I ventured onto the High Street later on to capture some street and candid shots. Each person then puts forward three of their shots for the competition, though that’s competition in the loosest sense of the word. In any case, here are the three shots I selected:

Foreboding:

Winchester Cathedral 5

Winchester Cathedral HDR:

Winchester Cathedral 1

Blind Busker:

Blind Buskers 2

I’m proud of the first one as the burning of the clouds in PSE really made it. The second one is a composite of 9 separate exposures bought together using FDRTools. The third is something a bit different from the last two, but not a bad shot all the same despite the camera shake.

Migrating from wordpress.com to .org – feeds and stats

This is probably the most interesting area of the whole move, and the one I’ve spent most time getting right.

One of the things I did like about wordpress.com was the statistics it showed you under the dashboard. They were quite comprehensive and also addictive. The feed stats on the other hand were not so good. I experimented with using FeedBurner, but as you have no control over the default feeds wordpress.com generates all you can do is add a link on the blog pages and hope people use it.

It is quite a shock when the stats tab disappears once you move to a self-hosted wordpress.org blog. As such, the general way people compensate is to use two external pieces of technology to provide much the same (and more) information:

FeedBurner

I’m not going to cover the process of obtaining a FeebBurner feed as that is well known and documented. However it is worth covering what you have to do to the blog to make sure people end up using it.

The first step is to use a plugin. I installed the FeedBurner Feed Replacement plugin. This will substitute the standard wordpress feed for your feedburner one whenever anybody clicks on the feed icon in the browser, or on the feeds under the Meta section.

However, there are still two possible ways that people can get to the original wordpress feeds. Firstly, they can copy and paste the links under the Meta section into their feed reader. The feed links that appear here are controlled by the PHP get_bloginfo() function. To modify what gets displayed, I went into the Theme Editor section under Presentation and edited the Sidebar theme file which originally looked like:


<?php wp_register(); ?>
<?php wp_loginout(); ?>
<a href="feed:<?php bloginfo('rss2_url'); ?>" title="<?php _e('Syndicate this site using RSS'); ?>"><?php _e('<abbr title="Really Simple Syndication">RSS</abbr>'); ?></a>
<a href="feed:<?php bloginfo('comments_rss2_url'); ?>" title="<?php _e('The latest comments to all posts in RSS'); ?>"><?php _e('Comments <abbr title="Really Simple Syndication">RSS</abbr>'); ?></a>
<a href="http://validator.w3.org/check/referer" title="<?php _e('This page validates as XHTML 1.0 Transitional'); ?>"><?php _e('Valid <abbr title="eXtensible HyperText Markup Language">XHTML</abbr>'); ?></a>
<?php wp_meta(); ?>

The interesting lines here are the ones that contain calls to get_bloginfo such as:

feed:<?php bloginfo('rss2_url'); ?>

I simply modified these lines to hardcode in the FeedBurner URLs for my main and comment feeds. However, if you are using widgets, then this does not affect the Meta widget and I’ve not yet tried to work out what needs to happen there.

The final way in which people may get to the original feeds instead of your Feedburner ones is via a feature called feed autodiscovery. This allows people to put in the site hostname (e.g. just http://www.adrianspender.com) into their feed reader and the reader then goes off to discover the correct feed URL. This will return the wordpress feed by default. To modify this, I went into my theme and edited the Header theme file in much the same way as for the Meta section. The difference this time is that there are three hrefs, each for different flavours of RSS and Atom feeds for the main site, not for comments. Simply replace the get_bloginfo() calls as before with your FeedBurner feed URL. I suggest you ensure your FeedBurner feed has the SmartFeed capability enabled.

Google Analytics

Analytics allows you to register your site with Google who will then provide you with very detailed site statistics. In order to do this, each page on the site needs to have a small piece of Javascript inserted into it. This contains your unique site key which is fed back to Google on each page visit.

For a dynamically generated site like a blog, this can be troublesome. However thankfully there are a number of plugins out there to help you. The one I used is called Ultimate Google Analytics. This automatically inserts the tracking Javascript into your blog pages and is configurable in a variety of ways to help you make sense of the stats.

Providing a dashboard stats view

Finally, there is a plugin called WordPress Reports which can collate your GA and FB stats into a single view within the wordpress admin, not too dissimilar to that provided by wordpress.com. It doesn’t look anywhere as good, nor give you as much detail, but may be useful nonetheless.