livz.org

Just testing

2009-09-11T22:48:00.001-07:00

Just testing
Originally uploaded by weblivz

steven

OpenID's hidden value in Application Integration

2009-05-04T02:41:00.000-07:00

I haven't seen any articles discussing this subject - but let me know if you know of any and i'll link.

OpenID is a hugely valuable standard protocol for federated authentication and by the looks of things adoption is and will coninue to grow at a phenomenal rate.

The advantage of OpenID over traditional authentication systems is well known - it saves you having to remember multiple usernames and passwords. An additional advantage of OpenID over other Internet authentication systems is that it is completely decentralized and anyone can create their own authentication provider.

Consider an enterprise that has some internal applications that can't be seen by the web itself. That enterprise can set up an OpenID provider to let all their staff log into these internal applications. Additionally however they can allow that provider to be seen from the outside world and so allow staff to log onto applications on the web using that same OpenID.

Cool! However there is an additional benefit when integrating applications which isn't quite as explicit and you really only notice at development stage. In particular this is very important for creators of 3rd party software. That is:

If a 3rd party application supports OpenID then no matter the language or platform it can be easily integrated into an existing software site or portal.

What do i mean by this? Well recently I created a web site that used a 3rd party open source system that had traditional basic authentication as well as OpenID. Great. However later on i wanted to add a nice blogging feature using another 3rd party ... but the problem was they didn't support OpenID.

Now, let's ignore all the technical detail here such as Single Sign On between applications - sure this is cool but i'd be willing to bet that even if the user had to log in using their OpenID twice into these two applications it is miles ahead of the common current issue where you suddenly need to remember a username and password again (and all the management that goes with that). Consider the case where most of us may integrate 4 or 5 separate systems to create a solution and suddenly the fact that ONE of then uses OpenID but the other 4 require old style username/password authentication really isn't much of a benefit.

In my case i chose not to go for them. I simply didn't have time to write OpenID support for their products. I was quite happy for the applications to run independently on the server with a simple link bewteen each ... even if they had to log in to each - so long as i didn't have to manage duplicate accounts. The great thing though is that the managament of their login details etc could all be done centrally. I know from my experience supporting OpenID.ORG that the time taken to support users dwarfs almost everything else you do.

So although it may sound obvoius that an OpenID account can really help you log into multiple applications with the same details, it's not until you start pulling together 3rd party applications you realize how valuable this is even on a single website or portal.

Going back to our enterprise - they can now create a cool portal that integrates custom software, 3rd party products & remote services, without worrying about user managament. And i know from current personal experience how big a win that will be.

xUnit & TeamCity

2009-04-14T04:25:00.000-07:00

Having used nUnit for some time and Visual Team System's test architecture i was recently interested in putting this to work on my own project as part of an automated build & test architecture. The test architecture of Visual Team system isn't particualy suited to Continuous Integration as you need to reference assemblies that only seem to be part of Visual Studio and you're not allowed to distribute these. I considered nUnit as this is what i am used to but as it's my own projected decided to give xUnit a shot as it seems to be the next logical step in good testing practices.

I had started using ThoughtWords CI server but after banging my head on the table a few times in my failed attempts to integrate xUnit, and on the advice of Brad Wilson, decided to look at TeamCity.

With some help from Brad I managed to get this working but as it was still a little tricky (i only rarely change this infrastruture of this kind of thing once it it set up) i decided to document how to do it. Note that you can't just point TeamCity at the .sln file in this case because you need to TELL it that it must run your xUnit tests and clearly the .sln file doesn't know about them. So what you need to do is create an MSBuild file where you can tell TeamCity to (a) build your solution file AND (b) run your tests via xUnit.

1. Download and Insteall team city. Configure this to get it building your base code ( i won't document this as there are loads of places to find out more).

2. Go to the folder where your .sln file sites and create a file called "mysolution.msbuild" (or something similar).

3. Enter the following Xml:

<project defaulttargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<usingtask assemblyfile="[RelativePathToXUnit]xunit.runner.msbuild.dll" taskname="Xunit.Runner.MSBuild.xunit">

<target name="Build">
<msbuild projects="mysolution.sln" targets="Build" properties="Configuration=Debug">
<xunit assembly="[RelativePathToProject]bin\Debug\MyProjectTests.dll">
</xunit></msbuild></target>

</usingtask></project>

* Remember to replace [RelativePathToXUnit] with the relative path from your solution file to where the xunit runner assembly can be found. Also make sure [RelativePathToProject] is changed to point to the project where your tests can be found.

4. Now via the TeamCity GUI in the "Build Runner" tab of your project, select MSBuild and add the relative path to your "mysolution.msbuild" file relative to your checkout directory (in my case it is often "/mysolution.msbuild").

5. Now you can run your build and you'll find all your unit tests are run.

Microsoft Tags, QR Codes and OpenID

2009-01-13T03:16:00.000-08:00

I have used Barcodes and QR Codes in the past on some projects and played around a bit. Well, this morning i saw Microsoft released "Microsoft Tags" as a beta at CES 2009.

Clearly from my own selfish perspective, what i'd like to see if OpenID (disclaimer, i run http://OpenID.ORG) go OPEN in the real world. So, at a conference you could put a QR Code or Microsoft tag on your badge, t-shirt, hat or what ever. Maybe on the back of a card, on your company documentation, promotional material and so on.

If your OpenID is encoded in there then people could save you as a contact direct from your t-shirt :)

Thoughts? Other ideas?

Here is my OpenID (http://weblivz.openid.org) as a Microsoft tag (which would be printed out of course):

... and here is my OpenID as a QR Code :

Update : Sure enough, i installed the Tag reader on my PDA and simply pointed the phone at the image and it took me to my OpenID page in the web browser. Very nice!

They say more formats may be supported in the future - hope they add QR Codes to that list!

Twitter phishing issues

2009-01-06T15:44:00.000-08:00

I agree with this article in the main, but there are some things worth pointing out… baring mind the majority and most concerning of these phishing issues came from people giving their credentials out and the 3rd parties assuming their identities and the associated trust with that.

1. oAuth could have allowed me to provide access for 3rd party systems in a case by case basis - not providing the credentials that could be stored and re-used.

2. oAuth would allow me to disable access immediately for a 3rd party site.

3. oAuth, as used in GMail, could be used to restrict the levels of access you permit of 3rd party applications. Therefore i may allow a site to read my contacts (which most do) but not sent replies or DM’s on my behalf (in fact this is what GMail can allow).

... I totally get that anyone can @ you on twitter - but i would be unlikely to click a random link. It it came from a friend - especially DM I am MUCH more likely to click it.

Only question i have is why did the phishing sites not use tinyURL or something to mask the link?

In the zone

2009-01-03T02:49:00.000-08:00

Every New Year we promise to change something about ourselves. Well, this year i'm THINKING of trying something new (for me). Starting Monday 5th Jan, I am going to change timezone.... well, for an experimental period anyway! I was inspired to try this by reading this article by @4entrepreneur ... and the fact that it's harder to work at home during the day when my daughter is around (not yet at nursery).

What does that mean? Well, for 2 weeks (gotta trial this and start somewhere!) i am going to work to PST time. So that means i will be working through the night, at the same time as most of the tech activity on Twitter. It also means i can see whether coding in the evenings really is more beneficial. I used to write all my code books into the early hours but when i had the kids things reversed and i starting getting up very early.

I will blog and tweet as i go.

But i need to get a few typical daily timetables from tech folk working in PST (when do you rise, work, go to sleep and so on?). If anyone can comment, email or DM on twitter it would be much appreciated!! I could easily subtract 8 from my normal day, but want something real.

Anyone NOT in PST already work to that timezone, perhaps out of necessity via remote working?

The Social Netbook

2008-12-29T02:21:00.000-08:00

It is perhaps reasonably well known that i won a Netbook courtesy of Vodafone last month. As soon as i got it I started to think about how it could make its way into the everyday lives of the people i know. So, here are a couple of suggestions:

Social Software
The first thing is that these Netboos need to come pre-bundled with social software. Most people won't buy a Netbook to write code or even compile documents. Most will buy them to "get on the web" as Netbooks are a great opportunity to point them at things to do.

I imagine a wizard in startup that suggests the services you may like to use, takes you through a signing up process and installs and configures the appropriate software for you.

An example is ...

Configure an OpenID account
Create Twitter account (they should be doing (1) by now!) and install software
Create a flickr and twitpic account and configure appropriately.
Set up friendfeed.
Create a MySpace/Facebook account and store shortcuts.
Install UStream/Qik and configure an account.
Email account.
blah dee blah ...

(Other suggestions ??)

Cloud Services
We're talking about a lightweight laptop that has limited disk space and less than the typical laptop in terms of memory and processor speeds. It is pretty much reliant on you being online. So why not use the cloud??

An example ...

An Amazon/Mesh storage account direct from your desktop ... via your OpenID account ;)
Allow direct storage and management of pictures, video and so on on the clound (maybe on top of (1)).
Perhaps even EC2 could be somehow synchd with Netbooks to allow complex calculations to be carried out on the cloud. Idea here?

In short there is potentially a huge opportunity for services to get on board this new wave of laptop and whilst rolling out an empty laptop works fine for me, there are many others who could climb on board the social wagon if they were given some help along the way!

Who else uses a Netbook? What do you think?

Xtranormal - create your own 3D animation

2008-12-07T02:41:00.000-08:00

Keeping the kids amuzed on a Sunday morning can be somewhat slow, but when @joecar mentioned Xtranormal to us this morning we tried it out... and how addictive it has become. To me, sure - but my 5 yr old loves it.

So we created a story (i helped him out but all the ideas, script, names and so on were his and you can perhaps guess someof his influences!).

Here is it below - you can also see it here and in youtube . I suggest you try it out - some things still to be worked on, but in general very, very well done.

Secret Santa on Twitter

2008-12-03T02:35:00.000-08:00

This Christmas i'm running a secret santa on Twitter.

The idea is that - you contact me with your twitter name and you will be added to the list - this will stop on Friday December 12 (I'll extend to December 19 if need be).

You will then be contacted with the twitter username of the person you are to get a "gift" for. The gift can be a cool URL, a connection with someone, a beta invite - anything useful (probably best avoid sending anything involving advertising).

When done (i.e. after i have contacted a secret link), you simply send the following:

@username #secretsanta [message]

The #secretsanta is just so we can follow this happening (and the person knows why you are contacting them).

If wished, i could publish the list after Christmas.

Update - Who's involved?

@weblivz
@katiebarrowman
@jenjeng
@KyleMacRae
@holeinhiseye
@salidatious
@garymurning
@peewii
@frontofficebox
@jimwolffman

A winner with Vodafone Live Guy

2008-11-18T08:46:00.001-08:00

Today i spent more than a few hours hunting for @vodafoneliveguy - you can see my twitter feed on the day at http://twitter.com/weblivz. This is a treasure hunt style contest where you need to find the vodafone live guy who is constantly moving throughout the city. It is a really nice idea, something i read about a while back in SmartMobs. You need to follow his updates via GPS, blog, twitter and more and try and guess where he is.

It was more than just winning the laptop i was interested in. I wanted to participate in something like this - as well as livestream it as much as possible, armed with a PDA, a laptop with some web access and my feet (i wasn't in a car, nor considered paying for taxi's to travel around).

The Morning
At 9.10 AM after dropping my son off at school, i got the train into Glasgow and the game started. I was pretty determined to find him - but i wanted to see how i would use technology to do so.

I went to Queens Street and immediately headed to Bothwell Street tracing the path from the old Herald offices opposite Livingstone Tower at Strathclyde Univeristy. I went to Bothwell Street and after looking around, figured he had went into the centre of town. So then i checked into Borders to fire up the laptop and check his map.... it said he was in Bothwell Street, but much further up. So back i headed and outside central approached a rather worried looking chap dressed in red and put my hand out to shake it. Perhaps he was a tourist thinking were were always thing friendly, but nevertheless he just kind of looked at me as I realized this was not my man. I felt pretty stupid, but onwards i went.

I was then at Bothwell Street and despite me being roughly where he was for about 15 minutes before anyone else turned up (i could tell the instant a challenger arrived, just outside Weatherspoons on Bothwell Street) and despite the picture of the car park, i just couldn't locate him.

I tried to think of where he could head... the Clyde was my thought - a fairly major part of Glasgow. So i walked down that way, only to discover he had been found. Fek... and i think it was the guy who ran past me who found him (i'd love to hear that story too!).

I was beaten but not out and the day was young. I recorded a video on my mobile where i talk about the fact he started at Colville Building of Strathclyde University (which is where the Headquarters are in detective series Taggart) and how he was at the Livingstone Tower, named after David Livingstone.

The Afternoon
As i sat in St Enoch (suggested by my wife), i read that he was at the Sas Raddison and spent 15 mins there before realizing i was never going to find him. I then read he had to moved into MY backyard - the West End! On the underground to Hillhead - guessing he would go to Glasgow University and the Hunterian Museum, only to find he was going to Kelvingrove.

So i walked pretty quickly to the museum and figured i'd go in the back way and surprise him. But as i went round the corner, what do i see? A big red jacket and a bunch of people around him. I approached to congratulate the winner(s) only to find none of them knew the phrase. I immediately said "You're the LiveGuy and I'm a Vodafone Winner" and sure enough, the prize was mine.

It turns out Jon Sykes, a lecturer at Glasgow Caladonian University had taxied two lots of students to Kelvingrove, but none of them knew the crucial phrase. So i was either lucky, or had studied harder! Put it this way, it wasn't exactly Maradona handball in the box - more like that second goal ;)

I had a nice chat with liveguy and friend for a while telling them of my day and the close misses and they went their way and i went mine. I recorded a video just after i had spoken with them.

I then went into the BeanScene cafe opposite the Galleries and relaxed for an hour or so, checked from mail, read some papers.

There's More
I do some work for Conscia and sometimes use their desks. Now, yesterday i had told them of the game and figured they had just ignored me - especially as i had heard nothing from any of them all day. Well, turns out they had been keeping tabs on the live guy.

I walked into the office and said "Yes, I won" only to be told, "Yes, so did Chris". Now, i assumed this to be a wind up, but it turns out that @vodafoneliveguy had left his GPS on and he made his way into town and stopped around the corner from the offices. About 10 of them apparently ran out in various directions and the first guy (Mark) didn't know the phrase either - "Can I have the laptop please" is apparently what he said. In fact the one that did (Chris) had made a call back to someone else (Dominic) in the office who told him what it was.

To end it, one of the guys (Jim) had went to the bank which is nearby where this all happened and came out only to see much of this going on. He had been there the whole time and despite HIM being the one i initially told AND HIM being the one that reminded everyone else this morning after i had told them yesterday, he had temporarily let his guard down. Otherwise he would be a happy owner of a new laptop.

Update: Apparently Mark had an iPhone with GPS tracking on it and could literally watch them drive into town, stop and emerge from the offices they had been in - with lots of people following him as they knew this! The only problem was he didn't know the winning phrase and Chris jumped in (ouch!).

What a mad day. Feet are sore, but happy that i won a laptop and even happier that i was part of all this, especially as by the time it hits London he's need to wear a disguise.

So i guess the moral to all of this is that social networking works - but only when you know a bit about the detail. Deeplinking is cool even in a social sense so long as you read what's at the end of that link.

OpenID "Friend" based Attributes

2008-11-05T07:30:00.000-08:00

Something i have been thinking about in line with some recent discussion on the OpenID mailing lists has been techniques to verify someone's identity and the attributes that are associated with it.

My initial thoughts comes up with the following:

Reputation
In this case, the longer you use your OpenID, the more people will come to know it is associated with you. I’d be interested in how we could explicitly extend this concept to support a distributed reputation system where you can attach OpenID Reputation points which are assigned from other sites. These could even be broken down into types of reputation. The longer and more you use it the more reputation points. Even sending an email that isn’t spam etc via Gmail or Windows Live could “increment” your reputation score (like pagerank, but for OpenID).

Friend Verification
Slightly more explicit and extension of the above whereby friends can verify things you write down about yourself so that others can trust them more. John says he works at Google Inc and Brian has verified this etc.

Central Validation
Shibboleth model and probably needed for institutions who perhaps verify things such as “Yes, this is definitely Dr. Livingstone” etc.

More thinking
However, i can't think of a reason why OpenID AX couldn't support attributes from third parties that are signed and stored with the OpenID. This way an institution could be given a users OpenID, make statements about them and return the signed statements/attributes which can then be stored anywhere with that users OpenID profile.

Someone who wants to get those attributes can easily check via the insituions publically available key that thse attributes have not been altered in any way.

Extending this you could have a plethora of OpenID attribute information that is both unverified and strongly verified.

OpenSearch and JSON

2008-11-01T07:15:00.000-07:00

In the limited spare time that i seem to have these days, one of the things i've been looking at is OpenSearch, "... a collection of simple formats for the sharing of search results".

OpenSearch is slowing gaining lots of traction and each of the major browsers now allow you to add an OpenSearch to your toolbar automatically (although the User Experience certainly needs work!).

In my current consultancy work for the NHS I was asked to extend the search API I wrote some months back which works against FAST, and enable any third party site to ended some script in their site and make Ajax style queries. Last week I wrote this functionality (which returns custom Xml and Json) and sure enough it works great.

However, I like open stuff because it makes it easy for others to pick up and run and so after I added OpenSearch support via the usual description file and tested this with Internet Explorer et al and it worked, i decided to look at whether i could use OpenSearch as the actual syndication format.

Now, if this all ran on the same domain there would be no problem in pulling back Atom or RSS and formatting it for display. But in a cross domain architecture you can't do this due to Same Origin Policy. So, using JSON-P we can enable OpenSearch queries over the wire.

To do that however, we need to have an OpenSearch format for JSON - in my case i simply took the Atom format as a starting point and mapped much of that and added in the OpenSearch elements and attributes.

So now you can take a few lines of code, add it to any page in the world and make an OpenSearch query using the template Uri and get an OpenSearch response which can be parsed and dislayed.

Early days but if it's useful to no-one else on the planet, it is useful to me :)

If interested, please try it out here. It queries over 100 million records so you will get results no matter what you type in, but the formats and data mappings are still being worked out.

PS. Would be *very* nice if we could query the OpenSearch discovery endpoint and ask for JSON to be returned. That way we could dynamically build all of the front end stuff that i can't do with the existing reponse (it returns an OpenSearch specific content type which is not trusted by browsers).

Grassroots rail support

2008-10-11T02:11:00.000-07:00

I've travelled on trains, planes and automobiles for so many years now and one thing that has yet to be resolved is telling you what is happening as it is happening. You are often left to try and guess what the problem is, or when the train will arrive - and this is worse at night when there's often no-one around to ask. Well, for years I always intended to build a grassroots support network for transport as the profilferation of mobile devices enabled more people to become involved using SMS and mobile browsing.

Well, below is a rough sketch (it's not too detailed as my son is pushing me to watch an Indiana Jones puppet show he has created, but you'll get the idea).

Thoughts on the Cloud API

2008-10-10T04:30:00.000-07:00

I have seen quite a few diagrams on cloud computing but not something like the one below - (please tell me if you know of more and i'll update!).

You see what many leave out is that if you build your service on a third party service in the cloud and it closes down, you're screwed. Well, currently you are. You need to find a new provider, swap your data, change the api, modify your internal code. Which in simple terms, means your're screwed.

Now, it's not such a big deal if Amazon is your service (although this should still be a consideration) but if the POINT of all this open web stuff and open source and api's etc etc is to allow the little guy to provide services, then it IT a big deal. The little guy will just NOT get the trust for many businesses and it is a shame. Once their ideas are bought by a bigger company then they get exposure. But surely this is a shitty way to do things. Why not let the little guy in?

A good way for this to happen would be for the "cloud" (and in Scotland we have cloud computing every day of the year, with torrential rain computing thrown in) to support common (i hate saying standard!) API's. oAuth is arguably the most successful example of that to date (with PoCo to come), but OpenID is also another good example (oAuth is not used seamlessly by many a big company).

But surely we need to look at common API's for all services - or at least the service providers do. I want to use provider A today and provider B tomorrow. It should be a configuration change for the client (and a "swap provider" checkbox ;) ).

Internally however there should be a service provider data exchange API that allows swapping of data (and arguably hot swapping for back up - imagine service A and service B backing each other's data up whilst providing the same service!?). So when i ask to move, i can have all my data moved to the new provider (for a fee of course). Here's that API :

data.Transfer(newProviderURI);

OK, so it's a little more complex than that, high level is a good place to start. I also like the idea of a higher level cloud API that provides common services for all kinds of cloud services. I know there has been lots of work on various SOAP style services at W3 and so on (transations, instrumentation etc) - not sure how this thinking all fits in with the cloud stuff but would love to hear/read about it.

One Web, Many Cultures - Reaching Out

2008-10-10T02:40:00.001-07:00

I'm not entirely sure where this post is going, nor do i have any conclusions in mind. I have simpy been thinking about this for some years and feel like writing it down. I'm not sure what tone it will have for the person reading, but it is intended to ask some questions rather than make any kind of statement! Let me first say that i am from Scotland and have live in England, Italy, Canada and Chile in the last 15 years - i wrote books and used the web constantly in each.

I listened to Kevin Marks interview with Jemima Kiss at the Guardian and other than Kevin sounding incredibly similar to a friend of mind, the part that jumped out of me was his comments that many of the big social networks have done well out of users they just didn't expect. US technology companies growing "execpectedly" from string user bases in Russia, Brazil, China and so on - in other words from users NOT in North America.

So, I wrote on Twitter :

"that was an interesting interview Kevin. Given the cultural luck many networks
have do you think we need to put more work in involving other cultures right
from the start rather than once the technology is ready?"

Kevin Marks of Google made the point to me on Twitter that :

"the technology is ready, and has been for a while. Finding people to
bridge app ideas to other cultures and language is still hard."

My wording wasn't great (140 characters isn't much!) so this is what i meant....

Much of the technology driving the web is coming from specific locations in the US - mostly California - there is some input from the rest of us through discussion groups and so on, but primarily it is driven offline. This is understandable on many points due to the huge concentration of top people over there who are much like myself - driven and excited by the potential of the web as a platform that can fit into the daily life of everyone on the planet - in the many different ways people will use it. Also, at the moment virtual working groups are still a work in progress.

Now, oftentimes i get frustrated that i'm not amongst it - i had hoped to some years back startig with OpenID - (every week on twitter there is another cool event somewhere in the US where all the guys and gals get together to push these things forward). However, something that must be more frustrating is little involvement other cultures in the rest of the world must feel - at least i speak (bad) English and can follow in the forums and blog posts (and every so often post in the forums). What if you speak Chinese and want to use oAuth? Or Spanish? Say you want to add to it or suggest a change? There are a few involved, but this is very much the exception and if you check the board membership for most of these initiatives you can see they are based in the US and often (but not always) part U.S. web companies.

I think it is a problem waiting for a solution - the problem is how to integrate other languages and cultures so that the open web can be something that fits for everyone. If Orkut surprisingly managed to get a dominant user base in Brazil, was there something different to what other social networks were offering which made this happen? Does anyone know? In Africa it is suggested the uptake of mobile phones will continue to be huge - how do these efforts fit into their plans for a social web?

There are many of these folks who do a lot of evangelizing for these efforts so perhaps they can get to know active people who could get involved in the core specifications from Europe, Asia, Africa, South America and so on. It's worth remembering this IS one web, but has many cultures and what fits one may not fit all.

Thoughts on Using Netflix’s New API

2008-10-09T10:07:00.000-07:00

OK - I have read through Using Netflix’s New API: A step-by-step guide by Joseph Smarr and tried out a few of the things mentioned.

My only irritation in the excellent efforts Netflix have done are around the anomalies you can see in the API workflow.

i'm sure they have some good reasons for diverting from the "standard" process and perhaps some view to these may be useful.

In addition it may suggest some extension points are needed that could be added to oAuth so that rather than hacking these things we could simply add some optional extensions (which are generic such as "dynamic parameter" rather than "userid") and can be plugged into any oAuth library.

The fact this can even be done is excellent but in using it it would be nice to just have a graphical pipeline for every site that shows where they may have used some of the optional filters during the oAuth process.

Copy and Paste between OpenID's

2008-10-08T15:54:00.001-07:00

In case you missed it i've added a new feature that allows you to Copy and Paste data between OpenID's.

More information can be found here.

TESCO miss the point on the web and PR

2008-10-04T12:08:00.000-07:00

Tonight we were expecting a home delivery from Tesco in Glasgow at 5pm. At 7.55 PM with the kids wandering where their milk was, we got a call saying they won't be coming due to "staffing issues". I am furious for obvious reasons - none worse than the fact i now need to go shopping to get breakfast for the kids for tomorrow and it is really late and pouring outside.

I am writing to them in response to an email i received so i will keep you updated and see how things progress (or don't).

--------------------------------------------------------------------------------------

Mr. Livingstone here. I included Terry Leahy in this and additionally will blog what happens at :
http://weblivz.blogspot.com/2008/10/tesco-miss-point-on-web-and-pr.html

Are you taking the piss? We have a 2 year old and a 5 year old who were waiting on milk before bed time because you were due to deliver before 7. You phone 1 HOUR late on a Saturday night to tell us this, asking if we can "come get it" !? You had from 5pm at the very,very latest to know of any apparent staffing issues.

What is WORSE is this pathetic general letter you have sent us - you never even took the time to send something personal. We never even had an opportunity to file any complaint and "you have been dissatisfied" is an understatement.

I would have been irritated at a cancellation earlier, but calling 1 hour later than the latest it was supposed to be here combined with a pathetic general letter (I assume the typists also had the evening off due to the rain) means we will no longer bother doing business with you and move to Asda or equivalent.

Additionally, I am a high profile online UK blogger and Internet consultant so you can bet a few people will hear about it and i will be looking out for someone at Tesco who can look into this.
Now I have got to go out into the pissing rain at 8 PM on a Saturday night to get milk for tonight and the kids breakfast for tomorrow morning.

For what it's worth - you don't need to concentrate on staff issues - you need to look at your PR... a phone call earlier in the day would have saved hassle for all of us.

steven
http://livz.org

________________________________
> From: online@tesco.co.uk
> Subject: Tesco Online Vouchers
> Date: Sat, 4 Oct 2008 19:51:19 +0100
>
>
> Dear Mr Livingstone
>
> Thank you for notifying us of your complaint. We value our customers' feedback, which helps us to monitor and improve our services.
>
> We are sorry to hear that you have been dissatisfied with the service you recently received from Tesco. As an apology we would like to offer you a ? 10.00 eCoupon, with our compliments, to spend on your next grocery order on Tesco.com. To use this eCoupon simply type in the code XXXXXXXXX when you reach Checkout. This eCoupon can be used from 4 Oct 2008 to 4 Jan 2009
>
>
> If you have any questions, you can also e-mail us at online@tesco.co.uk. Or you can contact our Customer Helpline on 0845 722 55 33 between 9am and 11pm Monday to Friday, 9am and 8pm Saturday and from 10am to 6pm on Sundays.
>
>
> Best wishes,
>
> -
>
> Tesco.com
>
>
>
> Tesco.Com Limited
> Company Number: 3942522
> Registered in England
> Registered Office: Tesco House, Delamare road, Cheshunt, Hertfordshire EN8 9SL

Syndication in 2009

2008-10-02T08:52:00.001-07:00

Had some thoughts on what could be done with Feed Aggregation sites such as http://weblivz.com/ and where it may all go. I decided to consider where we are, the problems we currently have and the inevitable move towards syndication for all.

This is a picture of where we stand today - a very much limited set of pre-defined feeds we can subscribe to (some such as twitter are doing some good work on improving this for their search).

So where are we going? Well, everyone needs to open up - and there are a whole load of good advantages to doing so. Personally, i think the fact we will be able to write complex queries is a major advantage - users are just never going to learn anything more than AND/OR - but we can improve their results by employing the advanced capabilities of search engine logic and API's.

Additionally we want to start using the Semantic Web, but how a user would work with Google Base or FreeBase etc isn't entirely straightforward. It really comes down to either providing an interface they can build upon or writing the queries for them - perhaps in a templated more to substitute their specific query (ala OpenSearch).

Whatever method is used people want RSS/Atom. Feeds just now are starting to get out of control because even a reliable source have a small percentage of data you really want. We now need to be able to write custom queries to request exactly what we want. To join, filter, deduplicate and so on.

I fully expect in the next year or so that the advances that have already started in these areas will only gain momentum and make it a hell of a lot easier for us to get results.

Will be see a "RSS" button on each page of the Google Search results by the end of 2009?

Happy Belated Anniversary Google

2008-09-29T00:49:00.000-07:00

I kinda wish i had spent more time on this the other day but i was busy doing other stuff. Thanks to a link from DeWitt Clinton (who now works at Google) on Google jobs back in 1999, i played around with archive.org for the first time in ages.

People may already have found this, but i had to blog anyway about some gems i found that blew me away.

1. The Google Beta Search Engine in 1998 - link

2. The "Google Friends" mailing list which i suspect was the first public message about Google. It was written by Larry Page himself on 28 April 1998 at 10.28 PM (so must have been a late one!). I love message 5 "Google gets funding" !!

The "Group Info" suggests 1 post a month. I like that either Larry or Sergey categorized the group as "Culture & Lifestyle : Gender : Women, Girls, Mothers, Daughters".

I did actually send a "Happy Birthday" message to "google-friends-subscribe@makelist.com" - it's nice to be nice. We made a big deal of my son's 5th birthday and i reckon i see Google as much as him these days! It hasn't bounced so maybe i'll get a reply ;)

3. This is the first public email (mentioned above) i am aware of by Larry Page himself about Google.

4. There is also the first email from Larry Page and another in July 1998 where he talks about the new features. For anyone stressing out over their servers, here is surely one of the historic paragraphs of the Internet:

Combined "our server" (some suggest there is now > 64,000 servers) and "try back in a minute or two". Come on guys - how did you get funding with that line ;) If this kind of thing doesn't inspire you as an entrepreneur you're maybe in the wrong job!

Look around and let me know if you find other similar gems.

This reminds us these were two guys who were like every other entrepreneur at the start - they had no idea where they would be in 10 years.

Hey, they'll likely never ready this, but Well Done - i consider myself inspired !!

Think i'm starting to really think about writing a book on this kind of thing. Amazing!

PS. I forgot to mention the Google Stickers... check this.

The art of a search query

2008-09-25T03:51:00.000-07:00

One thing that i don't see much coverage over is how non-technical users get the power afforded by advanced querying. Something that drive me to create something LIKE http://weblivz.com/ was through my experiences with querying in a project which involved many hundreds of millions of distributed records.

In that project we had a team over over 20 experts writing specific queries that were semantically relevant to the area they were in and the output of one search was actually a fairly complex backend query most of the time consisting of the union of multiple backend queries all reformatted for a specific output.

Today many sites - and emerging distributed query sites - are focused on simple queries, but this requires that you KNOW semantically what to look for and that you want to type it in all the time. In addition it assumes you know how to construct fairly complex (it's all relevant) queries.

So, YES, we need all these cool sites that do the keyword searching. But we ALSO need something a bit higher level. We need to hide the user from the complexity of searching and also make it easier for them to remember the kinds of searches they either constructed or used before.

A syndication formatting cache

2008-09-24T08:56:00.000-07:00

I'm really thinking about this stuff just now, so this note is as much use for me as anyone else.

We have a ton of sources all working with Atom/RSS formats but being semantically different and in cases extending the same concepts in different ways (e.g. Digg has it's own namespace in its Atom feeds for authors).

Imagine a service that indexed and transformed these sources to normalized formats. So you could basically do XPATH style queries (the interface wouldn't be so complex of course) on the RSS/Atom sources and not only get the data in a given element, but be semantically accurate on what you are getting.

In addition, extension namespaces could also be queried, so you could ask for media items from youtube, flickr, meefedia and so on and get an accurate result.

This service may even be useful if we were all using the same format, but at a time where joining feeds is near impossible when you are thinking about the user, it may be useful to have a service helping out.

I've already written a bunch of tranforms to do this and to be honest had to write an Xslt for every single feed i got (i think delicious was ok), so i know the headache as others look for more advanced syndication feeds!

Long tail of atom extended formats

2008-09-24T05:07:00.000-07:00

I picked up on a post by Andrew Turner on OGC Geospatial Search Summit

“Of course, a format can expand upon this and offer more complex formats that
conform to more complex specs. But by at least providing a common baseline means
that almost any service can easily interconnect with another service.”

I can see why we need to stop at some point! The issue is that in the long tail, these extended formats are quite prevalent and I’d like to see extended communities supporting people who want to extend. The reason I say that is that even in the rich media space I have numerous Xslt’s, function calls and so on to normalize what is essentially the same data. GeoRSS is an example of a specific community that does it well!

My thinking is that if (at the extreme) two companies in the world extended for a very specific topic, we could at least get some normalized view of the data for everyone as a response from an OpenSearch query.

Opening up advanced search syndication

2008-09-24T03:21:00.000-07:00

[update: 24 Sept @ 12:07 PM : Google Chrome also supports OpenSearch]

[update: 24 Sept @ 11:45 AM : Through the OpenSearch group, I see MS IE 8 Beta supports OpenSearch so perhaps even more sites will realize more complex searches are needed]

As you may well now be aware, I just launched an Alpha release of weblivz.com and what I tried to do there is (in many cases) write intelligent query sets against sites that provide the results as RSS or Atom feeds. So rather than just pulling in every feed we can find, we actually create a query such as "Europe and technology" and so on. It really isn’t easy and requires a lot more work that isn't visible up front. Here are the five issues:

1. Search, no feed
2. Query syntax
3. RESTful
4. The commercial clause
5. Semantics of response formats

I will provide one example in each section, but there are many others I have come across.

Search, no feed
In many cases you can get RSS or Atom feeds from static pages, but as soon as it comes to searching and gathering the results as a feed, you’re in trouble. One example if MeetUp.com.

I can do a bunch of querying to get certain feeds but as soon as I want to something such as "languages and Glasgow" I’m out of luck. In short, you get exactly what you want with most search queries on some of these excellent sites, but they only work when you are ON the site.

This misses the opportunity of syndicating the results to third parties to allow them to point at YOUR site. You end up with having to use the limited feeds available and most of the time this isn’t much use to anyone – especially in the era of content overload and the increasing importance of providing the user with what they want.

Requirement
Ensure all the results from your search query can be syndicated as RSS or Atom. The extra queries against your server will be balanced against higher profile or extra hits on your site.

Query Syntax
In short, query syntax is all over the place. In some cases you can only search for one term and in other cases you can’t use AND or OR. There is a real lack of support for doing interesting things – if you really want to customize the feeds, in most cases you are seriously limited by what you can achieve. One example is http://eventbrite.com.

If I want to search for Tech events in Glasgow or Edinburgh I really need to do 4 separate queries – "tech glasgow", "technology Glasgow", "technology Edinburgh" and "tech Edinburgh". This is a waste of resources all round for something that is relatively simple to achieve.

In some cases typing "or" doesn’t mean the same as "OR" and in others typing "and" gets interpreted as part of the querying rather than ANDing the terms.

Requirement
If we are not going to use http://www.OpenSearch.org then the site needs to at least provide a powerful search interface – perhaps more powerful than any individual is going to use, but in cases such as http://weblivz.com we are actively creating powerful queries against your backend database – saving everyone resources!

Give us a REST
For the majority of queries, a RESTful URL should be enough to get the results. Granted many sites already support this but there are others that provide access only through a POST API XML based syntax. This is good for more advanced queries but sometimes you want to write something simple that returns results in a given format, without passing an extended collection of parameters.

Requirement

If I can’t type (something like):

http://www.yoursite.com/search.atom?query=tech+Glasgow

… then you really need to think about adding this functionality.

The commercial clause
Now this is one I simply DO NOT GET. Many sites say the data can’t be used in a commercial context, without really defining what that means. I have some sympathy for this when it is data you have collated and published – such as a postcode search facility or something... syndicating that would render visiting your site close to pointless.

However, when it comes to user generated content I just cannot understand it. Many sites allow searching, feeds and then insert a clause saying you cannot syndicate the data without a license. The point of these feeds however is to provide a "teaser" to bring the people TO your site… so even in a commercial context, surely allowing your feeds to be displayed can ONLY work for you.

http://www.mystrands.com but it’s got "commercial" all over it (it also doesn’t have RESTful URL access to feeds). So do we write YET ANOTHER MyStrands or do they just provide intelligent syndicated search feeds we can all use and drive business to our and their site.

Requirement
Remove this kind of clause.

Semantics of response formats
This particular part almost drove me to distraction. We now have RSS and Atom as the key formats of feeds – sure there are some variations in versions but we are pretty close to two general formats.

So where is the problem with this? Well, the problem is twofold. The first is different interpretation of what goes into each field and the other is the extensions used within the feeds and the variation on how these are semantically interpreted.

The first of these is particularly an issue with "content" and "summary". Some people put in a short description, others put in formatted html. Some don’t’ put a summary and only add content so you need to parse this somehow if you want to display some kind of a summary.

In addition to this you may find some sites (such as FriendFeed) provide much of the information that should be in atom fields (such as the author) embedded within the content so you would need to parse that to give any kind of standard view.

Now, the extensions is altogether more of an issue. Just try combining some of the feeds using the xmlns:media (http://search.yahoo.com/mrss/ ) namespace. Sometimes the link is in the atom elements, others it’s with the media player element, sometimes the author is in the media credit and others it’s in the atom author field.

You need to parse some of these to death just to get a standard output – in fields where the output should really be a specific extension of the core RSS or Atom specifications. This is a nightmare when applied to video, photo’s, music and so on and makes intelligent search and syndication very difficult.

Call to action
We really need to change some of this. It’s not like we need any scientific breakthrough to make this work – we just need to come to some kind of agreement on the points I outlines above – all the difficult technical stuff has already been done. It doesn’t require ripping software – just extending it.

If you provide an Atom feed you may not want to change that but adding a version parameter in an API is easy. That way you can provide the "new" improved feed. Really the best option is to look at http://opensearch.org but there are any number of options I would happily accept.

We also need to generally improve search and syndication and realize it is not something that takes people away from your site, but rather drives them to your site. The better your search APi the easier it will be for sites like http://weblivz.com to integrate your feed with specialist and content sensitive queries. Users will like that and they will come to you through all sorts of gateways!

Please take away the commercial stuff, or at least tone it down. In most cases people will come to your site if they see a teaser of what they want no matter what site they are on.

Feedback
Thanks for reading. This was based on my experiences creating the weblivz website. I’d love to hear feedback – good or bad. If you have pointers or want to point out anything right/wrong or additions, please suggest and I will update the article.

Gazopa - it's fun too

2008-09-10T16:04:00.000-07:00

While Gazopa looked useful and i tried a few searches which i had mixed results with, what REALLY caught my imagination was its ability to allow YOU to draw and look for similar results.

Now, you need to remember I have the artistic ability of a drunk one handed (pawd?) fox wearing a blindfold so it really was a challenge.

Well what do you know, it actually brough something back for my, erm "spider" - and it wasn't too bad - mainly coz it DID actually show a spider (yes, i was as shocked as you are reading).

Look at that last result in the right - a frickin spider. And the others were not too bad - but a 20 points for anyone who can tell me why a LAMP is in there?!

So, being excited i decided to try searching for a more creative drawing. I thought, who will i search for that might come back in their database? Well, @scobleizer came to mind and i've watched enough of his videos to think i could have a good attempt at portrait. As my guide i used a fairly unusual picture of him (at least i'd imagine it's unusual)...

So below you can see my, erm, portrait (1000 apologies) and the results (using a facial search coz the results without it were just ridiculous).

It's perhaps fair to say there is still some work when it gets a little more advanced - but that's ok, coz it was the first thing ever to recognize something i drew!

Javascript object instance methods

2008-09-05T06:46:00.000-07:00

Some other styles i've seen in the creation of instance methods on Javascript objects are as follows:

NS.test = function() {
NS.test.prototype.doSomething = function()
{
...
};
};

NS.test2 = new function() {
this.doSomething = function() {
...
};
};

(function() {
var test3 = function() {};

test3.prototype = {
doSomething : function()
{
...
}
};

NS.test3.doSomethingStatic = function() {
...
}

NS.test3 = test3;

})();
var test1 = new NS.test();
test1.doSomething();
NS.test2.doSomething();

var test3 = new NS.test3();
test3.doSomething();

NS.test3.doSomethingStatic();

Variable scope in Javascript

2008-09-05T05:12:00.000-07:00

I'm from an old skool of JavaScript where it was a nice add-on and everything was done on the server. Sure, i've used all the JQuery, Ajaxy, Prototypey frameworks out there but i never went back to look at the core of the language - well at the moment i'm working on something and writing a lot of script.

So i learned something new. A lot of frameworks were using the following syntax in their libraries..

(function() {
...

})();

It's fair to say i was going mad trying to figure out why. I mean you put an alert in there and it pops up when the script loads. So why bother with the function? Why not just write your code as we used to - directly in the page?

Well, thanks to this paper, I have discovered why and this note is as much for my future reading as yours :)

Turns out that you can locally scope variables within these anonymous javascript functions which more importantly means you don't screw up global vars that are being used elsewhere. So you can confidently write the following ...

var person = 'steven';
(function() {

var person = 'xavier';

alert("Person 1 " + person);

})();

alert("Person 2 " + person);

... and you will get "Person 1 xavier" and "Person 2 steven" - notably the global variable "person" was redeclared within the scope of the anonymous function and the name changed, but as it was declared with "var" it does not overwrite the global value. If you did NOT use "var" in the function, you would change the value globally. So it works like every other programming language - just a wee bit different in term of syntax :)

V8, Chrome and the DOM

2008-09-03T02:36:00.001-07:00

V8 is the Javascript engine for Chrome, the new Google web browser. It is very fast.

V8 does not provide DOM or XML DOM support. This was confirmed to me by a member of the team at the V8 google group.

This leaves me with a question i hope someone can answer.

Most of the scripts i write either use the DOM (getElementById) or the Xml Dom (childNodes[0].nodeValue) and so on. That is a LARGE part of my scripts.

So if the engine does not provide support for this, what happens to my scripts? Do they go really fast until they need to query the DOM and then slow down? The net effect in my case could be a lot of slowing down. I my experience, querying the DOM is the slowest part of your scripts (unless you write non-typical).

Haven't had time (and won't today) to run tests so if you know the answer please let me know.

"Google Suggests ..." - don't use Chrome :)

2008-09-02T13:11:00.000-07:00

I had to put this one up. I was in Chrome and tried to change my pic in the Groups and here's what i got (changeing pic works fine in friendfeed, for example).

my first chrome weird thing

2008-09-02T12:47:00.000-07:00

This may just chance, but the only site i can't get to is webkit.org - which Chrome runs on top of :)

Not only that, it thinks it is "null" (other sites that can't actually be found give a DNS error).

Anyone else get to http://webkit.org ?

Chrome Auto-complete Ideas

2008-09-02T04:27:00.000-07:00

Whilst some of what i have seen from Chrome is what i would expect of a browser of this generation (much of it is in IE 8 too), there are some things such as the V8 Javascript compiler which looks to be very cool. I am only halfway through the book, but wanted to write a post on auto-complete.

Google say Chrome will mean you don't need to bookmark pages any more and will provide an improved auto-complete. Well, here's my thoughts...

"Auto-complete" should be much more expansive. I rarely remembers URL's and even things i have found and bookmarked can be a pain to find again. In fact i tend to bookmark things and never look at them again. So i may be searching for something i have already found. Auto-complete needs to add techniques to discover information you want to keep track of. Tags in a delicious style are an obvious way. Add to that some kind of personal strength and it may be very useful. So i type in "Javascript Cross Domain problem" and get a view of a set of links i previously found useful - coz let's face it, you'll end up going to google, and in the process of a single search modify the query, view certain pages, like some discard others and repeat. In that *session* I should be able to get back to all the pages i found useful. I don't mean caching the pages, i mean the semantic link to those pages.

Can you imaging a little package of links you found useful related to a search concept that you can reuse in the future. I would LOVE this. So where Chrome shows a new tab with the top links it it, the auto-complete concept would extend to the search package i discussed above.

ps. stuff such as gears, prototype and so on should be part of the environment itself... maybe that is later in the comic :)

pps. Is it true that the Joker appears in chapter 5??

Friendfeed custom display

2008-08-29T02:58:00.000-07:00

A while back i changed my homepage to be a custom view of my friendfeed activity so you could see what i was doing all day long. The only thing i don't like is that it puts my most recent items up which is great, but it means i have lots of twitter posts as i'm very active in there. You don't see my most recent images, videos and so on. I should have posted this ages ago, but i didn't, so apologies.

Here you go if you find it useful...

1. Put in the head element ...


    <script>

    function ParseFriend(feedid)
    {
        var prefeed = document.getElementById("prefriendfeed"+feedid);
        prefeed.style.display = 'none';
        
        var feeditem = document.getElementById("feed"+feedid);
        var divs = feeditem.getElementsByTagName("div");
        for(var i=0;i<divs.length;i++)
        {
            if (divs[i].className == "header") divs[i].style.display='none';
            
            if (divs[i].className == "friendfeed")
            {  
                var subdivs = divs[i].getElementsByTagName("div");
                for(var j=0;j<subdivs.length;j++)
                {       
                    if (subdivs[j].className == "feed")
                    {
                        subdivs[j].style.border='10px';
                        subdivs[j].parentNode.parentNode.style.display = '';
                    }
                }
            }
        }        
    }

    </script>

2. Place on your page and customize ...



            <div id="friendfeed" style="float:left;width:600px;">
                <div>
                      <a href="http://friendfeed.com/weblivz"><img src="images/friendfeed.gif" style="vertical-align:top;" border="0" /></a>
                      <a href="#feed101">location</a> 
                    | <a href="#feed1">blog</a> 
                    | <a href="#feed2">twitter</a> 
                    | <a href="#feed6">favourites</a> 
                    | <a href="#feed5">videos</a> 
                    | <a href="#feed3">photos</a> 
                    | <a href="#feed4">music</a>
                    | <a href="#feed7">books</a>
                </div>
                <br />
                <div name="feed101" id="feed101" style="float:left;margin-left:20px;"><img src="images/plazes.jpg" border="0" alt="" /> </div><div style="margin-left:40px;"><object type="application/x-shockwave-flash" data="http://plazes.com/flashes/badge.swf" width="316" height="146"><param name="movie" value="http://plazes.com/flashes/badge.swf" /><param name="allowScriptAccess" value="sameDomain" /><param name="swLiveConnect" value="true" /><param name="wmode" value="transparent" /><param name="FlashVars" value="key=fcffc504b834fcd539f4310515a31d5e&amp;dark=7cd9f7&amp;light=ff9900&amp;text=000000&amp;link=ffffff" /><p><strong><a class="external" href="http://www.adobe.com/">Download Flash plugin</a></strong></p></object></div>
                <div id="prefriendfeed2" style="float:left;width:600px;"><img src="images/loading.gif" style="vertical-align:middle;" /> loading livzstream tweets ...</div>
                <div name="feed2" id="feed2" style="display:none;"><script type="text/javascript" src="http://friendfeed.com/embed/widget/weblivz?num=2&service=twitter"></script></div>
                <script>ParseFriend(2);</script>
                <div id="prefriendfeed1" style="float:left;width:600px;"><img src="images/loading.gif" style="vertical-align:middle;" /> loading livzstream blogs ...</div>
                <div name="feed1" id="feed1" style="display:none;"><script type="text/javascript" src="http://friendfeed.com/embed/widget/weblivz?num=2&service=blog"></script></div>
                <script>ParseFriend(1);</script>
                <div id="prefriendfeed6" style="float:left;width:600px;"><img src="images/loading.gif" style="vertical-align:middle;" /> loading livzstream favourites ...</div>
                <div name="feed6" id="feed6" style="display:none;"><script type="text/javascript" src="http://friendfeed.com/embed/widget/weblivz?num=2&service=delicious"></script></div>                
                <script>ParseFriend(6);</script>
                <div id="prefriendfeed3" style="float:left;width:600px;"><img src="images/loading.gif" style="vertical-align:middle;" /> loading livzstream photos ...</div>
                <div name="feed3" id="feed3" style="display:none;"><script type="text/javascript" src="http://friendfeed.com/embed/widget/weblivz?num=2&service=flickr"></script></div>
                <script>ParseFriend(3);</script>
                <div id="prefriendfeed5" style="float:left;width:600px;"><img src="images/loading.gif" style="vertical-align:middle;" /> loading livzstream videos ...</div>
                <div name="feed5" id="feed5" style="display:none;"><script type="text/javascript" src="http://friendfeed.com/embed/widget/weblivz?num=2&service=youtube"></script><script type="text/javascript" src="http://friendfeed.com/embed/widget/weblivz?num=2&service=seesmic"></script></div>
                <script>ParseFriend(5);</script>
                <div id="prefriendfeed4" style="float:left;width:600px;"><img src="images/loading.gif" style="vertical-align:middle;" /> loading livzstream music ...</div>
                <div name="feed4" id="feed4" style="display:none;"><script type="text/javascript" src="http://friendfeed.com/embed/widget/weblivz?num=2&service=lastfm"></script></div>
                <script>ParseFriend(4);</script>
                <div id="prefriendfeed7" style="float:left;width:600px;"><img src="images/loading.gif" style="vertical-align:middle;" /> loading livzstream books ...</div>
                <div name="feed7" id="feed7" style="display:none;"><script type="text/javascript" src="http://friendfeed.com/embed/widget/weblivz?num=2&service=amazon"></script></div>                
                <script>ParseFriend(7);</script>

3. Put at the end of your web page the following ...


<script>ParseFriend(1);ParseFriend(2);ParseFriend(3);ParseFriend(4);ParseFriend(5);ParseFriend(6);ParseFriend(7);</script>

Make your API's easy to play with

2008-08-23T03:37:00.000-07:00

I'm looking at advertising API's and i've got a headache. Got a nice list from here, but ALL i want to do right now is test them out. I'm sitting here with my code calling some Amazon Web Services (excellent) and other simpler services but the Ad one is driving me nuts. Google's was way complex and involved signing up all over the place. Yahoo required me to be processed. Again Microsoft's looks fairly involved. (Update) Didn't realize that i need to use AdSense and not AdWords to actually get suggestions ! Never saw any pointers at the top of the AdWords. Anyway AdSense is a little better but it seems to return only HTML which kinds defeats the purpose of an API. I want to embed it my own way!

I want to pass some data and get some ads - maybe do something more in the future. More importantly i just want to test this just now. I don't want to commit to anything but things need to be simpler. I don't wanna spend hours registering, being verified and then coding against something far more complex than i need. I don't mind the complexity, but when i can't immediately see (and play with) what i need to do then, to me, it's complex.

Please - all - provide a sandbox for hacking. Make it open with an API key easy to get which allows X number of requests. Make the API simple - most could be RESTful gets for that. Make the reponse simple, but customizable.

I know from personal experience these things get complex quickly, but we need API's we can quickly run with.

Needz twitter

2008-08-21T13:21:00.000-07:00

As an experiment i've created an account on Twitter when you can add and request things ou need, such as positions to be filled, a job, directions and so on.

If you need something, send a message to @needz at http://twitter.com/needz

Reading from Freebase using C#

2008-08-19T10:22:00.000-07:00

Update : Seems Freebase no longer requires the cookie authentication - comment out that line and it works fine. Cool.

Freebase is one of my favourite things EVER on the web.

I'm working on a project just now and need to read from FreeBase. Alas I couldn't find any C# examples (if you know of any please let me know). As FreeBase is in early stages it required Cookie authentication - they say they will open up in the future.

Anyway, here is the code that does the same thing as the Perl, PHP and JavaScript code on this Freebase page. Enjoy :)


using System;
using System.IO;
using System.Net;
using System.Text;
using System.Security.Authentication;

namespace FreebaseSample
{
    public class FreeFind
    {
        private static string server = "http://www.freebase.com";
        private static string queryurl = "/api/service/mqlread";
        private static string path = "/api/account/login";
        private static string username = "";
        private static string password = "";
        private static CookieContainer authCookies = null;



        public void Ask(string band)
        {
            // This is our MQL query 
            // Find a band
            // With the specified name
            // We want to know about albums
            // Return album names
            // And release dates
            // Order by release date
            string query = "{\"type\": \"/music/artist\",\"name\": \"" + band + "\",\"album\": [{\"name\":null,\"sort\": \"release_date\", \"release_date\":null}]}";

            string envelope = "{\"a0\":{\"query\":"+query+"}}";
            string url = server + queryurl + "?queries=" + System.Web.HttpUtility.UrlDecode(envelope);

            HttpWebRequest request = (HttpWebRequest)System.Net.WebRequest.Create(url);
            request.Method = "GET";
            request.ContentType = "application/x-www-form-urlencoded";
            request.CookieContainer = authCookies;

            HttpWebResponse response = (HttpWebResponse)request.GetResponse();
            StreamReader reader = new StreamReader(response.GetResponseStream());

            Console.WriteLine(reader.ReadToEnd());
            reader.Close();
            response.Close();
        }

        /// <summary>
        /// Authentication.
        /// </summary>
        public void Authenticate()
        {
            string authURI = server + path;

            // Create the web request body:

            string body = string.Format("username={0}&password={1}", username, password);
            byte[] bytes = Encoding.UTF8.GetBytes( body );

            // Create the web request:

            HttpWebRequest request = (HttpWebRequest)System.Net.WebRequest.Create( authURI );
            request.Method = "POST";
            request.ContentType = "application/x-www-form-urlencoded";
            request.CookieContainer = new CookieContainer();
            request.ContentLength = bytes.Length;

            // Create the web request content stream:
            using ( Stream stream = request.GetRequestStream() )
            {
                stream.Write( bytes, 0, bytes.Length );
                stream.Close();
            }

            // Get the response & store the authentication cookies:

            HttpWebResponse response = (HttpWebResponse)request.GetResponse();

            if ( response.Cookies.Count < 2 )
                throw new AuthenticationException( "Login failed. Is the login / password correct?" );

            authCookies = new CookieContainer();
            foreach ( Cookie myCookie in response.Cookies )
            {
                authCookies.Add(myCookie);
            }

            response.Close();
        }
    }
}

You can call it as so ...


using System;

namespace FreebaseSample
{
    class Program
    {

        static void Main(string[] args)
        {
            FreeFind f = new FreeFind();
            f.Authenticate();
            f.Ask("U2");
            Console.ReadKey();
        }


    }
}

You get something like the following JSON response ...

{
  "status": "200 OK",
  "a0": {
    "code": "/api/status/ok",
    "result": {
      "album": [
        "Three",
        "Another Day",
        "11 O'Clock Tick Tock",
        "A Day Without Me",
    .
    .
    .
        "U2 Popmart - Miss Sarajevo (disc 2)",
        "Unplugged",
        "Zoo TV Tour (disc 2: Live Transmission)",
        "Shaped",
        "The Complete U2",
        "Wide Awake in America"
      ],
      "type": "/music/artist",
      "name": "U2"
    }
  },
  "code": "/api/status/ok"
}

Large Hadron Collider Nightmare scenario

2008-08-02T10:32:00.000-07:00

The pictures of the large hadron collider look amazing. Other than pulling us all into a black hole, surely scientists who wrote the software dread the following ....

c:\>run large hadron collider.exe

Error at line 4. The 'Higgs Boson' is not a recognized particle. Please try again.

Wall-E

2008-07-18T06:45:00.000-07:00

Son went to see Wall-E today. Complete waste of time ...

All starts off very nice in a research lab, but soon Microsoft come in and buy him, henceforth known as Wall-XE to support their “A robot in every home” initiative (IBM missed out pronouncing that there will be “only need for one robot in the world”). Then a bunch of other folks create an Open Source version (Wall-FREE) but no-one is around to support it and there ends up 800 versions of it. Google released their own “Wall-G beta” funding it through its new advertising platform Robo-Words (which no-one ever reads of course and so goes down the pan… but despite being perpetually in beta it DOES read your Gmail and integrate with every appliance in your kitchen). It finishes by being released on Bit Torrent and now you just download and install the software to have your own.

Sorry for spoiling it for everyone.

* Please note i haven't seen it - my son is watching it right now... and it looks , really good!

Powerset does, FAST is

2008-06-27T13:10:00.000-07:00

I haven't used Powerset much. I have used FAST a lot. I used their API fairly extensively. I could write a book about that (something i have considered - see my "previous life"), but i won't go into all that just now.

What i DID like when i used Powerset was the "factz" (in itself a slightly ironic spelling). You see FAST is all centred around Nouns - people, places, things, authors, books and so on. This works pretty good. (So for example) You search for Steven Livingtone and you get my books, my friends, my bookmarks, my relations and so on - nouns related to me - in facets.

However, what i liked about Powerset what the fact the use verbs in the facts. So again i search for Steven Livingstone (someone has to) and i maybe get "writes", "likes", "bookmarks" and so on - well something to that effect.

Unfortunately i'm not entirely sure how to get these factz appearing (you seem to need to do the search and then select something and even then it isn't always verbs), but i DO like the idea of making a search and combining the doing with the object.

"Steven Livingstone writes MSXML" and so on. One part powerset, one part fast.

I'm still looking into Powerset so things may become more obvious to me.

haircut

2008-06-27T06:21:00.000-07:00

Got my haircut - what's best?

Livz in the past

Livz in the present

10 startups to watch

2008-06-27T00:31:00.000-07:00

I read the list over at Technology Review and here's my take:

pinger.com
Nice idea - great idea in fact. Got an account but the software isn't available outside the US and Canada, but the number is (i would have thought it would have been the reverse!). As I am hopless at numbers and baring in mind when you are ON the mobile phone it's impossible to look up someone's number (you tried it?) - i am going to wait until i can use the software.

I tried their Outlook add-in, but it never seemed to do anything. Needs work.

qik.com
I just keep wandering why it took so long for Comvu to really push their offering.

Ushahidi.com
Another good idea which could be a platform for hyperlocal citizens jourlalism anywhere. Probably needs branded a bit but should be interesting to follow.

reQuall
Anything that helps me remember stuff is good - i like the idea but the interface was a little confusing and when i dialled the number it asked for for a pin- i gave it my mobile and added something but it then said "thanks for the comment for our support team". The UI was also a little confusing but will improve.

33Across
Tells you who is influential in the world. No idea how good it is. No signup i could see or way of registering interest which was slighlty ironic given its remit.

Peer39
I love anything semantic (but i bet it becomes the most overused word after Web 2.0 soon!). Advertising sure needs to get smarter and take in context. I'm not an advertiser so not sure how useful it is, but their list of publications looks a good read!

http://www.peer39.com/publications.html

Could be huge.

Mashery
Now this is interesting. It's not a million miles from something i am looking at just now (maybe a few thousand tho') and managed API's are important for enterprises. I'll be keeping my eye on these guys. Very interesting to see how mashups take a step inside the firewall!!

Great list. The other companies looked great too, but these are the ones that i could play around with and are closer to what i do.

Live Poll - Will Germany or Turkey win?

2008-06-25T11:57:00.001-07:00

Take a look at this:
http://sodahead.com/question/106797/

RESTful WCF Web Services

2008-06-25T02:36:00.000-07:00

I spent some time looking at how you could remove the ".svc" from WCF web services and looking at it last year i got quite close but just not close enough. Well, i'm back writing WCF services again to expose JSON for my REST API, and this time i was determined to spend time looking for a solution. I found some people who did a lot of great work but the solutions were all missing the extra i really needed. I needed an extensible method of re-writing any kind of URL.

The solution was to combine the ideas here and here using a mixture of IHttpModule and regular expressions to provide a solution allowing any URL to be mapped in the configuration file. The code is below and is made up of RestModule.cs and some rules in the web.config.

This code will actually work for any URL you wish to make RESTful but my target was WCF.

RestModule.cs


using System;
using System.Configuration;
using System.Web;
using System.Xml;
using System.Text.RegularExpressions;
using System.Xml.Xsl;

public class RestModule : IHttpModule
{
    public void Dispose()
    { }

    public void Init(HttpApplication app)
    {
        app.BeginRequest += delegate
        {
            HttpContext ctx = HttpContext.Current;
            string path = ctx.Request.AppRelativeCurrentExecutionFilePath;

            if (ctx.Request.Url.Query.Length > 0) path = path + ctx.Request.Url.Query;
            string zSubst = ReWriter.Process(path);

            //did we have a new path match?
            if (!zSubst.Equals(path))
            {
                int i = zSubst.IndexOf('/', 2);
                if (i > 0)
                {
                    string svc = null;
                    string rest = null;
                    string qs = null;

                    svc = zSubst.Substring(0, i);
                    rest = zSubst.IndexOf('?') > -1 ? zSubst.Substring(i, zSubst.Length - i).Split('?')[0] : zSubst.Substring(i, zSubst.Length - i);
                    qs = zSubst.IndexOf('?') > -1 ? zSubst.Split('?')[1] : null;

                    ctx.RewritePath(svc, rest, qs, false);
                }
            }
        };

    }
}

public class ReWriter : IConfigurationSectionHandler
{
    protected XmlNode _oRules = null;

    public string GetSubstitution(string zPath)
    {
        Regex oReg;

        foreach (XmlNode oNode in _oRules.SelectNodes("rule"))
        {
            oReg = new Regex(oNode.SelectSingleNode("url/text()").Value, RegexOptions.IgnoreCase);
            Match oMatch = oReg.Match(zPath);

            if (oMatch.Success)
            {
                return oReg.Replace(zPath, oNode.SelectSingleNode("rewrite/text()").Value);
            }
        }

        return zPath;
    }

    public static string Process(string requestpath)
    {
        ReWriter oRewriter = (ReWriter)System.Configuration.ConfigurationManager.GetSection("system.web/urlrewrites");

        return oRewriter.GetSubstitution(requestpath);
    }

    #region Implementation of IConfigurationSectionHandler
    public object Create(object parent, object configContext, XmlNode section)
    {
        _oRules = section;

        // TODO: Compile all Regular Expressions

        return this;
    }
    #endregion
}

Add a new section group to your web.config ...


<sectionGroup name="system.web">
   <section name="urlrewrites" type="ReWriter"/>
</sectionGroup>

Finally add some rules in the web.config in the system.web section ...


        <urlrewrites>
            <rule>
                <url>/MapService/GetLocation\?(.*)</url>
                <rewrite>/MapService.svc/GetLocation?$1</rewrite>
            </rule>
            <rule>
                <url>/MapService/GetLocation/(.*)</url>
                <rewrite>/MapService.svc/GetLocation?target={"Name":"$1"}</rewrite>
            </rule>
        </urlrewrites>

Now you can have ...

http://site.com/WebSite/MapService/GetLocation?target={"Name":"Starbucks%20Manhattan"}

or ....

http://site.com/WebSite/MapService/GetLocation/Starbucks%20Manhattan

Neat heh?

Vint Cerf @ Sharkfest 2008

2008-06-18T01:41:00.001-07:00

<div><a href='http://www.omnisio.com'>Share and annotate your videos</a> with Omnisio!</div>

social web, without the kids

2008-06-17T04:21:00.000-07:00

I listened to an interesting question at SuperNova that was put to Clay Shirky around the SPAM issues around Web 2.0.

He answer was based around reactive solutions managed by the crowd. That is YOU don't manage the comments etc but rather the members of that community do this for you. Which works great in the adult world (most of the time).

I DO have issues around this - more and more i am beginning NOT to trust the crowd. No necessarily the people i know, but rather the way that companies tend to promote media created by the crowd.

YouTube for example will show many videos that just should NOT be seen by kids (and some of them by adults!) by default until someone flags this. A recent experience taught me this. So, someone - or likely a number of people - need to flag a video before it is removed ... which is just too late when all you wanted to do was show your kid a "Puff the Magic Dragon" video.

How could this work better? Well, for a start ONLY show videos recommened by specific friends - all others are not seen by default. Links from videos work in a similar manner. Same with comments.

This stuff isn't hard and with OpenSocial i'd expect youtube to be making this higher priority than allowing MORE comments on videos.

It concerns me that the younger generation are being ignored in the social web and i'd like to see a hell of a lot more effort put into it than PICS which the social web seems to completely ignore.

Balancing act

2008-05-31T10:20:00.001-07:00

Balancing act
Originally uploaded by weblivz

Xavier and Francisca practicing for a future life in the circus !

Fingers in many API's

2008-05-28T08:49:00.000-07:00

Firstly, I love API's. When you don't have an API on the web you lock much of the data users may wish to move around and share.

I have previously documented my concerns with the API/Data issues when all this data is stored (ala Google maps, facebook etc) in write-only API's and getting it back has a whole bunch of legal (often non-commercial) issues associated with it.... even tho it's YOUR data.

However, let's move away from the legal stuff for a few minutes. There is some concern at the Yahoo Maps API may have changed somewhat http://snipurl.com/2b2q4 and more than a few users are suddenly experiencing issues with it and don't really know where to go next.

THIS is where i really DO see issues with API's on the web. You write a nice map annotation system, store all the data but suddenly the mapping API you are basing this content on breaks... and with it your business is screwed. This is of course not only an issue in mapping - it extends to any business using a 3rd party API as a core part of its application.

I had commented on a post by Dave Winer on data portability that ...

You DO need API's but they need to be balanced in a distributed sense so
you can get to your data through a common API no matter where it is. Sure,
Amazon is a solution, but in a pure web sense it would be nice if your data were
encrypted and stored in a quasi distributed fashion with storage servers YOU
decide to trust. Complex, but i don't want one person owning all my data.

In other words, i don't mind the API's as they let me get to the data... my issue is that i believe the data itself should be separate from the API's and the data should be distributed. Sure, it's a nirvana, but i'd like services to be the cool stuff that makes you use a product and not the data it holds. So if two mapping services agree on a baseline common annotation API, then i can easily switch between them if required - both of them sharing data but perhaps doing slightly different things with it.

Amazon has great web service storage services, but the first thing i did was wrote my own API that translates my requests to Amazon requests. It means in a second i can change the storage server i am using. Data replication is trickier but achievable.

We are trying out some ideas at http://openid.org - in the lab at the moment but we'll see how things progress, but i will be exploring how to provide an API that allows users to move the data they have anywhere they wish, even FOAF files are a nice start!

Customer Support - they have it wrong

2008-05-26T08:22:00.000-07:00

I've recently signed up for Pipex Broadband, a service i have been using for some years (but had to changed because i moved house). It requires BT to do some work on their end and the waiting is now at 7 days (it was typically two before).

So i email customer support from the email account i used in the first place and get the reply:

In accordance with data protection laws, so that we can access your Pipex
account and advise you on your query, please forward the following information:

- Your Full name
- Your Pipex account number
- Your date of birth
- Two digits from your customer care password (If applicable)
- Your Pipex phone number
- Your address and postcode
- Your bank name

So i replied saying that with this information i a plain text email it could easily be intercepted and if so, all of the above information would be enough for someone to perform fraudlent activities as me! In short, this is insane - and BT had asked me for the same information - and others too. All this despite the fact i'm not asking THEM to send me any sensitive information (other than the status of my order).

They replied, again asking me for this data to "access" the account (which we all know is bullshit).

This is seriously wrong. There are two ways to fix this.

1. If you send from the email you originally signed up with, providing the correct reference number then you should get a response (assuming you aren't asking for anything too sensitive).

2. A private, encrypted area where you can review and send messages to the support team. Surely this is an obvoius solution to the place text email issues we have.

In short, you can't say that for data protection issues please send (in plain text) all the data you are trying to protect!

Tommy Burns - Rest in Peace

2008-05-20T14:56:00.000-07:00

Today myself and the family went early to Celtic Park to pay our respects and say goodbye to Tommy Burns - one of the most commited Celtic players of my lifetime.

I used to watch him with my Grandad on the terracing of Parkhead - Paddie Bonner, Tommy Burns, Roy Aitken, Danny McGrain (who we met at Celtic Park on Sunday) and more - Tommy Burns being particularly memorable for me as he presented me with my first football medals as a 10 year old boy. Even then the thing that stood out (other than his head of hair!) was how pleasant and unrushed he was, despite the demand of his time.

I was particularly proud of my son Xavier who, as when we did the same with Jimmy Johnstone, asked loads of questions and even held his scarf aloft for You'll Never Walk Alone. I was the same when i watched Tommy as a player.

It was also poignant for me as from where i stood i could see Tommy on the big screens in the stadium. I don't mind saying I shed a tear for the great man. It's sad but i know he is up there watching and still nervous about Thursday.

When all is said and done, i can be very proud that i have the chance to meet and chat with a true Celtic legend. It's amazing how big you realize the club is when something like this happens - people are part of the club not only when alive, but for a hell of a long time after they have gone ... you realize this when on the tour.

Tommy will now be another of those. If you know your history.

Twists and Turns, Tommy Burns - gone but never forgotten.

OpenID

2008-05-16T13:10:00.000-07:00

So i now have 15000+ people registered at http://openid.org

We are having some issues with a few sites but most are working great.

I put the source to the portal over at http://code.google.com/p/dotnetopenidprovider/

This is based on the excellent OpenID library at http://code.google.com/p/dotnetopenid/

I'll be updating the provider in the coming weeks, so please download, contribute and if you speak another lanugage, you can help out there too!

Oh, What a night

2008-05-11T04:33:00.001-07:00

Last night myself and my wife Loreto went to see Oh What a Night, a musical production starring Kid Creole.

Fantastic night, some excellent music and afterwards we went for a drink and the crew happened to be at the other table across from us too.

It's a different way of life (my cousin in Ireland works for an Irish band and tours constantly so i know what it is like), but it must be amazing to see 400 people going crazy!

Comback Trail

2008-05-11T04:26:00.000-07:00

Not posted for quite a while for fairly good reasons. Took a bizarre reaction to some antibiotics - long story i won't go in to. It's being looked into anyway - i may post more... very frustrating period.

Still, been working for NHS developing and architecting their FAST search environment. Very powerful technology. More on that soon too.

Good to be back!

Interop with Membership Provider

2007-12-12T00:03:00.000-08:00

I have a situation where I have been using the Membership provider and wish to migrate to a custom format for the re-work i am doing on OpenID. There is no great reason for not using their structure other than it has a bunch of stuff I really don't need nor will be implementing.

My issue was that the user passwords are SHA-1 hashed and i wanted to simply migrate that data and provide the same implementation so that I don't have hassles checking password or creating new password.

The code below will do that (except for encrypted passwords).


using System;
using System.Text;
using System.Security.Cryptography;

public enum PasswordFormats
{
    Clear,
    Hashed
}

public static class MembershipManager
{
    public static bool IsMatch(string pass, string salt, string encodedPassword)
    {
        string newencoded = EncodePassword(pass, salt);
        return newencoded.Equals(encodedPassword);
    }

    public static string EncodePassword(string pass, string salt)
    {
        return EncodePassword(pass, PasswordFormats.Hashed ,salt);
    }

    public static string EncodePassword(string pass, PasswordFormats format, string salt)
    {
        if (format == PasswordFormats.Clear)
            return pass;

        byte[] bIn = Encoding.Unicode.GetBytes(pass);
        byte[] bSalt = Convert.FromBase64String(salt);
        byte[] bAll = new byte[bSalt.Length + bIn.Length];
        byte[] bRet = null;

        Buffer.BlockCopy(bSalt, 0, bAll, 0, bSalt.Length);
        Buffer.BlockCopy(bIn, 0, bAll, bSalt.Length, bIn.Length);
        HashAlgorithm s = HashAlgorithm.Create("SHA1");
        bRet = s.ComputeHash(bAll);

        return Convert.ToBase64String(bRet);
    }

    public static string GenerateSalt()
    {
        byte[] buf = new byte[16];
        (new RNGCryptoServiceProvider()).GetBytes(buf);
        return Convert.ToBase64String(buf);
    }
}

The dead web - Google + Archive.org?

2007-12-11T01:03:00.000-08:00

I have put this into this new blog from my old one where i originally posted this (actually that was also a re-hash of the same idea i wrote back in 2002, so been on my mind for a while!) a while back due to Dave Winer's recent post. He is spot on!

====

This was something i posted on an old blog some years back, but recent discussions have made me re-post just in case there is some new opinion!

Over the last 2 months I have been conducting research almost exclusively on the web.
What has really became obvious to me is the amout of dead material out there.

From web pages, that contain out of date information, to whole sites that stopped running years ago with no indication, to projects that seem to have been in flux for years, businesses that stopped trading years back and left their site on and even stuff written by people whom i'm almost read to email, only to find the passed away a couple of years back.

So is a new web needed to get us out of this? Can we see Google work with Archive.org and create a diary of the web? A time-aware searcheable web which allows some kind of time scale on the information out there, without requiring everyone to annotate their documents! Could i say "Only search content added/updated in the last year" ?

I hope so, because frankly it's getting ridiculous. 10 years ago i did some research on solitons for a Physics paper i wrote. Today some of that material returns seelingly as relevant as ever despite things continuing to evolve over the last decade. My File Exists article on 15 seconds at http://www.15seconds.com/issue/990401.htm is now over 5 years old, but still comes 8th in Google when i type "FileExists".

I don't know how many replies i have had indicating some academic moved on 3 years ago, or some project research was finished, or even links to other sites that closed their doors, re-organized or just changed their content to make it completely useless.

Could a hyped up archive.org challenge something like Google? I think so. Could we "Diff The Web" to make the content more relevant - noting that getting dublin core on everything is highly unlikely.

Anyone got answers?

Feelz - search with context

2007-12-08T03:46:00.000-08:00

Today sees the launch of an experimental idea i've created called Feelz at http://feelz.org

The idea is to allow you to label things on the web with emotions, audience, characteristics and so on - things that aren't easily understood by computers. It will be particularly useful for audio, video and products but through concepts such as "audience", can easily extend to the web in general.

I'm really interested in feedback about this and what direction you think it should or should not take.

You can find the blog at http://feelz.wordpress.com

Le Web and Techcrunch - 1 in 53

2007-12-04T09:35:00.000-08:00

At least check, there were 53 competing posts on the competition to win a ticket to Le Web next week. I am one of them.

So, to build my confidence i'd figure out what else has a 1 in 53 chance? ....

"The current probability of a pro-union worker being fired"

"there is a 1 in 53 chance of a Joker being dealt on the initial deal each game [of Jokers Wild]"

"[a black boy has] a 1 in 53 chance of becoming a teacher"

"... each [space] shuttle mission has a 1 in 53 chance of not making it back"

Ah crap, it just went to 54 comments :(

How far does collaboration stretch?

2007-12-03T06:08:00.000-08:00

A few years back I worked at a large government enterprise that attempted to implement a knowledge system that expected users to enter titles, descriptions, hierarchical categories, privacy and so on ...

.. it didn't work very well. The argument i heard was that too much was expected of the users who had to enter all this data - they had too many other things to do.

And THEN (roughly speaking) we moved into tagging (delicious kicking this off) and so on - really the first phase of proper online collaboration. This became popular with Tag Clouds (all Scottish ones had rain as a background of course) and so on being implemented by every single web page on the planet.

Of course the last 2 years or so has seen a movement beyond this to real social collaboration - much of it to this point about ourselves (save specifics such as blogging and wikipedia which is a very small proportion of the online community). The interesting thing (and i have noticed this mainly through non-tech family and friends) is that everyone is using linked-in, geni, facebook and so on. And they all required a LOT of work.

Furthermore, sites such as Digg etc are very easy for users to make a statement - pretty much a click and Google seem to be looking at adding similar features.

Here is my question though. How far will this go? We see Mahalo looking at something akin to where Yahoo started out many years ago - human managed information... although now there was is a much longer tail in content and so you needed a LOT of editors - something that is definitely getting easier!

There is also Freebase and Google base to name but a few. These all expect a lot more from the users than anything before - well at least in 7 years.

Will this fail again or are we just at the right time for this kind of user involvement to work? Yes I know the stats about we only need X users out of everyone to contribute, but the more content that is added, the more the authors and those with a vested social interest have seemed to become involved.

There was ALWAYS the issues of creating structures to hold metadata (such as Dublin Core and RDF) but so long as this stuff is abstracted in a nice UI, do we think it can now work in a distributed social environment? In other words, people seem to now be working harder (i.e. actually adding this metadata which in the past was always empty... ask anyone who ever maintained a database of it!) - is this a long term thing or are we just in a social high?

Will this extend to micro-content and the enterprise?

I do have a vested interest as i'm working on an experimental idea i wish to release in the next few days, so please post comments as they will be very useful!

OAuth for API Authentication

2007-12-01T13:26:00.000-08:00

I've defined a number of API's in my time and always seem to resort to a different kind of authentication mechanism and my latest API was likely to be no different. I looked at Flickr, Google and Amazon API's and how they did authentication and they all uses customer MD5 and/or HMAC-SHA-1 hashing with a private key with different parts of the query being hashed.

I thought, there should be a standards around this ... i had looked at OAuth but from an API veiwpoint never really made the connection. Until today! I am looking at a C# implementation by Eran Hammer and i see there is some work on it, but I need to do some more research to find out what i need to provide on top of this to actually implement it (i.e. client and server requirements).

I hope to learn more about what i need to do to just use this in my API and in particular how i may integrate the authentication in an AJAX client application without redirecting to the site.

Scotland to raise your management profile

2007-11-27T23:59:00.000-08:00

So it happened. I think when I first heard his "anything can happen" spiel I knew it was inevitable. The Scotland Manager (left) Alex McLeish has resigned.

He has moved to ~~Real Madrid AC Milan Manchester United~~ Birmingham!! Now I lived in Birmingham and feel the place gets a hard deal from people who haven't been there, but is McLeish for real?

Are we now at the point where someone who i would have regarded as a REAL Scottish manager - and I remember him from his playing days - is happy to dump the team after 10 months in charge??

Indeed Alex :"To be manager of your home nation is a very proud day indeed for Alex McLeish, my family and friends". *

[* footnote : Unless more money comes my way]

Walter Smith did something similar before, but although i don't think it was the right thing to do, moving to Rangers is probably the only thing that would have made his leave his post. But Alex McLeish - a Scotland "great" has left after 10 months.... and i know how short a time that is as my daugter is only 11 months old.

He has basically used the National team to springboard to a higher paid job. Don't tell me this isn't about money - i mean he must be really badly off after managing Rangers. Come on Alex - you could have been a huge figure in this country by guiding team to the next World Cup in South Africa... he even managed to screw us just a little bit more by even attending the DRAW in South Africa!

It pisses me of that these guys are always talking about player loyalty and yet this is the most obvious case of UNLOYALTY i have seen in years. 10 months. I could understand it if were were an awful team but we have progressed pretty well in recent years - which bring me to my next point....

If anything positive comes from this, it is that we will now see that we have a team pretty much independent of manager. Smith and McLeish did a good job obviously but (and I wish i had recorded this) I said a number of years back that we would have a decent team as you could see the young players coming through the ranks at club level. This was during the Berti era. It came as no shock to me that we started getting results. Vogts was a terrible man motivator which is what you got from the Scottish lads, but it's important to remember that it was under him that this young team came together.

My view is that we simply need someone passionate and let the team run itself. I have a 4 year old and once he realized he could read the alphabet he started making up all kinds of words - only every so often does he need to come to me for help.

And don't kid ourselves we run on passion - sure we have passion, but the second half against the Italians shows us we have no skill shortage either!

WhoIs On Seesmic

2007-11-27T09:24:00.000-08:00

I was interested in finding out a bit more about who is on Seesmic - so i asked people to make short videos of who there are and where they are from ... we have three thus far - hopefully get more!

You can see who is on Seesmic by hitting the following :

http://livz.org/seesmic/whois

If you wish it text format you can hit the following:

http://livz.org/seesmic/whois?f=txt

Hopefully a few more will join so we can follow who is on there! Just add "whois" to your Seesmic video subject line!

Gregg Brockway at Web 2.0

2007-11-27T08:44:00.000-08:00

I just watched the presentation given by Gregg Brockway of http://tripit.com at the Web 2.0 Conference last month on Blip.tv.

There is a part where his phone went off and he said it was his wife. It was really well done (despite the audio folks needing a prod) and having been in similar situations I actually thought his wife was maybe having a baby! Of course it was him demonstrating how simply someone could be directed to use the service on a mobile via email.

It was quite an original way of presenting and demonstrating an idea. Worth taking note.

Embed Seesmic in your blogs and so on

2007-11-27T03:09:00.001-08:00

So you use Seesmic and want to have the latest version of your videos, or other videos in your blog without having to copy, paste and edit each time. You can do it now.

There are some options. Whether you show the video preview or a text link is based on the querystring parameter "f" which takes the value "vid" or "txt".

So, say i want to just show a text list of the latest (20 by default) videos - i put this in my html page:

<script language="JavaScript" src="http://livz.org/seesmic?f=txt"></script>
Bon! But my dear Steven, i want to show only the last 5 posts - can you do THAT? Well, yes we can... as follows using the "c" querystring value:

<script language="JavaScript" src="http://livz.org/seesmic?f=txt&c=5"></script>Sacré bleu i hear you say. But you want more... you want to display only YOUR latest posts. So for myself who goes under the name "@weblivz" i would show the latest 5 posts as follows:

<script language="JavaScript" src="http://livz.org/seesmic?f=txt&u=@weblivz&c=5"></script>Ah, but video. Seesmic is about video, can i show my latest video?? Of Course - just change the "f" parameter to "vid" rather than "txt":

<script language="JavaScript" src="http://livz.org/seesmic?f=vid&u=@weblivz&c=1"></script>
Of course you can point at ANY username you wish by changing the "u" parameter. Additionally if you do NOT use the "u" parameter you can use the "t" parameter and add any text you wish to be matched as so:

<script language="JavaScript" src="http://livz.org/seesmic?f=vid&t=berners&c=1"></script>
Mon Deu! Can it tell me what I should say in my next video to make sure people watch???

Unfortunately I had to pull that feature at the last minute, but as a work around I suggest clicking the "I'm feeling lucky" Google button and just talking about the first page you come to.

You can see all of this in action at http://livz.org. Play away :)

Xavier wins Enjoy a Ball trophey

2007-11-27T01:07:00.001-08:00

Xavier won the Enjoy-a-ball trophey this week for kids so he decided to show it off and by the way he is holding it i get the feeling it may not be the last!! Well done wee man!

Question to Tim Berners-Lee

2007-11-27T01:07:00.000-08:00

I noticed Scoble Twittered that he is talking with none other than Tim Berners-Lee this evening!! Well, i have been reading his book Weaving the Web which he wrote a few years back and so even though i love all of Robert's shows, this one will be particularly interesting for me!

So he asked if anyone had a question for him, and although i have an infitinite owl:List of questions this is my first attempt (happens to be the similar to Sam's which was way freaky as it was also at the same time) .

On page 196 of "Weaving the Web" (which i am reading again just now) you voice your concerns specifically over the effect of patenting on reasoning engines for the Semantic Web and i guess the general effects of patenting on the fundamental Semantic Web technologies.

The Semantic Web is certainly progressing although perhaps not quite as fast as we may have expected (I was looking at RDF when writing my first book on Xml around 10 years ago now). In fact most of the good work I have used has come out of research labs in Universites in the US, UK and Germany.

Do you think the constant desire for patents around emerging Semantic Web concepts and the counter fact that a lot of the good work has actually come out of Universites and has been fairly open and flexible (and hence trickier - although not impossible - to build patents on top of), has slowed the rate of large scale commercial adotoption of Semantic Web technology development?

As a startup one of the first things i often hear is "What can you patent?".

Or perhaps more simply a large consumer facing Semantic application has just now taken hold yet? Rather it is dominated by mashups of existing standards - i.e. using the code to hack the meaning rather than the data containg it.
PS. When are you writing the follow up to Weaving the Web?

IF you have time for a follow up question Robert (and you read this!), I'd also like to know whether he can see the existing P3P standards emerging to fit into a Social Environment as privacy is perhaps the most fundamental issue of Social Networks that has not really been tackled yet (identity has been discussed much more than privacy) and a standard looks the best way to ensure this very important aspect is included!

Seesmic Feed

2007-11-27T00:44:00.000-08:00

Thanks for all the positive feedback on the work I did. Actually the first thing i did this morning was this:

http://livz.org/seesmic?u=@weblivz

and

http://livz.org/seesmic?t=rss

This returned no results, but i DID get some great feedback on the blogs and privately by email.

Anyway, glad to receive any feedback and hope it comes in useful. Back to my own plan to rule the world now ...

Playing with Seesmic feeds - 3 small features

2007-11-26T15:31:00.000-08:00

One of my frustrations with Seesmic was simply knowing what was going on without hitting the site all the time and filtering posts. The provide a Twitter feed which is pretty good but it's getting busy and although they have an RSS feed on the top of the Twitter feed i never knew this til i looked at their JSON code....

... in any case there are a number of things i wanted to do and will likely want to do in the future to the JSON to XML stuff i did will likely come in useful again.

ANYWAY - what have I done??

Well ....

1. An RSS feed of Seesmic - updates around every 15 minutes and can be found here :

http://livz.org/seesmic

2. You can discover posts relevant to you using the "u" querystring value. So

http://livz.org/seesmic?u=@weblivz

... will return feeds created by me - or directed to me (if you really only want things created by you, then add "by " to the query, such as http://livz.org/seesmic?u=by @weblivz).

3. You can filter based on content using the "t" querystring value. So you can find posts about Seesmic with the following query:

http://livz.org/seesmic?t=seesmic

THIS means you can see if anyone has replied to you by just appending "Re: " to the first few characters of your original subject line. So looking at the site just now, i can find replies to "Reload Seesmic for new features !" with the following query ....

http://livz.org/seesmic?t=Re: Reload

It's a bit of a start - hope it is of some use. I already use it to quickly check the last N posts - i guess the fact i can now translate their JSON to RSS will be more useful when it comes to RSS on a person basic - rather than public (which may be quite soon as it's getting busy). In any case hope the filtering features are of some use.

Seesmic is a conversation

2007-11-26T03:49:00.000-08:00

I watched a Seesmic video by Loic Le Meur of Seesmic trying to state the point that unlike YouTube, Seesmic is a conversation - much likfe Twitter.

I've been using it since the weekend now and it really is very cool and yes, it is a conversation (although the actual conversation part is something they need to work on - following parts of a conversation is tricky).

The difference from YouTube is that you are talking to your friends. Sure, just now everything is public, but over time it will be your friends because as the public timeline gets noisier, you want to follow only things interesting to you - i.e. your friends.

I have made many friends online recently and many through Twitter, blogs and even Seesmic - so the definition of a friend is pretty flexible.

We all feel a little self-concious when creating these videos but over time this (i hope) will improve and you (or I) will feel more confortable with it all. I DO think that feedback and people talking back to you on Seesmic in a response to your video is a big part of that.

Perhaps we should all be teamed with a mentor who is confortable with it all and can give some feedback. Or maybe just an abrupt "hot or not" system :)

Oh, Seesmic is COOL!

Optimizing NTFS Performance

2007-11-24T14:07:00.000-08:00

If you are writing any web services that store large files on an NTFS volume, this section is very useful - full article here.

Optimizing NTFS Performance

NTFS performance is affected by many factors, such as cluster size, fragmentation level, and the use of programs such as antivirus software. In addition, NTFS features such as compression and Indexing Service can also affect performance. Optimize the performance of NTFS volumes by using the following guidelines.
Determine Cluster Size

Before you format an NTFS volume, evaluate the types of files to be stored on the volume to determine whether to use the default cluster size. Some important questions to answer include:

* Are the files typically the same size?
* Are the files smaller than the default cluster size?
* Do the files remain the same size or grow larger?

If the files are typically smaller than the default cluster size (for example, 4 KB) and do not increase, use the default cluster size to reduce wasted disk space. However, smaller clusters can increase fragmentation, especially when files grow to fill more than one cluster. Therefore, adjust the cluster size accordingly when you format the volume. If the files you store tend to be large or increase in size, use 16-KB or 32-KB clusters instead of the default 4-KB cluster size.

Note Compression is supported only on volumes that use 4-KB or smaller clusters.

Cluster size is also an issue for volumes that were converted from FAT to NTFS in Windows 2000 or earlier because the default cluster size for converted volumes is 512 bytes, and the MFT was most likely fragmented during the conversion. For optimum performance, back up the data on the volume, reformat the volume, specify the appropriate cluster size, and then restore the data.

For more information about choosing a cluster size, see “Cluster Size” earlier in this chapter.
Use Short File Names

Every time you create a file with a long file name, NTFS creates a second file entry that has a similar 8.3 short file name. A file with an 8.3 short file name has a file name containing 1 to 8 characters and a file name extension containing 1 to 3 characters. The file name and file name extension are separated by a period.

If you have a large number of files (300,000 or more) in a folder, and the files have long file names with the same initial characters, the time required to create the files increases. The increase occurs because NTFS bases the short file name on the first six characters of the long file name. In folders with more than 300,000 files, the short file names start to conflict after NTFS uses all the 8.3 names that are similar to the long file names. Repeated conflicts between a generated short file name and existing short file names cause NTFS to regenerate the short file name from 6 to 8 times.

To reduce the time required to create files, use the fsutil behavior set disable8dot3 command to disable the creation of 8.3 short file names. (You must restart your computer for this setting to take effect.) For more information about disabling 8.3 short file names, see “MS-DOS-Readable File Names on NTFS Volumes” later in this chapter.

If you want NTFS to generate 8.3 names, improve performance by using a naming scheme in which long file names differ at the beginning of the name instead of at the end.

For more information about short file names, see “File Names in Windows XP Professional” later in this chapter.
Determine Folder Structure

NTFS supports volumes with large numbers of files and folders, so create a folder structure that works best for your organization. Some guidelines to consider when designing a folder structure include:

* Avoid putting a large number of files into a folder if you use programs that create, delete, open, or close files quickly or frequently. The better solution is to logically separate the files into folders to distribute the workload on multiple folders at the same time.

* If there is no way to logically separate the files into folders, put all the files into one folder and then disable 8.3 file name generation. If you must use 8.3 names, use a file-naming scheme that ensures that the first six characters are unique.

Warning The time required to run Chkdsk.exe increases with larger folders. For more information about determining how long Chkdsk takes to complete, see Chapter 28, “Troubleshooting Disks and File Systems.”

Seesmic Citizens Collective

2007-11-24T10:11:00.000-08:00

Thanks to Jeff Pulver and his tweet, I got a Seesmic invite and signed up and sent in my first video - it was all spur of the moment and the two kids were kicking off so it was slightly chaotic (whether it gets less chaotic, only time will tell!).

Anyway, i wanted to simply ask whether there were any other Scottish Seemsic folk and wasn't sure what you call a collection of Seemic folks - i.e. "Are there any other Scottish ..... out there"?

1. Seesmicites - i'm sure i saw one of these in a nature program

2. Seesmatics - sounds like a hoover

3. ?

Any ideas?

I should have posted this as a video but the kids are approaching bedtime and so all hell is breaking loose!

23AndMe, and Geni

2007-11-21T05:25:00.000-08:00

So I just visited the http://23andme.com site, a personal genome service which

"... is intended to help people understand their inherited traits and to allow them
to compare themselves with friends and family."

At first i thought WHAT?! OK, so I thought it was a gentic based dating site (perhaps that's in their future plans ;) ) but once i looked around i thought - COOL !

It's not cheap at $1k per person but based on the extremely qualified backers and advisors of this site i'd more likely believe it actually works pretty well. I think we'd need educated as to exactly what to take and understand from any results we were given - as well as perhaps some clue as to what to do with it (other than - "i told you you had your fathers stubborness"). Could it be beneficial in education, sports and health?? Even a marginal improvement here can only be a good thing (especially as a father with two kids)!

Now, imagine this.... They do a deal with Geni (one of my fav sites) and over time (as things get slightly cheaper) you get larger percentages of families getting involved. Imagine the kind of searches you could do - even within your own family.

I don't know how far this will extend from the family but clearly their are huge opportunities - with a whole set of new privacy issues to watch over. But clearly this is a direction these kind of services will move into.

Has anyone used the service? If so please let me know.

A startup idea to make £56 Million in one week

2007-11-21T01:28:00.000-08:00

Here is my idea to making £56 million in one week. Any investors out there?

1. Start a Credit Reference Agency

2. Get loads of people's credit information.

3. Wait 'til the government loses 28 million items of personal information.

4. Charge two pounds per reference check.

5. After one week, sell to someone else who can make cash the next time it happens.

6. Check the bank account to see your lovely £56 Million.

Oh crap, someone stole it when they got your credit details. Doh!

This is obvously tounge-in-cheek but this is what they are asking you to do. Credit Agencies must be loving it!

Alastair Darling - we have technology to fix this

2007-11-21T01:01:00.000-08:00

Having worked on a project for schools in Scotland that required the export and trasportation of some personal information I cannot believe the basic flaws that have been made by the "technical people" at HM Revenue and Customs.

Here is it in one line.

DO NOT ALLOW CLEAR TEXT EXPORT OF PERSONAL INFORMATION !!

This is not difficult. The technologies are far enough advanced. In an attempt to save the trillions of consultancy fees, here is how to fix this.

1. Give every member of staff a certificate and/or smart card.

2. Ensure they use a very strong password when using that certificate

3. Any EXPORT routine MUST ask for the private key of the user exporting the data and signs that data. This ensure you know who exported the data.

4. Additionally, any export routine MUST ask for the public key of the EXACT user you intend to send the data to and encrypt that data. Now only that person can read that data as only they have the private key for that certificate.

5. Now, with the IMPORT routine, when the person wants to read the data they need their own private key to decrypt the data.

6. The IMPORT routine should also check the signature an alert the user as to confirm who SENT them the data.

Additional steps should be :

7 .Use Key Revocation to ensure that should any key be lost, that key becomes IMMEDIATELY invalid and hence can't be used to view the data.

8. Know and ensure that every application that uses this data has a common import/export routine with encryption and logging as standard.

9. Log every interaction with that data. You don't even need the personal data to do this - just an identifier for the user (say the certificate hash) and an identifier for the data items.

This stuff sounds tricker than it is. I have now done this with the Government, Barclays and a few other places.... in short - you CANNOT just allow people to export this kind of data.

Now we are all checking our bank accounts - i have an additional post coming next on the business side of this ...

YouTube for Parents - you have it the wrong way around

2007-11-20T06:28:00.000-08:00

Update : Ian Hayward the founder of Glubble corrected me below in that you CAN indeed restrict on a page basis. We both agree the comments (and links with previews) by others can be quite unfortunate. Tricker to deal with outside of YouTube.

I've just spent 15 minutes trying to find a video for my 4 year old son to watch on YouTube. It's not the fist time. Simply put due to some idiotic and outdated rules on the US site which says only kids in the US can see a certain 5 minute cartoon, it leaves him asking why and me hunting for an equivalent on YouTube.

However, I would never let him use that site himself as some of the "cartoons" content and comments aren't stuff i want him to see. So i can't even use software like Glubble as it works on a site by site basis. In any case - this is for YouTube to sort out. Oh, and YouTube is not alone, please read on if you have any mixed content with some intended for kids.

Here is my guide:

1. Everything should be opt out
Yes, if i say i am under X years of age or go into "parent mode" (doesn't exist but should) then plesae make EVERYTHING opt in. That means when *I* hit the site there are NO videos. Well, perhaps you can suggest some that have been vetted. *I* will then decide precisely what my kids can see. PRECISELY. I will choose what comments and what content- and you will like to other stuff i have suggested. So that means NO previews of stuff you THINK may be suitable and no "That was fucking awsome. The older stuff was shit but this is good." comments.

2. Let you and your friends decide which moves you wish to set as opt in
OK, so i am willing to go one further step and I should be able to specify what friends i have who i also trust to recommend kids material (no offence to any friends :)). If i say my sons Grandma can recommend a kids video to him then i expect you to respect that.... but just because i may be friends with "womanizer77" (i made that up completely btw) does NOT mean i trust his movies by proxy for my kids.

3. Don't recommend clips from the Exorcist when showing a Ben 10 clip to a 4 year old
I really don't think much needs said here. It happened.

OpenID provider fitness

2007-11-14T01:32:00.000-08:00

I'm adding OpenID support to a site i am working on, but came up with an interesting question.

Typically i have the username/password/email and email verification process. Alternatively you enter your OpenID.... but this is where i started to think. Do I limit the OpenID to trusted OpenID providers or do i just accept ANY OpenID? How do i know they haven't juset set up their own provider and how do i know whether the email is valid??

Do I request their email and ask them for yet another verification before they can use the site.... something that could get quite annoying overall.

Would be nice to have an online service which you can query every so often to check the "fitness" of providers so you can decide whether a provider is valid or not. Similar to the certificate lists you get in your browser. This could be managed and be very beneficial for the community i would have thought!? Anyone working on such a service?

If you are reading and have implemened an openid client on your site, how have you gotten round this?

UK Postal Scam - spam goes offline

2007-10-24T07:21:00.000-07:00

"It has been confirmed by Royal Mail. The Trading Standards Office are making
people aware of the following scam:

A card is posted through your door from a company called PDS (Parcel
Delivery Service) suggesting that they were unable to deliver a parcel and
that you need to contact them on 0906 6611911 (a premium rate number).

DO NOT call this number, as this is a mail scam originating from Belize.

If you call the number and you start to hear a recorded message you will
already have been billed £15 for the phone call.

If you do receive a card with these details, then please contact Royal Mail
Fraud on 02072396655 or ICSTIS (the premium rate service regulator) at
www.icstis.org.uk

Please circulate this to avoid anyone else being ripped off. "

AI Web Services

2007-10-21T12:25:00.000-07:00

I asked Peter Norvig to point me towards sites that demonstrate real world applications of AI - mainly in the software world. As he wrote the book (literally in this case) and now works as Director of Research at a company called Google, i figured he'd be a good start.

He pointed me at a great resource here which has a list of projects as well as some here in Scotland.

I am interested in more however. Although i have a background in Applied Physics and software development, i do not have a very deep knowledge of algorithms and AI - however my experience over a number of years gives me a *feeling* that there are potential reusable components - or more likely - pluggable web services - that use AI and can be re-used in your application.

I want to get a list of these. Imagine a simple one - take a paragraph of text (say a blog post) and reccommend tags. This could be a library component as part of an application or a web service.

Anyone have pointers?

Applications of Intelligent Systems

2007-10-18T23:55:00.000-07:00

For a number of years - since actually doing some emergent fractals work in C++ back in 1993 or so - i have been interested in intelligent and emergent systems. I read Emergence a few years back and it only hightened my awareness and desire that rather than creating some elete intelligent system, rather i'd like to add intelligent components to my applications.

I watched the self-aware robots talk by Hod Lipson at TED. If i wasn't doing what i do now, THAT is what i'd like to do. Not so much the robots (although myself and son are playing with Lego MindStorms) but the software and ideas that create and apply this stuff. Amazing.

After looking for some books on writing this (couldn't find any in C#) i found the seminal book "Artificial Intelligence: A Modern Approach" by Stuart Russell and Peter Norvig. I will re-write their algorithms in C# as needed - there is a Java effort going on too that is cool.

HOWEVER, the single post that really illustrates what i want from this is in Peter Norvig's "Doing the Martin Shuffle".

He discusses and demonstrates a real world technique that could be used to improve how the iPod shuffles the songs it plays to you. How ironic as it's Google (who doing amazing rsearch) and Apple (who do amazing UX) kind of sums up my take on thing.. trying new concepts with actual people!

So the next step IMHO is to take something like the AI book and also take some ideas in the real world (web 2.0 apps are all the rage so why not start there!) and show how intelligent components could be plugged in.

Very cool stuff!

YouTube Video Identification Beta

2007-10-16T07:10:00.001-07:00

David King, the Product Manager of Google's YouTube just launched this beta.

The idea is you can identify your works on YouTube and i guess others can know it is identified as being from you.

I'm not sure whether it's really for security rather then simple identification - the idea of being able to know what videos have been published - perhaps even by others - on YouTube by a simple hash query over the video content is quite interesting.

Jeramiah has some interesting comments on it.

Peek-a-boo FREE Tricks & Treats for You! Get 'em!

Robot helecopter from Stanford

2007-10-16T04:41:00.000-07:00

Watched this video this morning with my son after Robert Scoble pointed me at it throw Twitter.

"Is that from Star Wars" said my four year old son Xavier.

It is very very cool but now puts a hell of a lot of pressure on us as we are building a robot using MindStorms.

I also watched Steve Gillmor and Dan Farber get kicked off Dow Jones property whilst discussing whether Facebook is worth $100 billion. Even my wife asked me what the heck was going on. Finally i caught Scoble talking with Six Apart. Good guys and some nice ideas.

Third World Wind Power

2007-10-13T11:59:00.000-07:00

I read this great post from Tim O'Reilly - Third World Wind Power

I love this kind of thinking - not only because of the vast amount of good it can do for the Third World, but also at the relative simplicity of it. Thinking different.

BTW - this isn't only for the third world - this could work in Scotland too - we certainly get more wind that many places in the world.

More detailed information can be found here.

re: E-mail Faces Deletion

2007-10-12T12:14:00.000-07:00

Scoble pointed me towards an interesting discussion over at Business Week entitled "E-mail Faces Deletion".

I am going to twitter him? No - there's not enough space in 140 chars. Am i going to email him? No. He'll never read it. A few people may read it and it there is enough interest someone may point him to it (or directly through Twitter).

The point is that email is not right for a conversation. 25 messages to arrange a meeting where the overhead is 6 times the content CAN'T be right... and on a mobile (come on SMS has proven this in the UK alone).

A great point by Scoble is

"But when I left, my e-mail account was turned off. I don’t have access to that
knowledge now."

.. and neither do THEY! Everyone loses.

Twitter, Jaiku and so on are not perfect - it gives people like me a chance to write something better. It's all a step in the right direction.

Email is dead as the primary communication method IMHO - i use it because, like letters, people still do things that way. But, for a few key emails, the rest is broadcast or spam. Communication is going to be about content and pointers to that content (tweets, jaiku's, others) - you write, suggest and maybe i read - kinda like reading an email subject line and deciding whether to read the email.

In saying that, anyone who thinks communication is going to be one dimensional probably thinks Google is the final answer in search.

Movavi - please add an API!!!

2007-10-06T00:10:00.000-07:00

After doing some research i came across an excellent service from Movavi which allows you to convert media files to flash formats to make it easy to play them back to users.

They even have an SDK and it's a great price too, but here's a suggestion. Add an API!!!!

There is no doubt a a hassle adding all the codecs and such for doing the conversion and when they add new codec support you need to download and update and so on.

However, if they could allow you to call there API as a service it would be very powerful indeed!! I know someone over at Amazon Web Services had created an AMI for transcoding - however the Movavi service is very extensive and supports exactly what i expect most developers would need.

They could likely integrate some payment method - but it would need to be around the level of Amazon's to keep it open and available to small startup companies like my own.

If you read this and agree then add a comment - maybe they're listening!

WCF inheritance in REST interop scenarios

2007-09-27T03:05:00.000-07:00

I am writing Xml that needs to be consumable by any client that can make an HTTP request. I am also trying to write a service API that using inheritance so that i can have a method such as Accept() that can take any object derived from MyBaseClass.

The answer turned out to be easier than i thought, but it was a but of a struggle getting there as most answers are tuned toward your CLIENT also being a WCF client (which is the case in about 5% of my use cases).

The answer is to construct your Xml and use the Xml Schema type attribute to define the type of the object when passing it to a method that accepts its base class.

So if i have a web service method "Accept":

public void Accept(MyBaseClass val);

... and if i have the class MyClass : MyBaseClass , then in code i can simply write:

MyBaseClass bc = new MyBaseClass();
Accept(bc);

Now, to do the same in an XML REST POST WCF scenario, you construct the following Xml:

<val xmlns="http://myns.org" i="http://www.w3.org/2001/XMLSchema-instance" i:type="MyClass"><br />...</val>

Notice the type attribute is used to state the derived class.

BBC Blast in Glasgow

2007-09-24T00:42:00.000-07:00

By chance yesterday we wandered into the BBC Blast event in Atlantic Quay, here in Glasgow.

My son Xavier was fascinated as soon as he saw one of our friends on the big screen (she presents the news on TV) and with that and getting a special badge which went around his neck, we weren't going home anytime soon. It wasn't too busy, but the quality was quite high with artists, musicians, reporters, producers and so on all doing various things to keep your attention.

My son even wrote his own news report abut saving the earth from a meteor which is printed and laminated. It was a dangerous walk home as he asked me to read it to him - along with his sister who, at 9 months, discovered the "Franci-saurus" in Chile.

I still think there must be better ways of raising awareness of events in your local area and i'm thinking hard about that just now.

Wipro and the elephant

2007-09-13T01:43:00.000-07:00

I'm reading Bangalore Tiger, the story of Wipro - one of the most successful technical services companies in India and indeed the world.

I have started to look outside of Silicon Valley for inspiration. The problem is that Silicon Valley is a fairly unique community and businesses coming from the Valley have soooo many resources available to them. I read about them and start realizing that most of what they take for granted just isn't available to most of us.

Now, i still love the stories of the big Valley companies, but when i started reading about Wirpo i started to see (and no disrespect intended) a company who hacked around until they got it right. They started off in oil and soaps and now run a 68,000 employee technology company. They are an amazing story of bootstrapping.

I also love the story by Vivek Paul, former vice chairman at Wipro.

He was at a circus on the outskirts of Bangladore and an elephant was tied to a
peg and wasn't trying to break free, despite the fact it easily could. When he
asked the trainer why, he simply said that it has been tied to that peg since it
was a baby and couldn't break free and stopped trying. So in adult life it
continued to believe this and so never tried.

I love this story, as coming from a relatively small country in Scotland there are too many who believe you can't be successful simply because we are small - negativety is a disease! The result against France last night is a good example of this not being the case.

SPAM Frustration

2007-09-13T01:38:00.000-07:00

Yet again i have been hit by the problem of emails that SHOULD be getting to me going into my Junk mail. It seems with Hotmail that you either get a load of SPAM in your inbox, otherwise most go into the junk mail - no middle ground. Their SPAM options don't work particularly well either.

A few years back i wrote a short internal paper on intelligent email based on a social network - might be worth looking at it again....

Celtic progress in Champions League

2007-08-29T16:26:00.000-07:00

Well, tonight was quite amazing. Celtic managed to beat Spartak Moscow, the best team we could have faced in the qualifiers and leaders of the Russian league.

They played very well, but we should have beaten them long beforehand - they had a chance or two but we had SO many but never had our finishing boots on despite playing well.

As always the atmosphere just can't be beaten - the reason we were voted the best venue in the UK was obvious... especially to the sorry Russian who missed the last penalty.

I got a few pics, the above was the best. Need a nice camera. I feel physically drained - never shouted and cheered so much. It's hard supporting the best club in the world!

Hail Hail!!!

Maps still need direction - Collaborative Maps?

2007-08-29T05:45:00.000-07:00

Well, what with Google, Microsoft, Yahoo and some other familiar names (even open such as OpenStreetMap) i figured Maps were dead. I mean dead in the sense of why would ANYONE write another one.

Well, that is until is actually tried to use them. The Google and Yahoo API's are nice, i couldn't even FIND an API for Windows Live and although OpenStreetMap is nice, it's API was almost impossible to figure out (i just want to use as part of my app).

Now, even though i can USE Google and Yahoo easily, it doesn't work for place names!! Yes, in most apps people are going to enter a place name "Mitchell Library" rather than "201 North Street, Glasgow" - but although this works great for the browser, it does NOT work as part of the REST interface either company provides.

Is this because they KNOW it is so key to future apps? Particularly in the mobile space (prior to us all having GPS)?

So here's a call to action. Why does every company who doesn't regards their location information as core to their revenue model not join together and create a universal database - like OpenStreetMap or something. Put on a nice API and let people integrate the data.

I don't mean with all the graphical maps etc - i mean just the data about placenames and where they are. Done together.

I'm working on something that really needs more than what these companies offer and i'd gladly put my location data into such an open system.

Anyone?

Initial API:

Put(placename, number, streetname, city, postcode, country, latitude, londitude);

Get(placename, country)
Get(placename, number, streetname, city, postcode, coutry)
Get (latitude, longitude)

Scoble video as a social conversation

2007-08-29T01:47:00.000-07:00

I've read with some interest the many of the posts "dissing" that search post by Robert.

Now, not saying I agree or diagree with some or all of the points he makes, but HOW he makes it is surely an interesting social experiement for us all - well, more specifically how the responses happened is more interesting.

You see, i believe brainstorming with smart minds is one of the most exciting things you can do in my field. But that only works if the bar is level - criticism is fine, but it needs to be something that is on a par with the original work. So a better response to Scobles post would be a set of social video responses - an ascychronous online conversation.

For example, most of the standards bodies and so on are outwith the reach of the majority of us and typically confied to established companies in that field (if fact you often miss the complete conversation as happened to me with Attenion Profiling) - we have a lot to say, but no way to say it (you sometimes get a public forum if you're lucky). However, the idea of being able to just brainstorm and see what people think through video is pretty exciting.

I'd like to see more ideas, whiteboards and rants about topics I am interested in. Wake up in the morning and there are 5 responses to what you said - not just criticisms in blogs that don't accept comments, but real views and opinions in the same manner you started.

So, why not have "comments" on Scobles video post ONLY be videos. That would be very interesting - it's a hard thing to do to just lay it on the line real time. But would be pretty exciting i'd think ....

Twitter as IM

2007-08-27T08:49:00.000-07:00

I added a new Twitter account so i can async IM with people as well as write random stuff i don't wanna bother others with. My new account is at http://twitter.com/ilivz .

If you want my regular stuff i am at http://twitter.com/weblivz

On my ilivz account i will be posting most of the day so expect a ton or random "stuff" ....

Celtic game on Wednesday

2007-08-27T07:40:00.000-07:00

So come this Wednesday there will be over 60,000 fans packed into Parkhead to watch Celtic play Spartak Moscow in a qualifier for the champions league. The atmosphere is pretty amazing by any standards and thankfully I will be there! Tickets arrived in the door and my and a mate are set.

I have been told I take on a new persona at football games - maybe we all need something we can vent our energy out on. A couple of beers and screaming at the top of my voice for two hours will be a start!

I will try twittering and blogging - i might even try a live video cast and see how that goes.

Hope i don't become the first guy to get arrested using twitter through some bizarre 200 year old legal broadcast rule (yep, why do you think it's called "Scotland Yard" ?!!).

Changing my viewing habits

2007-08-17T04:06:00.000-07:00

btw - blogger - I love you!! Thank God you added "autosave" as IE just died on my and i could see the content behind the "Send Error Message to Microsoft" and thought i had lost it!!

One of the real frustrations for me is that i miss out on some very cool content on the web. I have a wife, two kids, some consultancy and trying to create a startup at the same time (which is now coming on very well). Thankfully I am going to have a lot of free (i.e. my own) time soon (i hope) as the consultancy (to buy food which is apparently an "important part of childrens diet" etc) is coming to an end.

However, there is what seems like endless content out there - and there is a wealth of really good stuff too!! I have Gnomedex, Always On, Scoble Show and (thanks to Sam) Intruders.tv - as well as IT Conversations etc - and that is just rich media. My BlogLines keeps going up to 4000 posts... i secretly think Scoble is multi-threaded to get through all this.

So how to balance this out? Well, other than the short viewings that can be done between things or on my PDA, i have decided to dedicate at least 3 of my evenings watching online content (say between 7pm and 10pm). At the weekends I should also be able to catch up on some. I don't watch much TV anyway (other than football), but continuing to purely code can also work against you as you miss all this content.... and what i have found is that a lot of it actually feeds in to what you are doing and either changes what you are doing or the way you are doing it - and sometimes i creates completely new ideas or angles - this is exactly what happened after I listened to Robert Scoble talking with Scott Klemmer Stanford.

This will clearly affect my coding time, but some of the things I learn that have a direct impact on what i am interested in is just too important. I actually think the net effect will be better coding - or rather higher quality results.

How is anyone else coping with this (who is in a similar situation?). There is no way we can catch everything, but there is also no way some this can be missed .... and i have long since lost the illusion of remembering to go back to see it or checking my bookmarks! If i don't watch it there and then, i likely will never.

This isn't really work for me - it's like reading a good book when listening to some of these podcasts and videos so moving from traditional TV isn't hard (it's even easier with the stuff that they show over here). However, it's also very important that i spend plenty of family time with my wife and kids, who aren't quite as interested as me (yet [rubs hands as scary laughter roars out....]).

WCF RESTful GET queries using custom entity input parameters

2007-08-16T08:20:00.001-07:00

I am looking to map RESTful GET queries to custom objects in my WCF service layer and asked the folks over at Microsoft how to do it. To my surprise i got the reply that:

An HTTP GET request may come from a web browser, and all parameters that are
sent to the operation need to be specified in the URL. There isn't any set
way to represent a list of locations in the URL. [link]

Now, I am no expert in REST architecture, but I think of the URI template mapping stuff Microsoft are adding as much the same as RegEx which we have all used for years. I can think of a number of ways that you could map URL parameters to a custom object... even an option allowing you to define a custom mapping layer would be neat. I like the idea of using XPath, but anything that maps an input pattern to a custom object would do me.

Otherwise i need to define all my GET interfaces using simple value types rather than my custom entity objects - which can be returned fine, just not passed in...

I'm in view of flexibility and not having to think of REST interfaces in WCF as something special - other than the attributes you need to apply. I don't think anything i am asking for is difficult in any sense so i'd be interested in where any issue in doing this would lie?!

Son back at school

2007-08-16T08:07:00.000-07:00

Xavier went back to school today (Nursery really, but he calls it that) - he is only 4 and just missed out on the "big" school.

He has a great day apparently, very social as he is now the "big kid" and looked afer the little ones really well.

He ALSO corrected the teachers on his name, which they randomly spelled last year. Now he says "There is no need to look it up, it is X-A-V-I-E-R".

He also remembered every teachers name which was pretty impressive consdering i need a map and compass to get home at night.

Geni the next Linked In?

2007-08-04T12:31:00.000-07:00

First off, i love Geni. It has put me in touch with many family members and we now have more than 700 people in our tree, most of which has been viral.

I also like Linked In. Scoble mentioned to me on his blog that he hates linked in and will never go back. However, many of my contacts remain on Linked In and moving them all would be more hassle that it is worth. So while i use Facebook, i'd only start to use it as my main social networking solution IF it was part of a distributed open identity network where i could merge people in there with others in other networks.

However, Linked In has remained fairly static for a long time now. Sure, they have added some features, but they have stood still for so long they have lost ground on other networks that have seen the clear need for something after you have made contact... collaboration, community. Facebook have done very well in that area. FB has also added a "platform" that allows people to build on their services. Linked In could probably take this to the next step by completely opening up and acting as a social network platform for other communities. However, they will have to work fast for this to happen.

So how does this relate in any way to Geni? It's the same thing really, only the connections are family. Once that connection has been made, the major problem on Geni is that there is nothing else to do - not really any tools do make you return after you have connected. I have said in the past they should work with Dandelife (in fact i think Dandelife really need that connection too, but perhaps less than Geni).

At the very least, Geni needs to add family blogging style tools. My daughter said her first word today ... "dada" .. something many of my family would be interested in knowing, along with my video and a few pictures. Geni doesn't really provide any tools to enable that to happen. [ Update (based on Geni Team comment):While I know and have used their image upload and tagging tools which work well, i really mean a diary style tool - the reason blogging has become so popular; this would work really well with a family site and would save me having to "diff" the content of my family tree ] They could probably just provide an option to include external feeds from flickr, youtube, blogger etc and that would be a start.

There is a clear window of opportunity for someone to create what Geni has done, but add the stickiness factor to it. I really think they will need more than just connections... they have such a great opportunity and i love their service, so i hope they start brining some of these things to us soon.

Amazon FPS

2007-08-03T01:06:00.000-07:00

Update: Seems you need a U.S credit card to use this.... will this be extended soon, as i don't :)

Amazon's FPS (flexible payment system) was released about 10 minutes ago ... Jeff was waiting for about 2 days to post and eventually he posted and didn't disappoint. When Jeff was in Glasgow I had asked about improvig the way transactions could be done through services. It seems he got back to his team at Amazon and they wrote a 250 page spec for me in a week or so... thanks Jeff ;)

I look forward to reading about this through the day... this is really powerful stuff. I am particularly interested to see if there is anything for Micropayments (i haven't look yet) as i was discussing this at AO yeterday - i see this as crucial for pay as you go services... don't want to use my CC every time.

Lot of reading to do ...

HumanML - would be nice to see it gain popularity

2007-08-03T00:47:00.000-07:00

Many years ago i was a little involved but mainly following HumanML (or here)- the Human Markup Language.

At AO yesterday, i got talking with Rex Brooks, who was involved in starting the concept and vice-chaired it at OASIS (he is now co-chair of the Emergency Management Technical Committee)- he told me that due to lack of participation he had to close it down.

I'd like to see this come back. Why? Well, i'd like to be able to index my conversations online, Twitter and more specifically in second life. I'd like to be able to search for people i was happy talking with, or confused, or enlightened. "Find me people (or conversations) who englightened me in search technology". You can imagine this could get a little more complex and very powerful.

Emoticons in IM's, chat etc can easily map to HumanML and this could also be indexed and searched upon. It could also make images more searchable - based on your emotions in the image.

Surely Second Life would want us to be able to do this? Anyone able to ask them? I'd love to see this happen again as it's now the right time. Anyone???

Innovation and Technology on Second Life

2007-08-03T00:32:00.000-07:00

After some discussion in the Always On chat panel (all brilliantly hosted as far as i'm aware by Evan Donn, aka divergent), I really think Second Life (meetups, conferences etc rather than generally walking around) could be very useful for quite a lot of us.

Update : I created the Eventful group "SecondLifeTech" for second life technology events - the general one had over 5000 random events....

There was a general view that moving what we were doing into a "virtual" environment would be quite a neat thing to do.

So I am going to try and capture (a) active places, groups etc
in Second Life discussing innovation and technology, and (b) related innovation
and technology events. Follow their events listing on SL is too difficult.

So why not use Eventful, with a tag of "secondlife tech" to life tech and innovation events? Please let me know if you know of some. I will be adding the Amazon AWS event and meetup location first. Not sure where I'll list them, but i'd like to get more involved... there are some good people out there and i enjoyed talking with them on AO!! Contact me if you know anything i should at add weblivz AT hotmail DOT com

ps. next year i'd like to see AO open thing up so i can finally have a good reason for my webcam and mic and ask from questions direct to the panel. They answered three this year which was great!

Blognation

2007-08-02T07:27:00.000-07:00

update: FWIW, i'd like to see something similar for Latin America. I got my first ASDL line there long before i could get one in Scotland.

Sam Sethi et al are doing some very cool things over at Blognation - especially if you have ever wished there was something concentrating on the European startup scene. Our countries are generally smaller than some of the big players, but collectively we are quite large in comparison - however, most times i just assume a startup is from the US unless a notice it is not by chance (e.g. Plazes from Germany) or I am told directly (as happened with WeeWorld which is a stone throw away... I do actually know this after having to pay for a replacement window for them - sometimes theory is as far as it should go!).

You may ask why it matters where a startup is based. Well it does and it doesn't. In terms of technology, it probably doesn't (most of the time). However, US startups are closer to where the action has been traditionally. Even if you are not from the US, you often have US investors and in some cases end up over there. So these startups get more press and there is an uneven buzz about them .... as of course would happen when events are closer, timezones are more aligned (e.g. staying up 'til 3am to listen to always on an be part of the chat is tricky during the week!).

However, there is a lot going on in Europe and i want to hear more about it. In fact i want to hear more about startups closer to where i am (Glasgow/Scotland in my case) - for a different reason. They may not be competitors or even in the same field, but is it encouraging to know there are more of you out there and from this you can perhaps form smaller localized startup communities - share information - wisdom of crowds and all that! Earlier it was mentioned that with localized startup data and Google Maps Microformats support we may see this in Blognation sooner than later - now that is something that i think will be very cool indeed!

Global Patenting

2007-08-02T04:50:00.000-07:00

Yesterday i asked a question to the patenting panel at Always On and got a response - what is cool is that it was from David Hayes, Partner, Intellectual Property Group, Fenwick & West . he has represented companies such as Apple, Cisco and Google so really knows his business.

I simple asked where companies based outside the US should consider patenting - US, China, India or UK (based in the UK).

The obvious answer was "depends on your market" - but as he said that everyone in the forum commented that most businesses these days are global.... just as he said it :) Good to know we're all on the same page.

His answer was US, China, India and Europe (i assume you can get a pan-european patent).

I wonder how soon until China comes at the start of that list?!

Peer Patents

2007-08-01T13:50:00.001-07:00

After a chat whilst watching Always On today (where i got two questions answered by the panel!!), i decided to join the PeerToPatent project.

Peer-to-Patent involves 1) review and discussion of posted patent applications,
2) research to locate prior art references 3) uploading prior art references
relevant to the claims, 4) annotating and evaluating submitted prior art, and 5)
top ten references, along with commentary, forwarded to the USPTO. The goal of
this pilot is to prove that organized public participation can improve the
quality of issued patents.

I would ideally like to see a process that is startup friendly and allows you to have your patent validted by the community and reserved for one (or two) years until you ahve the money to pay for it. We all know that for all butthe larger companies, getting patents in more that one country is using finances you really can ill afford to spend on anything other then keeping your business alive!

Anyway, I hope to be able to help out.