Skip to: Site menu | Main content

>> Massive web growth

Wed, Feb 25th 8:33am 2009 >> Web Development

The February Netcraft Web Server Survey just came out, and the figures are rather dumbfounding. Netcraft perform an automated survey each month in an attempt to create a definitive list of every single website in existence, and along the way they collect a bunch of data about what is the most popular web server software, how many sites use SSL certificates, all that sort of thing. The Feb survey shows a big spike due to the addition of about 20 million sites hosted by QZHTTP, a big Chinese hosting provider. Added to the more "normal" growth of the web that means there were 30 million new websites added between January and February this year! That's a million new sites per day. Or 41,000 sites per hour. Or 700 sites per minute. 12 sites per second! Pretty crazy. Even if you remove the unusual spike caused by QZHTTP it's still amazing growth: nearly 14,000 new sites created every hour, not counting the massive growth in China. I don't think the web has quite run out of steam yet.

Bookmark and Share

>> The power of place

Thu, Aug 28th 2:44pm 2008 >> Web Development

Just like the hottest rock bands, the latest "hot new thing" in e-business may seem to suddenly appear over night when in fact it's been in gestation for years, bubbling away below mainstream consciousness and appreciated by just a few bleeding-edge early adopters who saw the potential before it became famous. Then at some point in time a confluence of events results in an "ah-ha!" moment in the collective psyche, and all of a sudden the latest hot new thing seems to be everywhere you look.

I believe we're currently right on the cusp of just such a moment, and in the next 12 months we'll see a certain concept go from obscure "why on Earth would anyone want that?" status to "can't live without it!" ubiquity.

That concept is location-based services.

The great catch-cry in the early days of the internet was globalisation: the concept that an obscure little company in a backwoods country town could throw up a website and gain instant access to a global market and compete with existing multinationals, with their location being irrelevant. Nice in theory, but of course there were all sorts of catches and even now it's a rare business indeed that can trade online without consideration for the geographic location of its customers.

So people are now starting to realise that things like search results really need to take into account geographic location. I can't even guess the number of times I've done a web search over the years and wished I could apply a rule such as "only show me results for businesses within 20km of my current location". When you're searching for somewhere to buy a washing machine it really matters where it's located: the electrical goods retailer with the best washing machine website in the world isn't going to be a lot of use to you if you're in Melbourne and they're in Minneapolis or Mayfair. I dream of the day that Google Maps and Google Search are merged into one, and I can select an area on the map and say "search for 'dog grooming service' right *there*", or "search for 'scuba dive operator' right around this area *here*".

Most of the time we *want* location to be relevant, but on the internet we've lost the sense of physical context and proximity that is so important when dealing with people and businesses in the real world. We've been stripped of something that is fundamental to the way our brains are wired.

Another application where location could provide enormous additional value is social networking. For years I've talked about a hypothetical device that people could carry around in their pockets that would be a sort of "proximity alert" that tells you when a friend or colleague is nearby, allowing you to stumble upon chance meetings that right now are probably passing you by. Walking down the street there are times you meet people you know, but it's likely that far more often someone you know will have walked down that same street one minute before, or one minute later, or have stepped into a shop while you walk past, or been on the other side of the street. Wouldn't it be cool to have a device in your pocket that could say "hey, your friend Mark is just over the street in that coffee shop!" rather than walk past totally unaware? And for someone like me who travels a lot it would be particularly handy, because by expanding the "alert" range from say 100 meters to perhaps 20km when I travel to another city it would help me catch up with people I don't get to see very often.

You know what? You may not have heard about it yet, but those devices are in mass production right now by big-name companies including Samsung, Sony, and Nokia. In fact you probably already have one in your pocket. It's called a mobile phone.

A moment ago I talked about location-based services suddenly becoming a recognised mainstream phenomenon, and there are a couple of trigger events that are bringing it about.

The first trigger event is that almost all new mobile phones now have GPS built in: a while ago it was unheard of for phones to include cameras, but now many of them have two. Likewise phones with GPS have been few and far between, but very soon it'll be almost impossible to buy one without it. GPS will be everywhere, in everyone's pocket, and nobody will think twice about it.

The second trigger event is the release of simple, easy-to-use software and online services that take advantage of the ubiquity of personal GPS. That's the stage we're at right now: the hardware platform is out there in people's pockets, and now enterprising developers are dreaming up new ways to utilise that platform. One perfect example of the sort of building block currently being put in place is a new service from Yahoo! called Fire Eagle (, which might sound a little bit obscure at first but has enormous potential to change the way online business is conducted. The simplest way to understand what Fire Eagle does is think of it as a pinboard where you can post a note stating your current location, and that information can then be used by third parties to provide you with more relevant services. Fire Eagle itself doesn't do anything "useful" as far as an end user is concerned, but as a building block for other services it's critical.

The way it works is that you create a Fire Eagle account, and then use one or more methods to regularly update your location within the Fire Eagle system. You can do it manually by logging into the website and setting it, or you can run a little program on your mobile phone that regularly checks your location by GPS and updates it automatically, or by linking your Fire Eagle account to a travel planning service like Dopplr that knows what cities you will be in and when.

With your current location in the Fire Eagle system you can then authorise other services to make use of that information. Examples include websites like, which looks up your location and cross-references it to Wikipedia articles related to places nearby - a great way to find random interesting things in your vicinity that you may never have been aware of previously! Or, which is a search engine that personalises your search results based on factors including interests of people within your social network and your current location. Or, which (provided you're in New York or San Francisco!) uses your location to find nearby restaurants, bars, movies, and shops. Or, which does the same thing but for people you know who happen to be nearby. Zkout even provides a live map which updates to show what's going on around you.

How does this relate to e-business? Right now: not much. In the near future: a lot.

As users become aware of the power of location-awareness we're going to see a lot more services spring up that take advantage of it, and people are going to start expecting websites to "locate" themselves geographically. For example, websites will need to have location metadata embedded so that they will appear in search results when users search for things like "cheap washing machines within 20km of my current location".

And if we're really lucky we'll see an end to those really annoying websites that ask you to select from a country before you can proceed: websites should just know automatically where you're located, and behave accordingly. As a side issue while talking about this, one of my all-time most hated websites is, and for one specific reason: it won't even let you get to the home page until you tell it your postcode! They've got the right idea, but it's implemented in such an obnoxiously obtrusive way that it drives me nuts.

So in a year or two when you're doing a web search and restricting the results to your local area, or walking down the street and your phone tells you that your best friend is two blocks away, it will probably seem like the most natural thing in the world. Once again science fiction becomes a typical everyday event.

Bookmark and Share

>> Social networking, online video, and brand promotion

Thu, Jul 31st 9:40am 2008 >> Web Development

Over the last couple of weeks I've been watching an interesting project unfold as Samsung run a promotion here in Australia called "Samsung People". The promotion has been carefully designed to use the power of social networking to increase awareness of their brand and particularly of a new product, the MX10 video camera. I'm not involved in running the project in any way but I am involved as a contestant, so I've been able to see a bit of what happens behind the scenes as they put all the pieces in place and it certainly makes an interesting case study of how companies are starting to use social networking for marketing purposes. This particular promotion has several stages, each designed to encourage people to use their own social networks to get others involved and therefore exposed to the brand. The basic premise is to promote the Samsung MX10 video camera, which uses a memory card instead of video tapes or hard disk and is designed to be very simple to operate. Samsung wanted to spread the message that this camera is ideal for people wanting a basic camera to post video diaries, put video online on sites such as YouTube, and generally communicate with their friends and family. So they hired a promotions company who equipped a number of staff with the MX10 and sent them out into the streets of Sydney to talk to random people and ask them the question "where is your mind?", recording the interviews and posting them on a special "Samsung People" website that they set up for the project. They also invited people to enter a competition by answering the "where is your mind?" question online at the Samsung People site. The next stage was selecting ten finalists from the people who entered the competition, and this is where the social networking effect really kicks in. They gave each of those ten finalists an MX10 camera and video editing software, with instructions that they have to keep a video diary until the end of the competition period at the end of August. The rules state that finalists are only allowed to use the camera and software supplied by Samsung, and the video diaries must be hosted on the Samsung People website. Finalists were each given a username / password so they can log in and upload new diary entries whenever they want to. During this stage of the competition the public can view the video diaries and vote for the finalist they think should win. This obviously means that it's in the contestants own self interest to promote the competition as much as they can, and reach out using their social networks (both in the real world and also online using sites such as Facebook and LinkedIn) to get as many people voting for them as possible. This turns the contestants (and, to some extent, their first and second degree friends who will want to help their friend out of loyalty) into active promoters of the competition. Each contestant's vested interest in winning the competition becomes their motivation for promoting it, and, in turn, promoting the company behind it. Very, very clever. Of course, *I* have a vested interest because I'm one of the ten finalists! The grand prize is quite significant (a trip for two to Tokyo, New York, and London) and I'd love to win it, but quite apart from that I'm fascinated by the way the competition has been engineered to maximise exposure for Samsung. There are some serious technical problems with the way they have implemented the diary system itself, but as a concept it's very interesting and probably a good indicator of the sorts of promotions we'll see more of in future as companies start to gain some understanding of what web 2.0 and social networking actually means. So please vote for me! I'll send you a postcard from London if I win ;-) Oh yeah, and you can find the competition site at

Bookmark and Share

>> Managing email overload

Thu, May 1st 1:35pm 2008 >> Web Development

One of the most fundamental tools used in any form of e-business is email, but most of us don't really think about it - we just use it out of habit, not with any real plan. And as business becomes ever busier it's easy to become inundated with email and fall so far behind that it becomes useless and customers get frustrated with lack of responsiveness. In "How To Build A Website And Stay Sane" I talked about some of the macro aspects of email in e-business, such as establishing an acceptable response time, delegating responsibility for inquiry responses, and use of role addresses such as "" so responsibility can be re-delegated when staff are away. But I didn't talk about how to actually manage what ends up in your inbox - how to file it, archive it, and prioritise it. Over the years I've tried a few different approaches to managing email. I personally receive about 1000 emails per day so my message load is probably a little bit higher than most, and that means I have to work really hard to keep up with it or I risk getting to the point where I have to declare email bankruptcy and just delete my inbox and start again. This morning I had 473 messages waiting for me when I got to work - and that was just new messages since I checked it before I went to bed last night! So staying on top of email is one of the biggest burdens I have to face, and it's a problem faced by everyone heavily involved in e-business. For many years my preferred approach was to use a highly structured filing system that had many different folders each with multiple levels. For example, I had a "Clients" folder, then inside that I had one folder for each client. As time went on that became unwieldy and just opening the Clients folder presented me with a list of hundreds of clients, so I broke it down further into Current, Inactive, and Archive - but then I was frequently shuffling folders around, and I had to remember where things were stored. I also wrote a whole bunch of filing rules on the mail server so that whenever email came in it would be automatically directed into the correct folder without my email software having to do anything. But eventually it got to the point where it would take over half an hour each morning just to open my email software because it took so long working through the enormous list of folders, and I still ended up with over 50,000 unread emails in my inbox at one point. Not fun. My next tactic was to just give in to the inevitable and let the avalanche sweep over my inbox. I gave up on filing entirely, leaving everything in my inbox and relying on search to locate messages related to particular customers or friends. That's the basis on which Gmail works, and it's actually a pretty cool approach. It worked for me for a while but it also had problems. Search became really slow because I had something like 400,000 messages in my inbox and it was very hard to pick out messages that still needed action. So to improve search performance I created one archive folder for each year of email, and put all messages from 2006 into one folder, 2005 into another, etc. My inbox became all messages from my current year, with anything earlier being in a year-based archive. Basically it was the "full inbox" approach with trivial archiving. After a few months of that approach Arjen Lentz introduced me to the concept of "inbox zero", a technique developed by Merlin Mann from 37signals. Inbox zero is based on the "Getting Things Done" methodology made famous by David Allen and it's designed to leave you with an empty inbox at all times. The promise of email nirvana! I was sceptical at first but when you're drowning it doesn't matter what comes into reach - you'll grab at anything you can! So I thought I'd give it a try. What I've ended up with is a system that's a blend of my previous year-based archive with the "inbox zero" prioritisation technique applied as a triage mechanism on incoming messages. I still have archives per year, but rather than use the inbox for all messages from the current year I also created an archive for the current year, and the objective of the game is to get every single incoming message into the archive as fast as possible by applying some rules to them. And that's where the "getting things done" mindset comes into play. The first discipline is to never read an email twice while it's in your inbox. My natural tendency is to read an email, think "ok, I'll get back to that", and leave it there - but that's bad, because then you have to re-read it later to regain the context and actually do something about it. For every message that arrives in the inbox, you have to read it *once* and then classify it immediately. Do *not* think "I'll get back to that": classify it *now*, and don't take more than about 3 seconds thinking about each one. Which leads to the second discipline: classification. I use a slightly modified version of Mann's classifications. For every email that comes in I mentally throw it into one of 5 piles: Do, Delegate, Delete, Defer, Archive. It's become a bit of a mantra that runs through my head at all sorts of odd times! If it's something quick and all you need to do is dash off a reply, just Do it. Then file the email in the current archive. If it's something that can be passed on to a colleague, forward the email and then file it in a "Delegated" folder. If it's something that you really don't care about, like time-wasting "look at these funny pictures!" messages from that cousin you haven't seen in 6 years, delete it. If it's something that will take a bit more care and time to actually follow up on, file it in a "Defer" folder. If it's an email you want to keep for future reference but don't actually need to do anything about, file it in the "Archive" folder for the current year. After a few minutes working through your inbox you should have every single message either Done, Delegated, Deleted, Deferred, or Archived. And your inbox will be empty. A miracle has occurred! The third discipline is then to go through your Defer folder several times per day and action the items that will take some time and care. If you've been brutal enough with your classification process this will hopefully be a relatively short list. Once each item has been actioned, move it into the Archive folder. Once again your objective should be to achieve an empty Defer folder, but that's often not practical. I've heard of people using this technique who write a mail rule that takes anything which has been sitting in the Defer folder for more than 30 days and simply deletes it. After all, if you've deferred it for more than a month it's not likely you'll actually do anything about it, so why not just accept the inevitable? Personally I don't do that but I can understand why people do. The fourth discipline is to regularly go through your Delegated folder and check if there's anything you need to follow up. Often you'll delegate a task to a colleague and never hear the outcome, so the messages in the Delegated folder can act as a prompt that you need to check the status of that task and then move it into Archive once it's done. Once again your objective is to have an empty Delegated folder: a sign that all the tasks you are delegating to other people are being completed quickly. So there you have it: my technique for drinking from a fire-hose without being washed away. Everything comes into the Inbox, then is redirected into one of several folders, and eventually filters through to the archive for long term storage. Conceptually simple, but the trick is applying the discipline needed to make it work! There's a very cool video online of Merlin Mann doing a Google Tech Talk about inbox zero. It's well worth a look if you're anything like me and wish you didn't have to spend so much time dealing with email.

Bookmark and Share

>> Quickstart Guide to Google AdWords

Mon, Feb 25th 2:12pm 2008 >> Web Development

At CeBIT last year I grabbed a bunch of little booklets called "Insiders Guide to Google AdWords" from the Google stand, and since then we've been giving them away to interested customers. Then about a month ago in eBusiness News I did a story on the Google AdWords bid optimizer and offered to send readers the last few copies I had of Insiders Guide. The response was unbelievable. My last few copies were claimed within five minutes of eBusiness News going out, and I ended up with a spreadsheet of people who wanted it but missed out. "No problem," I thought, "I'll just get on the phone to the AdWords marketing team and grab some more copies. A box or two should do it." No such luck. It turns out that Insiders Guide is out of print, and the AdWords marketing team has no intention of producing more - which is a pity, because it's a useful little guide. Oh well, since I talk to a lot of people about online marketing and many of them want to try out AdWords, perhaps I could arrange to have extra copies printed at my expense if Google didn't have plans to produce more? "Absolutely not. It's Google copyright material and nobody else may reproduce it under any circumstances." Wow. So they don't want me helping to advertise their services and send them more customers by reproducing what is effectively their marketing material for them at my expense? OK, I can understand them wanting to protect their corporate image, but still! That left me in a pickle. Many of my customers are hassling me for a quickstart guide to AdWords, but Google won't let me use theirs. What's a guy to do? Write one myself, of course! So over the space of a couple of weeks I spent many late evenings writing "Quickstart Guide to Google AdWords", and now I'm happy to say it's all done so maybe I can start getting some sleep again. The first hard copies should arrive in Oz late this week or early next week but you can already order it online, and it should appear on and B&N and other random places once the Books In Print database does it's slow-as-a-snail update thing.

Bookmark and Share

>> Day-trip to Brisbane without leaving the airport

Thu, Jun 14th 10:25am 2007 >> Web Development

Yesterday was quite a day: up at 3:30am, early flight to Brisbane (watched the sunrise on the flight up), a day in a workshop with Brisbane Airport Corporation operations and security staff going over some of their internal processes, then onto a flight back to Melbourne (watched the sunset on the flight back). It's actually pretty convenient to visit clients that happen to run an airport. The flight destination *is* the trip destination! Even better when it's for an interesting project and the client's staff are very focused and switched-on. Workshops examining business processes generally tend to drift off into minutia and bikeshedding, but that didn't happen with BAC at all. Smart people + time pressure + a specific required outcome = productive meeting. I'd love to tell more about the project, but then I'd have to kill everyone that reads this blog. Or maybe just myself. Either way, it's not a desired outcome.

Bookmark and Share

>> FileMaker, you're wiggin me out, man

Wed, Mar 14th 1:58pm 2007 >> Web Development

Way back in the day (a day well over a decade past, I might add) I spent a lot of time doing FileMaker Pro development. Most of what I did was hooking them up to the Internet using nasty little Applescript hacks and such, but then I saw the light and didn't look back. Until recently. I'm in the process of converting not one but two art dealerships / galleries from internally developed FMP stock management systems to SiteBuilder-based intranets. They both have thousands of stock records in FMP that I need to somehow liberate into a more accessible format, and along the way I've developed a bunch of little tricks like field-at-a-time conversion (more about that in a later post) but the one that has been bending my brain today is how to get images out of the DB into an actual image file. For background, FMP has a "container" field type that is basically a binary blob. You can stick anything in it you like, including raw images: with a container field in a layout you can copy and paste (or even just drag in) things like JPEGs. Sweet. But it's kinda one-way. In the first conversion, the client had stored stock images externally and included a text field in the DB that referenced the image file, so that was a piece of cake. The second client had all the images stored *inside* the DB. But the kicker is that there is no way (at least in FMP v7) to export container fields. It just flat-out refuses to do it. The only way to get them out is to go through every record, select the image, copy it, go to a document, paste it, and save the document. Manually. For every record. AAAGGHHHHHH!!!! A bit of quick Googling showed up this post by Marc Liyanage who went to the extreme of using FMP script to execute an internal calculation to dynamically generate AppleScript which then generated and executed Shell. Now *there's* an example of determination, and maybe just a little bit of insanity. It's right up there with my OSDC lightning talk last year demonstrating a PHP script that generated code in OSDcLang which is a 3-syntax-element Turing-complete language which was then executed by an interpreter written in Ruby. But back to the story. Marc's effort was heroic, but I just couldn't get it quite working in the way he wrote it. So I simplified it and actually got the same end result with one less AppleScript and no shell at all. If you don't mind losing a few braincells just follow the bouncing ball: 1. Create a directory called "picture_export" in your home directory for the images to go in. 2. Go to Scripts -> ScriptMaker -> New..., and make yourself a script called "ImageExport" or somesuch. 3. Create the basic script structure like this: Go to Record/Request/Page [First] Loop Copy [Select; STOCK LIST 1-9-04::PICTURE] Perform AppleScript[] Go to Record/Request/Page [Next; Exit after last] End Loop In this case the DB name is "STOCK LIST 1-9-04", and "PICTURE" is the name of the container field with the images that we want to liberate. 4. Now the funky bit: the calculated AppleScript. Edit the "Perform AppleScript" line and set it to "Calculated AppleScript", then "Specify..." to get a dialog box to put it into. Hold onto your pants (and put some on first if you need to) because this is going to get weird: "set thePic to the clipboard" & ¶ & "try" & ¶ & " set thePic to JPEG picture of thePic" & ¶ & "set thePath to (path to home folder as Unicode text) & "picture_export:" & "" & ${STOCK LIST 1-9-04}::Registration number & ".jpeg"" & ¶ & "set fileRef to open for access file thePath with write permission" & ¶ & "set eof fileRef to 0" & ¶ & "«event rdwrwrit» thePic given «class refn»:fileRef" & ¶ & "close access fileRef" & ¶ & "on error" & ¶ & "end try" & ¶ Basically the calculation is a simple text concatenation that dynamically generates the AppleScript on each execution. That's why there are bizarro Apple paragraph characters appended to each line: without them the AppleScript parser can't figure out where the lines end. Update the database name reference to suit, and in this case "Registration number" is a unique, auto-incrementing serial number field that we're using so the file names are unique and we can relate them back to DB records. 5. Run the script. Enjoy!

Bookmark and Share

>> Thinking on my feet

Thu, Nov 24th 9:47pm 2005 >> Web Development

I was invited to be a guest speaker tonight at a post-grad class at Swinburne Uni, talking to a bunch of people about eBusiness. Turns out to be not quite what I expected! I had prepared on the basis that I was to talk to students intending to become eBusiness professionals so I intended to do a talk about the life of a web developer, but when I got there I discovered it was a business management course and what they really wanted to know about was use of eBusiness in their businesses.

It's a good thing I didn't put any serious preparation into the slides because I had to just ad-lib it anyway. Still managed to talk for an hour and a half though!

Bookmark and Share

1 of 2 >