Journal tags: location

Yesterday when I mentioned my paranoia of third-party dependencies on The Session, I said:

I’ve built in the option to switch between multiple geocoding providers. When one of them inevitably starts enshittifying their service, I can quickly move on to another. It’s like having a “go bag” for geocoding.

(Geocoding, by the way, is when you provide a human-readable address and get back latitude and longitude coordinates.)

My paranoia is well-founded. I’ve been using Google’s geocoding API, which is changing its pricing model from next March.

You wouldn’t know it from the breathlessly excited emails they’ve been sending about it, but this is not a good change for me. I don’t do that much geocoding on The Session—around 13,000 or 14,000 requests a month. With the new pricing model that’ll be around $15 to $20 a month. Currently I slip by under the radar with the free tier.

So it might be time for me to flip that switch in my code. But which geocoding provider should I use?

There are plenty of slop-like listicles out there enumerating the various providers, but they’re mostly just regurgitating the marketing blurbs from the provider websites. What I need is more like a test kitchen.

Here’s what I did…

I took a representative sample of six recent additions to the sessions section of thesession.org. These examples represent places in the USA, Ireland, England, Scotland, Northern Ireland, and Spain, so a reasonable spread.

For each one of those sessions, I’m taking:

• the venue name,
• the town name,
• the area name, and
• the country.

I’m deliberately not including the street address. Quite often people don’t bother including this information so I want to see how well the geocoding APIs cope without it.

I’ve scored the results on a simple scale of good, so-so, and just plain wrong.

• A good result gets a score of one. This is when the result gives back an accurate street-level result.
• A so-so result gets a score of zero. This when it’s got the right coordinates for the town, but no more than that.
• A wrong result gets a score of minus one. This is when the result is like something from a large language model: very confident but untethered from reality, like claiming the address is in a completely different country. Being wrong is worse than being vague, hence the difference in scoring.

Then I tot up those results for an overall score for each provider.

When I tried my six examples with twelve different geocoding providers, these were the results:

Geocoding providers

Provider	USA	England	Ireland	Spain	Scotland	Northern Ireland	Total
Google	1	1	1	1	1	1	7
Mapquest	1	1	1	1	1	1	7
Geoapify	0	1	1	0	1	0	3
Here	1	1	0	1	0	0	3
Mapbox	1	1	0	1	1	-1	3
Bing	1	0	0	0	0	0	1
Nominatim	0	0	0	0	-1	1	0
OpenCage	-1	1	0	0	0	-1	-1
Tom Tom	-1	-1	0	0	-1	1	-2
Positionstack	0	-1	0	-1	1	-1	-2
Locationiq	-1	0	-1	0	0	-1	-3
Map Maker	-1	0	-1	-1	-1	-1	-5

Some interesting results there. I was surprised by how crap Bing is. I was also expecting better results from Mapbox.

Most interesting for me, Mapquest is right up there with Google.

So now that I’ve got a good scoring system, my next question is around pricing. If Google and Mapquest are roughly comparable in terms of accuracy, how would the pricing work out for each of them?

Let’s say I make 15,000 API requests a month. Under Google’s new pricing plan, that works out at $25. Not bad.

But if I’ve understood Mapquest’s pricing correctly, I reckon I’ll just squeek in under the free tier.

Looks like I’m flipping the switch to Mapquest.

If you’re shopping around for geocoding providers, I hope this is useful to you. But I don’t think you should just look at my results; they’re very specific to my needs. Come up with your own representative sample of tests and try putting the providers through their paces with your data.

If, for some reason, you want to see the terrible PHP code I’m using for geocoding on The Session, here it is.

The Session has been online for over 20 years. When you maintain a site for that long, you don’t want to be relying on third parties—it’s only a matter of time until they’re no longer around.

Some third party APIs are unavoidable. The Session has maps for sessions and other events. When people add a new entry, they provide the address but then I need to get the latitude and longitude. So I have to use a third-party geocoding API.

My code is like a lesson in paranoia: I’ve built in the option to switch between multiple geocoding providers. When one of them inevitably starts enshittifying their service, I can quickly move on to another. It’s like having a “go bag” for geocoding.

Things are better on the client side. I’m using other people’s JavaScript libraries—like the brilliant abcjs—but at least I can self-host them.

I’m using Leaflet for embedding maps. It’s a great little library built on top of Open Street Map data.

A little while back I linked to a new project called OpenFreeMap. It’s a mapping provider where you even have the option of hosting the tiles yourself!

For now, I’m not self-hosting my map tiles (yet!), but I did want to switch to OpenFreeMap’s tiles. They’re vector-based rather than bitmap, so they’re lovely and crisp.

But there’s an issue.

I can use OpenFreeMap with Leaflet, but to do that I also have to use the MapLibre GL library. But whereas Leaflet is 148K of JavaScript, MapLibre GL is 800K! Yowzers!

That’s mahoosive by the standards of The Session’s performance budget. I’m not sure the loveliness of the vector maps is worth increasing the JavaScript payload by so much.

But this doesn’t have to be an either/or decision. I can use progressive enhancement to get the best of both worlds.

If you land straight on a map page on The Session for the first time, you’ll get the old-fashioned bitmap map tiles. There’s no MapLibre code.

But if you browse around The Session and then arrive on a map page, you’ll get the lovely vector maps.

Here’s what’s happening…

The maps are embedded using an HTML web component called embed-map. The fallback is a static image between the opening and closing tags. The web component then loads up Leaflet.

Here’s where the enhancement comes in. When the web component is initiated (in its connectedCallback method), it uses the Cache API to see if MapLibre has been stored in a cache. If it has, it loads that library:

caches.match('/path/to/maplibre-gl.js')
.then( responseFromCache => {
    if (responseFromCache) {
        // load maplibre-gl.js
    }
});

Then when it comes to drawing the map, I can check for the existence of the maplibreGL object. If it exists, I can use OpenFreeMap tiles. Otherwise I use the old Leaflet tiles.

But how does the MapLibre library end up in a cache? That’s thanks to the service worker script.

During the service worker’s install event, I give it a list of static files to cache: CSS, JavaScript, and so on. That includes third-party libraries like abcjs, Leaflet, and now MapLibre GL.

Crucially this caching happens off the main thread. It happens in the background and it won’t slow down the loading of whatever page is currently being displayed.

That’s it. If the service worker installation works as planned, you’ll get the nice new vector maps. If anything goes wrong, you’ll get the older version.

By the way, it’s always a good idea to use a service worker and the Cache API to store your JavaScript files. As you know, JavaScript is unduly expensive to performance; not only does the JavaScript file have to be downloaded, it then has to be parsed and compiled. But JavaScript stored in a cache during a service worker’s install event is already parsed and compiled.

Over on the Failed Architecture site, there’s a piece about Kevin Lynch’s 1960 book The Image Of The City. It’s kind of fun to look back at a work like that, from today’s vantage point of ubiquitous GPS and smartphones with maps that bestow God-like wayfinding. How much did Lynch—or any other futurist from the past—get right about our present?

Quite a bit, as it turns out.

Lynch invented the term ‘imageability’ to describe the degree to which the urban environment can be perceived as a clear and coherent mental image. Reshaping the city is one way to increase imageability. But what if the cognitive map were complemented by some external device? Lynch proposed that this too could strengthen the mental image and effectively support navigation.

Past visions of the future can be a lot of fun. Matt Novak’s Paleofuture blog is testament to that. Present visions of the future are rarely as enjoyable. But every so often, one comes along…

Kevin Kelly has a new piece in Wired magazine about Augmented Reality. He suggests we don’t call it AR. Sounds good to me. Instead, he proposes we use David Gelernter’s term “the mirrorworld”.

I like it! I feel like the term won’t age well, but that’s not the point. The term “cyberspace” hasn’t aged well either—it sounds positively retro now—but Gibson’s term served its purpose in prompting discussing and spurring excitement. I feel like Kelly’s “mirrorworld” could do the same.

Incidentally, the mirrorworld has already made an appearance in the William Gibson book Spook Country in the form of locative art:

Locative art, a melding of global positioning technology to virtual reality, is the new wrinkle in Gibson’s matrix. One locative artist, for example, plants a virtual image of F. Scott Fitzgerald dying at the very spot where, in fact, he had his Hollywood heart attack, and does the same for River Phoenix and his fatal overdose.

Yup, that sounds like the mirrorworld:

Time is a dimension in the mirror­world that can be adjusted. Unlike the real world, but very much like the world of software apps, you will be able to scroll back.

Now look, normally I’m wary to the point of cynicism when it comes to breathless evocations of fantastical futures extropolated from a barely functioning technology of today, but damn, if Kevin Kelly’s enthusiasm isn’t infectious! He invokes Borges. He acknowledges the challenges. But mostly he pumps up the excitement by baldly stating possible outcomes as though they are inevitabilities:

We will hyperlink objects into a network of the physical, just as the web hyperlinked words, producing marvelous benefits and new products.

When he really gets going, we enter into some next-level science-fictional domains:

The mirrorworld will be a world governed by light rays zipping around, coming into cameras, leaving displays, entering eyes, a never-­ending stream of photons painting forms that we walk through and visible ghosts that we touch. The laws of light will govern what is possible.

And then we get sentences like this:

History will be a verb.

I kind of love it. I mean, I’m sure we’ll look back on it one day and laugh, shaking our heads at its naivety, but for right now, it’s kind of refreshing to read something so unabashedly hopeful and so wildly optimistic.

I was in Berlin for most of last week, and every day was packed with activity:

• Indie Web Camp on Saturday and Sunday,
• Accessibility Club on Monday,
• Beyond Tellerrand on Tuesday and Wednesday.

By the time I got back to Brighton, my brain was full …just in time for FF Conf.

All of the events were very different, but equally enjoyable. It was also quite nice to just attend events without speaking at them.

Indie Web Camp Berlin was terrific. There was an excellent turnout, and once again, I found that the format was just right: a day of discussions (BarCamp style) followed by a day of doing (coding, designing, hacking). I got very inspired on the first day, so I was raring to go on the second.

What I like to do on the second day is try to complete two tasks; one that’s fairly straightforward, and one that’s a bit tougher. That way, when it comes time to demo at the end of the day, even if I haven’t managed to complete the tougher one, I’ll still be able to demo the simpler one.

In this case, the tougher one was also tricky to demo. It involved a lot of invisible behind-the-scenes plumbing. I was tweaking my webmention endpoint (stop sniggering—tweaking your endpoint is no laughing matter).

Up until now, I could handle straightforward webmentions, and I could handle updates (if I receive more than one webmention from the same link, I check it each time). But I needed to also handle deletions.

The spec is quite clear on this. A 404 isn’t enough to trigger a deletion—that might be a temporary state. But a status of 410 Gone indicates that a resource was once here but has since been deliberately removed. In that situation, any stored webmentions for that link should also be removed.

Anyway, I think I got it working, but it’s tricky to test and even trickier to demo. “Not to worry”, I thought, “I’ve always got my simpler task.”

For that, I chose to add a little map to my homepage showing the last location I published something from. I’ve been geotagging all my content for years (journal entries, notes, links, articles), but not really doing anything with that data. This is a first step to doing something interesting with many years of location data.

I’ve got it working now, but the demo gods really weren’t with me at Indie Web Camp. Both of my demos failed. The webmention demo failed quite embarrassingly.

As well as handling deletions, I also wanted to handle updates where a URL that once linked to a post of mine no longer does. Just to be clear, the URL still exists—it’s not 404 or 410—but it has been updated to remove the original link back to one of my posts. I know this sounds like another very theoretical situation, but I’ve actually got an example of it on my very first webmention test post from five years ago. Believe it or not, there’s an escort agency in Nottingham that’s using webmention as a vector for spam. They post something that does link to my test post, send a webmention, and then remove the link to my test post. I almost admire their dedication.

Still, I wanted to foil this particular situation so I thought I had updated my code to handle it. Alas, when it came time to demo this, I was using someone else’s computer, and in my attempt to right-click and copy the URL of the spam link …I accidentally triggered it. In front of a room full of people. It was midly NSFW, but more worryingly, a potential Code Of Conduct violation. I’m very sorry about that.

Apart from the humiliating demo, I thoroughly enjoyed Indie Web Camp, and I’m going to keep adjusting my webmention endpoint. There was a terrific discussion around the ethical implications of storing webmentions, led by Sebastian, based on his epic post from earlier this year.

We established early in the discussion that we weren’t going to try to solve legal questions—like GDPR “compliance”, which varies depending on which lawyer you talk to—but rather try to figure out what the right thing to do is.

Earlier that day, during the introductions, I quite happily showed webmentions in action on my site. I pointed out that my last blog post had received a response from another site, and because that response was marked up as an h-entry, I displayed it in full on my site. I thought this was all hunky-dory, but now this discussion around privacy made me question some inferences I was making:

1. By receiving a webention in the first place, I was inferring a willingness for the link to be made public. That’s not necessarily true, as someone pointed out: a CMS could be automatically sending webmentions, which the author might be unaware of.
2. If the linking post is marked up in h-entry, I was inferring a willingness for the content to be republished. Again, not necessarily true.

That second inferrence of mine—that publishing in a particular format somehow grants permissions—actually has an interesting precedent: Google AMP. Simply by including the Google AMP script on a web page, you are implicitly giving Google permission to store a complete copy of that page and serve it from their servers instead of sending people to your site. No terms and conditions. No checkbox ticked. No “I agree” button pressed.

Just sayin’.

Anyway, when it comes to my own processing of webmentions, I’m going to take some of the suggestions from the discussion on board. There are certain signals I could be looking for in the linking post:

• Does it include a link to a licence?
• Is there a restrictive robots.txt file?
• Are there meta declarations that say noindex?

Each one of these could help to infer whether or not I should be publishing a webmention or not. I quickly realised that what we’re talking about here is an algorithm.

Despite its current usage to mean “magic”, an algorithm is a recipe. It’s a series of steps that contribute to a decision point. The problem is that, in the case of silos like Facebook or Instagram, the algorithms are secret (which probably contributes to their aura of magical thinking). If I’m going to write an algorithm that handles other people’s information, I don’t want to make that mistake. Whatever steps I end up codifying in my webmention endpoint, I’ll be sure to document them publicly.

Once I finished my workshop on evaluating technology I stayed in Nuremberg for that weekend’s Indie Web Camp.

Just as with Indie Web Camp Düsseldorf the weekend before, it was a fun two days—one day of discussions, followed by one day of making.

I spent most of the second day playing around with a new service that Aaron created called OwnYourSwarm. It’s very similar to his other service, OwnYourGram. Whereas OwnYourGram is all about posting pictures from Instagram to your own site, OwnYourSwarm is all about posting Swarm check-ins to your own site.

Usually I prefer to publish on my own site and then push copies out to other services like Twitter, Flickr, etc. (POSSE—Publish on Own Site, Syndicate Elsewhere). In the case of Instagram, that’s impossible because of their ludicrously restrictive API, so I have go the other way around (PESOS—Publish Elsewhere, Syndicate to Own Site). When it comes to check-ins, I could do it from my own site, but I’d have to create my own databases of places to check into. I don’t fancy that much (yet) so I’m using OwnYourSwarm to PESOS check-ins.

The great thing about OwnYourSwarm is that I didn’t have to do anything. I already had the building blocks in place.

First of all, I needed some way to authenticate as my website. IndieAuth takes care of all that. All I needed was rel="me" attributes pointing from my website to my profiles on Twitter, Flickr, Github, or any other services that provide OAuth. Then I can piggyback on their authentication flow (this is also how you sign in to the Indie Web wiki).

The other step is more involved. My site needs to provide an API endpoint so that services like OwnYourGram and OwnYourSwarm can post to it. That’s where micropub comes in. You can see the code for my minimal micropub endpoint if you like. If you want to test your own micropub endpoint, check out micropub.rocks—the companion to webmention.rocks.

Anyway, I already had IndieAuth and micropub set up on my site, so all I had to do was log in to OwnYourSwarm and I immediately started to get check-ins posted to my own site. They show up the same as any other note, so I decided to spend my time at Indie Web Camp Nuremberg making them look a bit different. I used Mapbox’s static map API to show an image of the location of the check-in. What’s really nice is that if I post a photo on Swarm, that gets posted to my own site too. I had fun playing around with the display of photo+map on my home page stream. I’ve made a page for keeping track of check-ins too.

All in all, a fun way to spend Indie Web Camp Nuremberg. But when it came time to demo, the one that really impressed me was Amber’s. She worked flat out on her site, getting to the second level on IndieWebify.me …including sending a webmention to my site!

It’s been just a week since Clearleft unveiled the Map Tales project that we built at Hackfarm and there have already been some great stories told with the site.

Paul documented his 2009 road trip to South by Southwest.

Alessio put together a photographic guide to his adopted home, showing the secrets of Barcelona.

Andy told two tales of two different trips: wine-tasting in California’s Dry Creek Valley and hanging with the hipsters in East London.

Fellow Brightonian Tom Prior has recreated the story of the famous Stirling Moss victory at the 1955 Mille Miglia, the legendary open-road endurance race in Northern Italy.

I love the simplicity of Oliver and Peter Walk to School that Peter Ruk has embedded on his site—beautifully simple psychogeography.

I’ve made a map tale of the voyage of The Beagle with material fromAboutDarwin.com.

Meanwhile Anna is putting together the tale of the Terra Nova expedition to the South Pole because—get this—a relative of hers was part of Scott’s team!

There’s plenty of room for improvement with Map Tales. It would be nice to have customisation options at some point—colours, fonts, maybe even map tiles. Some narratives would probably work better with aerial imagery, for example. In fact, that’s something that Andy has been tirelessly tinkering with. To get a taste of how that looks, check out Britain From Above, the epic map tale of the 2008 BBC documentary series.

We have some new location-centric toys to play with. Let the hacking commence.

Flickr has released its shapefiles dataset for free (as in beer, as in it would be nice if you mentioned where you got the free beer). These shapefiles are bounding boxes that have been generated by the action of humans correcting suggested place names for geotagged photos. Tom put this data to good use with his neighbourhood boundaries app.

Speaking of excellent location-driven creations by Tom, be sure to check out Clarke; a little OS X app that updates your FireEagle location every five minutes by triangulating your position with Skyhook.

Meanwhile, in another part of Yahoo, Placemaker has been released in Beta form. It looks very nifty indeed. Pass it some human-readable text and it will try to figure out what physical locations are mentioned in the text. You can help it along by using structured data like the geo and adr microformats, but it seems to be pretty good at natural language parsing. Christian has put together some good examples to illustrate his JavaScript Placemaker/YQL mashup.

Slowly but surely we’re heading towards a future where everything is geotagged.

There’s been some really interesting stuff coming out of Mozilla Labs lately. The latest toy is a plugin called Geode.

It’s based on the W3C editor’s draft geolocation API. In a nutshell, it allows you to provide your location to a website at the click of a button. You can try it for yourself on Pownce.

Now, I have no idea where it’s getting the location data from—probably a mixture of WiFi and network information a la Plazes—but I don’t need to know or care. What’s important is that it works. It works to such an extent that it’s close to being indistinguishable from magic. Sitting in the Clearleft office at 28 Kensington Street in Brighton, Geode updated my Pownce location as 9 Kensington Street in Brighton. That’s pretty damn close.

Little by little, we’re getting there:

I look forward to the day when geostamps are as ubiquitous as timestamps. If every image, every blog post, every video, every sound file had a longitude and latitude as well as a date and time… I can’t even begin to imagine the possibilities that would open up.

I’m looking forward to getting to San Francisco this weekend. Mostly that’s because I’ll be seeing so many of my friends there. But there’s a lesser reason that’s so geeky I’m almost ashamed to admit it…

At some stage while I’m online in San Francisco, I will, no doubt, visit my Pownce profile—where I post something almost every day—and I will take great delight in seeing my location listed as San Francisco, CA rather than the usual Brighton, UK because that’s what Fire Eagle will have told Pownce. Fire Eagle will know this because my visit to San Francisco is listed on my Dopplr account. Dopplr talks to Fire Eagle. Pownce talks to Fire Eagle. In a roundabout way, Dopplr talks to Pownce.

In case you missed it, Fire Eagle is out of beta. Go forth and explore the apps.

What’s missing from that list is a kick-ass iPhone app that would do its this app wants to know your location trick to update Fire Eagle (and therefore Pownce, Dopplr and soon, Twitter) on the go. I hereby invoke the LazyMobileWeb to build such an app. I wish I could offer some kind of modern day version of a longitude prize for geeks on the move.

At this year’s dConstruct, George treated us all to a sneak peak of a new location-based feature on Flickr designed to solve the sunset problem with Interestingness®. It’s launched a few weeks ago. It’s called Places and it’s basically a mashup of location and interestingness®. Kellan has written about it—revealing a nice secret feature—and Dan has given us an insight into the design of the URLs.

Like most people, the first thing I did was to look at my own town. I really like the “Featured Photographers” bit. That turns out to be especially useful or those places that bear watching for topical, rather than personal, reasons. Take a look at the page for Baghdad. It’s not quite citizen journalism—soldiers belong to a narrow band of citizenry—but it’s a great way of seeing pictures from the ground without the intervention of a media filter.

Speaking of interesting locations, Dopplr has now officially left Beta and opened up its doors to everyone. Like Tom, I’ve found it to be surprisingly useful. It’s already got some nice Flickr integration and Aaron has been playing around with some automated tagging between the two sites.

Today I travelled from home to work, from work to band practice, from band practice to an educational celebration: OpenStreetMap Brighton 1.0.

Ever since the mapping workshop after dConstruct 2006, Mikel and others have been out and about improving the mapping data for Brighton from the ground up. While a map can never be truly finished—it is, after all, a representation of a changing, evolving place—the data is now remarkably complete.

There’s a natural tendency for us to think in our own domains of experience so I usually only see the potential for OpenStreetMap data in web applications and mashups. But the launch event showed some wonderful use-cases in the real world: local councils, public transport… these are organisations that would otherwise have to pay very large sums (of taxpayer’s money) to the Ordnance Survey just to display a map.

OpenStreetMap is one of those applications of technology, like Wikipedia or BarCamp, that fills me with hope. On paper, the concepts sound crazy. In reality, they don’t just compete with commercial services, they surpass them.

I really need to get myself a GPS device.

A couple of months ago I wrote:

Jessica speculated a while back about reverse Google Maps. Suppose that when you entered an address, instead of just showing you the top-down view of that point on the planet, you also got to see how the sky would look from that point. Enter a postcode; view the corresponding starmap.

It isn’t in Google Maps yet but it is in Google Earth. The newest version features a button labeled “Switch between Sky and Earth”. This new Sky feature allows you to navigate photographs of space taken from the Palomar observatory and the Hubble telescope. It’s just one more example of what you can do with geodata.

Location information is the basis for a lot of the mashups out there—of which, Overplot remains my favourite. The possibilities in mashing up geodata with timestamps are almost limitless.

Getting datetime information is relatively easy. Every file created on a computer has a timestamp. Almost everything published on the Web is also timestamped: that’s the basis of lifestreams.

I look forward to the day when geostamps are as ubiquitous as timestamps. If every image, every blog post, every video, every sound file had a longitude and latitude as well as a date and time… I can’t even begin to imagine the possibilities that would open up.

I’m not the only one thinking about this. Responding to the question, what parts of the Web need to be improved or fixed in order for the Web of today to evolve into the Web of the future?, Jeff Veen writes:

I wish every device that was capable of talking to the network could send its geolocation. I’d like this to be fundamental—let’s send longitude and latitude in the HTTP header of every request. Let’s make it as ubiquitous and accessible as the time stamp, user agent, and referring URL.

This doesn’t necessarily mean that every electronic device needs to be geo-aware. As long as devices can communicate easily, you may ever only need one location-aware device. Suppose my phone has GPS or some other way of pinpointing location. As long as that device can communicate with my computer, perhaps using Bluetooth, then my computer can know my location: a very short string of two numbers. Once my computer has that data, my location can be broadcast and a whole ecosystem of services can be enhanced. Sites built around travel or events are the obvious winners but I can imagine huge benefits for music sites, photo sharing or any kind of social networking site that boils down to real-world activity.

The technology isn’t quite ubiquitous enough yet and there are privacy concerns (though the granularity of geodata negates a lot of the worst fears) but I hope that as the usefulness of geodata becomes clearer, location enhanced services can really begin to bloom.