Posts Tagged ‘tagging’

Tags in Space

A lot of you enjoyed our post (“Found in Space”) on the amazing astrometry.net project, and there have been some interesting followups.

A mysterious figure known only as “jim” paired up astronomy photos from Flickr with Google Sky. (You’re going to need the Google Earth plug-in for your browser — just follow the instructions on that page if you don’t have it.) In his technical writeup, “jim” explains how he used the Yahoo Query Language (YQL) to fetch the data. YQL is similar to the existing Flickr APIs, but it’s a query language like SQL rather than a set of REST-ish APIs. And both of those are really just ways to get data out of Flickr’s machine tag system, specifically the astro:* namespace. It’s turtles all the way down.

Who else is using astrotags? The British Royal Observatory in Greenwich is sponsoring a contest to determine the Astronomy Photographer of the Year and the whole thing is based on a Flickr group and extensive use of Flickr’s APIs. The integration is so seamless — galleries of photos and discussions are surfaced on their site as well as ours — you might as well consider Flickr to be their “backend” server. But they’ve also added much, such as great documentation about how to astrotag your photos as well as a concise explanation about how Astrometry.net identifies your photo, even among millions of known stars. (The sci-fi website io9 interviewed Fiona Romeo of the Royal Observatory about the contest; check it out.)

It’s dizzying how many services have been combined here — Astrometry.net grew out of research at the University of Toronto, web mashups use Google Sky for visualization in context, Yahoo infrastructure delivers and transforms data, the Royal Observatory at Greenwich provides leadership and expertise, and then little old Flickr acts as a data repository and social hub. And let’s not forget you, the Flickr community, and your inexhaustible creativity — which is the reason why all this can even come together.

All this was done with pretty light coordination and few people at Flickr were even aware what was going on until recently. I have no idea what the future is for APIs and a web of services loosely joined, but I hope we get to see more and more of this sort of thing.

Machine Tag Hierarchies

Untitled Compass #1228516024

something:somethingelse=somethingspecific

With apologies to Jeremy Keith

If you’re not already familiar with machine tags the easiest way to think of them is being like a plain old tag but with a special syntax that allows users to define additional structured data about that tag. If you’d like to know more, the best place to start is the official announcement we made about machine tags in the Flickr API group.

If you want to know even more, still, take a look at:

Okay! Now that everyone is feeling warm and fuzzy about machine tags: We’ve added (4) new API methods for browsing the hierarchies of machine tags added to photos on the site. These are aggregate rollups of all the unique namespaces, predicates, values and pairs for public photos with machine tags.

For example, lots of people have added exif: related machine tags to their photos but there hasn’t been a way to know what kind of EXIF data has been added: exif:model? exif:focal_length? exif:tunablaster? Or what about all the planespotters who have been diligently adding machine tags to their photos using the aero namespace: What are the predicates that they’re tagging their photos with?

Those are the sorts of things these methods are designed to help you find. Sort of like wildcard URLs but for metadata instead of photos. Uh, sort of.

Untitled Future #1223665161

Anyway, the new methods are:

flickr.machinetags.getNamespaces

This returns a list of all the unique namespaces, optionally bracketed by a specific predicate. For example, these are all the namespaces that have an airport predicate:

# ?method=flickr.machinetags.getNamespaces&predicate=airport

<namespaces predicate="airport" page="1" total="2" perpage="500" pages="1">
	<namespace usage="1931" predicates="1">aero</namespace>
	<namespace usage="3" predicates="1">geo</namespace>
</namespaces>
		

flickr.machinetags.getPredicates

Like the getNamespaces method this returns a list of all the unique predicates, optionally bracketed by a specific namespace. For example, these are all the predicates that use the dopplr namespace:

# ?method=flickr.machinetags.getPredicates&predicate=dopplr

<predicates namespace="dopplr" page="1" total="4" perpage="500" pages="1">
	<predicate usage="4392" namespaces="1">tagged</predicate>
	<predicate usage="1" namespaces="1">traveller</predicate>
	<predicate usage="7780" namespaces="1">trip</predicate>
	<predicate usage="4269" namespaces="1">woeid</predicate>
</predicates>
		

flickr.machinetags.getValues

At this point, the pattern should be pretty straightforward. This method returns all the unique values for a specific namespace/predicate pair. For example, these are some of the values associated with the aero:tail machine tag (yes, really, airplane tail models!):

# ?method=flickr.machinetags.getValues&namespace=aero&predicate=tail

<values namespace="aero" predicate="tail" page="1" total="1159"
	perpage="500" pages="3">
	<value usage="1">01-0041</value>
	<value usage="1">164993</value>
	<value usage="2">26000</value>
	<value usage="1">4k-az01</value>
	<value usage="1">4l-tgl</value>
	<value usage="1">4r-ade</value>
	<!-- and so on... -->
</values>
		

flickr.machinetags.getPairs

Finally, the getPairs method returns the list of unique namespace/predicate pairs optionally filtered by namespace or predicate.

Rather than including yet-another giant blob of XML, here’s a pretty picture of the metro stations in Munich instead:

ubahn

A few things to note

Certain namespace/predicate pairs have been special-cased to return a single value. As of this writing they are:

  • geo:lat (and variations)
  • geo:lon (and variations)
  • file:name
  • file:path
  • anything:md5

If people have particular reasons for needing or wanting these we’re open to the idea but otherwise the cost of storing all the variations and the dubious uses for returning them in the first place made us decide to exclude them.

Now what?

Some people take sandcastles pretty seriously...

photo by Daveybot

Well, that’s what we’re hoping you’ll tell us. Machine tags have been chugging away quietly since we announced them almost two years ago and despite being a bit nerd-tastic and awkward to explain we’ve been thrilled to see how people have been finding their own use for them.

And the list goes on.

The trick with machine tags has always been to make them both invisible (or at least barely visible) to those people who don’t care about them but also as easy as tags to pick up and use for those people who do or who wonder whether they might be the tool they were looking for. One thing we didn’t do very well, though, until the release of the machine tag hierarchy APIs was give people a way to learn about machine tags. The only way to find out which machine tags people were using was to hop-scotch your way around people’s photostreams or to be part of a larger community having a discussion about which tags to use. Oops.

Which is why it was extra-fantastic when a few short days after we announced the machine tag hierarchy methods on the API list, the ever prolific and awesome Paul Mison wrote back and said:

The obvious thing to build on top of these … is some sort of graphical machine tag browser, a bit like the Mac OS X / iPod column view browser. So I did.

http://husk.org/code/machine-tag-browser.html

This is entirely self-contained in one file (except for loading jQuery from Google and (cough) the pulser from Flickr). It uses JavaScript to get a full list of namespaces, giving you the option to drill down into predicates and the values available for that namespace/predicate pair.

We’re hoping that this provides a little more raw material to play with and maybe find some magic and that you’ll tell us what comes next.

Yay!

Oh yeah, the actual API methods

Enjoy!

In the coming weeks we’ll also try to gather most of the blog posts and other writings about machine tags and put them with the rest of the API documentation.

Wildcard Machine Tag URLs

Machine tags!

Photo by cackhanded

If you’re not already familiar with machine tags the easiest way to think of them is being like a plain old tag but with a special syntax that allows users to define additional structured data about that tag. In turn the magic space hamsters that run the site have been trained to recognize, index and allow for searches across multiple facets of a given machine tag.

Machine tags have three parts : a namespace which is like a subject or a topic; a predicate which is a like a property of that topic; a value which is … well, a value.

For a more thorough introduction to the subject I’d recommend reading the announcement
we made in the Flickr API discussion group
when machine tags were first added to the site. If you’d like to know even more, after that, there is good collection of links available on del.icio.us.

Which brings us to the part where I tell you that we’ve added the ability to search for machine tagged photos in plain old tag URLs (as well as in tag searches on the Flickr search page) using the facetted query syntax that has always been available in the API. For example :

That’s a trick, really. You’ve always been able to do this since machine tags are just
tags. The New-New means you can be even more granular in what you are looking
for. How about :

The wildcard URL syntax is also available for an individual user’s tags :

Now for the list of caveats and Known-Knowns :

  • At the moment it is still not possible to poke around the hierarchy of a given machine tag : all the predicates for a namespace; all the unique pairs of namespace and predicates; that sort of thing. It is On The List ™ and hopefully we can offer up something for you to play with, even if it’s just in the API to start with, shortly.

  • Values in wildcard URLs should are treated the same way regular tags are in URLs. That is “san francisco” becomes “sanfrancisco” or in machine tag speak : *:*=sanfrancisco.

  • In the examples above, I’ve illustrated namespaces that are used to denote one service or another. It is important to remember that there are no rules about what can or should be a namespace. Like tagging, the hope is that the various communities will arrive at and adapt a consensus according to their needs.

  • Untitled Souvenir #1173678685

    Photo by straup

    In the meantime, kick back and enjoy photos taken by people on their Dopplr trips, photos by people who really really like airplanes or photos by people who are interested in possums
    (not to mention all manner of marsupials) or whatever else comes to mind!

DevBlog: Recent posts

Flickr development team talks nerdy.






Hacking Uploadr: Recent posts

A place to moan about all things XULRunner.
more...


Random photos from FlickrHQ


more...