How Google pulled the plug on the public Jabber Network

I know I'm late but I still can't see this part of the deal showing up...

So, it's no news that Google first disallowed new s2s connections, then started their own protocol, which has no s2s connections at all.

With that, the largest node of the public Jabber network went down, incorporating the largest share of its users. They're pointing to spam:

Chee Chew said that "we haven't seen significant uptake" in federation with Google Talk via server-to-server connections. The majority of the uptake Google did see was from organizations or individuals looking to bombard Google Talk users with chat spam

Ars Technica

And also did the original dream of Jabber.

By assigning server identity as a hostname, servers are also able to route data directly to each other and their connected clients, thereby creating a new, inherently open IM network out of all the individual installed servers and clients.

Jabber internet Draft

There is no open IM network anymore: we have some walled gardens supporting the XMPP c2s protocol for easy access.

For years, I've been dreaming - and trying to get funds - for a mail system competing GMail, but being compatible with it both in e-mail and IM - that's not possible anymore.

I was urging Hungarian e-mail providers - some of them providing their own XMPP implementation - to allow external IM connections between each other and gmail - that's not possible anymore.

The open network of communication won't be brought by XMPP, that's for sure.

Jabber failed to provide good enough spam protection, failed to provide a scalable protocol, failed to provide easy transfer of accounts between providers (if I change e-mail address, I don't have to re-add all my friends, it's enough to set a simple forward or inbox pulling - that's not true for Jabber IDs!)

Besides, no matter what was the dream, it turned out, the complexity is mostly on the client side, and that IM networks are inherently complex. We had a lot of clients able to transfer basically text-only messages. Not even files.

Also, given the fragmentation of the community - it's not a product, it's a protocol community somehow - Jabber failed to gain developer interest it seems - instead of Jingle, the way for A/V communications seems to be WebRTC now.

For message broadcasting and queing, there are a lot of protocols available. XMPP is one of them. For me, XMPP was an open protocol and network for human-to-human communication, and it seems it has failed its mission. It's only used as a fallback protocol for IM clients.

So, it seems we're struck with walled gardens for a while again.

Jingle FT - the million dollar protocol

I was speaking with Florian, one of the board members of XSF, and showed him my pattern about HTTP-based filetransfer in IM clients, just as done by Beejive or Meebo.

As far as I remember, the filetransfer argument - namely: it doesn't work in XMPP with 5 parallel standards - was settled down by requiring everyone to implement the most complex one: Jingle File Transfer.

It's like saying we don't need filetransfer to be implemented at all.

Jingle is a such complex protocol, that most of the client developers won't even try to implement it. It's just huge. Even Google didn't really succeed with their own implementation (remember the Cricket:: namespace??) and we don't speak about server-side requirements yet.

The XMPP community is constantly ignoring the fact that we're living in - albeit perhaps the end of - the web age. XML was proven to be inefficient, no matter how you compress it, it's just isn't par with JSON or other formats. Pull parsing technique requires special parsers to be written for XMPP, meaning loss of reuse (although they give the stanzas to normal parsers afterwards), Nokia didn't take XMPP in its production phones for efficiency reasons and all the social networks - facebook, hyves, meebo, gmail - implement their own client2httpd solution instead of the inefficient BOSH.

You can joke about it, but the reality is: XMPP is likely to be superseeded sooner or later.

Google took the API step

Just to remind: even if nobody seemed to care after my controversial talk on XMPP Summit this year (not that most of you believed such at all), Google took the API step.

(For a rant on APIs, see my previous post and the ones listed at the beginning of that)

I know SamePlace was much before this, but, with all the credits to Massimiliano, GTalk has a much larger user base.

Now, what we need is a standard API above Meebo, SamePlace, and this stuff, like Ajax did for the XMLHttpRequest implementations

Active resource representations - aka OEmbed

(Note: this isn't a jabber-only posts, although one of the earliest implementations is in Google Talk Gadget, and the jabber community could benefit from this in general)



PHP-OEmbed project page

Remember when the newly introduced Google Talk Gadget displayed Youtube videos instead of showing their links? That's what I call "active resource representation", because it represents the given video more [inter]actively than the pure link - which is hardly showing anything in the case of youtube.

I believe that a resource usually can be equally represented in a small place: there's no need to load a full page to view a youtube video or a flickr photo, but there isn't need for it to look at a person's social network profile, to accept/reject a calendar invitation, etc - and all of these have URLs nowadays. These technologies could be used in e-mail systems, forums, and chat clients.

What OEmbed does, is that it tries to provide metadata about a certain resource found at the given URL. That is: you send the link of a flickr photo, and you get an XML or JSON-formatted structure back. While it can be argued, that these could be / should be / in an ideal world, would be solved by HTTP headers, this is not the case, and this solution isn't as bad.


I tried to do a server-side PHP framework, which talks the OEmbed protocol for clients to consume, however, it makes you be able to create non-oembed compatible wrappers too (the perfect example is youtube, of course). This could be used in a web-client, given that it talks back javascript to you.

Currently, it can handle (proxy) any OEmbed provider (given that you enter their respective address into the providers.xml file), and it has a Youtube provider, which uses some not-so-official part of the "official YouTube Data API". New providers can be added by looking at the provider API - oh, and there's no error handling yet, so you could understand the code :)

Note: I extended OEmbed protocol a bit, namely:

• There could be a description field for every OEmbed-entity (handy for tooltips)


• There could be a duration part (in seconds) for video embeds


• The original resource URL could be in the data response

Oh, and Flickr: please add thumbnails! Thanks :)

Widgets, Xiclets and the REST

It seems Fabio Forno and Pedro Melo are also brainstorming on some kind of "widget-technology" for jabber, and the same applies to me. I've written several articles in the theme, see:

Why xmpp4moz needs to be a standard: What's wrong with XEPs?
XMPP services as OpenSocial providers
Meebo invented everything for us in 2007(?!)

Why do we need such? Let's see:

• HTTP is one of the most succesful protocols


• It has the following extensions currently in use:
  
  
  
  • HTML 4.01 (or XHTML 1.0) for presentation
  
  
  • CSS 2.0 for design
  
  
  • ECMAScript 3 (JavaScript 1.6) for interactivity (with an included DOM API, and with the lack of threads, therefore it should use asynchronous requests)
  
  
  
  

See? Nothing else. No JSRs, no XEPs, no "MAYs", just these three.

Does it solve all problems? Not always, but mostly. Does XMPP solve all problems? Not yet.

Now let's get back to the topic specifically.

The thing is about code mobility, and now we can safely reference to the original REST article (which isn't as bad as restafarians usually are): http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm

Let's have an application, residing on a certain location (URL), being it chess, or a flash-based videoconferencing/VoIP tool; of course, for interactivity, at least two instances should be run, one at each participating user. We can use either the fact that modern IM-clients use an embedded browser to display conversations (official AIM, Y!M, WLM; Adium), or some kind of integration between the IM-client (eg. pidgin) and a browser (eg. firefox). What could this look like?

The design issues we met are:

1. Features provided by XMPP
   
   
   
   • (RealTime) Communication channel between two application instances from the same source
     
   • (RT) Communication channel between an application instance and the participating users
     
   • RESTRICTED? Presence information of the participating entities (users + bots)
     
   • STRICTLY RESTRICTED: Roster information + all contacts' presence of the user
     
   
   
   
2. Features provided by HTTP
   
   
   
   • Remote script inclusion (like: Google Gadgets)
     
   • Developers MUST be able to write XMPP-supported applications in pure web-style (eg. PHP scripts)
     
   • It SHOULD be easy to adopt already written applications for the new platform, either which uses real-time communication and those who aren't
     
   
   
3. Features derived from browsers:
   
   
   
   • Ability to run javascript applications
     
   • Ability to display XHTML + CSS
     
   • DOM API (perhaps E4X as well? at least, at a later time)
     
   • Sandboxing and cross-domain code execution
     
   • Some kind of interaction between the XMPP client (which SHOULD be able to be a web-based client too!) and the application
     
   
   

Restricted features could be accessed if certain security checks are met (like: the page is from HTTPS, signed by an acceptable authority, like the enterprise running the XMPP infrastructure, or an official XMPP extension); This was done for years with java applet/midlet technology.

Our current example is a Go (simpler than chess) client which should be written in pure PHP + JavaScript (not because we ourselves cannot use more sophisticated, but it's the lowest entry barrier)

Another good example is whiteboarding, it's done such way in Yahoo! Messenger.

We'll see if we could extend it further.

Hope we could meet at DevCon and have a conversation about that (but it shouldn't stop you to comment :)

Open Source Nights presentation

I presented a few hours ago on Open Source Nights, hosted by my long-time friend and ubuntu advocate Szilveszter "Phanatic" Farkas about jabber.

I quickly translated the slides, you can read them in English here or in Hungarian here.

All in all, it was a good evening, hope it will bear us some deployments, or at least, jabber users :)

Meebo invented everything for us in 2007(?!)

If there would exists a "Jabber Innovation Award 2007" I would definitely give it to Silicon Valley-based Meebo. Their platform, built on top of libgaim and jabber, offers the following things currently:

• Online Presence / chat widget embeddable at webpages


• Web interface and desktop client support for jabber (your account is user@meebo.org)


• Ad-hoc groupchat functions accessible from any network (through web interface) - most possibly based on MUC


• Embeddable groupchat (MUC) for webpages


• Media-recognition (youtube links are added to a media pool in a groupchat)


• Web-based filesending abilities, even for groupchat


• Platform-independent web-based plugin capabilities, including:


• Voice and videochat support with any users on any network

Pretty awesome list, isn't it?

I recommend to try it for yourself, as I said above, registering a meebo user means effectively having a jabber address - using other systems happen through gaim.

While we're trying to push out new XEPs, they realized two things:

• Web is the platform you should build on, because it's independent from nearly everything, yet quite popular


• If you have a disadvantage (being a browser-based client it isn't fashionable for geeks) you better start to innovate.


• Even if you're based on jabber, you should make sure protocol does not matter: it's the client which does the user experience

If we could only ask them, how they achieved this, and how we could make this a common knowledge of humanity and ourselves!

(The author still dreams of having an open meebo platform-like solution for desktop clients too through standardizing and WebKit/Gecko/CHtmlView)