I made a Twitter feed called @PDML. The letters stand for Pentax-Discuss Mailing List, which I read with pleasure; a high-volume, rowdy, enjoyable gaggle of camera geeks. One of the things they do is post nice pictures, which are identified with a subject line starting “PESO:” for Picture Every So Often (and sometimes “GESO:”, G for Gallery). @PDML has those posts’ first hyperlink and as much of their content as can be stuffed into 140 bytes. It’s a nice low-volume Twitter feed, less than five pix a day on average. I believe this is what the cool kids call a “Mashup”. A very slow one, but still.

[Update, 2009/03/08]: @PDML was broken but is back. It turns out that in Google’s opinion I was violating their terms of service; the Gmail account was disabled. I shifted it over to a POP account on one of my own domains, like I should have when I set it up.

Is This OK? · I didn’t ask the PDMLers, nor the Twitterers, nor the other silent players (see below). But I don’t expect any static; when you publish a pointer on the Web, I think it’s OK to repurpose it. A lot of the pictures themselves have some rights reserved, and are clearly marked as such, which is just fine.

This is one of the Good Things about the Web.

How It Works · First, I created a new Gmail account, subscribed it to PDML, and POP-enabled it.

Then I wrote a little Ruby program to read the mail. It turns out that the current Ruby 1.8.6 doesn’t support POP+SSL, so I had to use Ruby 1.9. The API is pretty simple.

The Ruby program figures out which posts are PESOs, distills out a message, and POSTs that via the Twitter REST API.

The hardest part was distilling the email content down to a useful 140-byte chunk. You have to get the hyperlink and title in, plus the author’s name if at all possible, and lots of times that still leaves room to fish around in the email body for more. Once I’d written some fuzzy unit tests, I lost my fear and just sorted that out.

I saved some space by calling out to is.gd for URI shortening.

So the Ruby program, now bloated up to a big 103 lines, is running as a cron job a few times an hour over on tbray.org.

That’s Nice · You don’t say. I had the idea mid-morning on Tuesday, and the basic structure working by late afternoon. And that’s leaving time for one teleconference, a visit to my lawyer’s office, and lunch. It took me till sometime Wednesday to get the rough edges filed down and things looking good. Live testing was very slow, because there are only a couple of PESOs a day and I couldn’t figure out how to read the post via POP without marking it as read, so I could use each only once.

I held off a couple of days announcing it so as to build up a few PESOs in the feed. There also remain a few tweets which are debugging grunge and which Twitter failed to delete no matter how politely I asked.

Any Problems? · Well, it does bother me that I’m not paying for any of this. I’m using Gmail as a handy POP service and never looking at an ad, which pretty well nukes their business model. This is particularly egregious since I have a few domains where I could set up a mailbox and POP server but that would be, you know, work, and Gmail is just, you know, there. I’m using the is.gd service similarly. It doesn’t cost anything to read PDML or post to Twitter.

Is this a problem?



Contributions

Comment feed for ongoing:Comments feed

From: Rich Miller (May 02 2008, at 15:46)

Thanks! The world needs more little mashups like this.

'Twill be interesting to see how well the links display on mobile devices, which is where I do most of my twittering.

[link]

From: Colin Barrett (May 02 2008, at 15:50)

Nope.

You just drove a bunch of traffic to is.gd by linking to them, and you did the same sort of thing for Gmail. Not that they need the publicity, but a positive mention is nice.

If you used their services and then totally trashed them publically, but continued to use them, that'd be a bit more of a problem, I think.

A real, positive review goes a long way, IMO. In fact, paying a periodical to review your product is a time honored tradition -- look at the Video Game industry, for one example.

[link]

From: Joe Pallas (May 02 2008, at 15:53)

Does Twitter's API really qualify for the REST label? When I see "create" and "destroy" in the URIs and there are no PUT or DELETE methods, I'm hesitant to call it REST. Especially if I can delete something by doing a GET.

[link]

From: Dan (May 02 2008, at 16:26)

Very cool. One question: why do you use GMail to read the RSS Feed and not just an RSS parser in your cron'd Ruby program?

[link]

From: Tim (May 02 2008, at 16:31)

Dan: I don't think there is a feed, it's a pure mailing list. I was going to say "A feed would be better/less work" but actually it wouldn't really make much difference.

Hmm, I could easily <em>generate</em> a feed...

[link]

From: luis (May 02 2008, at 17:05)

Doesn't the gmail inbox already have a feed you can use?

[link]

From: Gowir Sivaprasad (May 02 2008, at 21:57)

Tim, Nice! Could you also post your Ruby code?

BTW, how come the twitter feed has some tinyurls?

[link]

From: Tim (May 02 2008, at 23:35)

Gowir: I'm not 100% sure about the TinyURLs. There's a URI-shortening routine that sends the URL off to is.gd and if it gets a 200 back, uses that, otherwise just uses the long form.

Some of the PESOs have TinyURLs already. So when I send those off, is.gd is either throwing an error or intelligently just echoing them back; either would produce the observed result.

Publish the code? I suppose... but it's moronically simple.

[link]

From: Scott Johnson (May 03 2008, at 00:05)

@luis: Yes, Gmail does provide an Atom feed, but one must first pass the javascript-based authentication challenge to get to it. In other words, it's a lot easier to just POP the mailbox.

[link]

From: Marcus (May 03 2008, at 00:06)

While I think it is generally OK to reuse any resource published on the web somewhere, I really don't feel too comfortable about this. I am very selective about where I post my pictures, which I rarely do, especially when I'm asking for c&c.

The "PDML crowd" and the "Twitter crowd" are very different. I guess I'll be more reserved in the future posting pictures to PDML. I'll certainly not mark them as PESOs...

[link]

From: Marius (May 03 2008, at 04:22)

How does your script deal with HTML mails (or are they banned from the list perhaps).

I'm looking for a program/library/script to convert incoming mails to plain text, perhaps saving the attachments to a directory or something. I've managed to hack together a Perl script that tries to guess the character set and runs HTML mails through Lynx, but it's very messy. Perhaps you've found something in Ruby that makes it easier?

[link]

From: Mark Holder (May 03 2008, at 07:11)

From Google's terms of service layman's version: http://www.google.com/accounts/TOS

5.3 You agree not to access (or attempt to access) any of the Services by any means other than through the interface that is provided by Google, unless you have been specifically allowed to do so in a separate agreement with Google. You specifically agree not to access (or attempt to access) any of the Services through any automated means (including use of scripts or web crawlers) and shall ensure that you comply with the instructions set out in any robots.txt file present on the Services.

[link]

From: Daniel Haran (May 03 2008, at 08:24)

Mark Holder's got a point about the TOS.

There is an Atom feed though, and the resulting code could end up being a bit more reusable.

Although trivial, some of us, myself included, could find your code instructive. There are a lot of unspoken assumptions evident even in some trivial scripts.

[link]

From: David Terrell (May 03 2008, at 08:45)

Personally, I prefer delivering the mail to my domain so I can pipe the email through a script as soon as it arrives, instead of polling via POP.

I'm an event driven kinda guy...

[link]

From: Norman Walsh (May 03 2008, at 12:43)

Doesn't seem like a problem to me. That said, Twitter looks like it must cost real money to run and yet I can't see how any money is made.

[link]

From: Scott Johnson (May 03 2008, at 20:24)

The above comments about the TOS ignore the very words of the TOS. It says that you may not access the service by anything other than the interface provided by Google. The POP access is provided by Google. Further, it states that you cannot access in an automated fashion. This is certainly vague, as I'm sure nobody telnets to Gmail port 110 and types in POP3 commands. There's always automation involved. As with most things, the TOS is open to interpretation.

[link]

From: Tim (but not THE Tim) (May 04 2008, at 10:21)

(sorry, but a straight line for this does not come up every day)

No, real geeks don't telnet to 110 and enter POP3 commands, we call on the phone and whistle Hayes AT-compatible tones :-)

I think how you're doing it is probably fine - I mean, how is anyone going to know your POP access isn't a user's Outlook program, anyway?

The salient point is the ease with which we can take existing tools and make something useful

[link]

author · Dad · software · colophon · rights
picture of the day
April 30, 2008
· Technology (77 fragments)
· · Web (385 more)

By .

I am an employee
of Amazon.com, but
the opinions expressed here
are my own, and no other party
necessarily agrees with them.

A full disclosure of my
professional interests is
on the author page.