Technobabbles I try to sound like I know what I'm talking about. Don't be fooled.

17Jan/103

How I Beat Blogger and Redirected to WordPress

Yes, I really and truly have migrated to WordPress from Blogger. I know there are a lot of guides out there, but I think the method I used was different enough to warrant a new post. (I won't duplicate the steps of setting up the new WordPress site and importing the content from Blogger; that's pretty obvious, and it's well covered elsewhere. Redirecting is the art form that's been changing every few months.)

Obligatory Rant

Getting away from Blogger is really much harder than it should be. There should be a process to verify with Google that one is not a spammer trying to get free link juice etc. and have the 301 redirects with the Custom Domain feature specifically enabled for the blog one wishes to move. Hackery like what I came across, and what I used, shouldn't be necessary. Blogger should natively support moving away from its platform, even if it might require a little human intervention on Google's end.

It's really too bad that Google is so horrible with customer support. Their products are, generally, really great, but if you run into a problem it had better be something common that others have run into. Your only option, in most cases, is to post in a public support forum and cross your fingers. Fortunately the problems I've encountered have been few, though I am getting increasingly irritated with every issue.

The Premise

Way back in the day, Blogger used to offer 301 Moved Permanently redirects via its Custom Domain feature, even for domains not set to CNAME to ghs.google.com, but I guess spammers were abusing the feature because that ability went away. For a while now, it's been showing an interstitial page to visitors instead of redirecting, warning visitors and crawlers that the site is now redirecting to an untrusted domain. This passes no link juice, and is incredibly inconvenient and unsightly for everyone.

I did a lot of research beginning when I returned from my long summer. In between college applications, family trips, and so on, I performed search after search for methods of migrating away from Blogger to a self-hosted WordPress site. I found a plugin to restore the old Blogger post slugs (to make redirection easier), but every redirect method I came across was in some way outdated and/or broken — until I happened to find Digital Inspiration's method: A Classic Blogger template specially configured to issue canonical links tags, JavaScript redirects, and a big <h1> anchor link to the new site.

Digital Inspiration redirected all traffic to the old site to a special page on the new server, built to handle redirections. A PHP script was offered, but I didn't take that bit. I found a nifty WordPress plugin instead: Redirection by urbangiraffe.com. With two plugins (Redirection and Maintain Blogger Permalinks, mentioned below) and the Digital Inspiration template generator, I was ready to develop my method.

Introduction

Before you do this kind of thing, make sure you run a plugin like the one mentioned here to convert the slugs for posts imported from Blogger to the Blogger format, so they'll match up with the redirects. If you don't, any posts with "small words" ("and", "the", etc.) in the title and posts with lengthy titles will have issues once you switch to the new site.

Also note that you may need to set the number of posts to display on the home page to 1 on Blogger's end. I had a last-minute bug when I implemented this for real that I didn't catch on the test site because I only created one nonsense post. Imagine how frustrating it was to suddenly have seven copies of each <Blogger> section on the page. (Now that I think about it, that's probably why the DI generator kept the homepage stuff out of those sections. But you should change the setting anyway just to be safe.)

Finally, note that, contrary to what a lot of migration guides say, you don't have to set your permalink structure in WordPress to match Blogger's. If you don't want your post URLs to end in .html, they don't have to. Apache .htaccess rewrites and/or the Redirection plugin will be your friend(s). I chose to simply convert the .html into a / but you can pretty much do whatever you want. I've also found that WordPress is capable of redirecting links of the format http://domain.tld/post-slug-here/ to the fully-qualified permalink all by itself, so use that as the target if you want to use a permalink format that doesn't match up with Blogger's. Note: I haven't personally tested this, but given what I know of how WordPress handles things, it should work fine. Use at your own risk, etc. etc., don't blame me if it won't work, etc. etc. But you're welcome to ask for help, of course. ;)

Development

I started with a redirect template generated using Digital Inspiration's generator and set up a Blogger handler in Redirection on the new WordPress site, modifying the newly generated Blogger template accordingly. (DI's generator defaults to /blogger/ which I didn't want; it might eventually conflict with something. I used a more action-oriented /handle-blogger, removing the trailing slash.) The redirect I used was: /handle-blogger\?q\=http\:\/\/voyagerfan5761\.blogspot\.com(\/.+) -> $1

I began developing this solution well before discovering the magic bullet at Digital Inspiration. One of my first steps after installing WordPress was to add mod_rewrite redirects in my .htaccess file so I'd be prepared for redirections from Blogger. Once the handler was in place, I transferred my Apache redirection rules to the Redirection plugin in WordPress:

  • Posts: /([0-9]{4})/([0-9]{1,2})/([^/]+)\.html(\?.+)?$ -> /$1/$2/$3/$4
  • Categories (labels): /search/label/(.+)(\?.+)?$ -> /category/$1/$2+
  • Monthly archives: /(\d{4})_(\d{2})_\d{2}_archive\.html(\?.+)?$ -> /$1/$2/$3

+ – Still need to figure out how to transform this to lowercase and change %20s to hyphens.

Update (02/14): I discovered that query strings killed the redirects and led to 404s. I changed $ to (\?.+)?$ at the end of each regex as well as added extra corresponding backreferences ($4 for posts, $2 for categories/labels, and $3 for monthly archives) to fix it. All query strings are kept, even if they're meaningless on the new site, but otherwise it would be a pain to whitelist only the useful ones. I'll update again with blacklisted query parameters if I discover a Blogger query parameter that breaks something in WordPress.

Once I had all the handling pieces in place on the new site, I tested the Blogger template on a test blog. During my testing, I found room for improvement. For instance, I read in my research that a meta refresh tag, used with a delay of 0, is often treated as a 301 redirect by search engines. I added refresh tags to the template.

I also found that Blogger tries to thwart JavaScript calls to window.location.href by placing script tags on the line following the call and invalidating the entire block. I defeated that by placing a multi-line comment starter after the semicolon, wrapping the block in <!>, and creatively faking out Blogger. It still inserts the extra tags, but they don't harm anything because I used trickery to force Blogger to place them inside a comment block (where they're just ignored).

In case future changes to Blogger break the template — and I'm sure that Google will do something to break my setup eventually — I added a link to my new contact page for users to tell me if it's not redirecting.

The template does everything I need it to do. With the Redirection plugin, I didn't need to use the PHP script provided in the Digital Inspiration post. I should also get better results from the migration because of the extra work I put into my template; feel free to get a copy for yourself. If you use it, please leave the existing comment block at the top intact. Both the original authors at Digital Inspiration and myself will appreciate it. I also plan to share my improvements back to DI.

Conclusion

Satisfied with my testing and confident of the results, I implemented the template on my real old Blogger site a couple days ago, where it seems to have been chugging along nicely ever since. If it breaks, and I don't notice, I'm sure someone will take the time to use the contact link I added and let me know. The link might cause a few issues with search crawlers, but I think I'm OK with it. Misdirected link juice is better than broken functionality.

Of course, this is the technique that worked for me. Like all the others I found, I'm sure this one too will become oudated and/or broken with time. But for now, it seems to be the best method for escaping the clutches of Blogger.

Update (01/23): A reader pointed out in the comments that IE6 might not follow the redirect. If your site has a significant fraction of readers using IE6, keep that in mind. I won't worry about it for my site just because most of you use something more recent.

6May/091

Updated Topify Gmail Filters

Well, Twitter got prettier follow and direct message notifications today. Bully for them. Now I have to publish this update.

Update (05/07, 22:45): My update was broken, so the update had to be updated. The filter should now catch DMs, too. Believe it or not, I was wrong that Twitter changed the address that direct message notifications come from; it stayed the same. So that part of the filter didn't need to change. All's well that ends well, right?

Update 2 (05/07, 23:05): Well, scratch the new filters. Twitter went back to the old From addresses tonight. Y'all can use the old filters again. What a fast-paced 24 hours, eh? :P I'll leave this post up just for posterity, but please don't try to use these new filters and then complain that they don't work. ;-)

To make things easier for myself, I'll assume that everyone's seen the old filter setup I published at the end of last month.

The old method was quite convenient for those of us with multiple Twitter accounts, because the email addresses in the From headers changed depending on the address associated with each account (after October 30, 2008 and before this afternoon). Now they all come from [email protected] (as they used to last year), with the account-specific email addresses tucked away in the reply-to headers (which I can't filter on in Gmail, so that sucks).

Not only did the addresses again become uniform, but that was basically the only easy way to tell the difference between my personal account (which has Topify set up) and the others I run (which don't). Now I have to go through several hoops, and the filter string is longer.

Anyway, here's the updated updated filter string; put this all in the "Has the words" field in Gmail's filter settings:

(to:([email protected]) from:([email protected]) subject:"is now following you on Twitter") OR from:twitter-dm-you=yourdomain.tld

As before, [email protected] is the email address set in your Twitter account settings, the address to which all your notifications are sent.

I won't bother making an XML file for the new filters, because it's only one field. I'll probably leave the old one for posterity — at least until my Google Page Creator account is completely borked by the transition to Google Sites — because it's easier than deleting it and then updating my old post to reflect that.

Just for the record, Twitter, I'm not happy that I'm having to retool my filters this soon. If you want to make me happy again, put back your email headers the way they were last week. kthx

27Apr/094

How To: Safely Use Twitter Notification Enhancement Services

Latest update: 2010-11-20: Twitter tweaked the addresses again. Remove the twitter- part of the addresses in your filters if you've set them up.

Update (2010 – 11-11): Twitter changed their addressing scheme again. See below for instructions.

As you all have probably heard, Twitter is gaining popularity in leaps and bounds. All the new users mean more follower notifications arriving in my inbox, and Twitter's default messages aren't very useful. (The direct message notifications are pretty bare-bones, too, but I don't get many of those so it wasn't a priority.)

Update (05/06): Twitter prettified their emails, but I still think Topify's are better. Unfortunately, Twitter also went back to using the same address ([email protected]) for all users' notifications, putting the email-address – specific addresses in the reply-to header. So the filter setup in this post doesn't work any more. I had to come up with a new, more complicated filter… Stupid Twitter…

Update (05/07): Twitter went back to the old From addresses, so the filters from this post should now work again.

The Beginning

The first enhanced-notification service I discovered was Twimailer, created by a British developer named Jon Wheatley (and apparently later sold—shortly after I signed up — to a Romanian named Toni).

Despite the admonitions in the article above (on ReadWriteWeb) about changing passwords and all kinds of security precautions, I'm not worried about my own account. There's one simple reason for that: I never actually switched my email address in Twitter's settings. Instead, I created a Gmail filter to auto-archive follow notifications from Twitter and forward them to Twimailer. That way, I:

  • had all my follow notifications even if the service went down (it did for several days) or glitched (sometimes I get messages with no information)
  • only forwarded the messages Twimailer needed to be useful, rather than everything
  • made sure to keep password resets (which I haven't used for my main account in the last few months anyway) completely out of Twimailer's hands

I was very comfortable with this system. I can only guess that Jon's original intent was to simplify the setup process. After all, most people don't bother with email filters, and wouldn't necessarily know how to set one up. Changing settings on Twitter's website is a lot easier.

A New Age

Thanks to a TechCrunch post about fighting Twitter spam I read tonight, I discovered Topify, an invitation-only (for now) service based in Israel that offers all of what Twimailer did — and more.

I found an invite on the Topify blog (sorry, no link; you gotta dig through their site too so it's fair for everyone) and quickly signed up. The Twitter password field distressed me a little, but it's obviously necessary for all the extra features (like follow-back, reply to direct message, and block), all of which can be done via email with Topify. (In the future, I hope Topify will implement support for Twitter's OAuth authentication and delete users' passwords from their system. Consider this a request, Arik. :-)

Anyway, switching was pretty painless. All I had to do was change the address to which my Gmail filter forwards and add my direct-message notification From address to the filter. I'm currently waiting for something to happen on my Twitter account so I can try out the new service. (I considered running Twimailer and Topify side-by-side for a bit, but decided against it; redundant emails would increase my processing time, the opposite of the intended effect.)

Filter Details

For those who want to copy my setup (I'm telling you, it's a lot more resilient than the default instructions from either service), here are the filter settings to enter.

Update (2010 – 11-11): Twitter changed their addressing scheme. The old addresses don't work any more. Make sure to use the inserted instructions. :) (Updated again 2010-11-20 when Twitter removed the twitter- part of the address.)

As of November 11 20, 2010, you'll have to copy the From address of at least one Twitter notification email before you can set up the filter. The address should look like follow-<lots-of-letters-and-numbers>@postmaster.twitter.com or dm-<lots-of-letters-and-numbers>@postmaster.twitter.com

The random letters and numbers appear to stay the same whether it's a follow or a DM, so you only need to copy one address, then paste it twice and change one instance of follow or dm to the other.

In the filter's From box, enter:

  • Topify: follow-<lots-of-letters-and-numbers> OR dm-<lots-of-letters-and-numbers> twitter-follow-you=yourdomain.tld OR twitter-dm-you=yourdomain.tld
  • Twimailer: twitter-follow-you=yourdomain.tld I'm not supporting Twimailer any more (is it dead?) so no updates here.

Replace you=yourdomain.tld with your email address, using = in place of @.

That's all you need to do for filter criteria. (If you have only one Twitter account coming into your inbox, it's even easier; you can omit the -you=yourdomain.tld part(s) of the filter criteria. It doesn't hurt to include them, though.)

For actions, I selected "Skip Inbox" and "Mark as read", and told Gmail to forward these messages to my secret Twimailer/Topify address.

Click the Create filter button, scroll down your filter list, and you should see something like the following (image is linked to full-size version):

(There's also an XML file available to import, for those with the Filter Import/Export feature enabled in Gmail Labs, but creating the filter from scratch is pretty easy. No reason to recreate the importable file. The new solution is so copy-and-paste – heavy that it wouldn't be worth it. The file link might go dead in a month or two when my Google Page Creator site is moved to Google Sites, but I'll know because things like the site logo will stop working. If that happens, I'll definitely fix it. Well, since I moved to self-hosted WordPress, no links broke. :D )

Note: As I was writing this, I discovered Chris Messina's post about this, published almost two months ago. My little hack is nothing new, I guess; but I'll publish anyway because his instructions are focused on Twimailer and Twimailer only.

Wrap

Let me know if you find this little hack useful. I haven't time to make a bunch of pretty screenshots (unlike Chris ;-), so if you have questions, post in the comments.

Incidentally, this is my 500th blog post. If that means anything.

20Jan/090

FeedBurner Migration Troubles on feeds.swrobotics.com

Server Chaos auf dem Dachboden.Image by rudolf_schuba via FlickrI didn't exactly blog about it outright, but I do remember expressing a reasonable amount of uncertainty when I announced that I moved my FeedBurner account to Google's servers. Yes I am an early adopter; the deadline may not be until the end of February, but do you really want to make something of it? Apparently Google did.

As part of the migration process, I had to ask the DNS administrator for swrobotics.com (I wish I could do things like this myself, but I can't) to change the entry for the feeds subdomain from feeds CNAME feeds.feedburner.com. — the easy-peasy old way — to <special-sub-sub-sub-domain>.feedproxy.ghs.google.com. — the (unnecessarily, I think) long new way.

The change seemed like a simple matter, and indeed I was emailed back the next day with a note that the modification was complete. Google, though, seems to have a glitch. Trying to access the feed at its usual home (feeds.swrobotics.com/swrobotics) turned up an error: "404 Server Not Found".

Research I did tonight turned up a thread on Google Groups with a solution to the annoying 404 Server Not Found errors I was getting. Apparently there are widespread issues with preexisting MyBrand (what FeedBurner's Custom Domain feature is referred to as) domain configurations. The easy solution is to deactivate the MyBrand service and re-enter the settings. Seconds after I implemented this fix, I was pulling up the feed once more.

So the moral of the story is, don't trust that automatic migration tools like FeedBurner's will fix everything up exactly right. Sometimes, hacks, kludges, and/or workarounds will be necessary to make sure things work properly.

Enhanced by Zemanta
21May/080

Gmail Tips

There are tips and then there are tips. I was just emailed by someone from another site, which yesterday published a lengthy list of tips for using Gmail effectively.

Some highlights include a few I've already heard of (Zoli Erdos' guide to importing old mail comes to mind), and a few ingenious hacks (like embedding MP3s in email; no specific link, just go near the bottom of the above-linked list). While the list mentions using Gmail's labels, it didn't seem to explain anything about them; so my views on using labels properly might complement those other tips.

It's not something I usually do here, blogging just a link — I think such practices contribute too much to the "echo chamber" view many people hold of the Internet — but rather than just submission to StumbleUpon, etc., I thought this one deserved a mention. Partly because it's a great list, and partly because the author took the time to reach out and ask me if this list would be useful to my readers (the few of you that there are, heh heh).

I note that the Gmail tips list is that blog's inaugural post, and I just left the third comment ever on the site (judging from its fragment ID, #comment-3). It might still be awaiting moderation; I haven't time to check.

But anyway, go read the list. Even I learned a few things, and I (at least like to think I) am an expert Gmail user. There's something in it for everyone.

As a side note, I probably will be continuing my sparse blogging schedule, as this is tech week for one theatrical production and next week is another, then I play in the orchestra at a graduation ceremony for my local high school. Plus I am behind in schoolwork. So all this Internet stuff is going to have to go on the back burner until a few of these commitments are out of the way.

20Mar/080

Remember The Milk's Flawed Sharing Model

Two days ago, I discovered my first glaring error in Remember The Milk. Two days ago, my mom signed up for the service. Two days ago, I shared a few lists with her as an experiment. Two days ago, I had to come up with a good hack, and fast.

That hack had to do one thing, and do it well: It had to get the shared lists out of her account. In other words, it had to unshare those lists.

The Dissertation

One would think that the ability to stop sharing something is an integral part of the sharing model. If you share something with a person, and then decide you no longer want them to have access to that data, you should be able to lock them out. True? Apparently not, even though Google Calendar and Google Docs (two examples that I use) both provide sharing and both let you remove other users from the sharing list.

Remember The Milk thinks that once you share something, the other person should have access to it for the rest of time, regardless of what in your relationship has changed:

Once you have shared a list with a contact, you cannot 'unshare' it.

The reason for this is that your contact may have added or changed some of the tasks, and 'unsharing' the list would mean that they would lose the changes they made.

Obviously my relationship with my mother won't change much, but their decision has serious implications for the business world. And that "reason" for not allowing unsharing is complete crap.

Granted, Remember The Milk is not to be used commercially. For people informally collaborating on a project, though, it's still an issue. One member may decide to leave, and the group task list would be stuck in their account. What's more, they retain the power to disrupt the work of the rest of the group, as all people with whom a list is shared are granted the ability to do everything the owner can.

That brings me to another point: Only a list's owner should be able to delete that list. The problems that arise when a user with whom a list is shared deletes the list have wide implications. For instance, none of the other members of that list can view the tasks any more. What's worse, though, is that the tasks are transferred to only one Inbox: that of the user who performs the deletion. Everyone else loses access to the tasks unless they are put in another list and reshared (or shared individually).

Granted, these flaws can be used for good, to transfer a list from one account to another (by modifying the unshare hack I'm about to share ). However, if list transfer is to be supported, it should be a separate feature, not based on hacks using existing flaws.

The Email

I wrote an email to Remember The Milk as well, explaining the problems I see in their sharing model. Here is what I sent to them:

Let me just state that as far as I'm concerned, the sharing feature is broken. Specifically, the fact that tasks and lists cannot be un-shared once contacts have been granted access is absurd. There is no way that everybody will always share something with the right person, or that the other people will be part of the project associate with that list forever.

Say a pair of people who have started a small company together decide to bring in a third person temporarily. They share with that person a list of tasks related to the business. Then the third person quits their job and moves on. That third person should no longer have access to the business's task list. While it could be said that it is up to the user to make sure they avoid situations like this, the real world is never perfect.

Where possible, software should smooth over the experience of using the service, and this includes un-sharing. For now, my mother will have three useless lists in her task screen that were meant to be published, not shared. I have tried deleting contacts on both ends, only succeeding in making the RTM interface say the lists are not shared even though they appear in her view. Deleting the list and then restoring it also failed.

For what it's worth, my RTM username is voyagerfan5761 and my mother's is <scrubbed>. If there's a way you guys can clear the sharing status between the two accounts, that would be wonderful! Implementing un-sharing would also be absolutely priceless (and might earn you guys another Pro subscriber, wink wink :D ).

Anyway, I'm done with my rant. Please keep me apprised of the status of this ticket.

Voyagerfan5761

Note that the letter was written before I figured out that the word 'unshare' was supposed to remain unhyphenated. Oh well. I'll remain true to the quote rather than edit myself.

The Hack

Anyway, I did manage to hack together a workaround to unshare a list in RTM. Here are the steps:

1. Clear your Inbox list by moving everything to a temporary new list to make room for Step 2

2. Take the shared list you want to unshare and delete it in the Settings

3. Create a new list with the same name

4. Go to the Inbox and move all incomplete and complete tasks to the new identically-named list

5. Delete the temporary list created to hold any Inbox tasks (which moves them back to the Inbox list)

6. With that, you should have your list intact, and with you as its only user again

The contacts with whom the list was shared will have to delete it from their accounts themselves, unless they have not yet accepted the list. (I consider it another flaw, BTW, that deleted lists are merely emptied for users with whom they are shared, rather than being removed completely.)

Do not — repeat: do not — delete contacts in an effort to unshare something. That will only make matters worse, as RTM will continue sharing but not display the fact (shared items will display as unshared even though the other users(s) still ha(s|ve) access). However, even if you have, this hack will still work. I know because deleting contacts was the first thing I tried.

Conclusion

So anyway, this is a pretty complete write-up of all the things I found wrong with RTM's sharing model. I just hope someone (Emily and/or Omar, preferably) takes notice of this post…

10Nov/070

Blogger Apostrophe Workaround

I think I've found a workaround for the apostrophes-in-widget-titles problem. I tried using an apostrophe character at the place I wanted the apostrophe, and it turned into a quote after save, but I got an idea from one of Blogger's comment notification emails to replace the apostrophe character position with &#39;. That appears to display an apostrophe instead of getting converted to a quote. I'll see if it holds.

9Nov/070

Buzz Around New Gmail Macros

As many hardcore Gmail users know, Mihai Parparita, one of the Google Reader developers, wrote a script nearly two years ago named, aptly, Gmail Macros. Since then, the members of the GMail Power Users group on Google Groups have made countless modifications, and released several different versions. Some of the modifications were integrated into Mihai's script by Mihai himself.

Fast-forward to this month, when Mihai re-wrote the "official" version to be compatible with the new Gmail code. Since then, Brent Nef has updated his version, and a new GreaseMonkey (forgive the pun) going by the name of GreenPlastik has also been making his own version.

Lifehacker has begun separate development on a new version of the Better Gmail Firefox extension, cunningly named Better Gmail 2. There have been flurries of comments going on at both Mihai's blog and Lifehacker around various updates. Mihai's new Macros actually prompted the release of Better Gmail 2, according to the extension's author, Gina Trapani.

Interestingly enough, there is a sort of unofficial API that can be loaded by Greasemonkey scripts. It gives API methods to aid in accessing Gmail functions. It is somewhat fragile, and could change at Google's whim, but for now it should make writing Greasemonkey scripts for Gmail a little bit easier.

Better Gmail 2 is currently at release 0.1, but will probably get some updates as script authors re-write their existing scripts to work with Gmail's new code. I anticipate multiple versions of Gmail Macros, as with earlier releases of the first Better Gmail. The flakes are still falling here, so stay tuned for more updates.

9Nov/072

New Gmail + Tab Mix Plus = Problem?

It is possible for users of both the new Gmail version and the Tab Mix Plus extension for Firefox to have problems downloading attachments. When clicking the "Download" link, the Gmail tab is blanked, and can only be restored by reloading. Tab Mix Plus developers are working on a fix.

In the mean time, there are three possible workarounds:

  1. Go into Tab Mix Plus settings -> Prevent blank tabs when downloading files -> EDIT and remove the entries /disp=attd&view=att/ and /view=att&disp=attd/ from the list.
  2. Lock the Gmail tab.
  3. Middle-click or Ctrl+click on the link.
30Oct/070

Hacker Software Jailbreaks iPhones Without a Computer

This seems to be an Apple banner day: three out of five news items concern the company. The latest news? A hacker program for the iPhone named AppSnapp installs itself on iPhone and iPod Touch devices by exploiting a vulnerability in the embedded Safari browser, and then patches the vulnerability after installation. A rather interesting idea, that is.

The website for the program boasts that your device is more secure with AppSnapp than it is without, as the vulnerability used is a long-standing TIFF handler problem. The program does not unlock the phone; users must use anySIM for that. The firefight between the cunning hackers and the deep-pocketed corporation continues…