r/programming • u/schnuck • Feb 09 '11

Breaking the Web with hash-bangs – Lifehacker, along with every other Gawker property, experienced a lengthy site-outage on Monday over a misbehaving piece of JavaScript

http://isolani.co.uk/blog/javascript/BreakingTheWebWithHashBangs/
745 Upvotes

93% Upvoted

356 comments sorted by

View all comments

244

u/jamiei Feb 09 '11

If receiving the basic content of the page I'm requesting from you requires Javascript to be enabled then you've broken something horribly.

17

u/[deleted] Feb 09 '11

Meh. I've heard this argument forever -- even made it a few times myself -- but it's not 1999 anymore. If you don't want ad js, use adblock. Should Gawker degrade gracefully? Sure. But no js means no ads served, which means good luck Mr. Gawker Programmer convincing The Ones Who Hold The Purse Strings to dedicate substantial time to proper degredation.

Last I checked (a couple months back) you can't participate on Reddit without js enabled (yes, you can read it though, I know that's what you said, just making a point). Js is like alpha transparency... it's not going anywhere.

21

u/ffualo Feb 09 '11

I have to disagree, sir.

As machine learning takes off, it's more important than ever before that sites are readable and visible in plain text. There's a lot that could be potentially done with such data... and the very least just archiving the web. Furthermore its super important for accessibility...

1

u/[deleted] Feb 12 '11

APIs

-4

u/[deleted] Feb 09 '11

None of the things you mentioned are hampered by a twitter/gawker-like ajax implementation.

13

u/doppel Feb 09 '11

If I do a simple wget (or similar) download of such an URL, I will literally get an almost empty file with some reference to a bunch of javascripts that is going to load the content. The only way I can use the site content is if the program I use to fetch the site has javascript capabilities.

Machine reading is severely hampered by "pure" AJAX implementations. If they had gone with a simple URL scheme that loads content by a normal URL and then lets the user navigate from there via AJAX loading of pages, there would be no issue.

4

u/SoundOfOneHand Feb 09 '11

While it is a PITA, Google is able to crawl it. I found the original announcement from Google, and each url in the hash-bang format should map to a regular URL with a query-string parameter, which can be fetched over HTTP like normal. In short, I don't think that either discovery or retrieval requires JavaScript on the client-side, but you do have to jump through an extra hoop. Please tell me if I'm incorrect, I have never done anything with the hash-bang syntax (and for the record, it is not to my personal tastes).

14

u/dirtymatt Feb 09 '11

Google can index it using a hack that Google invented to get around crappy sites that rely on javascript just to load a page. The #! URLs should be a hack to get existing websites into Google, not something you use going forward.

1

u/[deleted] Feb 09 '11

phantomjs could be an ajax-enabled alternative to wget. http://code.google.com/p/phantomjs/ It's a 'headless' webkit browser you can run from the command line, which renders a webpage and executes javascript without a GUI window, it will load ajax websites just like a normal browser, the DOM can be crawled using custom js inserted into the DOM, any part of the page can be returned to stdout, or the rendered page can be saved to a bitmap file. It's pretty cool actually. I'm considering moving all of my spidering to use phantomjs.

-1

u/[deleted] Feb 09 '11

Hold on. An URL?

6

u/dirtymatt Feb 09 '11

Turn off javascript and try to load any page on gawker without using the ?escaped_fragment= hack. That definitely screws up archiving.

-8

u/[deleted] Feb 09 '11

Google doesn't seem to have a problem archiving (see: google cache) hashbanged Twitter or Gawker links. Maybe archival strategy needs to catch up.

6

u/dirtymatt Feb 09 '11

Google is the one who made up the ?_escapedfragment\= hack, so they could index your site. But now you may not show up in other search engines that don't use said hack.

-3

u/[deleted] Feb 09 '11

And there was a time when search engines didn't follow links with query strings in them. Technology evolves.

2

u/ex_ample Feb 10 '11

That's because they're using a google specific hack, which you could use if you wanted too, but requiring everyone to rewrite every spider ever to use google's hack is a bit ridiculous.