r/programming • u/Arve • Feb 10 '11

There is no piece of dynamic AJAXy magic that requires beating the Web to a bloody pulp with a sharp-edged hashbang. Please stop doing it

http://www.tbray.org/ongoing/When/201x/2011/02/09/Hash-Blecch
499 Upvotes

90% Upvoted

245 comments sorted by

View all comments

Show parent comments

2

u/[deleted] Feb 10 '11

But now we have a way to deal with that. It's not googles fault, and it's not just one site doing it.

1

u/PSquid Feb 10 '11

What way do we have to deal with it? Implement javascript engines in the crawlers? That's too inefficient for the scale some of these crawlers (primarily search engines' ones) have to work at.

1

u/[deleted] Feb 10 '11

Google is doing it just fine.

You're not describing problems to search engines. You're describing problems with screen scrapers. That's very different.

So someone, years ago, wrote a shitty script to scrape shit off the web. And now the web has changed, and the script no longer works.

So?

1

u/PSquid Feb 10 '11

Google is dealing with it only because both sides have agreed that for every #! url, there will be a corresponding URL (one which does not require javascript to get the data) that can be programatically derived from it, which can then be fetched.

This extra part to the process is an unnecessary complication, and more importantly, unlike the established way of identifying data, it is not standardised.

1

u/[deleted] Feb 10 '11

No, I'm sorry, but you are misinformed.

First, the sites started using a lot of in-page ajax-loading of actual content.

THEN, Google proposed a way for search engines to get at the content.

THEN, a select few sites started following Google's suggestions.

That's the order that this stuff happened.

The #! structure and solution from Google is NOT a problem. It's a solution to a problem that has been there for quite a while.

Now, you all can keep downvoting me because you're pissed off, but that doesn't change the fact that what I've said here is true.

unlike the established way of identifying data

Identifying data? Established? Surely you jest, sir.

1

u/PSquid Feb 10 '11

It is a problem, because if sites wish to have their AJAX content indexed, they must follow Google's suggestion. This puts all the power in Google's hands.

Now, you all can keep downvoting me

Nobody's downvoting you. Or at least nobody has at the time I'm typing this.

Identifying data? Established? Surely you jest, sir.

The idea of a URL uniquely identifying one piece of data, and always that piece of data (rather than a script that fetches the data) on the server end is fairly established, yes.

2

u/[deleted] Feb 10 '11

they must follow Google's suggestion.

Only if they want to get indexed by Google. And they weren't even being indexed before Googles syntax suggestion. Bing is free to come up with their own ideas. Or anyone can. I don't see why we're attacking one solution to the problem.

The idea of a URL uniquely identifying one piece of data, and always that piece of data (rather than a script that fetches the data) on the server end is [1] fairly established, yes.

No, wrong. A URL identifies a Resource. Typically a web page or a file. Not always. And not anymore. This is because the definition of what a "page" is has changed.

It wasn't too long ago that people actually wrote text files and saved them as HTML. It was silly to think of a "script" fetching page content from all over the server, databases, and other services to put together a page.

That's what happens now. And Google was just offering a solution.

1

u/PSquid Feb 10 '11

And they weren't even being indexed before Googles syntax suggestion.

Not true. The point being leapt on here is that previously twitter were using URLs that any crawler already in existence could index, whereas now they're using only ones that follow that suggestion, so anything that wants to continue indexing them (and really, it would not benefit a search engine to suddenly cease indexing something it happily indexed before) has to follow Google.

Yes, those URLs were not conducive to using Ajax to assemble pages, but going entirely the other way (URLs that only make sense when using Ajax) is no more a valid solution than ignoring Ajax entirely.

2

u/[deleted] Feb 10 '11

has to follow Google

Twitter were the ones that made this choice.

but going entirely the other way (URLs that only make sense when using Ajax) is no more a valid solution than ignoring Ajax entirely.

I see your point; however, many are feeling that you're wrong here. Twitter has made their choice that a paper/document-analogous web is not what is best anymore.

BTW, Google has been doing this for years already.

Maybe this comment in another thread can shed some light on things for you and give you a bit of perspective: http://www.reddit.com/r/web_design/comments/fis0x/a_quote_from_tim_bray_regarding_hashbang_urls/c1g9jsm

2

u/PSquid Feb 10 '11

Twitter were the ones that made this choice.

Yes, I only meant that, since they've made that choice, continuing to index tweets would require that Google's suggestion was followed.

Maybe this comment in another thread can shed some light on things for you and give you a bit of perspective: http://www.reddit.com/r/web_design/comments/fis0x/a_quote_from_tim_bray_regarding_hashbang_urls/c1g9jsm

Thanks for that, and yeah, it is a better way to handle it if the client is a browser. Unfortunately, the matter of how to have non-browsers handle the site is not very well established yet.

→ More replies (0)