Can Zoom Search find Ajax content?

[jerry2]

I am not asking about links, but content in general. Content that is loaded "on page load" to a div?

The search log says it did index this page I have but the results are not found, so I guess it opens an empty page as the results are ajax loaded into a div (no user interaction).

Should this work? Maybe I am doing something wrong.

[David]

If there is content on your page that requires Javascript to execute in a browser before the content is visible, then the content won't be indexed by search engines.

Here is our advice,
http://www.wrensoft.com/zoom/support...avascriptmenus

Here is more of less the same advice from Google,
http://www.google.com/support/webmas...y?answer=35769

"Use a text browser such as Lynx to examine your site, because most search engine spiders see your site much as Lynx would. If fancy features such as JavaScript, cookies, session IDs, frames, DHTML, or Flash keep you from seeing all of your site in a text browser, then search engine spiders may have trouble crawling your site."

[jerry2]

Huh... Yes, the content and pagination is dome via jquery .load method. This is standard way to create ajax pagination. Are you sure google still doesn't use javascript for contents? I understand this for the links and user behaviour, but to load the page as is without cookies etc... just that part of the page is ajax loaded... then EVERYTHING on the web that is ajax loaded like all ajax pagination contents in lists would not be crawled by google, do you really think that is the case?

[David]

I did some searching for you. Google has a proposal on how to handle it. But it probably means rewriting your application to support it and redoing your URLs.
http://code.google.com/web/ajaxcrawl...g-started.html

[jerry2]

Thanx, but Zoom Search doesn't use the #! Ajax crawl as far as I know.

I see two solutions:

1. On each Ajax page make noscript tags with links to ajax content
2. Use Dynamically generated sitemap for spider

Can you advise me which route seems better to you, you are more qualified in search engines than I am, that's for sure.

[David]

Yes, if your site has the ability to generate meaningful content and URL for browsers (and spiders) with JS off then you could do a site map with all the possible URLs.

Having a sitemap means you no longer need to ensure you have links between all the pages for the spider to follow.

[jerry2]

I would need to make that option. I only now realize how "bad" Ajax really is for search engines. Ajax can be used only in rare occasions I think. Any loading of context inside DIV's that most of programmers use will not be indexed.

Are you 100% sure google won't find such data? I understand the user response part (forms etc.), but loading a content to DIV does not require user input, it could be crawled if search engine would execute javascript.

[David]

You really need to ask Google what they do. We didn't design Google and aren't privy to their algorithms.

But yes, AJAX is bad news for search engines. Or if you look at it from the other point of view, AJAX is bad news for sites that want all their content indexed.

If your content loads automatically, without any user intervention, why even use AJAX? There are lots of other options to include content on the fly server side.

[jerry2]

To answer: because loading just part of the page in context is much faster than reloading whole page. Many lists of products now use this approach, I wonder how they get searched for. But yes, Ajax is bad news for history (back in many cases, and the solutions are only hacks) and for ad serving also...

[David]

But if the loads automatically, without any user intervention, then you hit the server twice. Once for the page and once for the DIV. If this was in response to some trigger, like a user click it might make sent, or a timer re-loading the DIV every second. But for a single page for a single div it doesn't make sense.

[jerry2]

The page loads and the 10 items listing on the page loads into div. When user clicks page2 the div is reloaded not the whole page.

[David]

When user clicks page2...

I see. I wouldn't have described this as without user intervention.

If this was the 2nd page the user looked at then the page header and footer graphics, and any external CSS, JS files, would have been cached by the browser. So doing a full load on page 2 should be a similar speed as just loading the middle of your page.

[jerry2]

Well, it isn't the pages using Ajax for content are much faster because no reload and no redraw is necessary. I have the links set up to work without ajax also. Problem is Zoom search engine doesn't trigger the ajax load into div, so the content is not visable.

But you told me why it is this way, so I am not arguing.

But I tested today my site and i see my content on this site is somehow cached by google. So the google seems to support div load somehow. I don't know how. I haven't set anything in google.

[Ray]

It would be easier for us to see what you mean if you can give us the actual URL to the pages in question. PM or e-mail if you don't want to make it public.