Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
Home
Discussion GroupsGeneralPHPASPPerlColdFusionFlashHTML, CSS, ScriptsBrowsers

Webmaster Forum / HTML, CSS, Scripts / JavaScript / July 2007



Tip: Looking for answers? Try searching our database.

How do I hide javascript content from googlebot?

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
Ciaran - 28 Jul 2007 16:19 GMT
I have a piece of code that I'd rather google's spider did not follow. Is
this
possible please?
Evertjan. - 28 Jul 2007 16:54 GMT
Ciaran wrote on 28 jul 2007 in comp.lang.javascript:

> I have a piece of code that I'd rather google's spider did not follow.
> Is this possible please?

Only link URLs are "FOLLOWED", code is not.

Or do you want it not to be "INDEXED"?

<http://www.google.com/support/webmasters>

Not really on topic for javascript, meseems.

Signature

Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

Ciaran - 28 Jul 2007 17:10 GMT
> Ciaran wrote on 28 jul 2007 in comp.lang.javascript:
>
[quoted text clipped - 8 lines]
>
> Not really on topic for javascript, meseems.

Sorry, you're right. I have a bit of javascript code that I dont want to
google to index. Is there a Javascript method to hide this code from the
crawler?
Thanks
Evertjan. - 28 Jul 2007 17:25 GMT
Ciaran wrote on 28 jul 2007 in comp.lang.javascript:

>> Ciaran wrote on 28 jul 2007 in comp.lang.javascript:
>>
[quoted text clipped - 12 lines]
> google to index. Is there a Javascript method to hide this code from the
> crawler?

No, not with clientside javascript.

You could test for the word "googlebot" with serverside jscriptin
Request.ServerVariables("HTTP_USER_AGENT"),
but I doubt Google will ever visit your site, if they find out.

Anyhow, why would you want to do that?
Is your scripting so unique and valuable,
that anyone would be interested?

If so, use serverside techniques, which codes are hidden by default.

Signature

Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

Ciaran - 29 Jul 2007 01:54 GMT
> Ciaran wrote on 28 jul 2007 in comp.lang.javascript:
>
[quoted text clipped - 26 lines]
>
> If so, use serverside techniques, which codes are hidden by default.

Ah yes! Good thinkin - I'll do it with PHP. Thanks!
Bart Van der Donck - 29 Jul 2007 08:28 GMT
>> I have a piece of code that I'd rather google's spider did
>> not follow.

> You could test for the word "googlebot" with serverside jscriptin
> Request.ServerVariables("HTTP_USER_AGENT"),
> but I doubt Google will ever visit your site, if they find out.

You shouldn't do that. The Robots Exclusion Protocol contains widely
accepted rules how to keep crawlers away from your website, or from
some parts of it. Just create a file named "robots.txt" in your
website's root directory.

This "robots.txt"-file keeps all robots out of .js files:

 User-agent: *
 Disallow: /*.js$

This "robots.txt"-file tells (only) Google to stay away from
directory /javascript/:

 User-agent: Googlebot
 Disallow: /javascript/

Information:

 http://en.wikipedia.org/wiki/Robots_Exclusion_Standard
 http://www.google.com/support/webmasters/bin/answer.py?answer=33570

--
 Bart
Evertjan. - 29 Jul 2007 16:08 GMT
Bart Van der Donck wrote on 29 jul 2007 in comp.lang.javascript:

>>> I have a piece of code that I'd rather google's spider did
>>> not follow.
[quoted text clipped - 4 lines]
>
> You shouldn't do that.

Do what? Ever visit his site when you are Google?

> The Robots Exclusion Protocol contains widely
> accepted rules how to keep crawlers away from your website, or from
[quoted text clipped - 16 lines]
>   http://en.wikipedia.org/wiki/Robots_Exclusion_Standard
>   http://www.google.com/support/webmasters/bin/answer.py?answer=33570

My interpretation of "a piece of code" is inHTML code,
not a complete .js file.

Signature

Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

Bart Van der Donck - 29 Jul 2007 21:17 GMT
> Bart Van der Donck wrote on 29 jul 2007 in comp.lang.javascript:
>
[quoted text clipped - 8 lines]
>
> Do what? Ever visit his site when you are Google?

I think it's better to use robots.txt rather than reading out the UA,
and then decide to (whether or not) output javascript code based on
that info.

The general goal is not clear for me. Crawlers don't do much with
javascript; at best they may detect some hyperlinks to index them. So
what's the use of hiding js code for crawlers ? Preventing the code to
be displayed in search results ? I'ld say a well-programmed crawler
wouldn't do that. Preventing source code to influence algorithms like
PageRank ?

> My interpretation of "a piece of code" is inHTML code,
> not a complete .js file.

When I read the orignal poster's sentence "I have a piece of code that
I'd rather google's spider did not follow", I surmised he meant a a
separate .js file, because that's the only thing a bot could follow.

--
Bart
Ciaran - 31 Jul 2007 14:31 GMT
>> Bart Van der Donck wrote on 29 jul 2007 in comp.lang.javascript:
>>
[quoted text clipped - 29 lines]
> --
> Bart

Hi guys, I've already sorted this out using PHP but to clarify:
I have a JS link that records votes on my site. googlebot was clicking it
and voting over and over so I wanted to hide it. Now hold that thought! I
know I shouldn't be doing stuff like that with javascript but I did it years
ago when I didnt really know what I was doing. It's not a big important
project and I'm far to lazy to go back and change the whole system now.

Thanks for the help!
Ciarán
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2009 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.