• OzzModz is no longer taking registrations. All registrations are being redirected to Snog's Site
    All addons and support is available there now.

webmasters error

S

stratos

New member
Often I see errors lately in google webmaster.
Here are some with error 404

Code: 
/cache/lang_cache/2/core_public_login.php
/cache/lang_cache/2/core_admin_posts.php
/cache/lang_cache/2/core_admin_tools.php
/cache/lang_cache/2/forums_admin_forums.php
/cache/lang_cache/2/core_admin_logs.php
/cache/lang_cache/2/forums_public_post.php
I press that they are fixed and after a while I see them again. Any clue why this issue

I have vb4.2.0pl4 the only thing is that I see one folder with name "cache" its empty in my ftp

Thanks for any help...
 
This is what I have now
Code: 
User-agent: *
Disallow: /admincp/
Disallow: /archive/
Disallow: /attachment.php
Disallow: /cgi-bin/
Disallow: /clientscript/
Disallow: /cpstyles/
Disallow: /cron.php
Disallow: /customavatars/
Disallow: /customprofilepics/
Disallow: /editpost.php
Disallow: /modcp/
Disallow: /calendar.php
Disallow: /inlinemod.php
Disallow: /image.php
Disallow: /images/
Disallow: /memberlist.php
Disallow: /misc.php
Disallow: /moderator.php
Disallow: /newattachment.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /online.php 
Disallow: /poll.php
Disallow: /postings.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /profile.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /register.php
Disallow: /usercp.php
Disallow: /usernote.php
Disallow: /scripts/
Disallow: /search.php
Disallow: /sendmessage.php
Disallow: /subscription.php
Disallow: /threadrate.php
Disallow: /tmp/
What else I must? or any of above to delete?
 
stratos said:
This is what I have now
HTML: 
User-agent: *
Disallow: /archive/
Disallow: /attachment.php
Disallow: /cgi-bin/
Disallow: /clientscript/
Disallow: /cpstyles/
Disallow: /cache/
Disallow: /cron.php
Disallow: /customavatars/
Disallow: /customprofilepics/
Disallow: /editpost.php
Disallow: /calendar.php
Disallow: /inlinemod.php
Disallow: /image.php
Disallow: /images/
Disallow: /memberlist.php
Disallow: /misc.php
Disallow: /newattachment.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /online.php 
Disallow: /poll.php
Disallow: /postings.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /profile.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /register.php
Disallow: /usercp.php
Disallow: /usernote.php
Disallow: /scripts/
Disallow: /search.php
Disallow: /sendmessage.php
Disallow: /subscription.php
Disallow: /threadrate.php
Disallow: /tmp/
What else I must? or any of above to delete?
Click to expand...
I added /cache/ to it.
Resistance said:
remove admincp and modcp no need for them there
Click to expand...
And definitely yes this is right. The robots.txt file is a publicly viewable document (it has to be for the bots to access) and all you're doing by having those on the list is telling script kiddies where these pages are. The bad bots that ignore robots.txt are going to index these anyway, so no point in having them in there. I removed those from yours above, as well.

Here is the robots.txt file I use, for vB4:

HTML: 
User-agent: *
Disallow: /ajax.php
Disallow: /asset.php
Disallow: /assetmanage.php
Disallow: /attachment_inlinemod.php
Disallow: /attachment.php
Disallow: /attachments/
Disallow: /blog_ajax.php
Disallow: /blog_external.php
Disallow: /blog_inlinemod.php
Disallow: /blog_report.php
Disallow: /blog_search.php
Disallow: /blog_subscription.php
Disallow: /blog_usercp.php
Disallow: /calendar.php
Disallow: /clientscript/
Disallow: /cache/
Disallow: /converse.php
Disallow: /cpstyles/
Disallow: /cron.php
Disallow: /customavatars/
Disallow: /customgroupicons/
Disallow: /customprofilepics/
Disallow: /editor.php
Disallow: /editpost.php
Disallow: /external.php
Disallow: /forumrunner/
Disallow: /global.php
Disallow: /group_inlinemod.php
Disallow: /groupsubscription.php
Disallow: /images/
Disallow: /img/
Disallow: /includes/
Disallow: /infraction.php
Disallow: /inlinemod.php
Disallow: /install/
Disallow: /joinrequests.php
Disallow: /list.php
Disallow: /login.php
Disallow: /member.php
Disallow: /memberlist.php
Disallow: /members/
Disallow: /misc.php
Disallow: /mobiquo/
Disallow: /moderator.php
Disallow: /newattachment.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /online.php
Disallow: /packages/
Disallow: /payment_gateway.php
Disallow: /payments.php
Disallow: /picture_inlinemod.php
Disallow: /picture.php
Disallow: /picturecomment.php
Disallow: /poll.php
Disallow: /posthistory.php
Disallow: /postings.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /profile.php
Disallow: /register.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /search.php
Disallow: /sendmessage.php
Disallow: /showgroups.php
Disallow: /showpost.php
Disallow: /signaturepics/
Disallow: /store_sitemap/
Disallow: /subscription.php
Disallow: /tags.php
Disallow: /threadrate.php
Disallow: /threadtag.php
Disallow: /usercp.php
Disallow: /usernote.php
Disallow: /vb/
Disallow: /visitormessage.php
Disallow: /widget.php

KEEP IN MIND robots.txt isn't a rule, or any kind of blocker or even a script. It is just a publicly viewable list you provide for legitimate search engine spiders to "know" what pages, files and folders you don't want them to archive or index.

THIS DOES NOT MEAN they still won't - Google is pretty good about following it, so is yahoo and some others, but many, many spiders ignore this.
 
You must log in or register to reply here.
Back
Top