• OzzModz is no longer taking registrations. All registrations are being redirected to Snog's Site
    All addons and support is available there now.

Robots.txt What are you using?

simpleguy

New member
I am trying to rank my new forum in google and i notice that the robots.txt is missing from the forum root.
So in google webmaster it say is missing too.

So i am curios what you guys use for the robots.txt?

My other question is:
Not sure what will be the best approach to rank a forum? i am what plugins will be best to use,and other Technics
like getting few gig's from fiver,etc.

Thank you
S.
 

Resistance

New member
robots.txt is used to stop bots crawling parts of your site, you don't want them on but most bots don't follow it. But if you want one use mine
 
Last edited:

Max Taxable

The Wyatt Earp of Anti-Spam
I use this:
HTML:
User-agent: *
Disallow: /ajax.php
Disallow: /asset.php
Disallow: /assetmanage.php
Disallow: /attachment_inlinemod.php
Disallow: /attachment.php
Disallow: /attachments/
Disallow: /blog_ajax.php
Disallow: /blog_external.php
Disallow: /blog_inlinemod.php
Disallow: /blog_report.php
Disallow: /blog_search.php
Disallow: /blog_subscription.php
Disallow: /blog_usercp.php
Disallow: /calendar.php
Disallow: /clientscript/
Disallow: /converse.php
Disallow: /cpstyles/
Disallow: /cron.php
Disallow: /customavatars/
Disallow: /customgroupicons/
Disallow: /customprofilepics/
Disallow: /editor.php
Disallow: /editpost.php
Disallow: /external.php
Disallow: /forumrunner/
Disallow: /global.php
Disallow: /group_inlinemod.php
Disallow: /groupsubscription.php
Disallow: /images/
Disallow: /img/
Disallow: /includes/
Disallow: /infraction.php
Disallow: /inlinemod.php
Disallow: /install/
Disallow: /joinrequests.php
Disallow: /list.php
Disallow: /login.php
Disallow: /member.php
Disallow: /memberlist.php
Disallow: /members/
Disallow: /misc.php
Disallow: /mobiquo/
Disallow: /moderator.php
Disallow: /newattachment.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /online.php
Disallow: /packages/
Disallow: /payment_gateway.php
Disallow: /payments.php
Disallow: /picture_inlinemod.php
Disallow: /picture.php
Disallow: /picturecomment.php
Disallow: /poll.php
Disallow: /posthistory.php
Disallow: /postings.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /profile.php
Disallow: /register.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /search.php
Disallow: /sendmessage.php
Disallow: /showgroups.php
Disallow: /showpost.php
Disallow: /signaturepics/
Disallow: /store_sitemap/
Disallow: /subscription.php
Disallow: /tags.php
Disallow: /threadrate.php
Disallow: /threadtag.php
Disallow: /usercp.php
Disallow: /usernote.php
Disallow: /vb/
Disallow: /visitormessage.php
Disallow: /widget.php
Interestingly, this also makes the google mobile friendly test pass, even if you're not really mobile friendly at all.

Keep in mind this is a publicly viewable document, as it has to be in order for it to work at all - and so therefore keep your admincp and modcp and any other area you don't want to announce publicly, OFF this list. The bad bots are gonna index those anyway.
 
Last edited:

simpleguy

New member
robots.txt is used to stop bots crawling parts of your site, you don't want them on but most bots don't follow it. But if you want one use mine
Thank you for providing this file Friend!
Now will this file , have the trick to pass "google mobile friendly test" ?

Thank you!
S.
 

simpleguy

New member
I use this:
HTML:
User-agent: *
Disallow: /ajax.php
Disallow: /asset.php
Disallow: /assetmanage.php
Disallow: /attachment_inlinemod.php
Disallow: /attachment.php
Disallow: /attachments/
Disallow: /blog_ajax.php
Disallow: /blog_external.php
Disallow: /blog_inlinemod.php
Disallow: /blog_report.php
Disallow: /blog_search.php
Disallow: /blog_subscription.php
Disallow: /blog_usercp.php
Disallow: /calendar.php
Disallow: /clientscript/
Disallow: /converse.php
Disallow: /cpstyles/
Disallow: /cron.php
Disallow: /customavatars/
Disallow: /customgroupicons/
Disallow: /customprofilepics/
Disallow: /editor.php
Disallow: /editpost.php
Disallow: /external.php
Disallow: /forumrunner/
Disallow: /global.php
Disallow: /group_inlinemod.php
Disallow: /groupsubscription.php
Disallow: /images/
Disallow: /img/
Disallow: /includes/
Disallow: /infraction.php
Disallow: /inlinemod.php
Disallow: /install/
Disallow: /joinrequests.php
Disallow: /list.php
Disallow: /login.php
Disallow: /member.php
Disallow: /memberlist.php
Disallow: /members/
Disallow: /misc.php
Disallow: /mobiquo/
Disallow: /moderator.php
Disallow: /newattachment.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /online.php
Disallow: /packages/
Disallow: /payment_gateway.php
Disallow: /payments.php
Disallow: /picture_inlinemod.php
Disallow: /picture.php
Disallow: /picturecomment.php
Disallow: /poll.php
Disallow: /posthistory.php
Disallow: /postings.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /profile.php
Disallow: /register.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /search.php
Disallow: /sendmessage.php
Disallow: /showgroups.php
Disallow: /showpost.php
Disallow: /signaturepics/
Disallow: /store_sitemap/
Disallow: /subscription.php
Disallow: /tags.php
Disallow: /threadrate.php
Disallow: /threadtag.php
Disallow: /usercp.php
Disallow: /usernote.php
Disallow: /vb/
Disallow: /visitormessage.php
Disallow: /widget.php
Interestingly, this also makes the google mobile friendly test pass, even if you're not really mobile friendly at all.

Keep in mind this is a publicly viewable document, as it has to be in order for it to work at all - and so therefore keep your admincp and modcp and any other area you don't want to announce publicly, OFF this list. The bad bots are gonna index those anyway.

Thank you!

So regarding the admincp and modcp i see that they not on your list, so you took them out already ?

I need to figure out 2 things regarding the backend security for my vb4 , for me as admin i can rename the admincp to something else, but for modcp will this affect the moderators in the forum? if they need to log into
the backend?

Thanks for that tip regarding the "google mobile friendly test"! I don't even have a mobile theme/skin yet, still looking for a away to add the mobile side of the forum,as for cell phones is hard to surf.

Thanks!.
 

Max Taxable

The Wyatt Earp of Anti-Spam
Thank you!

So regarding the admincp and modcp i see that they not on your list, so you took them out already ?
No, they exist. They're just not on this list.

Renaming them is purely personal preference - it really doesn't help security. You're far better off just making sure your mod staff use secure passwords.
 
Top