Current time: 02-21-2020, 01:14 PM Hello There, Guest! (LoginRegister)

Post Reply 
 
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Sitemap contains urls which are blocked by robots.txt - Help needed
02-03-2014, 03:26 PM
Post: #3
RE: Sitemap contains urls which are blocked by robots.txt - Help needed
(02-03-2014 02:14 AM)robzilla Wrote:  Your current robots.txt file tells Googlebot it's not allowed to access those pages.

Quote:User-agent: *
# disallow all files in these directories
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /archives/
disallow: /*?*
Disallow: /page/
Disallow: /comments/feed/
Disallow: /index.php
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: *?wptheme
Disallow: /tag
Disallow: /author
Disallow: /trackback
Disallow: /*trackback
Disallow: /*trackback*
Disallow: /*/trackback
Disallow: /*?*
Disallow: /*.html/$
Disallow: /feed/
Disallow: /xmlrpc.php
Disallow: *?nomobile
Disallow: ?comments=*
Disallow: /search?

A /*?* rule will match /?p=222, for example. This isn't strictly a problem, since you seem to be using static URLs like "/install-external-apps-games-android-smartphones/" rather than "/?p=222", but apparently Google encountered URLs of the "/?p=222"-kind in your sitemap.xml file. Judging from the current look of your sitemap file, however, it looks like you've fixed this by now?


Thanks dear robzilla,
It seems your suggestion worked.
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Sitemap contains urls which are blocked by robots.txt - Help needed - bkgroup - 02-03-2014 03:26 PM

Forum Jump:


User(s) browsing this thread: 1 Guest(s)