Difference Between .HTACCESS and ROBOTS.txt File

It gets tough when you are a total beginner at the technical sides of running your own blog or website. I remember it took me a while to figure out the difference between the .htaccess file and robots.txt, because even though I always loved anything IT and computers, the first time you encounter a … [Read more...]

Robots.txt Meta Tag Versus X-Robots-Tag HTTP Header .htaccess

It has become common practice to implement robot.txt files to prevent the search engine spiders from crawling your webpage and having it show up in the search engine results page (SERP). Unfortunately, robots.txt does not prevent your site from showing up in the search engine results. That means … [Read more...]