What is robot.txt?
You might have heard of robot.txt before when you are trying to build a website or looking over a webpage’s root directory, but what exactly is this robot.txt? Robot.txt protocol, also known as the robots exclusion standard, is a file to instruct the search engine robots which pages and files to be indexed or ignored. Robot.txt can prevent disclosure of privacy from search engines and reduce unnecessary bandwidth load to the server. A web administrator can decide what instructions to give to the robots by filling in the robot.txt. Just like the META tags, robot.txt is not a necessary component of the webpage, but it doesn’t hurt to create one, as robot.txt will give the search engines an idea how should they treat your webpage, which might help your page ranking a bit!
Create your own Robot.txt
You can create robot.txt with any text editors, such as notepad and wordpad. There are only two fields in the text file. They are user-agent and disallow. User-agent represents the name of the spider/bot, while disallow stands for the directory or file you want to exclude from crawling. The basic format of robot.txt is as follow: Read more…
Meta elements are special HTML codes that contains the metadata, which is the”data describing data itself”. Unlike regular codes, meta elements do not affect the appearance of the website. Its purpose is to specify and provide certain information of the page such as author and copyright, identify and assign proper values.
Once the meta tags play a major role in the algorithm of the search engines. Although they are no longer the primary factor, they are still important to a website and good to have regardless, since they can give your page a unique description to the search engines. Meta tags are relevant for a successful website SEO process.
There are four valid attributes of meta elements. They are : content, name, http-equiv and scheme.”Content” provides the property of the meta information, while”Name” provides property name for the corresponding value. The”scheme” formats and interprets the property value.” Http-equiv” is used to indicate the property as a HTTP header for HTTP servers to gather informantion and respond.
The <meta> tags should always stay inside the <head> element.
|
<head>
<title>Mavista - Web Site Management Solutions</title>
<meta http-equiv=”Content-Type” content=”text/html; charset=utf-8″>
</head>
|
Here are some common usage of the meta tags:
Read more…