History log of /web-php/robots.txt (Results 1 – 15 of 15)
Revision (<<< Hide revision tags) (Show revision tags >>>) Date Author Comments
# dd8d8bf8 17-Oct-2018 Peter Kokot

Sync final and leading newlines

This patch adds some missing newlines, trims some multiple redundant
final newlines into a single one, and trims few redundant leading
newlines.

Sync final and leading newlines

This patch adds some missing newlines, trims some multiple redundant
final newlines into a single one, and trims few redundant leading
newlines.

According to POSIX, a line is a sequence of zero or more non-'<newline>'
characters plus a terminating '<newline>' character. [1] Files should
normally have at least one final newline character.

C89 [2] and later standards [3] mention a final newline:
"A source file that is not empty shall end in a new-line character,
which shall not be immediately preceded by a backslash character."

Although it is not mandatory for all files to have a final newline
fixed, a more consistent and homogeneous approach brings less of commit
differences issues and a better development experience in certain text
editors and IDEs.

[1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_206
[2] https://port70.net/~nsz/c/c89/c89-draft.html#2.1.1.2
[3] https://port70.net/~nsz/c/c99/n1256.html#5.1.1.2

show more ...


# 009b722c 21-Jul-2017 Connor A. Gurney

Remove source.php from robots.txt

I've removed source.php from robots.txt as source.php no longer exists, and was still in the list from the previous version of PHP.net.


# 5f6005af 24-Mar-2013 Sherif Ramadan

Added /manual/vote-note.php to robots.txt to prevent search engines hammering it.


# 18a564c5 03-Sep-2011 Hannes Magnusson

Do not index the apache status page (reported by Paul King)


# 3a27310c 27-Jul-2010 Hannes Magnusson

Make sure the robot obeys his laws


# 73495e14 01-Nov-2003 Gabor Hojtsy

There is nothing to index for search engines in distributions


# 1f1bb002 24-Aug-2003 Gabor Hojtsy

Add some sensible information on non-php.net pages when mod.php is visited.
Also disallow the indexing of mod.php and remove duplicate exclusion
of search.php (it is already included in robot

Add some sensible information on non-php.net pages when mod.php is visited.
Also disallow the indexing of mod.php and remove duplicate exclusion
of search.php (it is already included in robots.txt)

show more ...


# c2ec04ce 15-Mar-2003 Jacques Marneweck

* Updated to disallow /backend/ as per BUG #22707 seeing that google is
crawling the /backend/ directories.

--jm


# 85e464b5 01-Jun-2002 jim winstead

disallow robots from indexing stats


# 247107a6 21-Nov-2001 jim winstead

remove non-existent files from robots.txt


# 4ea886cd 04-Oct-2001 Rasmus Lerdorf

No point in crawling the search


# 847373f6 22-Sep-2001 jim winstead

more stuff for robots to not index


# 48ede9ae 22-Sep-2001 jim winstead

well-behaved robots shouldn't go in these areas....


Revision tags: POST_MERGE_FROM_NEW_LOOK_2001_TAG_1, NEW_LOOK_2001_TAG_1, MERGE_FROM_NEW_LOOK_2001_TAG_1
# 3eb3f00b 20-Jan-2001 jim winstead

not used


# e7e266df 07-Feb-2000 Sascha Schumann

Prevent further `attacks' from dumb indexers.

This is a reaction on "Teleport Pro/1.29" from 131.175.23.82