The first step is to download the source code for the player database and ban system. This system is written in the Java programming language. There are no external dependencies for deploying the player database (e.g. no external database software needed). All you need is a JVM (Java Virtual Machine).
You need to make some modifications to the ioUrbanTerror server code (this code is essentially ioquake3). These modifications will have the game server contact the player database when players connect to the game server.
Communication with the player database happens via network UDP packets. This section explains the nature of this protocol.
You have to install a JDK (Java Development Kit) in order to compile and run the player database
code. Installing Java works differently for every operating system. You need Java 5 or newer. I would recommend
either OpenJDK or Sun's JDK. You need to have the commands java, javac, and jar available in
your command shell.
You also need to have Subversion installed; you need the svn command at your disposal.
To get the source code and documentation for the player database:
rambetter@porky% svn checkout svn://svn.nerius.com/repos/urt-playerdb ./urt-playerdb
The most recent version of this HTML document that you are now reading is available as ./urt-playerdb/README.html .
Note that all of the standalone player database source code is distributed under a BSD-style license, which means that you can do pretty much anything you want with the source code, even sell it.
You should have a look at the file ./urt-playerdb/src/net/clanwtf/server/PlayerDBBanServer.java .
This is the main class that launches the player database.
Read the code comments near the top. They explain some important things regarding running the player database.
You probably don't need to modify anything in the source code, but you may want to consider modifying any of the following
constants in the source code:
DEBUG |
LOG_AUTHORIZE_PLAYER_DENIED |
USE_INTERNED_NAME_STRINGS |
OUTGOING_BUFF_LEN |
ACCEPT_DATABASE_WRITES |
ACCEPT_DATABASE_QUERIES |
FILL_GAPS_IN_C_BANS |
COMPILE_ADDITIONAL_BANS_USING_DATABASE |
The comment above each constant describes what it does. You most likely want to leave the constants unmodified.
rambetter@porky% cd ./urt-playerdb/src/ rambetter@porky% javac `find . -name '*.java'` rambetter@porky% jar cvf urt-playerdb.jar `find . -name '*.class'`
The file urt-playerdb.jar now contains the compiled form of the player database server.
You will need to have this file handy whenever you want to run the player database or any of its helper programs.
You'll want to create a directory specifically for running the player database. Move the file urt-playerdb.jar
into this directory. For example:
rambetter@porky% mkdir ~/urt-playerdb-run rambetter@porky% mv urt-playerdb.jar ~/urt-playerdb-run/ rambetter@porky% cd ~/urt-playerdb-run/
Now you need to set a password of your choice for the player database:
rambetter@porky% echo "pa55w0rd" > ./.password
Before you run the player database you need to set the CLASSPATH environment variable. For example:
rambetter@porky% CLASSPATH=`pwd`/urt-playerdb.jar rambetter@porky% export CLASSPATH
You are now ready to start the player database:
rambetter@porky% java -Xmx256m -XX:MaxPermSize=64m net.clanwtf.server.PlayerDBBanServer ./ 10030 127.0.0.1 >> playerdb.out 2>> playerdb.err
You can change the IP address (127.0.0.1 in above example)
to be any single network interface you want to listen on, or you can omit that
third argument to listen on all available network interfaces. I would suggest only listening on the loopback interface for
security reasons if your game servers are running on the same box.
You can omit the options -Xmx256m and -XX:MaxPermSize=64m . I've included them for reference. You
only need to set them if your database is very large.
You most likely will want to start the player database as a background process; you won't want to stay logged in for
the database to run in your command shell. To do this, you can either use something like the screen utility,
or you can run the process as a daemon using a utility that is specific to your operating system (for example,
/usr/sbin/daemon on FreeBSD).
The creation of ban lists is already discussed in the comments near the top of PlayerDBBanServer.java,
but I will go over that prodedure again here.
Creating, deleting, or modifying a ban list is really easy and can be done anytime (even while the player database
server is running). You simply have to create a file with the .banlist suffix and place that into your
urt-playerdb-run/ directory.
Let's create a file cheaters.banlist with the following contents:
// This is a comment and will be ignored. // Blank lines such as the following are ignored too. // When you place a full IP address in the ban list, it will // be treated as a class C ban. For example, the following: 190.229.148.198:-1 // jorge, wallhack // is exactly the same as: 190.229.148.*:-1 // alternate form of ban // The two ban lines above will be treated identically. You can // use the star notation if you like, but it's not necessary. The two // examples above will ban IP addresses between 190.229.148.0 and // 190.229.148.255, inclusive. If COMPILE_ADDITIONAL_BANS_USING_DATABASE // is set to true, for every player touching this IP address range, // all of their known IP addresses will be placed in the internal // ban list. // To do a class B ban: 207.6.*.*:-1 // aLeK, aimbot // The above will ban the range of IP addresses from // 207.6.0.0 to 207.6.255.255. Internal bans for players hitting // these IP addresses are not computed in the case of a class B. // The following is NOT a class B ban: 207.6.0.0:-1 // The above line is equivalent to: 207.6.0.*:-1 // and it will only ban the IPs from 207.6.0.0 to 207.6.0.255. // I will now describe interval banning. It only works if // FILL_GAPS_IN_C_BANS is set to true. Let's say we have the // following 2 bans: 71.98.66.*:-1 71.98.71.*:-1 // These 2 Ip addresses are fairly close together. Then, // the IP address range 71.98.66.0 to 71.98.71.255 will be added // to the internal ban list.
Modifications to ban list files will be automatically detected by the player database within a minute. New files and removed files will likewise be detected.
It's important to shut down the database cleanly in order to have it save player data to disc on shutdown.
Do NOT use CTRL+C or the kill command to shut the database down. To shut the server down
cleanly, you need to send a well-formed UDP packet to the database server that means "quit", and you need to do it
from the same host that the database is running on.
An example command that will shut down the server on a FreeBSD operating system:
rambetter@porky% PLAYERDB_HOST="localhost"
rambetter@porky% PLAYERDB_PORT="10030"
rambetter@porky% PLAYERDB_PASSWORD="pa55w0rd"
rambetter@porky% REQUEST_BODY=`printf "playerDBRequest\n${PLAYERDB_PASSWORD}\nquit\n"`
rambetter@porky% echo "$REQUEST_BODY" | nc -w 1 -u "$PLAYERDB_HOST" "$PLAYERDB_PORT"For a complete reference on all commands available to the database, see Section 3.
Note that netcat (the nc command) typically has an input buffer of size 1024. This means that
incoming UDP packets larger than this size would be truncated to 1024 bytes. Therefore, I would not recommend
using netcat for operations which return large packets, such as the database query operations. You should write
your own custom programs (using C, PHP, Python, or Java for example) that send UDP packets.
The source code for the UrT game server side of things is here:
rambetter@porky% svn checkout svn://svn.nerius.com/repos/ioUrT-server-4.1
You'll want to read the README-extra-patches.txt file in ./ioUrT-server-4.1/extra-patches/ . The patch you're
looking for in particular is playerdb.patch .
More information about this SVN repository for the game server code can be found on this page.
The README-extra-patches.txt file already explains the new server cvars that are defined with this patch, but
I will go over that in more detail here. Here is a sample server.cfg file with the new server cvars:
set sv_requireValidGuid "1" // Require that a cl_guid be present for every player, and
// that it be a string of length 32 consisting of the characters
// '0' through '9' and 'A' through 'F'. The default for this cvar
// is 0, which means don't require the valid cl_guid.
// The database won't store player information for invalid guids.
set sv_playerDBHost "localhost:10030" // Host and port of the player database and ban system.
set sv_playerDBPassword "pa55w0rd" // Password for the player database and ban system.
set sv_playerDBUserInfo "1" // Send ClientUserinfo strings to player database every time a player changes it.
// By receiving this information the database builds player data. The default value
// for this cvar is 0, which means don't send.
set sv_playerDBBanIDs "cheaters, asshats" // Require that all connecting IP addresses are not on these ban lists.
// If this property is the empty string, all players will be allowed to
// connect, and no authorizePlayer packets will be sent to the
// player database server. However, the native UrT banlist.txt banning
// system will still be in place whether or not we use ban lists here.
set sv_permaBanBypass "pa55" // Bypass the ban system by typing this in your client:
// \setu permabanbypass "pa55"This section describes the communication over network UDP that happens with the player database server.
For all of the commands below (except for largePacketTest) that respond with a single packet, if the information to be returned cannot
fit in one packet (length is greater than OUTGOING_BUFF_LEN), then the string
"\n<< snipped >>\n" will be tacked on to the end of the response packet.
The first command defined below describes something called a challenge. A challenge is optional on all of the commands below.
A game server will send a UDP packet like this to the player database:
@@@@playerDBRequest pa55w0rd authorizePlayer:0afc5e92 cheaters, asshats 99.50.206.241
@@@@
The string "0afc5e92" in the request can actually be any 32 bit hexadecimal quantity. It must consist of exactly 8 characters, where each character is a digit or a lowercase letter between 'a' and 'f'. This field is called the challenge, and it can be present on all request types made to the server, not just authorizePlayer. It gets echoed back in the response (see example response below). Its purpose is to prevent spoofed packets from being sent to the person making the request. The challenge is optional on all of the commands (you must leave out the colon before the challenge as well if the challenge is to be omitted).
The lines of text in the request are separated by a single newline character. There must be a trailing newline after the IP address (the last line).
The playerDB responds with a UDP packet like so:
@@@@playerDBResponse "authorizePlayer:0afc5e92" "99.50.206.241" "denied"
or
@@@@playerDBResponse "authorizePlayer:0afc5e92" "99.50.206.241" "allowed"
Note how the challenge is echoed back in the response. The response is not followed by a newline character.
If one of the supplied ban lists does not exist, the request still gets processed, but that missing banlist does not prevent the player from being approved.
On ClientUserinfo change a packet is sent to the player database:
@@@@playerDBRequest pa55w0rd clientUserInfo \ip\87.208.10.234:27960\name\Kagh_NL_\... << snipped >> ...\cl_guid\D8FDC4796B55536CFF0EA6EB4328A341
There is no response to this packet type. A challenge can be included in the request, but it serves no purpose since there is no response.
To figure out why an IP address is on an internal ban list:
playerDBRequest pa55w0rd banCausedBy cheaters 99.50.206.241
This will first determine if the supplied IP address (99.50.206.241) is on the specified ban list (cheaters). Only a single ban list can be specified. If the IP is indeed on the internal ban list, a packet such as the following will be returned:
playerDBResponse banCausedBy cheaters 99.50.206.241 banned 99.50.*.*:-1 // MaestroHalo 160.33.43.*:-1 // huh
If FILL_GAPS_IN_C_BANS is set to true in the server, it's possible that a player will be banned but there may not be a direct reason for that ban. The reason for the ban may be that the player falls in the interval between two close internal bans. If this is the case, a packet such as the following will be returned:
playerDBResponse banCausedBy cheaters 99.50.206.241 banned
If a player is not affacted by this ban list, the following packet structure is returned:
playerDBResponse banCausedBy cheaters 99.50.206.241 clean
If there is no ban list by the specified name, the request is silently ignored.
Note that a challenge may accompany this request type, but has been omitted in the example.
A useful way to debug this server's ban list compilation algorithms is by sending packets such as this one:
playerDBRequest pa55w0rd dumpBanLists
First, this will force a reload and/or pruning of ban lists if they have been modified on disk (just like the auto-load feature). Then, the internal ban list for each .banlist file is saved to the data directory; the internal ban list file is named with the original ban list name plus "-internal". So for example if you have a ban list called cheaters.banlist in your data directory, this will generate a file cheaters.banlist-internal containing all of the ban lines that were internally generated by the ban list compiler.
This request will be ignored if the request packet does not originate from the host on which the server is running. If the server is listening on the wildcard interface (all interfaces), then the packet must originate from the loopback address.
There is no response packet for this request type. Note that a challenge may accompany the request, but it serves no purpose.
To query all players in the database by IP address, send this:
playerDBRequest pa55w0rd queryByIP:64ea25a6 99.50.206.241
This would return a packet that looks like this:
playerDBResponse
queryByIP:64ea25a6
99.50.206.241
cl_guid:
5212B71033CDDCE449A4DDD99649647E
IPs:
99.50.206.241
99.50.206.243
names:
Rambetter@sam
booby
n00bsy
wTf|Rambetter
cl_guid:
0E60A7B8C6039878AA480A9E7F596A42
IPs:
99.50.206.241
names:
Rambetter@hugoThe lines that are indented above have the tab character at their beginning. If there are no matching players, there will be only one newline after the header in the response packet, not two.
Note that a challenge has been included in the example above, but is not required.
To get just a list of guids on a specific IP address, send this:
playerDBRequest pa55w0rd queryByIPShort 99.50.206.241
This would return a packet that looks like this:
playerDBResponse queryByIPShort 99.50.206.241 5212B71033CDDCE449A4DDD99649647E 0E60A7B8C6039878AA480A9E7F596A42
The lines following the header are a list of guids. If there are no matches, there will only be one newline after the header, not two.
To query all players in the database by a range of IP addresss, send this:
playerDBRequest pa55w0rd queryByIPRange 99.50.206.128-99.50.208.255
This would return a packet that looks like this:
playerDBResponse
queryByIPRange
99.50.206.128-99.50.208.255
cl_guid:
5212B71033CDDCE449A4DDD99649647E
IPs:
99.50.206.241
99.50.206.243
names:
Rambetter@sam
booby
n00bsy
wTf|Rambetter
cl_guid:
C0E6F20ACE21F3AFF73B7E417D1A8560
IPs:
99.50.207.13
names:
Rambetter@porkyThe begin and end IP address is in the range are inclusive boundaries.
Note that a challenge may accompany this request type, but has been omitted in the example.
To get just a list of guids on a range of IP address, send this:
playerDBRequest pa55w0rd queryByIPRangeShort 99.50.206.128-99.50.208.255
This would return a packet that looks like this:
playerDBResponse queryByIPRangeShort 99.50.206.128-99.50.208.255 5212B71033CDDCE449A4DDD99649647E C0E6F20ACE21F3AFF73B7E417D1A8560
To query a player by cl_guid, send this:
playerDBRequest pa55w0rd queryByGuid 5212B71033CDDCE449A4DDD99649647E
This would return a packet that looks like this:
playerDBResponse
queryByGuid
5212B71033CDDCE449A4DDD99649647E
cl_guid:
5212B71033CDDCE449A4DDD99649647E
IPs:
99.50.206.241
names:
Rambetter@sam
booby
n00bsy
wTf|RambetterPackets with invalid guids will be silently ignored. A valid cl_guid must contain 32 characters consisting of '0' through '9' and 'A' through 'F'. If no player with the given guid exists, just the headers are returned.
Note that a challenge may accompany this request type, but has been omitted in the example.
To query a player by name using a regular expression:
playerDBRequest pa55w0rd queryByNameRegexp Rambetter
This might return the following:
playerDBResponse
queryByNameRegexp
Rambetter
cl_guid:
5212B71033CDDCE449A4DDD99649647E
IPs:
99.50.206.241
99.50.206.243
names:
Rambetter@sam
booby
n00bsy
wTf|Rambetter
cl_guid:
C0E6F20ACE21F3AFF73B7E417D1A8560
IPs:
99.50.207.13
names:
Rambetter@porkyAs you can see, the names "Rambetter@sam", "wTf|Rambetter", and "Rambetter@porky" match the regular expression "Rambetter". To perform a case-insensitive search, precede your regular expression with "(?i)", for example "(?i)rAmBeTtEr" would match "Rambetter@sam" and the other two names listed above. To match an exact name, the expression is for example "^Rambetter\@sam$"; The '^' matches the beginning of a string, the '\' is an escape character, and the '$' matches the end of the string. This expression would match the name "Rambetter@sam" and nothing else.
Note that a challenge may accompany this request type, but has been omitted in the example.
Another way to query players by name:
playerDBRequest pa55w0rd queryByNameRegexpShort Rambetter
This would return something like the following:
playerDBResponse queryByNameRegexpShort Rambetter 5212B71033CDDCE449A4DDD99649647E C0E6F20ACE21F3AFF73B7E417D1A8560
In case you're too afraid to understand regular expressions, you can query the player database by exact player name:
playerDBRequest pa55w0rd queryByNameExact Rambetter
This works just like the regular expression name query except that the name must match exactly.
Returns just the client guids that have a name matching exactly:
playerDBRequest pa55w0rd queryByNameExactShort Rambetter
Your database of players may grow to be very large. To truncate the database, send a packet similar to the following:
playerDBRequest pa55w0rd truncateDatabase 2880
Because this is an important operation, there will be a return packet even though that packet will not contain any important information. The return packet will look like this:
playerDBResponse truncateDatabase 2880 success
The numeric value ("2880" in this example) must be a positive integer no greater than 305760 (approximately 35 years); it specifies the number of hours of data to keep. All data older than this many hours will be discarded. All ban lists will be reloaded and recompiled.
If the database was compiled with ACCEPT_DATABASE_WRITES as false, the database truncation request will be silently ignored. Otherwise, if you do not get a return packet for this operation, it probably means that the truncate operation was not successful, and that the database server probably shut down as a result (if it was running). The only other reason why this operation would fail would be due to dropped network packets.
Before the server even attempts this operation, it does an autosave. You will have the autosave backup until the next autosave takes place (about one hour after the trucate operation). The truncated database will only reside in memory at first. It will not be persisted until either the next autosave or server shutdown.
This request will be ignored if the request packet does not originate from the host on which the server is running. If the server is listening on the wildcard interface (all interfaces), then the packet must originate from the loopback address.
Note that a challenge may accompany this request type, but has been omitted in the example.
To shut the server down, send this kind of packet from the loopback or local address:
playerDBRequest pa55w0rd quit
This request will be ignored if the request packet does not originate from the host on which the server is running. If the server is listening on the wildcard interface (all interfaces), then the packet must originate from the loopback address. There is no response to this packet type.
Note that a challenge may accompany this request type, but it serves no purpose because there is no response.
Use this to test out how large outgoing packets can be.
playerDBRequest pa55w0rd largePacketTest 4096
The above request will trigger a response packet to be sent that looks similar to the following:
playerDBResponse largePacketTest 4096/9216 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa << snipped >>
The third line of output consists of the number that was input (4096 in this case), followed by the forward slash, followed by the value of OUTGOING_BUFF_LEN. The total size of the returned packet (the headers plus the 'a' characters that follow) will be the minimum of these two numbers.
Note that a challenge may accompany this request type, but has been omitted in the example.
Now you may want to generate a Google Maps interface showing where players are located. To do this, you can push the following request to the player database server:
playerDBRequest pa55w0rd IPMap:7fc8b9b0 24
This would return information for players who have been active in the past 24 hours (you can change "24" to be any positive value less than or equal to 305760 specifying a number of hours, or zero to mean an infinite number of hours). The information returned looks like this:
playerDBResponse IPMap:7fc8b9b0 24 1/2 << binary data >>
playerDBResponse IPMap:7fc8b9b0 24 2/2 << binary data >>
In the example above, 2 UDP packets would be returned, not just one. The line in the response headers "1/2" means the first packet out of 2. Likewise, "2/2" means the second packet out of 2. Multiple packets would be generated when not all data fits in one packet. The binary data after the headers contains IP addresses. The binary data has a size which is a multiple of 4, and every 4 bytes makes up an IP address; the first byte of a 4-byte quantity is the most significant octet of the IP address and so on. The algorithm for determining which IP addresses make the response is as follows. First, the database finds all players (all unique guids) that have been seen in the past 24 hours. Then, for each player, one of their IP addresses is chosen, but that IP address must be one that has been seen in the past 24 hours. That IP address is put in the response. So, the response contains exactly one IP address for each player seen in the past 24 hours.
If there are no IP addresses to be returned, the last line of the headers will read "1/1" and there will only be one newline after the headers, not two.
Please note that this operation should be used sparingly. It will likely take many CPU cycles to finish.
You may want to parse all of your UrT server logs to build an initial database of players.
There is a tool to do this. The Java program to do this is net.clanwtf.playerdb.LogParser .
This program takes an arbitrary number of arguments, where each argument is a directory in which to search
for server log files. A server log file is defined as any file with the .log suffix, except for
qconsole.log or botlib.log . The program reads from standard in, which should be an
existing player database file. If no standard in is supplied, the program creates a new player database. The program
writes to standard out a player database which is the result of combining the input with the parsed log files.
Last-modified timestamps of the individual log files are used to determine internal timestamps for the player data.
I've written a shell script that makes use of the LogParser program. You can save this script
as urt-playerdb-run/generate-fresh-database . Here is the shell script:
#!/bin/sh
# Arguments are directories which should be recursively searched for logs.
# For example, let's say you log files *games.log in the following directories:
#
# ~urt1/.q3a/q3ut4/
# ~urt1/.q3a/q3ut4/bak/2010-02-15_01:11:29_logs/
# ~urt2/.q3a/q3ut4/
# ~urt2/.q3a/q3ut4/bak/2009-09-25_20:07:18_logs/
#
# Then you could simply run this script like so:
#
# generate-fresh-database ~urt1/.q3a/q3ut4/ ~urt2/.q3a/q3ut4/
#
# which would create a file playerdb.bin-fresh. You would want to
# rename this to playerdb.bin to start using it before you start the
# player database server.
set -e
THIS_DIR=`pwd`
DIRNAME=`dirname "$0"`
cd "$DIRNAME"
SCRIPTDIR=`pwd`
CLASSPATH="${SCRIPTDIR}/urt-playerdb.jar"
export CLASSPATH
cd "$THIS_DIR"
java net.clanwtf.playerdb.LogParser > "${SCRIPTDIR}/tempdb.bin"
for LOGSDIR in "$@"; do
echo "Recursively finding log files in ${LOGSDIR}."
cat "${SCRIPTDIR}/tempdb.bin" | java -Xmx256m net.clanwtf.playerdb.LogParser \
`find "${LOGSDIR}" -type d` > "${SCRIPTDIR}/tempdb.bin-new"
mv "${SCRIPTDIR}/tempdb.bin-new" "${SCRIPTDIR}/tempdb.bin"
done
mv "${SCRIPTDIR}/tempdb.bin" "${SCRIPTDIR}/playerdb.bin-fresh"