Difference between revisions of "Dedicated server"

Revision as of 15:26, 24 July 2020

Setting up a dedicated server

SCAM WARNING: If you find [Pavlov VR server hosting - Zap-Hosting - ZAP-Hosting.com][1] which is currently top hit for "pavlov server hosting" this is a SCAM with no ability to actually host Pavlov. Will take your money and provide no service. They have recently started calling this a "pre-order"

If you don't want to go through the hassle yourself and want to pay for a hosting service Arctic VR provides a server hosting service and is trusted in the community. https://www.arctic-vr.com/server-hosting

There are no Windows binaries, this guide is exclusively for Ubuntu/Debian/Debian-like Linux distros. Possible to run with other distros, but some of the commands won't work and you will need to know enough to figure out the software install of dependencies on your own.

This guide assumes you know the basics of using a bash terminal: ssh'ing into a server, running bash commands, editing text files, etc.

If you have no experience using bash, maybe start [here][2]

Minimum Specs

Believe (but haven't gotten confirmation) that 64bit OS is required. Ubuntu 18.04 x86_64 and Ubuntu 19/04 amd64 are confirmed working. Ubuntu 15.04 x86 is known to fail to install using this guide

2GB RAM
2.3Ghz Dual Core CPU for a 10-15 player server (depends on the Maps)

Prerequisites

Install dependencies and create the `steam` user. (You only need to do this once):

sudo apt update && sudo apt install -y gdb curl lib32gcc1  && sudo useradd -m steam

Login as the steam user. (You will need to do this each time you connect to your server):

sudo su -l steam -s /bin/bash

Install Steam:

mkdir ~/Steam && cd ~/Steam && curl -sqL "https://steamcdn-a.akamaihd.net/client/installer/steamcmd_linux.tar.gz" | tar zxvf -

Install the Pavlov Server (You will need to to do this each time there is a Pavlov Server update):

cd ~/Steam && ./steamcmd.sh +login anonymous +force_install_dir /home/steam/pavlovserver +app_update 622970 +exit

Make the PavlovServer script executable. (You only need to do this once):

chmod +x ~/pavlovserver/PavlovServer.sh

Server Configuration

Before starting the server we need to configure it, create some directories and a file at /home/steam/pavlovserver/Pavlov/Saved/Config/LinuxServer/Game.ini and open it with your preferred text editor, Vim in this case.

mkdir -p /home/steam/pavlovserver/Pavlov/Saved/Logs
mkdir -p /home/steam/pavlovserver/Pavlov/Saved/Config/LinuxServer
vim /home/steam/pavlovserver/Pavlov/Saved/Config/LinuxServer/Game.ini

if your server is not showing up in the list make sure you forward server port +400 (default 8177 if running on port 7777)

[/Script/Pavlov.DedicatedServer]
bEnabled=true
ServerName="My dedicated server"
MaxPlayers=50
bSecured=true
bCustomServer=true 
bWhitelist=false 
RefreshListTime=120 
LimitedAmmoType=2 
TickRate=90
TimeLimit=60
Password=0000 
MapRotation=(MapId="UGC1668673188", GameMode="SND")
MapRotation=(MapId="datacenter", GameMode="SND")
MapRotation=(MapId="sand", GameMode="DM")

bEnabled - whether the server appears in the server list (unconfirmed)
ServerName - name it will be listed as. (approx 35 chars max)
MaxPlayers - See suggested specs for limits
bSecured - enabled Valve Anti-Cheat (VAC)
bCustomServer - activates some new modding tools (save and http functions)
bWhitelist - Only allow users in whitelist.txt to join
RefreshListTime - sets how often the running server checks for updates in ban/whitelist/mod files
LimitedAmmoType - sets limited ammo mode (experimental) 0-2 unlimited,generic,specific
TimeLimit - sets time limit for each map
TickRate - sets the server tickrate. (so far minimum stable seems around 50 and max around 120)
Password - PIN number for server
MapRotation - you can add multiple of these. The MapId must be “UGC” followed by the map ID. you can find the map id in the URL of the steam workshop page.

Available game modes as of 10th of May 2019: SND, TDM, DM, GUN.

Maps running a Custom Game Mode (TTT, BR, Duel, etc.) don't care which GameMode you chose.

UGC###### are for custom maps from steam workshop. Find the map you want in the workshop. This is the URL for Mirage (https://steamcommunity.com/sharedfiles/filedetails/?id=1661803933). To put Mirage in your rotation UGC1661803933 would be the correct MapId

Admin/Moderator list (optional)

If you add moderators to your server, they will have an "admin panel" in their menu which will allow them to:

kill player
teleport to player
kick player
ban player (note this ban is only in memory and clears when server restarts. If you want it to be permanent, use blacklist.txt file (below) and/or Rcon ban command)
rotate to next map

vim ~/pavlovserver/Pavlov/Saved/Config/mods.txt

Example Mod List:

76541198397498572 #Optional Comment: Player name
76541198397498573 #Some Other Player

ArcticVR Admin Menu

Arctic-VR Offers a more advanced In-Game Admin menu that is exclusive to their hosting service. Features include:

Switch Maps+Modes
Teleport Players to Players
Change Player Skins
Control Player Buying+Cash
Spawn Items
Clean up corpses + guns
Spawn Bots
Spawn Items for other players
Control Teams + Balance/Shuffle Teams
ResetSND
And More

You can read more about the ArcticVR admin menu on their website: https://www.arctic-vr.com/server-admin-menu

Blacklist/Whitelist (optional)

If you want the ability to ban players or only allow players based on their steamID you can use blacklist.txt and whitelist.txt to define these players

Both files should be written in ~/pavlovserver/Pavlov/Saved/Config/ and have same format as Mod list above.

Note that blacklist.txt has to exist if you want to use Rcon ban and whitelist.txt is only used if bWhitelist=true in Game.ini

Currently bans actioned by admins in-game are only held in memory and not written to blacklist.txt. As such, they go away when server is rebooted.

Handy tools for finding steamIDs

Steam ID finder: https://steamidfinder.com/lookup/

You want to enter in the steamID64 format.

This tool can convert or lookup IDs: https://steamid.io/

If you are looking for a steamID of a player that has been on your server recently this is technique to find them:

grep -i playername ~/pavlovserver/Pavlov/Saved/Logs/* | grep TicketValidation

Firewall/Port forwarding

Using the default settings there are two UDP ports (7777, 8177) that need to be allowed to access the server either by firewall setup or port forwarding.

You may also need to allow Pings (ICMP) for the server's true latency to appear in the master list.

If you are running multiple servers and have set additional ports (see http://wiki.pavlov-vr.com/index.php?title=Dedicated_server#Running_multiple_servers_on_one_host) then you need to allow access to the defined port plus the port 400 higher. So if you use 7000 as your port, then UDP 7000 and 7400 need to be open

If you have setup rcon via RconSettings.txt (below) then that port also need to be allowed access via TCP

Hosting at home

When hosting at home there are additional requirements regarding networking: Your router has to support port-forwarding and hair-pinning, techniques necessary to allow your server to be found and to talk back to the client. please see https://en.wikipedia.org/wiki/Port_forwarding and https://en.wikipedia.org/wiki/Hairpinning for more info and google for these terms in addition to your router model to find guides on how to enable/configure it properly. If you have modern Router, it may have these features already enabled, in this case you just have to configure port-forwarding.

A second challenge to hosting at home is CGNAT (Carrier grade NAT: https://www.a10networks.com/blog/carrier-grade-nat/). Just like NAT on your home network that has to be gotten around w/ port-forwarding, CGNAT breaks the ability of external connections to be made back to your server, but the controls are solely in your ISPs hands. Sometimes if you ask, you can be removed from CGNAT pools and/or assigned a static IP. CGNAT rollouts are becoming very common now, so if your server worked then doesn't allow connections now, your ISP may have made this change.

You can check whether you’re on a CGNAT network by visiting a Web site such as ipaddress.com or whatsmyip.com and comparing the IP address listed to the public/WAN IP address assigned to your Internet gateway. To find your gateway’s public IP address, check the status page in the administrative interface. If the two IP addresses differ, you’re likely on a CGNAT network. If so, this problem must be resolved before you go any further.

Not understanding these requirements and configuration is a frequent cause of failed setups and questions in discord. Don't skip this if you are trying to host from home

There are serious security implications of hosting this software at home. None of the devs are security experts and there has been no security review of the code. You are opening up your system to the world and publicity advertising this fact. Expect that any system hosting this software is open to compromise from the Internet and act accordingly. If your router offers a DMZ network, use it. Consider hosting on a cloud based server and think twice before running this software on any system that contains data you care about or has access to networks and computers you care about.

Starting the server

If you are running a server for ""Pavlov Shack"" (and only Shack) run this before starting the server. This only needs to be done once for every game update or beta update.

./steamcmd.sh +login anonymous +force_install_dir /home/steam/pavlovserver +app_update 622970 -beta shack +exit

And Start the server with

cd ~/pavlovserver && ./PavlovServer.sh

That's it, your server will be automatically broadcast to the master server! :)

Note that when you close your terminal, the server will go down. See the "supervisors" section below for how to prevent this.

Supervising the server using systemd

Running ./PavlovServer.sh in your interactive shell is easy, but the server will shut down when you close your terminal. If you want to run the server in the background, you'll want to run it under a supervisor. There are many options, ranging from the simple (nohup, screen or tmux), to the more robust (supervisord, systemd).

Systemd is running by default on Ubuntu Linux, so we'll use that here.

To run the server as a systemd service, create the file /etc/systemd/system/pavlovserver.service (as root) by typing "sudo nano (or other editor you want to use) /etc/systemd/system/pavlovserver.service" with the contents

[Unit]
Description=Pavlov VR dedicated server

[Service]
Type=simple
WorkingDirectory=/home/steam/pavlovserver
ExecStart=/home/steam/pavlovserver/PavlovServer.sh
# XXX the server will often crash on round end/map change with a segfault
# until dave hires more programmers, just restart the server every time
RestartSec=1
Restart=always
User=steam
Group=steam

[Install]
WantedBy = multi-user.target

Change the user or paths as appropriate. After saving the service file, to start, restart, stop, or monitor the server:

sudo systemctl start pavlovserver
sudo systemctl restart pavlovserver
sudo systemctl stop pavlovserver
sudo systemctl status pavlovserver

To view the logs:

sudo journalctl -u pavlovserver
# to live-tail the logs
sudo journalctl -u pavlovserver -f

If you want to run the pavlov server as soon as your server boots (e.g. a VPS you turn on and off), add pavlov as a startup service:

sudo systemctl enable pavlovserver

Here's some executables you can create to help, after setting these up they should be the only commands you need to run on your server. Descriptions on what they are, and how to install and use them are in the comment blocks.

#!/bin/bash -e
# Copy txt of this script to /usr/bin/pav-edit
# chmod a+x /usr/bin/pav-edit
# usage: pav-edit
# stops the server, edits the config, updates the server, starts the server, then tails the logs
# doesn't need to be run as the "steam" user
# does assume you've setup systemd as per the wiki

echo "stopping server..."
sudo service pavlovserver stop

echo "opening config..."
sudo vim /home/steam/pavlovserver/Pavlov/Saved/Config/LinuxServer/Game.ini

echo "updating server binary..."
cd /home/steam/Steam
sudo -u steam ./steamcmd.sh +login anonymous +force_install_dir /home/steam/pavlovserver +app_update 622970 +exit

echo "starting server..."
sudo service pavlovserver start

echo "tailing logs..."
/usr/bin/pav-logs

echo "pav-edit complete"

#!/bin/bash -e
# Copy txt of this script to /usr/bin/pav-logs
# chmod a+x /usr/bin/pav-logs
# usage: pav-logs
# tails the logs


tail -f -n1000 /home/steam/pavlovserver/Pavlov/Saved/Logs/Pavlov.log | grep -v 'Sending heartbeat to masterserver'

Rcon Overview and Commands

To enable Rcon access create config file "/home/steam/pavlovserver/Pavlov/Saved/Config/RconSettings.txt"

Example RconSettings.txt file:

Password=ChangeThisPassword
Port=9100

You must specify a port for the Rcon server to use as well as a Password (use a different port per sever instance if running multiple server instances off the same hardware)

Ban/Unban

If you want Ban/Unban to work create file /home/steam/pavlovserver/Pavlov/Saved/Config/blacklist.txt writable by user running server.

Commands:

Help                                -Returns the full list of commands
Ban {SteamID}                      -Bans the player from the server (adds them to blacklist.txt)
Kick {SteamID}                     -Kicks the player from the server
Unban {SteamID}                    -Unbans the player from the server (trys to remove the unique id from blacklist.txt)
RotateMap                           -Rotates to the next map in the game.ini
SwitchMap {MapName/ID} {GameMode}   -Switches to the map/gamemode specified, ex "SwitchMap DataCenter TDM" or "SwitchMap UGC1080326389 DM"
SwitchTeam {SteamID} {TeamId}      -Switches the specified players team
GiveItem {SteamID} {ItemId}        -Give the specified player the Item
GiveCash {SteamID} {CashAmt}       -Give the specified player cash
GiveTeamCash {TeamId} {CashAmt}     -Gives all members of the teamId cash
InspectPlayer {SteamID}            -Returns information about the player specified (PlayerName, Cash, KDA, TeamId etc)
RefreshList                         -Returns a list of players on the server (unique/steamId)
ServerInfo                          -Returns information about the server (ServerName, CurrentMap, PlayerCount/MaxPlayerCount etc)
Disconnect                          -Disconnects the player from Rcon
ResetSND                            -Resets the match back to round 1 keeping the same teams
SetPlayerSkin {SteamID} {SkinId}   -Sets the skin for the specified player (clown, prisoner, naked, farmer, russian, nato)
SetLimitedAmmoType {0-2}            -Sets the limited ammo type for the duration of the match

Make sure whatever port you used is open in your firewall for TCP connections.

To test connecting to the rcon port use netcat:

nc {ip address} {port}

For the password use the md5 hash of the password in the file (this may change in the future). You can get this using

echo -n password | md5sum

If successful you should get an Authenticated=1 reply

Tools available to access Rcon interface

At the simplest, using netcat (nc) as described above will give you basic access to all commands

If you want to batch commands (i.e. for moving players on to teams and triggering ResetSnd) the following script can be useful to call a pre-written batch files of commands. Call with your batch file as the first argument to the script. The file should have one command per line with first line being the hashed password as above.

 
#!/bin/bash
function slowcat(){ while read; do sleep .05; echo "$REPLY"; done; }
cat  $1 | slowcat | nc {ip address} {port}

Example batchfile:

8b96753b29612d8c98d6b696140b5d9f
ResetSnd
Disconnect

At the next level of complexity and utility you can checkout either of the two commandline programs available on github. Both of these provide a simple interface to Rcon via interactive text console interface:

https://github.com/vankruptgames/PavlovVR-Rcon

https://github.com/d0n/pavlovadm

For the ultimate in both control and complexity of setup, use pavlov-bot. This is a discord bot that will listen for commands in your discord channel and execute them. Supports all Rcon commands with handy features such as discord role based authentication and authorisation, player, map and team aliases (No more SteamIDs or UGC####) and future roadmap for autobalance of large player number servers and autokick/ban of TKers.

https://github.com/makupi/pavlov-bot

Rcon access from within VR

There are currently no known tools allowing direct access to Rcon commands while in VR, but there is a workable method using either built in access to your windows desktop via SteamVR or other tools like OVRDrop. Details are a bit complex, but the general overview is as follows:

On your gaming rig install Cygwin (https://cygwin.com/install.html). This is a well known opensource program to give you linux tools on your windows system. During install, there is an option to install additional packages. Find the "nc" package to install netcat.
Once cygwin is installed, open up a cygwin terminal window and test access to the server using nc as described above. Once that is proven working, create wrapper script given above in /usr/local/bin/rcon_file.sh (within Cygwin) and a batchfile as described above and test. A very handy one for SND allows you to call ResetSnd so that once teams are sorted the game can start.
Create a desktop shortcut that calls Cygwin, then your script and batchfile with following: C:\cygwin64\bin\mintty.exe -i /Cygwin-Terminal.ico /bin/bash -l -c "/usr/local/bin/rcon_file.sh ~/rcon_reset.txt" where rcon_reset.txt is your batchfile. Test this shortcut. It may be then handy to pin this to taskbar for easier access when viewing virtual desktop from within VR.
If all goes well, you can now trigger a set of commands from within VR after a quick glance to your desktop.

Hosting Providers

Considering the suggested server specs, you may be interested in the following hosting providers:

Provider              | Base CPU | Details
OVH                   | 4.5 GHz  | https://www.ovh.com.au/dedicated-servers/game/
Arctic VR             | 4.5 GHz  | https://www.arctic-vr.com/server-hosting
Amazon Web Services   | 4.0 Ghz  | https://aws.amazon.com/ec2/instance-types/z1d/
packet.com            | 3.5 GHz  | https://www.packet.com/cloud/servers/c1-small/
Cherry Host           | 3.0 GHz  | https://cherry.host/cloud-vps/#pavlov NOTE: Cherry Host has not contacted the Pavlov developers and has not been verified as legit.
Google Cloud Platform | 2.6 GHz  | https://cloud.google.com/compute/docs/cpu-platforms

ARCTIC VR

Arctic VR is a hosting provider specific to Pavlov. They offer both EU and NA servers. The In-Game Advanced Admin Menu is exclusive to the ArcticVR hosting service.

You can find more about them here: https://www.arctic-vr.com/server-hosting

Cherry Host

NOTE: Cherry Host has not contacted the Pavlov developers and has not been verified as legit. This disclaimer will be removed after they have exemplified their legitimacy to the devs or other prominent members of the community.

Cherry Host, like Artic VR, is a hosting provider that operates out of North America.

Pavlov servers are created on VPS's that are spun up upon request. Find out more here: https://cherry.host/cloud-vps/#pavlov

Amazon Web Services

Login -> Change Region: Choose your region
Services -> Compute -> EC2 -> Create
Choose an AMI: Ubuntu 18.04
Instance Type: z1d.large

Instance Details -> Tenancy: Dedicated (optional: this will more than triple the cost)

Storage: 50gb disk (or whatever, you're in control of the map roster)

Security Groups:
Create a new Security Group
Security group name: pavlov-server
Description: pavlov server
Rules:
Type: Custom ICMP Rule - IPV4 / Protocol: Echo Request
Type: Custom UDP Rule / Port Range: 7777 / Source: 0.0.0.0/0
Type: Custom UDP Rule / Port Range: 7977 / Source: 0.0.0.0/0
Type: SSH (optional, but I assume you wanna SSH in..)

Launch -> Create new Keypair -> Download -> Launch

Once it's started, SSH in like:
$ ssh -i your-key.pem ubuntu@you-server-details.amazonaws.com

(I use cmder on Windows, if you're using putty or mobaxterm or something, might be different)

Google Cloud Platform

The below configuration is successfully running 4 pavlov instances with ~15 total maps and 15-20 players total. If you want more maps use larger boot disk. Both can dynamically be scaled based on popularity.



Menu -> compute engine -> VM instances
create instance
- region: wherever you like
- machine type: n1-highcpu-4 (4 vCPUs, 3.6 GB memory)
- boot disk: Ubuntu 18.04
- boot disk -> size: 25gb
- network tags: pavlov-server
- [create]

- Menu -> Compute Engine -> Metadata -> SSH Keys
- [edit] -> [add item]
- "Enter entire key data": the syntax must be `<key> <username>`
- [save]

Menu -> Vpc network -> firewall rules
[Create firewall rule]
Name: pavlov server
Target Tags: pavlov-server
Source IP Ranges: 0.0.0.0/0
Protocols and Ports -> Specified protocols and ports -> UDP: 7777, 7977
[save]

Advanced Server Administration

Running multiple servers on one host

Make multiple server installs by defining a separate install dir for each server.
Run each server with a unique port. eg: ./PavlovServer.sh -PORT="Desired Port"
The Server will use the Desired Port and the Port 400 above (Example: You choose 8200 as Port. So Port 8600 will be used as well)
Ensure the ports are forwarded and opened in your firewall

To duplicate directories:

be in the steam home folder

cp -R pavlovserver pavlovserver1

/home/steam/pavlovserver1/PavlovServer.sh -PORT=[Desired Port]

Default Map ID's

Siberia = prisonbreak

Troubleshooting

To run the server without outputting the heartbeat:

./PavlovServer.sh | sed '/heartbeat/d'

To monitor the logs (without heartbeat) from another terminal:

tail -f -n10000 /home/steam/pavlovserver/Pavlov/Saved/Logs/Pavlov.log | grep -v 'Sending heartbeat to masterserver'

Check your CPU Clock Speed with:

lscpu | grep MHz

Map wont load (kicks to datacenter)

try deleting all the downloaded workshop maps with:

sudo rm -rf /tmp/workshop/7777/content/555160/*

for all maps or

sudo rm -rf /tmp/workshop/7777/content/555160/whatever-UGC

for specific map

Server kicks all players at end of SND game with "Connection Lost"

This looks to be a bug related to PavTV. If you don't care about PavTV, you can stop this behaviour by either firewalling off egress to port 80 of 147.75.74.75 or blackhole the traffic with

sudo route add 147.75.74.75 gw 127.0.0.1 lo

This IP address may change, so if this stops working raise the issue in #pc-custom-servers

Getting Help

There's a #pc-custom-servers channel on the official Pavlov Discord - https://discord.gg/Pavlov-VR. Lots of helpful people there!

@@ Line 129: / Line 129: @@
 76541198397498573 #Some Other Player
 </pre>
+==== ArcticVR Admin Menu ====
+Arctic-VR Offers a more advanced In-Game Admin menu that is exclusive to their hosting service. Features include:
+* Switch Maps+Modes
+* Teleport Players to Players
+* Change Player Skins
+* Control Player Buying+Cash
+* Spawn Items
+* Clean up corpses + guns
+* Spawn Bots
+* Spawn Items for other players
+* Control Teams + Balance/Shuffle Teams
+* ResetSND
+* And More
+You can read more about the ArcticVR admin menu on their website: https://www.arctic-vr.com/server-admin-menu
 ==== Blacklist/Whitelist (optional)  ====