Dedicated server

From Pavlov VR Wiki
Revision as of 19:30, 23 October 2019 by Invicta nz (talk | contribs) (Server segfaults on round end/map change)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Setting up a dedicated server

SCAM WARNING: You have to setup your own VPS to run a community server. If you find [Pavlov VR server hosting - Zap-Hosting - ZAP-Hosting.com][1] which is currently top hit for "pavlov server hosting" this is a SCAM with no ability to actually host Pavlov. Will take your money and provide no service. They have recently started calling this a "pre-order"


There are no Windows binaries, this guide is exclusively for Ubuntu/Debian/Debian-like Linux distros. Possible to run with other distros, but some of the commands won't work and you will need to know enough to figure out the software install of dependencies on your own.

This guide assumes you know the basics of using a bash terminal: ssh'ing into a server, running bash commands, editing text files, etc.

If you have no experience using bash, maybe start [here][2]


Minimum Specs

Believe (but haven't gotten confirmation) that 64bit OS is required. Ubuntu 18.04 x86_64 and Ubuntu 19/04 amd64 are confirmed working. Ubuntu 15.04 x86 is known to fail to install using this guide


2GB RAM
2.8Ghz CPU for a 10 player server
4.2Ghz i7 CPU for a 50 player server

Prerequisites

Install dependencies and create the `steam` user. (You only need to do this once):

sudo apt update && sudo apt install -y gdb curl lib32gcc1  && sudo useradd -m steam

Login as the steam user. (You will need to do this each time you connect to your server):

sudo su -l steam -s /bin/bash

Install Steam:

mkdir ~/Steam && cd ~/Steam && curl -sqL "https://steamcdn-a.akamaihd.net/client/installer/steamcmd_linux.tar.gz" | tar zxvf -

Install the Pavlov Server (You will need to to do this each time there is a Pavlov Server update):

cd ~/Steam && ./steamcmd.sh +login anonymous +force_install_dir /home/steam/pavlovserver +app_update 622970 +exit

Make the PavlovServer script executable. (You only need to do this once):

chmod +x ~/pavlovserver/PavlovServer.sh

Server Configuration

Before starting the server we need to configure it, create a file at /home/steam/pavlovserver/Pavlov/Saved/Config/LinuxServer/Game.ini and open it with your preferred text editor, Vim in this case.

mkdir -p /home/steam/pavlovserver/Pavlov/Saved/Config/LinuxServer
vim /home/steam/pavlovserver/Pavlov/Saved/Config/LinuxServer/Game.ini

Below is an example of a server config file.

[/Script/Pavlov.DedicatedServer]
bEnabled=true
ServerName="My dedicated server"
MaxPlayers=50
bSecured=true
MapRotation=(MapId="UGC1668673188")
MapRotation=(MapId="datacenter", GameMode="SND")
MapRotation=(MapId="sand", GameMode="DM")
MapRotation=(MapId="bridge", GameMode="TDM")
  • bEnabled - whether the server appears in the server list (unconfirmed)
  • ServerName - name it will be listed as. (approx 35 chars max)
  • MaxPlayers - See suggested specs for limits
  • bSecured - enabled Valve Anti-Cheat (VAC)
  • MapRotation - you can add multiple of these. The MapId must be “UGC” followed by the map ID. you can find the map id in the URL of the steam workshop page.

Available game modes as of 10th of May 2019: SND, TDM, DM, GUN.

Maps running a Custom Game Mode (TTT, BR, Duel, etc.) don't care which GameMode you chose.

Admin/Moderator list

(optional) If you add moderators to your server, they will have an "admin panel" in their menu which will allow them to:

  • kill player
  • teleport to player
  • kick player
  • ban player
  • rotate to next map
vim ~/pavlovserver/Pavlov/Saved/Config/mods.txt

Example Mod List:

90000000000000111 # Some Player
90000000000000112 # Some Other Player

Steam ID finder: https://steamidfinder.com/lookup/



Starting the server

cd ~/pavlovserver && ./PavlovServer.sh

That's it, your server will be automatically broadcast to the master server! :)

Note that when you close your terminal, the server will go down. See the "supervisors" section below for how to prevent this.


You have now completely setup the server, Check below for Advanced Troubleshooting / Administration

Hosting Providers

Considering the suggested server specs, you may be interested in the following hosting providers:

Provider              | Base CPU | Details
OVH                   | 4.5 GHz  | https://www.ovh.com.au/dedicated-servers/game/
Amazon Web Services   | 4.0 Ghz  | https://aws.amazon.com/ec2/instance-types/z1d/
packet.com            | 3.5 GHz  | https://www.packet.com/cloud/servers/c1-small/
Google Cloud Platform | 2.6 GHz  | https://cloud.google.com/compute/docs/cpu-platforms

Amazon Web Services

Login -> Change Region: Choose your region
Services -> Compute -> EC2 -> Create
Choose an AMI: Ubuntu 18.04
Instance Type: z1d.large

Instance Details -> Tenancy: Dedicated (optional: this will more than triple the cost)

Storage: 50gb disk (or whatever, you're in control of the map roster)

Security Groups:
Create a new Security Group
Security group name: pavlov-server
Description: pavlov server
Rules:
Type: Custom ICMP Rule - IPV4 / Protocol: Echo Request
Type: Custom UDP Rule / Port Range: 7000 / Source: 0.0.0.0/0
Type: Custom UDP Rule / Port Range: 7200 / Source: 0.0.0.0/0
Type: Custom UDP Rule / Port Range: 7777 / Source: 0.0.0.0/0
Type: SSH (optional, but I assume you wanna SSH in..)

Launch -> Create new Keypair -> Download -> Launch

Once it's started, SSH in like:
$ ssh -i your-key.pem ubuntu@you-server-details.amazonaws.com

(I use cmder on Windows, if you're using putty or mobaxterm or something, might be different)

Google Cloud Platform

The below configuration is successfully running 4 pavlov instances with ~15 total maps and 15-20 players total. If you want more maps use larger boot disk. Both can dynamically be scaled based on popularity.



Menu -> compute engine -> VM instances
create instance
- region: wherever you like
- machine type: n1-highcpu-4 (4 vCPUs, 3.6 GB memory)
- boot disk: Ubuntu 18.04
- boot disk -> size: 25gb
- network tags: pavlov-server
- [create]

- Menu -> Compute Engine -> Metadata -> SSH Keys
- [edit] -> [add item]
- "Enter entire key data": the syntax must be `<key> <username>`
- [save]

Menu -> Vpc network -> firewall rules
[Create firewall rule]
Name: pavlov server
Target Tags: pavlov-server
Source IP Ranges: 0.0.0.0/0
Protocols and Ports -> Specified protocols and ports -> UDP: 7, 7000, 7200, 7777
[save]

Advanced Server Administration

Running multiple servers on one host

  • Make multiple server installs by defining a separate install dir for each server.
  • Run each server with a unique port. eg: ./PavlovServer.sh -PORT="Desired Port"
  • Ensure the ports are forwarded and opened in your firewall

To duplicate directories:

be in the steam home folder

cp -R pavlovserver pavlovserver1

/home/steam/pavlovserver1/PavlovServer.sh -PORT=whatever



Supervising the server

Running ./PavlovServer.sh in your interactive shell is easy, but the server will shut down when you close your terminal. If you want to run the server in the background, you'll want to run it under a supervisor. There are many options, ranging from the simple (nohup, screen or tmux), to the more robust (supervisord, systemd).

Using Systemd

Systemd is running by default on Ubuntu Linux, so we'll use that here.

To run the server as a systemd service, create the file /etc/systemd/system/pavlovserver.service (as root) with the contents

[Unit]
Description=Pavlov VR dedicated server

[Service]
Type=simple
WorkingDirectory=/home/steam/pavlovserver
ExecStart=/home/steam/pavlovserver/PavlovServer.sh
# XXX the server will often crash on round end/map change with a segfault
# until dave hires more programmers, just restart the server every time
RestartSec=1
Restart=always
User=steam
Group=steam

[Install]
WantedBy = multi-user.target

Change the user or paths as appropriate. After saving the service file, to start, restart, stop, or monitor the server:

sudo systemctl start pavlovserver
sudo systemctl restart pavlovserver
sudo systemctl stop pavlovserver
sudo systemctl status pavlovserver

To view the logs:

sudo journalctl -u pavlovserver
# to live-tail the logs
sudo journalctl -u pavlovserver -f

If you want to run the pavlov server as soon as your server boots (e.g. a VPS you turn on and off), add pavlov as a startup service:

sudo systemctl enable pavlovserver

Here's some executables you can create to help, after setting these up they should be the only commands you need to run on your server. Descriptions on what they are, and how to install and use them are in the comment blocks.

#!/bin/bash -e
# Copy txt of this script to /usr/bin/pav-edit
# chmod a+x /usr/bin/pav-edit
# usage: pav-edit
# stops the server, edits the config, updates the server, starts the server, then tails the logs
# doesn't need to be run as the "steam" user
# does assume you've setup systemd as per the wiki

echo "stopping server..."
sudo service pavlovserver stop

echo "opening config..."
sudo vim /home/steam/pavlovserver/Pavlov/Saved/Config/LinuxServer/Game.ini

echo "updating server binary..."
cd /home/steam/Steam
sudo -u steam ./steamcmd.sh +login anonymous +force_install_dir /home/steam/pavlovserver +app_update 622970 +exit

echo "starting server..."
sudo service pavlovserver start

echo "tailing logs..."
/usr/bin/pav-logs

echo "pav-edit complete"
#!/bin/bash -e
# Copy txt of this script to /usr/bin/pav-logs
# chmod a+x /usr/bin/pav-logs
# usage: pav-logs
# tails the logs


tail -f -n1000 /home/steam/pavlovserver/Pavlov/Saved/Logs/Pavlov.log | grep -v 'Sending heartbeat to masterserver'

Troubleshooting

To run the server without outputting the heartbeat:

./PavlovServer.sh | sed '/heartbeat/d'

To monitor the logs (without heartbeat) from another terminal:

tail -f -n10000 /home/steam/pavlovserver/Pavlov/Saved/Logs/Pavlov.log | grep -v 'Sending heartbeat to masterserver'

Check your CPU Clock Speed with:

lscpu | grep MHz

Map wont load (kicks to datacenter)

try deleting all the downloaded workshop maps with:

sudo rm -rf /tmp/workshop/7777/content/555160/*
for all maps or
sudo rm -rf /tmp/workshop/7777/content/555160/whatever-UGC
for specific map

Server segfaults on round end/map change

If your server hangs at the end of the round and you see errors in the log similar to

Unhandled Exception: SIGSEGV: invalid attempt to read memory at address 0x000000000000390a

It's a known issue. There's no real workaround except to restart the server once it fails. The config in #Using Systemd will do this for you, or other supervisors can do the same.

After running servers with 1, 3, 5 and 10 maps in active use, I am very confident that the chance of crash is directly related to number of maps in rotation. With one map < 5%, with 3-5 maps %25-40, with 10 maps 100%. I have taken to running 3-4 instances with maps we are interested in playing and rotating by changing instances instead of within server.

If you have a map rotation, a simple workaround to the rotation starting at the first map after every round is to randomize map rotation every time the server restarts. e.g.

In PavlovServer.sh before the last line that launches the server insert the following 2 lines. Note that the command is one long line... copy paste may add line break which you will need to remove

 

# shuffle map rotation on start to workaround crashing bug
cat "$UE4_PROJECT_ROOT/Pavlov/Saved/Config/LinuxServer/Game.ini" | awk 'NR<2{print $0;next}{print $0| "shuf"}' > $UE4_PROJECT_ROOT/Pavlov/Saved/Config/LinuxServer/Game.ini_tmp && cp $UE4_PROJECT_ROOT/Pavlov/Saved/Config/LinuxServer/Game.ini_tmp $UE4_PROJECT_ROOT/Pavlov/Saved/Config/LinuxServer/Game.ini

There's still a chance you'll get a map repeat of course, but it'll be rarer.

Getting Help

There's a #custom-servers channel on the official Pavlov Discord - https://discord.gg/Pavlov-VR. Lots of helpful people there!