Unity Matchmaker Deep Dive
warning
This documentation is now deprecated. If you are using Matchmaker Self Serve though UDash, please use the documentation here.
This page covers advanced scenarios for working with the Multiplay matchmaker.
Matchmaker overview#
Multiplay matchmaking provides secure, fast, custom, and consistent matchmaking for your online game service backend.
Click the following image to view a YouTube video that provides a detailed demonstratation of how matchmaking works with Multiplay's game server hosting.
The matchmaking system features customizable, highly-concurrent, and distributed match functions that process pools of tickets, which represent matchmaking requests for a player or a group of players. The system processes the proposed matches, allocates dedicated game servers on Multiplay’s cloud-scale hybrid game server hosting, and then updates the tickets with connection details so game clients can poll for their assigned server, connect, and play.
Unity packages#
Multiplay matchmaking is completely engine-agnostic. Some Unity packages are provided to make development easier. Similar to other Unity packages, the source code is provided with the package.
| Package | Latest Version | Description |
|---|---|---|
| com.unity.ucg.qos | 0.1.1-preview.4 | A matchmaking client with samples and a request flow |
| com.unity.ucg.matchmaking-client | 0.2.1-preview | A client for communicating with Multiplay QoS servers |
| com.unity.ucg.usqp | 0.1.1-preview.3 | A Multiplay server query protocol package for communicating with game servers |
note
This list represents the most up-to-date list of packages and supersedes any packages that are included with matchmaking samples.
These packages are not currently visible in the Unity package manager. To reference them in your project, you need to add entries directly to your package manifest.json.
Tickets#
A ticket is the basic unit of a matchmaking request. A ticket represents the gameplay intentions for a player or a group of players, and includes any data that is needed by your matchmaking logic to put players together into games.
Tickets include attributes and properties. The contract for a ticket can be found at Ticket Contract
Attributes - indexable fields to use when querying for tickets from inside of match functions. Example attributes include map, mode, platform, players in the ticket, team skill, and games won. Note that attributes are currently limited to
doublevalues.Properties - a map of data that is available after a match function has retrieved tickets from a database. In the APIs, the property map is represented with the format
<string, byte[]>. Thebyte[]must be a Base64-encoded string (see the CreateTicket API documentation).
For more information, see the Tickets API documentation.
Created attribute#
The created global attribute is automatically indexed and made available on all tickets for querying purposes. It consists of a long of Unix UTC milliseconds for when the ticket was created. This allows match functions to consider the pools of tickets that have been waiting for matches for a set amount of time.
Matchmaking Logic#
The following links provide more information about match functions.
Configuration#
Configs are intended to be the management component to matchmaking logic. With a config, you can perform the following actions:
- Define the matchmaking logic
- Determine a pool of tickets for the function to operate over
- Handle any scaling or speed concerns through pool segmentation
- Specify which fleet and profile to allocate dedicated Multiplay servers to for the resulting matches
For more information, see the Configs API documentation. For the config contract, see the Config Contract section
Platform and iteration#
Matchmaking supports the idea of having dev, QA, and release populations, in addition to platform isolation and sandboxing. Use config pools for hard filters and isolation, especially if populations are pointed to different versions of the game server build or fleet.
The following example details the process for performing sandboxed isolation on a single matchmaker.
If your scenario calls for a multitude of isolation requirements, you can use segmentations to generate auto-pools for performing the same style of targeting.
The following example details another method for coding the process for performing sandboxed isolation on a single matchmaker:
The format in this example generates three different pools targeting roles between [1,2), [2,3), and [3,4).
This strategy also works for platform isolation if there is no cross-platform play. However, in many scenarios, it might be easier to write function logic to handle isolation instead of using pool filters to perform this task. Choosing a method for this action depends on whether the isolation scenario is supportable by using static logic (pool filters) or dynamic custom-logic (match functions).
Segmentation and scale#
Segmentation is a method for triggering multiple concurrent runs of a function with distinct query segments. The backend uses the DistributionType with the min and max settings to generate a reasonable curve that represents the population, and then uses the SegmentBehaviorType and the Value settings to generate a new set of ranges.
The supported DistributionType settings are Uniform and Normal.
note
The Auto and LinearApproximation DistributionType settings are currently not supported.
The supported SegmentBehaviorType settings are TargetPercentage, BucketSize, and BucketCount. These settings divide the distribution into new min and max settings.
For example, the preceding example code generates 12 concurrent runs of the function. Each run is activated with generated static filters in the pool, specifically, a skill attribute filter that renders with the min and max settings that are detailed in the following image.
The primary advantage of creating more function runs is to reduce the pool size that each function is responsible for addressing. This process also reduces CPU and memory pressure, alleviates bottlenecks in processing, and enables faster, more predictable burn-through of the total ticket population.
Zero-downtime game server builds#
By uploading a new config file with different multiplay properties, you can achieve Zero-downtime upgrades for your game. profileId is the server configuration + binary that is configured on Multiplay. By swapping profiles, you can perform an in-place server rollout to start routing matches onto the new server builds.
Backfill#
The basic concept behind backfill is that a dedicated game server (DGS) can represent its need for tickets as a special type of ticket (backfill ticket). These tickets can be created, queried and updated by matchmaking logic. The DGS is required to approve these tickets, or replace them with whatever its latest desired state is. There are some caveats to this that will be discussed below. The DGS may also create or delete backfill tickets at any point in time in order to start/stop backfilling.
The main goal of this architecture is to remove the possibility of accidentally creating more servers than are required; all existing DGS with capacity can be represented and observable to matchmaking logic so that it can make the best decision about allocating new servers vs utilizing existing servers via backfill tickets.
Backfill General Overview#
Backfill differs from the general matchmaking case in that a typical matchmaking scenario is some set of players and servers with no current association that are paired together based on various custom algorithms. Backfill has the implication that a match of this sort has already been created, but for some reason, has been rendered incomplete due to a loss of players or a failure of an assigned player to connect. In any event, a key difference is that for the backfill case, a server has already been selected to host a match, and is now expressing a need to get new players.
The following image details methods for how backfill is typically used:
For more information, see the Backfill API documentation.
The following sections detail some backfill use cases by category.
Backfill to keep target player count#
A server might backfill to keep target player count. This could happen for a variety of reasons, such as the following scenarios:
- To replace any players that drop during a match
- To ensure full utilization of a server to reduce costs
- To ensure a target number of players have connected before beginning a match (for example, consider a battle royale where players connect to a holding area while all players are given time to connect or be replaced by backfill before the match starts)
Backfill to balance teams#
In team-based modes, when one team gains a large lead over another team, it is common for players to disconnect before a match is complete. However, many games are designed so gameplay cannot continue fairly or at all without the correct number or type of players. This is especially true if players are tied to specific roles (for example, in class-based game modes).
Example use cases of this scenario include the following events:
- Replace players who drop without rebalancing teams
- Replace dropped players as fast as possible
- Replace dropped players on the losing team with higher skilled players to compensate for the winning team's lead
- Rebalance players between teams, and then replace players
- For example, if there is a team imbalance of 8 to 4, the server could move 2 players so the teams are 6 to 6, and then perform backfill to find comparably skilled players until teams are full again
- Pause the game while waiting for backfill to complete or time out
- For example, role-based games that require specific pairings (such as a driver and shooter pair) where the game cannot continue fairly or at all if a required role is not filled
Backfill Lifecycle Overview#
The below diagram outlines the flow of a BackfillTicket:
In backfill v2, tickets have the following lifecycle:
- Created
- Observed by a matchmaking logic and associated with a backfill ticket
- Tickets wait for the backfill ticket to be approved by the DGS
- If tickets time out, they go back into the matchmaker pool. This occurs if the DGS does not approve the BackfillTicket for various reasons.
- If the DGS rejects the backfill ticket (either by deleting it or updating the backfill ticket with the latest server state), the tickets are also released back into the matchmaker ticket pool
- If the DGS approves the backfill ticket, the tickets receive the assignment data associated with them when they were created and are given the server connection and assigned just like the normal ticket-assignment flow
Note: When a backfill ticket is removed, it will stop receiving backfills from the matchmaker. To start backfilling again, the DGS should create a new backfill ticket.
Matchmaking Logic Example#
The following diagram illustrates the steps matchmaking logic and DGS go through to allocate tickets via a backfill request:
In the example above matchmaking logic executes and finds 5 players available, and no existing backfill tickets. The matchmaking logic can assume “no other server exists at the moment that handles this type of ticket” and make a new backfill ticket associated with the 5 players.
At that point, a new server allocation request associated with that backfill ticket will be made.
While that server is starting, the matchmaker may execute a few more cycles of matchmaking.
When matchmaking logic runs again, they will observe the new backfill ticket and whatever state was written by the matchmaking logic, in this case, the current and needed player counts.
The matchmaking logic can then prioritize filling that backfill ticket by adding all 10 new tickets to a proposal associated with that backfill ticket.
The DGS finally starts and immediately calls the ApproveTicket API which assigns the 15 tickets associated with the backfill ticket to the DGS. The ApproveTicket ticket request returns the latest version of the backfill ticket to the DGS. At this point the DGS now has full access to latest backfill data property written by matchmaking logic.
The matchmakiong logic will continue to add tickets to the backfill ticket, until it is full. When the DGS observes that the backfill ticket state indicates its request for additional tickets has been satisfied, it should delete the ticket. Additionally, the server may at this point have lost players and either update its existing backfill ticket to reflect the new server state or create a new backfill request if one does not already exist. (Covered Below)
DGS Needs to Update Backfill Ticket#
The bellow chart shows what happens when the DGS makes an update to the backfill ticket via the update API
In the above example, the DGS updates the ticket to show that the server has lost 3 players, when matchmaking logic runs next, it picks up this state, and adds 3 more players, on the next few DGS approvals, these new tickets should show up.
NOTE: The matchmaking logic may execute as frequently as every .3 to 3 seconds depending on current load. Typically things run in 1 second or less, but that means the DGS may need to approve tickets multiple times before an updated ticket is observed.
DGS Updates Backfill Ticket Too Frequently#
The below chart shows what happens when the DGS makes an update to the backfill ticket via the update API at the same time the matchmaking logic is attempting to update the last version of the backfill ticket it observed.
The diagram above shows that at the same time the matchmaking logic was about to add 5 tickets, the DGS performed an update that increased the number of tickets desired from 10 to 13. The backfill tickets are versioned to detect these types of stale read/write conflicts and matchmaking logic will always fail as it is not the authority on the state of the ticket!
In the example above, the match tries again, but the same thing happens again!
There is in fact no guarantee things will ever progress if the DGS continues to update tickets faster than the matchmaking logic can execute!
caution
To avoid this situation, have the DGS hold non critical updates until the next approval observes that matchmaking logic has made a change, or at least longer than the maximum matchmaker cycle of 3 seconds. This eliminates the possibility of getting stuck in a state where the DGS is updating tickets and the matchmaker is constantly proposing changes to stale tickets.
Ticket Lifecycle With Backfill#
For any tickets associted with a backfill tickets, the following actions and inactions will update, assign, or release the tickets back into the matchmaking pool:
- The minimum reservation time for a ticket assigned to a backfill server expires, releasing all tickets into the pool. This prevents tickets from being captured permanently by non responsive servers.
- The server accepts the backfill ticket, which assigns all the tickets.
- The server updates/deletes the backfill ticket, which releases all tickets
Backfill API#
Contracts / API#
Backfill involves a few different APIS and contracts, for more information, see the Backfill API documentation and Backfill Contracts.
SDK Support for backfill#
Make sure you have the latest version of the SDK downloaded.
BackfillTicket#
The sdk contains the new backfill ticket (which also appears as json in the backfill API):
The Id, Created, and RecordVersion properties are read-only and should be kept at whatever value they have when a backfill ticket is received via the api. Attempts to set RecordVersion may cause updates to fail, attempts to update the others will simply be ignored.
Attributes and Properties#
Backfill tickets look a lot like normal tickets with respect to attributes and properties. However, backfill ticket attributes and properties are fully mutable.
When creating a backfill ticket, take care to give it attributes that reflect the standard pool filters used in your configuration files "standard" pool.
MatchProperties#
The MatchProperties class that is returned by the sesion API has field for backfill ticket id.
When the DGS starts, it should fetch its session data record from the session data service to see what the matchmaking logic has written there. If the backfill ticket ID is not null, this means the DGS was created via a backfill ticket and there are currently tickets waiting for the DGS to approve them. The typical best course of action here would be to immediately call approve on that backfill ticket to reduce the time to match for those tickets. See section on Calling the Session API from the DGS for more details.
Quality of service#
This section explains how the matchmaking client can provide QoS results when submitting a ticket. It is not assumed that the client is using the provided matchmaking or QoS client sample packages, but operates on the same principles.
This section assumes that you are using Multiplay dedicated server hosting. Accordingly, there is some data that is required for allocating or joining a Multiplay dedicated server. In addition, you should be familiar with the common Multiplay hosting terms.
For QoS API information, see the Multiplay QoS User Guide.
QoS overview#
Allocating a server on Multiplay requires an authenticated client and input for several fields, such as the Fleet ID, Profile ID, and Region ID. The Multiplay QoS protocol ensures that clients can discover the currently active regions where a game server can be allocated or joined, and that players who all have excellent connection quality to a particular region can be grouped together.
Reporting QoS results to match functions#
If you want to use QoS results to group players together by region and dynamically allocate servers from the best available regions, you need to attach QoS results when submitting matchmaking tickets.
When attaching QoS results, you should include the Region ID for each result, which is returned by the Discovery service. When your match function determines which region to use when allocating a server for a match, it can use the Region IDs that are provided from clients when making the Multiplay allocation call.
C# example#
In this example, QoS data is attached to a ticket's properties dictionary as a JSON object. This makes serializing a structure into JSON a natural match. An example structure might be something like the following code snippet (shown in C#):
In this example, the StandardQosResults (the class to be attached to the ticket) contains a dictionary of StandardQosResult with the RegionId as the key. Each StandardQosResult contains the Region ID from the Discovery service, in addition to the average latency and packet loss per-region, which is computed by performing a QoS check through the Multiplay QoS service.
When creating a ticket, the entire StandardQosResults object is serialized into a JSON object and attached to the ticket with "QosResults" as the property name. For example, the code to attach your QoS data to a ticket might look like the following code snippet:
In this example, ticket is the created ticket for the Multiplay matchmaking service, and the JSON object is added to the Properties that are submitted with the ticket and given to the match function.
Determining acceptable QoS limits#
Most real-time games have some thresholds for latency and packet loss above which gameplay is considered unacceptable (gameplay quality is obviously reduced, but is still technically playable) or impossible (players are unable to maintain connection or the game logic does not function properly). Using QoS results in matchmaking can help to prevent players from playing in regions where these limits are exceeded.
When determining these limits, remember to consider both the experience of the player with a poor connection and the experience of other players when playing in a game alongside a player that has a poor connection. For example, some game engines are able to provide a reasonably good experience for a player with high latency or packet loss, but to other players, that player might appear to teleport to a degree that makes gameplay with that player impossible. In practice, this means that you have to process the per-region QoS results that are gathered by the client and then filter out the regions that are unacceptable or impossible for them to play in.
- When filtering QoS results, you can either perform this action before submitting your ticket, which saves some processing time in the match function, or you can submit all of the unfiltered results and then perform the filtering on the match function, which is potentially a more flexible option.
- If a player's best available QoS result is outside of the "acceptable" latency or loss range for your title, but is within the "technically possible" range, you might want to consider filtering out all results except for the player's best result. This allows the player to at least find a match even if it is not a match with a good quality connection. In these scenarios, providing a "connection quality" warning to the player can help to set expectations that there might be gameplay issues.
- If a player's best available QoS result exceeds your impossible threshold, you should generally provide appropriate error messaging and prevent the player from playing.
QoS best practices#
Ensure that you consider the following best practices when working with the Multiplay QoS protocol.
- The latency and packet loss values per region that are included in a ticket should generally be an average or flattening of all responses for each QoS server that is contacted within that region. A weighted average of results that gives more weight to the most recent results might show a more accurate depiction of the current network quality.
- Review your packet loss versus latency ratio. For example, for real-time games, a very low latency connection that would normally result in a standard game session could be impacted by even minimal packet loss.
Multiplay#
Getting the session details#
The session service stores match details, such as config, ticket stats, and properties from a match, every time that a match is created. The dedicated game server (DGS) can access this data after the server starts.
Both the session service and backfill service require authentication. A prerequisite to using the session service is to get a JWT into your Multiplay profile by providing your project ID during the initial onboarding process. The DGS needs to read a config file that contains the session ID and the JWT to use as a bearer auth token when contacting the backfill and session services.
note
The Ping Server Sample has code that demonstrates this flow.
Required setup#
To enable JWT-based authentication, part of setting multiplay server allocations involves providing the matchmaker project ID (UPID) and the Multiplay account service ID (ASID) during the initial setup process. This data generates a JWT that is provided to the server at runtime.
Reading session service JWT and ID on the DGS#
The DGS should be configured to take a path to a config file as a startup argument (for example, -config <path>). The config file can have any format or data in it that you want to include, however, its main purpose is to provide runtime-replaced values for key Multiplay token values.
For the Ping Server Sample, the config has the following format (as config.json):
When the server receives an allocation, the values are replaced at runtime with the real values that are associated with the current allocation:
The Ping Server Sample code polls the file periodically to detect when a new game session has been created.
When the session service JWT and ID ("multiplay_uuid") have been populated, they can make calls against the session or backfill service.
Calling the Session API from the DGS#
Calls to the Session API are expected to be made by the DGS, and assume that you have processed a config file that contains a multiplay_uuid field and a session_auth field.
Base URL: https://server-session.multiplay.com/v1/
GET /session/{multiplay_uuid}
Get stored data for the current session.
multiplay_uuid: The value of the multiplay_uuid field from the processed config file
Authorization: Authorization <bearer JWT> (JWT from the session_auth field of the config file)
Response: 200 (MatchProperties)
For example: GET https://server-session.multiplay.com/v1/session/d2adb890-b423-4f08-9a6b-55eff0e79a4d
Extra contract dependencies for MatchProperties:
Calling the Backfill API from the DGS#
The backfill endpoint uses the same auth (JWT from the session_auth field, which is detailed in the Reading session service JWT and ID section) as the auth that is required by the session service.
The backfill endpoint takes a subset of the match configuration that is stored in the session service. This can be data that is retrieved from the session service or is created specifically by the DGS based on custom logic, provided that it conforms to the extensible contract that is defined by the Backfill API.
For API details and contracts, see the Backfill API documentation.